Uploading Multiple Assets from a Globus Endpoint - 2021-08-04

You can upload one or more folders from a Globus endpoint to a study in MoDaC. Each folder from Globus becomes an asset in that study. 

When you upload data, you become the owner of that data. (Ownership cannot be changed.) 

1. If you want to provide metadata for each asset (or each file), prepare that metadata in a Microsoft Excel XLSX file. 

   Instructions to prepare metadata file

   1. Download the example metadata file from the following GitHub location:
      
      https://github.com/CBIIT/nci-doe-data-sharing/blob/master/doc/training/samples/example_MoDaC_bulk_metadata.xlsx
   2. Revise the file to reflect the assets (and files) you intend to specify for registration:
      • Specify a row for each of your assets and each of your files.
      • In the path column, specify the location of each asset and each file within your Globus endpoint. 
      • Specify required metadata for each asset:
        • In the access_group column, specify either public or the group name. (TBD: Separate group names with commas?)
        • In the collection_type column, always specify Asset as the collection type.
        • In the asset_type column, specify either Model or Dataset. 
        • If Model is the asset type, the system requires platform_name, framework, and domain. 
          • Valid values for platform name are AMPL, Candle, None, or Other. 
          • Valid values for framework are Tensorflow, TensorRT, PyTorch, Caffe, Caffe2, Scikit-learn, Keras, DALI, ONNX, or Other. 
        • For both asset types, the system requires asset_identifier, asset_name, and description. 
      • (Optional) In additional columns, specify user-defined attributes for each asset and each file. 
      • Each file can be any file type.  

   For questions, select About > Contact. 

2. Log in, as described in Logging In. 
3. Click Upload in the header. The upload page appears. Navigate to the study in which you intend to create new assets:
   1. Select an existing program or add a new one. 
   2. Select an existing study or add a new one. 

   For instructions, refer to Adding a Collection.

4. Click Upload Asset(s). The Globus panel appears. 
5. Select Select Assets from Globus Endpoint. A Globus page appears. 
6. In Globus, select an endpoint. Select the folders that you want to upload. Click Submit. The upload page reappears with the Globus endpoint ID and path. (Depending on your selections in Globus, the upload page might also list the selected data files.) 
7. Specify whether you want to upload a bulk metadata file:
   • If you select Yes, then click Choose File. Navigate to and select your file. The filename appears next to the Choose File button.
   • If you select No, then select the asset type and specify all required metadata. If you want to add metadata, click Add Metadata. The system adds a row to the metadata table. Specify an attribute name and an attribute value. (Repeat for additional attributes. To delete an attribute, click the X.) If an attribute value begins with http or https, MoDaC displays it as a hyperlink in read-only fields.
8. Click Upload. The system transfers the data based on your selections. 
9. When the system displays the task ID, consider viewing the status of the upload. For instructions, refer to Viewing Download or Upload Status.