NIH | National Cancer Institute | NCI Wiki  

This page describes how to register multiple files. For prerequisites when using Globus with DME, refer to Preparing to Use Globus. If you want to provide metadata for each object (data file) or collection, also refer to Preparing a Metadata File for Bulk Registration.

For narrated slides demonstrating these instructions, refer to upload-Globus-slides.pptx.

To register data files:

  1. Log in as described in Logging In via the GUI. The Dashboard appears.
  2. Browse for the data destination, as described in Browsing for Data via the GUI. Navigate to and right-click the collection where you want to register your data files. Click Add Bulk. (Another option is as follows: Click Register tab > Bulk.) The top portion of the Register Bulk Data page appears.

    The top portion of the Register Bulk Data page.

  3. Specify the data source: 

    1. Select Globus and click Select Data from Globus Endpoint. A Globus page appears. 

    2. In Globus, select an endpoint.

    3. Select the files or folders that you want to register into DME.
    4. Click Submit. The Register Bulk Data page reappears, with the Globus endpoint ID and path. (Depending on your selections in Globus, the Register Bulk Data page might also list the selected data files or folders.)

    The Register Bulk Data page with data selected from Globus.

  4. If you want to provide required metadata for each object or collection, click Choose File. Navigate to and select the prepared metadata file. (For details, refer to Preparing a Metadata File for Bulk Registration.) 
    Choose File button for the bulk metadata file.
  5. Scroll down to the Filter panel. 
    The middle portion of the Register Bulk Data page.

    Consider the following filter options:

    FieldInstructions
    Include CriteriaUse patterns to specify the source files to include. If you specify more than one pattern, the system considers a union of all patterns. For details, refer to Specifying Include Criteria.
    Exclude CriteriaUse patterns to specify the source files to exclude. If you specify more than one pattern, the system considers a union of all patterns. For details, refer to Specifying Include Criteria.
    Criteria Type

    Specify the type of patterns in your criteria:

  6. Scroll down to the remaining portions of the Register Bulk Data page. If you browsed to the data destination, that portion of the page has only the Collection Path field, with the path already specified. 
    The remaining portions of the Register Bulk Data page.

    1. If necessary, specify the data destination for the parent collection (the collection that will contain all of the new data):

      1. If the Base Path field is available, select the base path specified by your group administrator. An information icon (The information icon for the base path.) appears next to the Base Path field and the system begins to populate values in the Collection Type field.  
        A portion of the Register Collection page with the hierarchy icon.

        Consider examining the valid hierarchy for the selected base path. To do so, click the information icon next to the Base Path field. A Data Hierarchy and Metadata Structure chart appears. For details, refer to Viewing the Data Hierarchy and Metadata Structure for an Archive. 

      2. If the Collection Type field is available, and if there is more than one collection type, select the one in which you want to register data. For guidance on selecting a collection type, refer to your group administrator. For some collection types, the system displays a list of required metadata attributes. 

      3. In the Collection Path field, specify the full path, including the base path and the name of the collection in which you intend to register bulk data. Avoid using invalid characters such as the space character, question mark (?), semicolon (;), backslash (\), or double quote ("). Consider the following example: 

        PathExample
        Base path/SAMPLE_Archive
        Collection path/SAMPLE_Archive/Sample_Collection_Name

        The last collection in the path can be new or existing.  

    2. Specify the metadata for the parent collection. The system applies this metadata to the entire collection, not to individual files:

      1. To add a metadata attribute: 
        1. Click Add Metadata, visible on the right or left side of the page. A blank attribute row appears. 

          Attribute portion of the page with a new attribute.
        2. Specify a unique attribute name. 

        If you change your mind about adding an attribute, click the trash can icon next to that attribute. If you proceed to update the collection with a new attribute, the attribute name is permanent.

      2. In each attribute row, specify a unique value that describes the content you are registering. The character limit for each metadata value is 2700.
        Example AttributeExample Value
        data_ownerJane Doe
        project_id1234567890
        sample_nameL1
        project_start_date2020-12-31

        For some date attributes, such as project_start_date, the system expects the "yyyy-MM-dd" format, as in the above example. 

  7. If you want to preview a list of the source file(s)/folder(s) that the system would register based on what you specified in the Data Source fields, click Dry Run. The system displays a list of the source file(s)/folder(s) that it would have registered based on what you specified in the Data Source fields. If necessary, revise your entries and click Dry Run again until you are satisfied with the dry run list. Keep in mind the following points:

    • This option tests only the Data Source entries. It does not test the Data Destination entries.

    • If you specified a metadata file, specify it again after each dry run. 

  8. When you are ready to perform the registration, click Register. The system responds as follows:

    • The system checks whether it can access the objects and collections you have specified, using the data you have entered:

      • If not, the system displays an error message. 
      • If so, the system responds based on your selections and displays a message at the top of the Register Bulk Data page with the task ID of the registration request. 
    • Depending on your event subscriptions, the system might send you an email notification of the registration status. For instructions on subscribing, refer to Subscribing to Download and Registration Notifications.
  9. When the system displays the task ID, consider clicking that link to visit the Data Registration Task Details page and view the progress of the registration. If you provided a metadata file, this page indicates any difficulty processing that metadata. For instructions, refer to Viewing the Details of a Registration Task.