NIH | National Cancer Institute | NCI Wiki  

If your user account has the Read permission level on various collections, you can download one or more data files in those collections, from DME to a Globus endpoint. 

To download one or more data files to a Globus endpoint:

  1. Prepare a Globus endpoint, as described in Preparing to Use Globus.

  2. Consider how you want to specify the Globus endpoint:

    • If you want to use the default Globus endpoint in your hpcdme.properties file, as described in Getting Started with DME CLU, plan to omit the endpoint from the command. 
    • If you want to specify a different Globus endpoint, plan to include the endpoint in the command.

  3. Consider whether you want to download a single data file or multiple data files: 

    • To download a single data file: Plan to specify the path for that data file in the command. 
    • To download multiple data files: In your local system, use a command line editor (such as vi editor) to create a file that lists the paths for all of the DME data files you want to download, delimited by newline. Plan to use the -f option to specify that file in the command. 

  4. Run the following command:

    dm_download_dataobject_globus [optional parameters] [DME data path] <globus-destination-path> [globus-endpoint]


    The following table describes each parameter:

    ParameterDescription
    [-D <REST-response>]

    An optional parameter, specifying a path and filename in your local system. The system always creates a response file:

    • If you specify this parameter, the system saves the response from the server to the specified file in the specified location.
    • If you omit this parameter, the system saves the file as download-dataobject-response-header.tmp in your home directory.
    [-o <output-json-file>]

    An optional parameter, specifying a path and filename in your local system. The system always creates an output file:

    • If you specify this parameter, the system saves the output to the specified file in the specified location.
    • If you omit this parameter, the system saves the output as download-dataobject-response-message.json.tmp in your home directory.

    If the command is successful, the output file is empty.

    [-f <paths-file>]
    or
    [DME data path]

    One or more paths within DME. Select one of the following methods to specify the data file or data files that you want to download:

    • To specify multiple data files, use the -f parameter to specify a path and filename in your local system, of a file that lists the paths for all of the DME data files you want to download, delimited by newline.
    • To specify a single data file, specify the path for the DME data file you want to download.
    <globus-destination-path>
    		A path from the Globus endpoint. Specify where you want the system to create the downloaded file or files. If the destination folder structure you specify does not already exist, DME creates it. 
    [globus-endpoint]
    		The UUID of the Globus endpoint that you want to use, if not the one specified in your hpcdme.properties file, as described in Getting Started with DME CLU. (The following instructions may be helpful: Copying the Globus Shared Endpoint UUID.)

Single File Example

The following example downloads a data file from DME. 

dm_download_dataobject_globus /Example_Archive/PI_Lab1/Project_1/data.txt /folder1/subfolder1/file1.txt


In this example, the command performs the following:

  • Locates or creates a folder1 folder at the root of the Globus endpoint specified in the hpcdme.properties file.
  • Locates or creates a subfolder1 folder within the folder1 folder.
  • Downloads the data.txt file from the Project_1 collection in DME.
  • Saves that file in the subfolder1 folder as file1.txt.

Multiple Files Example

For another example, consider the following command.

dm_download_dataobject_globus -f file-list.txt /


With the following code in the specified file-list.txt file, the above command downloads multiple data files from various locations in DME.

/Example_Archive/PI_Lab2/Project_1/data.txt 
/Example_Archive/PI_Lab2/Project_2/image.jpg 
/Example_Archive/PI_Lab3/Project_1/image.png

In this example, the command performs the following:

  • At the root of the Globus endpoint, locates or creates a folder structure matching the DME folder structure as listed in the specified file-list.txt file.
  • Downloads from DME all files listed in the specified file-list.txt file.
  • Saves those files in the new folder structure with the same file names they have in DME. 


  • No labels