Page History

...

The dm_register_dataobject_multipart command gets a pre-signed URL from DME and uses it to upload data directly to the NCI Data Vault, instead of through DME. If the file is 50 MB or larger, the command registers the file using a multipart presigned URL. If the file is smaller than that threshold, the command registers the file using a single part presigned URL.

To register a data file:

1. Choose whether to upload synchronously (from your file system) or asynchronously (from a Globus endpoint). 
   • To upload from your file system, plan to use the source file parameter to specify the file that you want to upload.
   • To upload from Globus, plan to use a JSON file to specify the file that you want to upload. 

2. In your file system, create a JSON file that specifies the metadata for the new data file. The contents of this file depend on the source of your data:

   If you are uploading from your file system, specify the metadata that you want to upload.

    Click the following link to view the syntax:

   Code Block
   collapse true
   { "metadataEntries": [ { "attribute": "description", "value": "my-dataObject-description" }, { "attribute": "my-second-attribute-name", "value": "my-second-attribute-value" } ] }

3. If you are uploading from Globus, specify the Globus endpoint, the file path on that endpoint, and the metadata that you want to upload. Click the following link to view the syntax:

   Code Block
   collapse true
   { "source": { "fileContainerId": "globus-shared-endpoint-uid", "fileId": "file-path-on-shared-globus-endpoint" }, "metadataEntries": [ { "attribute": "description", "value": "my-file-description" }, { "attribute": "my-second-attribute-name", "value": "my-second-attribute-description" } ] }

4. Run the following command:

   Code Block
   dm_register_dataobject_multipart <description.json> <destination-path> [source<source-file]file>

   The following table describes each parameter:

   Parameter	Description
   [-h]	If you want to print a usage (help) message for this command, specify this option.
   <description.json>	A path to the JSON file that specifies the metadata for the new data file.
   <destination-path>	A path within DME, including the name of the file you intend to upload. Specify where you want the system to create the new data file. (If you specify an existing data file, this command updates the metadata for that data file. For details, refer to Updating Data File Metadata via the CLU.)
   [source-file]	A path to a file in your file system: <source-file>	If you are uploading from your file system, use this parameter to specify the file that you want to upload .If you are uploading from Globus, omit this parameter .

For example, the following command uploads the data.txt file from the JaneDoe folder in the file system to the Project_New collection in DME:

dm_register_dataobject_multipart /cygdrive/c/Users/JaneDoe/my-metadata.json /Example_Archive/PI_Lab1/Project_New/Data.txt /cygdrive/c/Users/JaneDoe/data.txt

...