Page History

If you need to move a large volume of data, you can do so using an Amazon Web Services ( AWS ) S3 bucket. DME uses AWS for high performance transfer of large files.

To prepare an S3 bucket:

 If you intend to use an S3 bucket with the DME CLU, prepare as follows:

1. Use

2. In AWS, create a bucket, create an access policy, create an IAM user, get access keys for that user, and set up permissions for the bucket, as described in AWS user documentation:
    https://docs.aws.amazon.com/s3/ 

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true PageWithExcerpt wikicontent:Exit Disclaimer to Include

3. Gather required information: 
   • The name of the destination S3 bucket
   • The path to and the name of the file or folder to be created in the destination bucket 
   • The access key and secret key
   • The S3 region

   For instructions on locating this information, refer to AWS user documentation. 

4. In your ~/.aws/ folder, use a command line editor (such as vi editor) to create a file with the filename credentials that specifies S3 credential information, as follows:

   Code Block
   [default] aws_access_key_id = <access key> aws_secret_access_key = <secret access key> region = <s3 region>

5. Consider placing the credentials file in the default location for CLU commands, as follows:

   code

   Panel
   borderColor silver borderStyle solid
   Clipboard AllowLineWrap true
   ~/.aws/credentials

Now you can download data as described in the following pages:

• Downloading Multiple Collections via the GUI
• Downloading a Single Collection via the GUI
• Downloading Multiple Data Files via the GUI
• Downloading a Single Data File via the GUI
• Downloading One or More Collections to S3 via the CLU — dmUsing dm_download_collection_s3Downloading One or More Files to S3 via the CLU — dm 
• Using dm_download_dataobject_s3