![]() |
Page History
If you need to move a large volume of data, you can do so using an Amazon Web Services ( AWS ) S3 bucket. DME uses AWS for high performance transfer of large files.
To prepare an S3 bucket:
If you intend to use an S3 bucket with the DME CLU, prepare as follows:
Use a command line editor (such as vi editor) to create a
In AWS, create a bucket, create an access policy, create an IAM user, get access keys for that user, and set up permissions for the bucket, as described in AWS user documentation:
https://docs.aws.amazon.com/s3/Multiexcerpt include MultiExcerptName ExitDisclaimer nopanel true PageWithExcerpt wikicontent:Exit Disclaimer to Include - Gather required information:
- The name of the destination S3 bucket
- The path to and the name of the file or folder to be created in the destination bucket
- The access key and secret key
- The S3 region
For instructions on locating this information, refer to AWS user documentation.
In your ~/.aws/ folder, create a file with the filename credentials that specifies S3 credential information, as follows:
Code Block [default] aws_access_key_id = <access key> aws_secret_access_key = <secret access key> region = <s3 region>
Consider placing the credentials file in the default location for CLU commands, as follows:
Code Block ~/.aws/credentials
If you are using Cygwin on Windows, run the following command in Cygwin to change end-of-line delimiters from Windows standard to Unix standard in the credentials file:
Panel borderColor silver borderStyle solid Code Block sed -i "s/\r$//" Clipboard AllowLineWrap true ~/.aws/credentials
Now you can
...
download data as described in the following pages:
- Using dm_download_collection_s3
- Using dm_download_dataobject_s3
- Downloading a File or Collection to S3 via the CLU - 20190513
- Downloading One or More Collections to S3 via the CLU
- Downloading One or More Data Files to S3 via the CLU