If you need to move a large volume of data, you can do so using an Amazon Web Services (AWS) S3 bucket. DME uses AWS for high performance transfer of large files.
To prepare an S3 bucket:
In AWS, create a bucket, create an access policy, create an IAM user, get access keys for that user, and set up permissions for the bucket, as described in AWS user documentation:
https://docs.aws.amazon.com/s3/- Gather required information:
- The name of the destination S3 bucket
- The path to and the name of the file or folder to be created in the destination bucket
- The access key and secret key
- The S3 region
For instructions on locating this information, refer to AWS user documentation.
If you intend to use an S3 bucket with the DME GUI, plan to enter the above information in the GUI. If you intend to use an S3 bucket with the DME CLU, continue as follows:
Use a command line editor (such as vi editor) to create a file with the filename credentials that specifies S3 credential information, as follows:
[default] aws_access_key_id = <access key> aws_secret_access_key = <secret access key> region = <s3 region>
Consider placing the credentials file in the default location for CLU commands, as follows:
~/.aws/credentials
Now you can download data as described in the following pages:
- Downloading Multiple Collections via the GUI
- Downloading a Single Collection via the GUI
- Downloading Multiple Data Files via the GUI
- Downloading a Single Data File via the GUI
- Downloading One or More Collections to S3 via the CLU — dm_download_collection_s3
- Downloading One or More Files to S3 via the CLU — dm_download_dataobject_s3