If you decide to use the CLU to perform various activities in DME, the following steps are pre-requisites:
Check which version of Java is currently installed:
java –version |
The command output should display the version of Java. DME requires Java 1.8.x.
Download Java 1.8.x from the following page: https://www.java.com/en/download/
Check whether you have performed the previous steps correctly by repeating the java -version command.
If the latest version of JDK is not installed, perform the following steps:
In a cmd or shell window, type the following command in your working directory to clone DME utilities to your local machine:
Git clone https://github.com/CBIIT/HPC_DME_APIs |
The system creates a subdirectory (HPC_DME_APIs) containing all files from that GitHub repository.
In the HPC_DME_APIs/utils/ folder, create a copy of the sample configuration file (hpcdme.properties-sample) and rename it (as hpcdme.properties). Edit your hpcdme.properties file to make the following configuration changes:
Property | Instructions | |
---|---|---|
Server settings | If you want to connect to the UAT tier, uncomment the UAT server settings and comment out the Production server settings. By default, the Production server settings are enabled. | |
hpc.ssl.keystore.password | Contact NCIDataVault@mail.nih.gov to request this password. | |
hpc.user | Specify your NIH user ID. | |
hpc.default.globus.endpoint | If you intend to transfer files using Globus, specify the UUID of the Globus endpoint that you want to use. (The following instructions may be helpful: Copying the Globus Shared Endpoint UUID.) | |
hpc.server.proxy.port | If you intend to use Biowulf, specify the proxy port. To determine the correct port, run the following echo command:
The output includes the port. For example, if the output of the echo command is http://dtn08-e0:3128, then your port is 3128. | |
hpc.server.proxy.url | If you intend to use Biowulf, specify the proxy URL. To determine the correct URL, ping the output of the echo command. For example:
The output includes an IP address. Specify that IP address as the URL. | |
hpc.error-log.dir hpc.log.file | If you are using Windows, consider specifying a non-default location for the error log file. For example:
|
.
Configure environment variables:
Navigate to your ~/.bashrc or ~/.profile file, where ~ represents your home directory. For example:
C:\cygwin64\home\<username> |
Edit your ~/.bashrc or ~/.profile file to append the following commands, where <client_utils_home> is the absolute path to the HPC_DME_APIs\utils\ folder. The following commands define the HPC_DM_UTILS environment variable and update the PATH environment variable to include the path to the scripts folder:
export HPC_DM_UTILS="<client_utils_home>" export PATH=$HPC_DM_UTILS/scripts/:$PATH |
If you are using Cygwin on Windows, run the following commands in Cygwin to change end-of-line delimiters from Windows standard to Unix standard in any files involved in CLU setup:
Adjust the line delimiters in the file with your export commands (~/.bashrc or ~/.profile). For example:
sed -i "s/\r$//" ~/.bashrc |
Adjust the line delimiters in all files in the scripts folder. Use the following command:
sed -i "s/\r$//" $HPC_DM_UTILS/scripts/* |
To make the above changes take effect in current Bash session, source the modified file (~/.bashrc or ~/.profile). For example:
source ~/.bashrc |
Generate a DME API authentication token, as described in Generating a DME API Authentication Token via the CLU.
To view usage information for any DME CLU command, type the command with the help (-h) parameter. For detailed information on each of the available DME CLU commands, refer to the following pages:
Getting Started
Generating a DME API Authentication Token via the CLU — dm_generate_token
Finding Data
Using Bookmarks
Adding a Bookmark via the CLU — dm_add_bookmark
Retrieving Metadata
Retrieving the Metadata of a Collection via the CLU — dm_get_collection
Retrieving the Metadata of a Data File via the CLU — dm_get_dataobject
Registering Data
Creating an Empty Collection via the CLU — dm_register_collection
Registering Directory Contents from Your File System via the CLU — dm_register_directory
Updating Collection Metadata via the CLU — dm_register_collection
Downloading Data
Using DME as an Administrator
Registering a User via the CLU — dm_register_user
Adding a Bookmark and Permissions via the CLU — dm_add_bookmark
|
If you want to use DME CLU on a Mac (version 10.12.4/10.12.6) and the version of curl on your machine is OpenSSL 0.9.8zh 14 Jan 2016, perform the following steps to rebuild curl.
From a terminal prompt, execute the following command:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install) |
This command downloads and installs Homebrew, a package manager to download, build, and install Mac software.
Follow the prompts. The system might request your username/password multiple times.
Execute the following command:
brew install curl --with-openssl |
This command invokes Homebrew to build and install curl in /usr/local/opt with the latest version of openssl. At the end of the installation, a message appears indicating that this is keg-only. It provides the command to update the PATH so that the system uses your new curl installation, rather than the Mac-supplied one.
Execute the command as provided at the end of curl installation, which should be the following command:
export PATH="/usr/local/opt/curl/bin:$PATH"' >> ~/.bash_profile |
Confirm the installation: Open a new terminal and check the version of curl with the following command:
curl –-version |
The following information should appear:
curl 7.58.0 (x86_64-apple-darwin16.7.0) libcurl/7.58.0 OpenSSL/1.0.2n zlib/1.2.8 Release-Date: <today’s date or the date you install your new curl> Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp Features: AsynchDNS IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets HTTPS-proxy |