NIH | National Cancer Institute | NCI Wiki  

If you decide to use the CLU to perform various activities in DME, the following steps are pre-requisites: 

  1. If you are using Windows, install Cygwin as described in the following site: https://www.cygwin.com/
    Exit Disclaimer logo
    For best results, run Cygwin.bat from C:\cygwin64\ rather than from the Windows Start menu.
  2. Check which version of Java is currently installed:

    java –version

    The command output should display the version of Java. DME requires Java 1.8.x.

  3. If Java 1.8.x is not installed, perform the following steps:
    1. Download Java 1.8.x from the following page:  https://www.java.com/en/download/ Exit Disclaimer logo

    2. Install Java, as described in the following installation instructions:  https://www.java.com/en/download/help/index_installing.xml Exit Disclaimer logo
    3. Update your PATH system variable with the Java executable folder (such as c:/jdk1.8_13/bin). For details, refer to the following instructions: https://java.com/en/download/help/path.xml Exit Disclaimer logo
    4. Check whether you have performed the previous steps correctly by repeating the java -version command. 

  4. If the latest version of JDK is not installed, perform the following steps:

    1. Download JDK from the following page: 
      https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html Exit Disclaimer logo
    2. Install JDK. For information, refer to the following page:
      https://www.oracle.com/technetwork/java/javase/documentation/index.html Exit Disclaimer logo
  5. In a cmd or shell window, type the following command in your working directory to clone DME utilities to your local machine:

    Git clone https://github.com/CBIIT/HPC_DME_APIs

    The system creates a subdirectory (HPC_DME_APIs) containing all files from that GitHub repository. 

  6. In the HPC_DME_APIs/utils/ folder, create a copy of the sample configuration file (hpcdme.properties-sample) and rename it (as hpcdme.properties). Edit your hpcdme.properties file to make the following configuration changes:

    PropertyInstructions
    Server settings

    If you want to connect to the UAT tier, uncomment the UAT server settings and comment out the Production server settings. By default, the Production server settings are enabled.

    hpc.ssl.keystore.passwordContact NCIDataVault@mail.nih.gov to request this password.
    hpc.user

    Specify your NIH user ID.

    hpc.default.globus.endpointIf you intend to transfer files using Globus, specify the UUID of the Globus endpoint that you want to use. (The following instructions may be helpful: Copying the Globus Shared Endpoint UUID.)
    hpc.server.proxy.port

    If you intend to use Biowulf, specify the proxy port. To determine the correct port, run the following echo command:

    echo $http_proxy

    The output includes the port. For example, if the output of the echo command is http://dtn08-e0:3128, then your port is 3128.

    hpc.server.proxy.url

    If you intend to use Biowulf, specify the proxy URL. To determine the correct URL, ping the output of the echo command. For example:

    ping dtn08-e0

    The output includes an IP address. Specify that IP address as the URL.

    hpc.error-log.dir
    hpc.log.file

    If you are using Windows, consider specifying a non-default location for the error log file. For example:

    hpc.error-log.dir=log
    hpc.log.file=log/hpc-cli.log

    .

  7. Configure environment variables:

    1. Navigate to your ~/.bashrc or ~/.profile file, where ~ represents your home directory. For example: 

      C:\cygwin64\home\<username>

    2. Edit your ~/.bashrc or ~/.profile file to append the following commands, where <client_utils_home> is the absolute path to the HPC_DME_APIs\utils\ folder. The following commands define the HPC_DM_UTILS environment variable and update the PATH environment variable to include the path to the scripts folder:

      export HPC_DM_UTILS="<client_utils_home>"
      export PATH=$HPC_DM_UTILS/scripts/:$PATH
  8. If you are using Cygwin on Windows, run the following commands in Cygwin to change end-of-line delimiters from Windows standard to Unix standard in any files involved in CLU setup: 

    1. Adjust the line delimiters in the file with your export commands (~/.bashrc or ~/.profile). For example: 

      sed -i "s/\r$//" ~/.bashrc

    2. Adjust the line delimiters in all files in the scripts folder. Use the following command:

      sed -i "s/\r$//" $HPC_DM_UTILS/scripts/*

  9. To make the above changes take effect in current Bash session, source the modified file (~/.bashrc or ~/.profile). For example:

    source ~/.bashrc

  10. Generate a DME API authentication token, as described in Generating a DME API Authentication Token via the CLU

  11. If you want to use Globus or AWS S3, follow the instructions in the following pages:

To view a list of DME CLU commands, type dm_list_commands . To view usage information for any DME CLU command, type the command with the help (-h) parameter. For detailed information on each of the available DME CLU commands, refer to the following pages:

Important Note for Mac Version 10.12.4/10.12.6

If you want to use DME CLU on a Mac (version 10.12.4/10.12.6) and the version of curl on your machine is OpenSSL 0.9.8zh 14 Jan 2016, perform the following steps to rebuild curl. 

  1. From a terminal prompt, execute the following command: 

    /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)

    This command downloads and installs Homebrew, a package manager to download, build, and install Mac software.

  2. Follow the prompts. The system might request your username/password multiple times.

  3. Execute the following command:

    brew install curl --with-openssl

    This command invokes Homebrew to build and install curl in /usr/local/opt with the latest version of openssl. At the end of the installation, a message appears indicating that this is keg-only. It provides the command to update the PATH so that the system uses your new curl installation, rather than the Mac-supplied one. 

  4. Execute the command as provided at the end of curl installation, which should be the following command:

    export PATH="/usr/local/opt/curl/bin:$PATH"' >> ~/.bash_profile

  5. Confirm the installation: Open a new terminal and check the version of curl with the following command: 

    curl –-version


    The following information should appear: 

    curl 7.58.0 (x86_64-apple-darwin16.7.0) libcurl/7.58.0 OpenSSL/1.0.2n zlib/1.2.8
    Release-Date: <today’s date or the date you install your new curl>
    Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
    Features: AsynchDNS IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets HTTPS-proxy


  • No labels