Page History

If you need to move a large volume of data, you can do so using a Globus endpoint. DME uses Globus for high performance transfer of large files. Endpoints represent locations on a file transfer service server, such as Biowulf or gridftp. In DME CLU or GUI, you can choose to use Globus for file transfer. A prerequisite is a guest collection.

For narrated slides demonstrating these instructions, refer to prep-Globus-slides.pptx. Include Pageshared info - Globusshared info - Globus

To prepare a Globus endpoint:

1. If you do not already have a shareable Globus endpoint, contact NCIDataVault@mail.nih.gov and request one. 

   HTML Comment
   hidden true
   When people send messages to the NCI Data Vault inbox, requesting an endpoint, Udit usually asks them various questions. I asked about including those questions here, but Sunita said most of them have never seen this page. People usually see this page after the DME team has talked with them.

2. Log into Globus at the following site:  
   http://www.globus.org

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true PageWithExcerpt wikicontent:Exit Disclaimer to Include

   If prompted for login credentials, specify
   Use your NIH user account . Begin the process of sharing your shareable Globus endpoint as credentials as described in the following page:
   https://docs.globus.org/how-to/shareget-filesstarted/

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true PageWithExcerpt wikicontent:Exit Disclaimer to Include

3. Specify that you want to share with a group.  

4. If you do not already have a guest collection in Globus, create one:

   Panel
   borderColor black borderStyle solid
   Expand title Instructions to Create a Guest Collection In the Globus File Manager, search for NIH HPC Data Transfer (Biowulf). This is a public endpoint.  Image Added In the search results, in the row for NIH HPC Data Transfer (Biowulf), click the collection details icon (Image Added). The details page appears. On the Collections tab, click Add Guest Collection.  Image Added On the Create new guest collection page, browse to or specify the path to the directory where you want to save your data. Click Create Guest Collection.  Image Added

5. Share your Globus endpoint:

   1. In the Globus File Manager, navigate to the list of collections by opening the side bar, clicking Collections, and then clicking Shareable by You. 

      Image Added

   2. Select your guest collection. Access the sharing configuration for that collection by clicking Permissions. 

      Image Added

   3. Add permissions by clicking Add Permissions - Share With. In the Share With field, select the option to share with a group. Click Select a Group.

      Image Added

   4. Search all groups for "HPCDME-" and select the HPCDME-PROD

   Search all groups for "HPCDME-" and select the appropriate Globus group for your DOC, as indicated in the following table: 

   DOCGroup for the Production Tier of DME Group for the UAT Tier of DME
   FNLCRHPCDME-PROD-App-Accts-Pool-FNLCRHPCDME-DEV-App-Accts-Pool-FNLCRHiTIFHPCDME-PROD-App-Accts-Pool-HITIFHPCDME-DEV

   1. -App-Accts-Pool-

   HITIF
   CCBRHPCDME-PROD-App-Accts-Pool-CCBR HPCDME-DEV-App-Accts-Pool-CCBR
   CCR_SBLHPCDME-PROD-App-Accts-Pool-CCR_SBL HPCDME-DEV-App-Accts-Pool-CCR_SBL
   

   1. FNLCR group. 

      Panel
      borderColor black borderStyle solid
      Expand title Instructions to Use UAT If you intend to use the UAT tier of DME, select the HPCDME

   All OthersHPCDME-PROD-App-Accts-Pool-DefaultHPCDME

   1. -DEV-App-Accts-Pool-

   Default

   1. FNLCR group.

      Image Added

   2. Select the appropriate permission for the selected group:
   
   
   1. If you intend
   to download
   1. to download data from DME to the Globus endpoint, select the write permission. 

   If you intend to only upload data from the Globus endpoint to DME, select the read permission. 

   1. Image Added

   2. Click Add Permission. The system adds the HPCDME Globus group to the list for the current share.  
   Each time you need to download data from DME, perform the following steps to copy the shared endpoint UUID:
   1. Navigate to the list of endpoints that are sharable by you. The shared endpoint you have just created is available in that list with the following description:
      Shared Endpoint on <name of shareable Globus endpoint>
   2. Click the shared endpoint link. A page appears with details about that endpoint. 
   3. Next to the Endpoint UUID field, click the copy icon. 

Now you can upload register or download data in the GUI as described in the following pages:

• Registering Data in Bulk from a Globus Endpoint via the GUI
• Downloading to a Single Data File Globus Endpoint via the GUIDownloading a Collection via the GUI

To register or download data via CLU, refer to the additional prerequisites described in Getting Started with DME CLU.