The following table lists some frequently asked questions about DME:
| Question | Answer |
|---|---|
| Why should I use DME? | You can safeguard your data and easily access it. DME is a metadata-based data management platform that provides secure, virtualized storage. Its reliable storage mechanism and the ease of accessing and sharing large datasets eliminate the need for users to maintain copies of datasets in their local or network-attached storage. |
| What kind of data can be stored in DME? | You can store high-value scientific datasets generated at NCI. |
| How can I store data in DME? | You can store an individual file (up to 10 GB) from your local system or store data in bulk from Globus, AWS S3, Google Drive, or Google Cloud storage. You can store data using the web application's graphical user interface (GUI), Command Line Utilities (CLU), or the REST API. If you want fine-grained control of the activity, or if you want to integrate it into your scientific workflow, consider using CLU or API. Otherwise, consider using the GUI. |
| How is my data secured? | To access DME data, users must have an NIH account and request a DME account from NCIDataVault@mail.nih.gov. DME employs geographical redundancy to protect data against accidental deletion. Also, if your user account has the Own permission level on a collection or a data file, you can manage permissions on that data. You can manage permissions on one file or collection at a time, or in bulk. |
| How will I be billed for the service? | There is no charge for the storage or service. |
| How is data organized in DME? | We provide domain-specific templates to help you define your data hierarchy. These are fully customizable and can be tailored according to your unique data management needs. To obtain the maximum value out of this platform, we recommend choosing a data hierarchy and metadata structure that aligns with how you need to search and locate your data in DME. |
| What type of metadata does DME expect with the data? | DME requires minimal provenance metadata to establish the data's traceability. We recommend adding domain-specific metadata also that identifies the data's characteristics, and the processes, platforms and instruments used to generate it. |
| What interfaces does DME provide for managing the data? | DME provides a web-based graphical user interface for uploading and browsing the datasets and performing metadata-based searches. You can easily download the data to multiple endpoints such as Globus endpoint, AWS S3 bucket, Google Cloud, and Google Drive. REST APIs and command-line utilities are available for programmatic integration with bioinformatics pipelines and workflows. |
| How do I upload my data to DME? | You can upload data using the web application, DME REST APIs, or command-line utilities. Additionally, the DME team operates an archival workflow service that can scan your source directories on a defined schedule or as needed and copy the data and metadata to DME. |
| How do I supply metadata to DME? | The REST APIs and command-line utilities accept the metadata in JSON files. The web application accepts metadata for single collections and files through forms, and for bulk data through spreadsheets. All mandatory metadata must be supplied along with the data. You can add optional metadata anytime later. If you have requested the DME archival workflow service, then the workflow will upload all your data and metadata. |
| How long will my data remain in DME? | The initial storage is provided for seven years, which can be extended depending on your needs or federal requirements such as FDA mandates. We do not delete any data from DME without the documented consent of the data owner, irrespective of how long the data has been there. |
| How can I share my data in DME with external collaborators? | DME provides multiple options for sharing data with external collaborators. You can transfer the data to the collaborator's Globus endpoint, AWS bucket, Google Cloud Platform, or Google Drive. |
| Can I download my data in DME to other servers and platforms in NCI and NIH? | Yes, you can download your data in DME to any server with a Globus endpoint that is accessible to you. The FRCE Batch cluster in Frederick, the Biowulf cluster, and the Helix system provide Globus endpoints for users. You can also download data to your local machine directly (for individual files) or using the free Globus Personal Connect software. |
| You can access your datasets in DME from the NIH Integrated Data Analysis Platform (NIDAP). | Using the DME web application, you can perform dbGaP submissions of your datasets residing in DME. |
| Where can I find more information on DME? | More information on DME can be found <here>. You can also reach out to ncidatavault@mail.nih.gov . |
