NIH | National Cancer Institute | NCI Wiki  

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: For CDP-1471: Updated download statements.
Panel
titleContents of this Page
Table of Contents

The CSSI DCC Portal is a public repository of experiment-related information describing cancer research investigations. You can use the portal to browse, search, and download Physical Sciences Oncology Network (PSON) data sets. As of January 2017, there are 8 PSON data sets in the portal. These data sets are access data generated through CSSI funded projects and other user uploaded data sets. This data is in ISA-Tab format, which organizes investigation, study, and assay data according to the rules in the ISA-Tab specification

Multiexcerpt include
MultiExcerptNameExitDisclaimer
nopaneltrue
PageWithExcerptwikicontent:Exit Disclaimer to Include
 For more information about the ISA-Tab format, refer to the Understanding ISA-Tab page.  

Each data set contains three files--investigation, study, and assay--that conform to the ISA-Tab structure and naming conventions. Within this structure are fields that are standard for each type of file, though null values are allowed; that is, not every data set includes values for each field. The portal allows you to filter these fields in an interactive way so that you can visualize the data in a pie chart or list.

You can also search investigations, studies, and assays using any keyword. You can download selected files, the entire data archive, or only the metadata associated with a study.

This chapter provides detailed instructions on how to browse, search, and download data.

Browsing Investigations

You can browse and explore investigation data contained in investigation, study, and assay files.To browse investigations

...

The

...

Understanding the Pie Charts and Investigations List

The Browse Investigation page has two interactive components:

...

Info

If you add or remove fields from the pie charts, the Investigations list fields immediately reflect the same changes.

...

Adding and Removing Fields

You can customize which pie charts appear at the top of the page. Since the pie charts control how you filter the investigation data, you may prefer something other than, or in addition to, the three default fields of Study Protocol Name, Study Assay Technology Type, and Study Assay Measurement Type.

You can add and remove fields

To add or remove fields

...

Expand fields by clicking the plus signs. Note that in the STUDY PROTOCOLS and Study Assays section, the field values of Study Protocol Name, Study Assay Technology Type, and Study Assay Measurement Type are already selected. These field values are the ones that appear in the three default pie charts on the Browse Investigations page. If you clear these check boxes, those pie charts disappear from the page. When you reset all of the pie charts or select Browse > Investigations, however, they reappear.

...

Exploring Investigation Details

...

To explore investigation details

  1. Browse investigations or search investigations until you find an investigation, a study, or an assay in which you are interested.
    The search results appear.
    Image Removed

  2. Click the link corresponding with the investigation, study, or assay you are interested in exploring.
    The respective investigation details, study details, or assay details page appears.

  3. You can do the following from the details pages:

Investigation Details Page

The investigation details page shows the investigation name at the top followed by a visualization of the investigation filename, study filename and number of samples in the study, and assay filename and number of files (and total file size) in the assay. Below the visualization are links to download all or part of the investigation, its identifier, and its description. All of the icons on the page are clickable links.

Info

The currently selected entity has a green box behind its icon in the visualization. When the investigation is selected, its study/studies, and assay(s) are also selected. In the following screenshot, the investigation is selected.

Image Removed

Study Details Page

...

Assay Details Page

The assay details page shows the study name at the top followed by a visualization of the investigation filename, study filename and number of samples in the study, and assay filename and number of files (and total file size) in the assay. Below the visualization are links to download all or part of the assay, its file name, measurement type, and technology type. In the Processes and Filters section, the relationship of the study to its processes, and its processes to its files, are depicted in clickable icons. Click any icon to further filter the investigation data and download only a selected portion of it.

Assay Details pageImage Removed

Visualizing and Filtering Data

Once you browse or search the CSSI DCC data sets and reach a selected investigation details, study details, or assay details page, you can continue exploring the data. The ISA-Tab format is hierarchical, with investigation components becoming more granular as you proceed down the hierarchy. The largest organizing entity is the investigation, which holds one or more studies. Each study includes one or more assays. Assays are composed of samples, which in turn are composed of protocols. Data files are often associated with a protocol.

The following image depicts the hierarchy, without the samples and protocols.

Structure of ISA data model as described in the text on this page.Image Removed

Visualizing and Filtering Investigations

The investigation details page shows icons that represent the relationship of the investigation to its studies and assays. In the case of PSON Cell Line Genomic Characterization - mRNA, the investigation has one study and one assay.
Image Removed

You cannot filter data currently at the investigation level any further in the CSSI DCC Portal. You can only download the investigation's full data at this point, or start exploring its studies and assays.

Visualizing and Filtering Study Data

If you select the study in the PSON Cell Line Genomic Characterization - mRNA investigation or arrive at any other Study Details page through a search, you can visualize the study's file structure and filter on any field.

To visualize and select study data

...

Image Removed

In the Visualize and Select area, click one of the entities.

Info
titleStudy Hierarchy

The hierarchy of entities for studies according to the ISA-Tab standard is as follows, from less granular to more granular:

Source > Protocol > Sample

For example, click Source. Metadata for that source appears.

Info

Follow this same procedure if you want to filter on Protocol or Sample instead of Source.

...

CSSI DCC Portal allows you to perform various tasks:

  • Search investigations, studies, and assays using any keyword.
  • Download the data.
  • Download the metadata.
  • Download selected files, with the metadata.
  • Upload your own investigation data to the portal and request open access for your uploaded data. (For more information, refer to Uploading Investigation Data and Requesting Open Access.)

The following page family describes how to browse, search, and download data.

Page Tree
expandDepth3
root@self

Visualizing and Filtering Assay Data

If you select the assay in the PSON Cell Line Genomic Characterization - mRNA investigation or arrive at any other Assay Details page through a search, you can visualize the assay's file structure and filter on any field.

To visualize and select assay data

...

Image Removed

In the Visualize and Select area, click one of the entities. Note that the width of some visualizations require you to scroll by clicking the arrows at the bottom of the page.

Info
titleAssay Hierarchy

The hierarchy of entities for assays according to the ISA-Tab standard is as follows, from less granular to more granular:

Sample > Protocol > Data File

For example, click the sample. Metadata for the sample appears.

Info

Follow this same procedure if you want to filter on protocol instead of sample.

...

Click one or more values in the list to select them. To select multiple values, click one, wait for it to appear on the Metadata for Sample page, and then click the arrow again to select another value. To clear a selection, click that value again.
Image Removed

Info

Some entities do not have associated metadata fields on which you can filter the study or assay. In that case, when you click the icon for that entity, you see a message letting you know that no metadata fields are available.

Image Removed

...

Searching Investigation Data

You can search for investigations, studies, or assays in the CSSI DCC Portal by:

Search by Keyword or Phrase

You can search all investigations, studies, and assays in the CSSI DCC Portal by keyword or phrase. A search looks for matches in all file columns, field definitions, and placeholders in those investigations, studies, and assays.

To search investigation data by keyword or phrase

...

In the Search for box, select the context of the search. Options include Investigations, Studies, or Assays. Your search will be restricted to the context you select.

In the following example of search results, the keyword for the search is cell and the context is Investigations. All investigations that include the word cell anywhere in the investigation metadata appear in these search results.

Search Investigations page, search resultsImage Removed
If the context were Studies or Assays, the results would include information specific to these components, as follows

...

Search results from a search with a context of Studies display the study name, name of the investigation the study is associated with, number of assays, number of files, and description. In these cases, the investigation name and study names are identical.

...

Click any investigation name, study name, or assay filename to explore the data further.

Search by Related Terms

You can search all investigations, studies, and assays in the CSSI DCC Portal by terms found in related ontologies. The two ontologies available for searching are the Ontology for Biomedical Investigations

Multiexcerpt include
MultiExcerptNameExitDisclaimer
nopaneltrue
PageWithExcerptwikicontent:Exit Disclaimer to Include
and the NCI Thesaurus (NCIt). A search looks for synonyms and subclasses of the keyword you entered in the ontology(ies) you selected and matches them to investigations, studies, and assays in the CSSI DCC Portal. The matching ontological terms can exist in file columns, field definitions, and placeholders of the investigations, studies, or assays.

To search investigation data by related terms

...

  • Synonyms include terms in an ontology that are closely related to the keyword or phrase you entered.
  • Subclasses are lower categories associated with the ontology term.

...

Downloading Investigation Data

After reaching an investigation entity (such as investigation, study, assay, source, protocol, sample, or data file), you can download the full data, selected metadata, or selected data associated with that entity. The full data is always associated with the investigation as a whole. All of the data currently in the portal is public.

Metadata describes the structure of the data collected in an investigation and translates to the file columns, field definitions, and placeholders that appear in a spreadsheet. It is in Investigation-Study-Assay tab-delimited format (ISA-Tab), which is based on the ISA-Tab specification

Multiexcerpt include
MultiExcerptNameExitDisclaimer
nopaneltrue
PageWithExcerptwikicontent:Exit Disclaimer to Include
.

The data files often contain image files and spreadsheets and can be a large file size.

Each download option has a button on the Investigation Details, Study Details, and Assay Details pages.

Download Full Data (13.48 GB zip) Download Selected Metadata  Download Selected Data Image Removed

  • Download Full Data downloads metadata and data files for the entire investigation. Since this can be a large file size, the file size appears on the button.
  • Download Selected Metadata downloads only the metadata of a selection you make after filtering a study an assay.
  • Download Selected Data downloads both the metadata and the data files of a selection you make after filtering a study or an assay.
Info

An investigation's full data file is always available for download. However, due to the processing resources required, Selected Data downloads are currently limited to 30GB.

Downloading Full Data

You do not have to log in before downloading the full data from an investigation. If you are not logged in, when you request the full data, you are prompted to provide your email address. You will receive a link at that address you can use to access and download the data. If you are logged in, you have the option of using Globus to download the file.

To download full data

...

Click the Download Full Data button.
The Request Data Files dialog appears. It offers different options depending on whether or not you are logged in to CSSI DCC.

...

Multiexcerpt include
MultiExcerptNameExitDisclaimer
nopaneltrue
PageWithExcerptwikicontent:Exit Disclaimer to Include

...

Info

It may be useful to rename the .zip file as you save it to include the name of the investigation so that you can identify it more easily. For example, miRNA_full.zip.

...

Downloading Selected Metadata

Metadata files are usually small text files, so you can download them directly to your computer. You do not need to log in before downloading metadata files.

To download selected metadata

  1. Filter a study or an assay until you reach a selection of investigation data you are interested in downloading. You may be interested in only the metadata so that you can see how that selection of entities was structured.
  2. On the Investigation Details page, click Download Selected Metadata.
    Your browser prompts you to open or save the .zip file. Follow your browser's instructions to open or save the file.
    FireFox open or save download dialog boxImage Removed

    Info

    It may be useful to rename the .zip file as you save it to include the name of the investigation so that you can identify it more easily. For example, miRNA_metadata.zip.

    Info
    titleArchive File Contents

    The archive file is in compressed format. When you download it, it may be a single compressed folder or .zip file. When you open it, you see at minimum three text files at the root of the folder or file. An example of these follow. 

       a_10290.txt

       i_10290.txt

       s_10290.txt

    The text files describe the investigation, study or studies, and assay or assays. In this example, 10290 represents the file identifier but in practice, each file identifier may be named differently, even in the same investigation. Only the a, i, or s prefix is required. If you download full data or selected data, the archive may also contain other files or folders as appropriate for the investigation; for example, images. If you download metadata, the archive only includes the a, i, and s files.

Downloading Selected Data

You do not have to log in before downloading selected data from an investigation. If you are not logged in, when you request selected data, you are prompted to provide your email address. You will receive a link at that address you can use to access and download the data. If you are logged in, you have the option of using Globus to download the file.

To download selected data

  1. Filter a study or an assay until you reach a selection of investigation data you are interested in downloading.
  2. On the Investigation Details page, click Download Selected Data.
    The Request Data Files dialog box appears.
    Request Data Files dialog boxImage Removed
  3. You have two options:
    • Transfer the file with Globus, which is useful when the file is very large. If you choose this option, click the checkbox and then click Request Download. The Transfer Files page in Globus opens. For more information about downloading files with Globus, see Chapter 2: Browsing, Searching, and Downloading Investigations or Globus Support
      Multiexcerpt include
      MultiExcerptNameExitDisclaimer
      nopaneltrue
      PageWithExcerptwikicontent:Exit Disclaimer to Include
      .
    • Receive a link through email that you can use to download the file to your computer. If you choose this option, enter your email address in the box, click the I'm not a robot box, and then click Request Download.
      The Download Requested window appears.
      Download Requested window. The system is processing your request. You will receive an email when your download is ready.Image Removed
      Click Ok. Periodically check your email inbox for the link to the file.

Downloading Large Files with Globus

Globus is a service that enables large file transfers securely. You must have an account with Globus and install

Multiexcerpt include
MultiExcerptNameExitDisclaimer
nopaneltrue
PageWithExcerptwikicontent:Exit Disclaimer to Include
Globus Connect Personal to use it to download investigation files to CSSI DCC. If you do not already have an account, you are prompted to create one when you start the download process.

To download files using Globus

  1. If you haven't already, start Globus Connect Personal.
  2. Filter a study or an assay until you reach a the investigation data you are interested in downloading.
    The Request Data Files page appears. It looks slightly different if you are downloading the investigation's full data or selected data.
  3. Continue downloading the investigation's full data or continue downloading selected data from the investigation.

To download full data using Globus

    1. Click the Transfer with Globus checkbox.
      Request Data Files, full data Image Removed
       
    2. Click Download.
      You are prompted to log into Globus. The Transfer Files page appears.

      One of the Endpoints you configured when you installed Globus Connect Personal is already populated, though you can change it.
      Globus Transfer Files pageImage Removed
    3. Select the starting endpoint (on the right) where the file(s) you want to download reside(s). Narrow down to the path if necessary.

    4. Confirm or change the destination endpoint (on the left). 

    5. Click the left arrow button that points to the destination to begin the transfer request. 

    6. A message appears on the screen when the transfer request is submitted successfully.
      Globus Transfer Files page, transfer request submitted successfullyImage Removed
       

      You will receive an email when the request is granted and the transfer succeeds.

To download selected data using Globus

...

Click the Transfer with Globus checkbox.
Request Data Files windowImage Removed

...

Click Request Download.
The Globus Upload window appears, asking you to confirm your Globus ID.
Globus Upload windowImage Removed

...

Enter your Globus ID (an email address) and click Confirm.
The Download Requested window appears.

Download Requested window. The system is processing your request. You will receive an email when your download is ready.Image Removed

...

Globus Transfer Files pageImage Removed

...

Select the starting endpoint (on the right) where the file(s) you want to download reside(s). Narrow down to the path if necessary.

...

Confirm or change the destination endpoint (on the left).

...

 Click the left arrow button that points to the destination to begin the transfer request.

A message appears on the screen when the transfer request is submitted successfully.
Globus Transfer Files page, transfer request submitted successfullyImage Removed

...

You will receive an email when the request is granted and the transfer succeeds.