NIH | National Cancer Institute | NCI Wiki  

Error rendering macro 'rw-search'

null

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 223 Next »

Contents of this Page

The CSSI DCC Portal is a public repository of experiment-related information describing cancer research investigations. You can use the portal to browse, search, and access data generated through CSSI funded projects and other user uploaded data sets. This data is in ISA-Tab format, which organizes investigation, study, and assay data according to the rules in the ISA-Tab specification Exit Disclaimer logo .   

Each data set contains three files--investigation, study, and assay--that conform to the ISA-Tab structure and naming conventions. Within this structure are fields that are standard for each type of file, though null values are allowed; that is, not every data set includes values for each field. The portal allows you to filter these fields in an interactive way so that you can visualize the data in a pie chart or list.

You can search investigations, studies, and assays using any keyword. You can download selected files, the entire data archive, or only the metadata associated with a study. You can also upload and publish your own investigation data to the portal.

The following sections provide detailed instructions on how to browse, search, and download data.

Accessing Investigation Data

To access investigation data in the CSSI DCC portal, the prerequisites are as follows:

  • URL or web address. The URL might be from the Browse page, the Search page, an email message, or a browser bookmark. The URL for a specific version of an investigation remains the same over time. (For information on the Browse or Search pages, refer to Browsing Investigations or Searching Investigation Data.)
  • Permission to view the data. When a user uploads an investigation to the CSSI DCC portal, that user specifies who can access the investigation data. For details, refer to Controlling Access to Investigation Data

If you are unable to find the data for an investigation, consider the following points:

  • A typo might have been introduced in your copy of the URL.  
  • The investigation embargo date might still be in the future. 
  • The user who uploaded the data might not have made it open access yet or might not have given you permission to view it. 
  • The investigation might have been deleted. 

Browsing Investigations

You can browse and explore investigation data contained in investigation, study, and assay files.

To browse investigations

    • From the CSSI Data Portal home page, click the Browse button or select Investigations > Browse.
      Browse button
      The Browse Investigations page appears with two interactive components:

      • A list of all investigations currently included in this release of the CSSI DCC Portal

      • Pie charts at the top showing fields from the investigations

A subset of the page appears in the following screenshot. (Click the image to expand it.)

Browse Investigations page showing filters and investigations.

For more details, refer to the following pages:

Understanding the Pie Charts

When you first open the Browse Investigation page or reset it to clear all selections, three pie charts appear at the top. These pie charts represent three of the fields that occur in the metadata of the investigations currently in the portal:
  • Study Protocol Name
  • Study Assay Technology Type
  • Study Assay Measurement Type

The values for those fields, as well as a count for each value, appear in the pie charts. The count represents the number of times that field value occurs in all of the investigations currently in the portal. (Click the following image to expand it.)

Browse Investigations page showing filters and investigations.

For example, the default pie charts represented in the above screenshot show the following information:

  • The Study Protocol Name pie chart shows that four investigations used a blood draw protocol.
  • The Study Assay Technology Type pie chart shows that two investigations used light microscopy. 

The sum of numbers in each pie chart can exceed the total number of investigations in the system, because each investigation can have more than one value in each pie chart. You can determine the values for each investigation by clicking on each pie slice and reviewing the details in the list below the charts.

Click one or more field values in the pie charts ("slices") to filter the data by that/those field value(s). The more field values you select, in one or more pie charts, the more narrowly you filter the investigation data and the fewer investigations match your selections. The investigation list refreshes each time you filter the data in this way. You can also customize which fields appear in the pie charts and how many pie charts appear by adding and removing fields. (For instructions, refer to Adding and Removing Fields.)

In the following screenshot, a user has selected at least one field value in each pie chart. The selected values are: 

  • Scanning in the Study Protocol Name chart
  • CNV analysis in the Study Assay Measurement Type chart
  • Next Generation Sequencing in the Study Assay Technology Type chart

Only one investigation matches all of these selections and appears in the list below the charts. To reduce the amount of filtering, you can take one of the following actions:

  • To reset an individual pie chart, click reset on that chart. 
  • To reset all pie charts to their defaults, you can take one the following actions:
    • Click Reset All.
    • Select Investigations > Browse

(Click the following image to expand it.)

Browse Investigations page with values selected in each pie chart.

Understanding the Investigations List

The Investigations list shows details associated with the investigations that match your pie chart selections. For example, consider the pie charts in the following screenshot:
  • In the Study Protocol Name pie chart, a user has selected one field value nucleic acid extraction. The number 3 on the selected pie slice means that three investigations are associated with the Study Protocol Name field value of nucleic acid extraction. 
  • In the Study Assay Measurement Type pie chart, all investigations use the same value of nucleotide sequencing.
  • In the Study Assay Technology Type pie chart, two investigations have a value of transcription profiling and one investigation has a value of genome sequencing

(Click the following image to expand it.)

Browse Investigations page with values selected in one pie chart.

The fields in the Investigations list match the pie chart selections. The list includes only those three investigations that have a Study Protocol Name field value of nucleic acid extraction. Details for each investigation in the list includes the same fields represented by the pie charts: Study Protocol Name, Study Assay Technology Type, and Study Assay Measurement Type, plus the study name, description, and public release date. The list also shows additional data about each investigation, such as the other keywords used in the Study Protocol Name field. You can return to the default (full) list at anytime by selecting Investigations > Browse.

You can control various aspects of the list:

  • To control how many investigations appear in the list, select a value from the Max Display box. 
  • To change how the list is sorted, select a value from the Sort By box. 

If you add or remove fields from the pie charts, as described in Adding and Removing Fields, the Investigations list fields immediately reflect the same changes.

Adding and Removing Fields

You can customize which pie charts appear at the top of the page. Since the pie charts control how you filter the investigation data, you may prefer something other than, or in addition to, the three default fields of Study Protocol Name, Study Assay Technology Type, and Study Assay Measurement Type.

To add or remove fields

  1. On the Browse Investigations page, click Add/Remove fields. The Metadata Selection panel appears to the left of the pie charts. (Click the following image to expand it.)
    Metadata Selection panel.

  2. You can either search or browse for a field of interest in the Metadata Selection panel:

    • Expand fields by clicking the triangle icon next to each item. 
      An excerpt of the Metadata Selection panel with triangle icon for expanding list.

    • In the Search field, start typing any part of a field name. The system filters the list of fields by that search term. (To clear a search term, click clear. The system unfilters the list of fields.)
      Metadata Selection panel with param in search field.

  3. Note the default selection of some fields: 

    • Studies > STUDY PROTOCOLS > Study Protocol Name

    • Study Assays > Assay > Study Assay Technology Type

    • Study Assays > Assay > Study Assay Measurement Type

    These field values are the ones that appear in the three default pie charts on the Browse Investigations page. If you clear these check boxes, those pie charts disappear from the page. When you reset all of the pie charts or select Investigations > Browse, however, they reappear.
    Metadata Selection panel with default selections.

  4. Click any check box to select the corresponding field. For example, select Study Protocol Type and click Submit. The pie charts update to include one for Study Protocol Type. 
    Study Protocol Type selected in the menu and the resulting pie chart.

    Correspondingly, the Investigations list updates to show Study Protocol Type in the fields available for the investigation.
    Study Protocol Type selected in the menu and Study Protocol Type in the investigation information.

    You could also opt to clear all of the check boxes except for Study Protocol Type. In this case, that is the only pie chart that would appear.
    Browse page with only one pie chart, for Study Protocol Type.

    When you clear a field, it also disappears from the Investigations list.

  5. When you have selected or cleared as many fields as you want from the pie charts, click Hide select menu. The Select menu moves back to its original position.

Exploring Investigation Details

You can continue exploring investigation data by clicking a link to investigation, study, or assay details. Links to these details are on the Browse Investigations page or on the Search Investigations page, after you search on a keyword or phrase. These details include counts of studies, assays, samples, and files. The metadata available for an investigation determines if other entities, such as sources and collections, are available for counts. From these details pages, you can visualize the structure of the investigation and download selected study files, download the full archive, and download only the metadata.

To explore investigation details

  1. Browse investigations or search investigations until you find an investigation, a study, or an assay in which you are interested.
    The search results appear.
    Example investigation as listed in search results.

  2. Click the link corresponding with the investigation, study, or assay you are interested in exploring.
    The respective investigation details, study details, or assay details page appears.

  3. You can do the following from the details pages:

Investigation Details Page

The investigation details page displays information about the investigation, in the following order:
  • The investigation name is at the top. 

  • A visualization includes the following icons and information:
    • Icon and filename for the investigation node.
    • Icon, filename, and number of samples for each study node.
    • Icon, filename, number of files, number of links (external file links), and total file size for each assay node.

    All of the icons in this visualization are clickable links.

  • Below the visualization are buttons to download all or part of the investigation.

  • Below the download buttons are the investigation identifier and its description.

The currently selected icon has a green box behind its icon in the visualization. When the investigation is selected, its studies and assays are also selected. In the following screenshot, the investigation is selected.

An example investigation details page

If an investigation has many studies or assays, the visualization is horizontal and can be zoomed in and out. For example, the following investigation has 29 study files with 10 assay files each, totaling 290 nodes. The zoom out and zoom in buttons are below the visualization, as indicated below.

The investigation details page with zoom out and zoom in buttons.

Study Details Page

The study details page shows the investigation name at the top followed by a visualization of the investigation filename, study filename and number of samples in the study, and assay filename and number of files (and total file size) in the assay. Below the visualization are links to download all or part of the investigation, its identifier, and its description. All of the icons on the page are clickable links.

Assay Details Page

The assay details page shows the study name at the top followed by a visualization of the investigation filename, study filename and number of samples in the study, and assay filename and number of files (and total file size) in the assay. Below the visualization are links to download all or part of the assay, its file name, measurement type, and technology type. In the Visualize and Select section, the relationship of the study to its processes, and its processes to its files, are depicted in clickable icons. Click any icon to further filter the investigation data and download only a selected portion of it.

Assay Details page

Visualizing and Filtering Data

Once you browse or search the CSSI DCC data sets and reach a selected investigation details, study details, or assay details page, you can continue exploring the data. The ISA-Tab format is hierarchical, with investigation components becoming more granular as you proceed down the hierarchy. The largest organizing entity is the investigation, which holds one or more studies. Each study includes one or more assays. Assays are composed of samples, which in turn are composed of protocols. Data files are often associated with a protocol.

The following diagram depicts the hierarchy, without the samples and protocols.

Structure of ISA data model as described in the text on this page.

Visualizing and Filtering Investigations

The investigation details page shows icons that represent the relationship of the investigation to its studies and assays. In the case of PSON Cell Line Genomic Characterization - mRNA, the investigation has one study and one assay.
Investigation Details page

You cannot filter data currently at the investigation level any further in the CSSI DCC Portal. You can only download the investigation's full data at this point, or start exploring its studies and assays.

Selecting Multiple Objects

If any investigation has more than one study and assay, you can select which ones you want to filter by clicking the Select Multiple Objects box.

For example, the following investigation has many studies and assays, so it displays horizontally with a slider on the side to move up and down. It also has zoom out and zoom in buttons to see more or fewer objects.

Investigation with multiple studies and assays

Select the boxes next to the objects you want to visualize and filter. If you download these objects, all of these objects will be included in the download.

Visualizing and Filtering Study Data

If you select the study in the PSON Cell Line Genomic Characterization - mRNA investigation or arrive at any other Study Details page through a search, you can visualize the study's file structure and filter on any field.

You can also select multiple objects to visualize and filter within a single investigation.

To visualize and select study data

  1. Open a Study Details page.

    An example Study Details page.

  2. In the Visualize and Select area, click one of the entities.

    Study Hierarchy

    The hierarchy of entities for studies according to the ISA-Tab standard is as follows, from less granular to more granular:

    Source > Protocol > Sample

    For example, click Source. Metadata for that source appears.

    Follow this same procedure if you want to filter on Protocol or Sample instead of Source.


    Metadata dialog box for the selected source.

  3. From the Source Name list, click the arrow to open the list of values. Each value is a Source Name from the study file, which in this case is s_mrna.txt.
    Source Name drop-down list within the Metadata dialog box.
  4. Click one or more values in the list to select them. Each value appears immediately below the Source Name box. To clear a selection, click the value again.
    Metadata for Source with multiple source names selected
  5. Click Filter Results.
    The Source Count reduces from the 40 original values in the unfiltered study file, s_mrna.txt, to the three selected in this procedure.
    Source Count 3, sample collection, Samples 3
  6. Options you have at this point include continuing to explore this data set, downloading the metadata of the three values you selected, downloading the selected data of the three values you selected (which includes the metadata), or clicking Clear Filters to filter a different entity.

Visualizing and Filtering Assay Data

If you select the assay in the PSON Cell Line Genomic Characterization - mRNA investigation or arrive at any other Assay Details page through a search, you can visualize the assay's file structure and filter on any field.

You can also select multiple objects to visualize and filter within a single investigation.

To visualize and select assay data

  1. Open an Assay Details page.

    A sample Assay Details page.

  2. In the Visualize and Select area, click one of the entities. Note that the width of some visualizations require you to scroll by clicking the arrows at the bottom of the page.

    Assay Hierarchy

    The hierarchy of entities for assays according to the ISA-Tab standard is as follows, from less granular to more granular:

    Sample > Protocol > Data File

    For example, click the sample. Metadata for the sample appears.

    Follow this same procedure if you want to filter on protocol instead of sample.


    Metadata dialog box for the selected sample.

  3. From the Sample Name list, click the arrow to open the list of values. Each value is a Sample Name from the assay file, which in this case is a_mrna_transcription_profiling_nucleotide_sequencing.txt.
    Drop-down list within the Metadata dialog box.
  4. Click one or more values in the list to select them. To select multiple values, click one, wait for it to appear on the Metadata for Sample page, and then click the arrow again to select another value. To clear a selection, click that value again.
    Metadata dialog box with selected sample names.

    Some entities do not have associated metadata fields on which you can filter the study or assay. In that case, when you click the icon for that entity, you see a message letting you know that no metadata fields are available.

  5. Click Filter Results.
    The Sample Count reduces from the 39 original values in the unfiltered assay file, a_mrna_transcription_profiling_nucleotide_sequencing.txt, to the 5 selected in this procedure.
    Assay Visualization, 5 Samples Selected
  6. Options you have at this point include continuing to explore this data set, downloading the metadata of the 3 values you selected, downloading the selected data of the 3 values you selected (which includes the metadata), or clicking Clear Filters to filter a different entity.

Searching Investigation Data

You can search for investigations, studies, or assays in the CSSI DCC Portal by:

Performing a Basic Search

You can search all investigations, studies, and assays in the CSSI DCC Portal. By default, a search looks for matches in all file columns, field definitions, and placeholders in those investigations, studies, and assays.

To perform a basic search

  1. From the CSSI Data Portal home page, click the Search button or select Investigations > Search.
    Search button

    The Search Investigations page appears.

    Search Investigations
  2. In the Search box, enter a keyword or phrase. Search is not case-sensitive. Search results appear immediately, using the default context of Investigations.
  3. To change the context, from the Search for list, select Studies or Assays. The system restricts your search to the context you select.

To clear search items, click Reset.

Searching by Keyword or Phrase

Unable to render {include} The included page could not be found.

Searching ISA Tab Fields

Unable to render {include} The included page could not be found.

Searching by Related Terms

Unable to render {include} The included page could not be found.

Viewing Detailed Search Results

In the search results, click View next to a matching term to see where in the investigation this term occurs.

Investigation listing in search result with cell as the matching term

A detailed search results window appears. The matching term, cell, is highlighted in the metadata of each file in the ISA-Tab archive.

A detailed search results window.

To close the window, click Ok at the bottom or the x in the upper-right corner of the window.

Saving Search Parameters

To save search parameters

  1. Log in to the CSSI DCC portal. For instructions, refer to Logging In to the CSSI DCC Portal. Your email address appears in the upper-right corner of the screen.
  2. Perform a search, as described in one of the following sections:
  3. Click Save Search. The Saved Search dialog box appears with details about the search you are saving.
    Saved Search dialog box.
  4. Specify a name for this search. Click Save. A message appears indicating that the system has saved your search. 
    Saved Search message.
  5. Click Ok

For instructions on managing your saved searches, refer to Managing Saved Search Parameters.

Managing Saved Search Parameters

To manage your saved search parameters

  1. Log in to the CSSI DCC portal. For instructions, refer to Logging In to the CSSI DCC Portal. Your email address appears in the upper-right corner of the screen.
  2. Click your email address and then click Saved Searches. A page by that name appears, listing the searches you have saved. Initially, the most recent entry is the default. (The system loads the default saved search on login.)
    Saved Searches page.

  3. You can take the following actions:
    • To sort the list, click the Name, Date, Search Term, or Search Context column heading. 
    • To rename a saved search, click the current name and type the new one. Click elsewhere. A message appears confirming the name change. Click Ok
    • To specify a different saved search as default, click the radio button under Default in the row for that search. The system immediately saves your change. 
    • If a saved search has more than three key-value pairs for advanced search criteria and you want to see all of them, click Show more.
    • To delete a saved search, click the trash can icon in the row for that search. Your browser prompts you to confirm the deletion. 
    • To run a saved search and view the results, click the triangle icon in the row for that search.


Downloading Investigation Data

After reaching an investigation entity (such as investigation, study, assay, source, protocol, sample, or data file), you can download the full data, selected metadata, or selected data associated with that entity. The full data is always associated with the investigation as a whole. All of the data currently in the portal is public.

Also note that if you selected multiple objects to visualize and filter, a download includes all of those objects.

Metadata describes the structure of the data collected in an investigation and translates to the file columns, field definitions, and placeholders that appear in a spreadsheet. It is in Investigation-Study-Assay tab-delimited format (ISA-Tab), which is based on the ISA-Tab specification Exit Disclaimer logo .

The data files often contain image files and spreadsheets and can be a large file size.

Each download option has a button on the Investigation Details, Study Details, and Assay Details pages.

Download Full Data (13.48 GB zip) Download Selected Metadata  Download Selected Data

  • Download Full Data downloads metadata and data files for the entire investigation. Since this can be a large file size, the file size appears on the button.
  • Download Selected Metadata downloads only the metadata of a selection you make after filtering a study an assay.
  • Add to Download downloads both the metadata and the data files of one or more selections you make after filtering a study or an assay.

An investigation's full data file is always available for download. However, due to the processing resources required, downloads via the Add to Download button are currently limited to 30GB.

Downloading Full Data

Unable to render {include} The included page could not be found.

Downloading Selected Metadata

An entity's metadata shows how that entity is structured. Metadata files are usually small text files, so you can download them directly to your computer. You do not need to log in before downloading metadata files.

To download selected metadata

  1. Filter a study or an assay until you reach a selection of investigation data you are interested in downloading.
  2. On the Investigation Details page, click Download Selected Metadata.
    Your browser prompts you to open or save the .zip file. Follow your browser's instructions to open or save the file.
    FireFox open or save download dialog box

    It may be useful to rename the .zip file as you save it to include the name of the investigation so that you can identify it more easily. For example, miRNA_metadata.zip.

    Archive File Contents

    The archive file is in compressed format. When you download it, it may be a single compressed folder or .zip file. When you open it, you see at minimum three text files at the root of the folder or file. An example of these follow. 

       a_10290.txt

       i_10290.txt

       s_10290.txt

    The text files describe the investigation, study or studies, and assay or assays. In this example, 10290 represents the file identifier but in practice, each file identifier may be named differently, even in the same investigation. Only the a, i, or s prefix is required. If you download full data or selected data, the archive may also contain other files or folders as appropriate for the investigation; for example, images. If you download metadata, the archive only includes the a, i, and s files.

Downloading Selected Data

Unable to render {include} The included page could not be found.

Downloading Large Files with Globus

Unable to render {include} The included page could not be found.

Managing Your Downloads

When you download data from CSSI DCC, keep in mind the following points:
  • Each time you download data while logged in, the system archives that download.
  • You can view a list of your previous downloads.
  • Partial or filtered downloads expire in seven days. Full downloads never expire.
    • A full download is an unfiltered download of an entire investigation.
    • A filtered download is one created by selecting metadata in the Visualize and Select section of the investigation, study, or assay details page. For instructions, refer to Visualizing and Filtering Data.
    • A partial download is a filtered download, or a download at the study or assay level.
  • If an archived download has not expired, you can download it again. If a download has expired, you can download it only one more time.
  • If something interrupts your download, you can resume the download.
  • The system constructs download zip filenames with the following components:
    • Investigation ID (for example, 123456789a123456789b1234)
    • Investigation title 
    • "Full" or "Partial"
    • "data" if the zip file contains only data 
    • "metadata" if the zip file contains only metadata 
  • When constructing the zip filename, the system omits spaces and special characters from the investigation title. If the resulting investigation title is more than 40 characters, the system truncates the title part of the filename as follows:
    • The first 19 characters
    • Two underscores
    • The last 19 characters
  • Example download filenames include:
    • 123456789a123456789b1234_ExampleInvestigationTitle_Full_data.zip
    • 123456789a123456789b1234_ExampleInvestigationTitle_Full_metadata.zip
    • 123456789a123456789b1234_ExampleInvestigationTitle_Partial.zip
    • 123456789a123456789b1234_ExampleInvestigationTitle_Partial_metadata.zip
    • 123456789a123456789b1235_ImagingofTCellsafte__mmunologicalSynapse_Partial_metadata.zip

To manage your downloads

  1. Log in to the CSSI DCC portal. For instructions, refer to Logging In to the CSSI DCC Portal. Your email address appears in the upper-right corner of the screen.
  2. Click your email address and then click My Downloads. The My Downloads History page appears with a list of your archived downloads. For each download, the list indicates the requested date, a download ID, and a status.  
    My Downloads History page.
    The following table describes each possible status:

    StatusDescription
    ProcessingThe initial status of each download.
    ReadyThe status indicating you can repeat the download.
    ExpiredThe ultimate status of each partial or filtered download.

    If a download request contains a partial download, the status reflects that download. Therefore, a download request may have a status of Expired, even if the request also contains a full download.

  3. In the list of downloads, you have the following options:
    • To search for a download, in the Search box, start typing a date or investigation title. The system filters the list of downloads. For example, typing "5/21" displays all downloads requested on May 21st.
    • To sort the list, click the Requested Date or Status column header.
    • To repeat a download, click the row for the download you want to repeat. The Download Details section appears, below the list of downloads. The following screenshots demonstrate some examples: 
      Download Details section with Download All Archives and click here options.  Download Details section with Download All Archives and re-request options.
  4. In the Download Details section, you have the following choices:
    • To download just one investigation, click the click here link for that investigation. Follow your browser's instructions to open or save the file.
    • To download all content from the selected download record (or to resume an interrupted download), click Download All Archives. (This button is not available for Globus downloads.) The response from the system depends on whether you have installed the DCC Download Manager:
    • If a partial download has expired, click Re-request. (This option is available only once for each download.) The system processes the request and sends you a message with "CSSI DCC Portal File Download Ready" as the subject line. 


Installing the DCC Download Manager

The DCC Download Manager allows you to repeat an archived download, if it has not expired.

To install the DCC Download Manager

  1. Attempt to download all archives, as described in Managing Your Downloads. If you have not yet installed the DCC Download Manager, a dialog box provides various links.
    DCC Download Manager dialog box
  2. Click a link to install the appropriate download manager application for your operating system: Windows or Mac. Follow your browser's instructions to open or save the file. Run the executable. Another dialog box appears.
    DCC Downloader dialog box.
  3. Click Quit.
  4. Repeat the attempt to download all archives, as described in Managing Your Downloads.
  5. Follow your browser's instructions to associate dccdl links with the DCC Download Manager application. Another dialog box appears, specifying a download directory. For instructions, refer to Using the DCC Download Manager.


Using the DCC Download Manager

The DCC Download Manager allows you to repeat an archived download, if it has not expired, and resume a download, if it had been interrupted.

To complete a download

  1. Attempt to download all archives, as described in Managing Your Downloads. A dialog box appears, specifying a download directory. The default download directory is C:\Users\<user name>\Downloads.
    DCC Downloader dialog box
  2. To specify a different download directory, click Set Download Directory. Follow your operating system's instructions to navigate to and select a folder.
  3. Click Start Download. A window appears, indicating the download status for each component.
    DCC Downloader window
    The following table describes each possible status:

    StatusDescription
    DownloadingThe initial status of each download.
    ErrorThe status indicating a problem. If you need support on an error, contact the Center for Strategic Initiatives (CSSI) Data Coordinating Center (https://cssi-dcc.nci.nih.gov/cssiportal/contactus) or click the Contact Us link at the bottom of any DCC page.
    InterruptedThe status indicating you can resume the download.
    DoneThe ultimate status of each download.


Using a Download Summary

Unable to render {include} The included page could not be found.

  • No labels