Topic: caArray Usage

Release: caArray 2.0 and above

Date entered: 04/01/2009

Question

caArray Status of Importing: Imported vs Imported Not Parsed

Browsing through NCI's caArray experiments, I noticed that the status for array design files and for experiment data files can be either "Imported" or "Imported (Not Parsed)". What are the differences between these two states?

Status Designation for Files Imported into caArray

Answer

Imported and Imported (Not Parsed) describe the import status for array design files and for sample-related array data or annotation files (See [caArray016] File Supported by caArray). All images and experiment-level related information should be added to Supplemental Files ([caArray 014 - Which files can be uploaded into caArray?]).

File Import versus File Parsing

Depending on the type of import, there are two stages to the adding a file to caArray: importing and parsing.

All files being imported pass through the "importing" state. It is a pure data storage process: the data file uploaded will be stored into the caArray database in its original data format (as it is).

If the files submitted are supported by caArray (see [caArray016] File Supported by caArray), they will be further parsed into a caArray table. During the file parsing, values from each data column, such as signal values, or log ratio values from an Affymetrix's CHP file, are extracted into a file-type specific table structure in caArray and associated to probe sets. These parsed files will be marked as Imported at the end of process.

caArray has the ability to upload the array design files or experiment data from many array providers, even if it doesn't have a parser available yet. Those files will ONLY be imported into caArray, without being validated or parsed. The end state of those files is imported not parsed as shown in the illustration with the question. Even if a file is not parsed, the user will still be able to download its contents (through caArray's user interface as well as through the programmatic API), and associate the file to samples, extracts, and hybridizations. This caArray feature allows data to be shared and helps the system identify which new parsers need to be developed in the future.

File Upload Process # 1: Array Design files

The following illustration shows the data flow of array design file processing in caArray.

Data Flow of Array Design Processing in caArray

File Upload Process # 2. Non Array Design Files

The following illustration summarizes the general data flow for uploading a file into caArray. It depicts when a file should be loaded as a supplemental file and when it would be imported, either with a final status of Imported or Imported Not Parsed.

Data Flow for Uploading a File into caArray

Have a comment?

Please leave your comment in the caArray End User Forum.