NIH | National Cancer Institute | NCI Wiki  

Some of the features in caDSR II allow you to process content from a CSV file for bulk/batch importing to create new content. To view or use these features, log in as described in Logging In. These features need specially formatted templates. 

Contents of This Page


Common usage details for the templates are that due to inclusion of the following Data Validations please follow these conventions: 

  • Basics
    • When the provided Template has .xls or .xlsx file extension, always start with a fresh Template for each import
      • This insures that the Data Validation and Formulas that are embedded in the template for drop-downs and auto-populate are correct for each Import and rows/valid choices have not been inadvertently deleted.
    • Save-As the completed Template as .cvs before importing
      • NOTE: This is a common step to forget and if you try to import a .xsl or .xslx file it will cause an immediate exception "error" when clicking "Import", so check the file extension if you have a problem
    • While editing the Template do not delete entire Rows or Columns
      • This is due to the Data Validation, if you delete rows some of the valid choices will be deleted
      • If you delete an entry in a cell that was auto-populated, you may lose the formula. These formulas are not essential for importing, but are helpful to avoid typos or other errors
      • You can use the Delete Key to clear the contents of individuals cell or cells but do not delete entire rows or columns
  • Templates contain Data Validation for several columns that are found on several of the templates end users avoid errors and streamline completing the spreadsheets. 
    • All templates - Context
        • Drop down of all valid Contexts
    • Templates with Macros
      • CDE Match: "Opt"+"CMD"+"t" to transform content into the correct format for CDE Match. After filling in your column headings and permissible values, the macro will take each column heading and permissible values and create a new "Transform" sheet containing one row per heading/unique permissible value combination. For example, if the column has 5 permissible values, there will be 5 rows in the transformed sheet. Please remember to save as cvs before running CDE Match.
      • Form Import: "Shift" + "CMD" + "t" to transform content into the correct format for importing. If you are using a REDCap data dictionary csv file, cut and paste your content into the template starting on Row 3, Column E "Variable / Field Name".

        • The macro will:
          • Add 4 columns to the worksheet. (Leave "DO NOT USE" empty, enter your "Batch User Name", a "Batch Name" on each Row. The combination of "Batch User Name" AND "Batch Name" must be unique for each imported template.)
          • Generate a unique "Seq ID" for each row
          • Copy "Form Name" into Column G, Section Header for each row since this is a required field in caDSR.  
DO NOT USEBatch UserBatch NameSeq ID

    • Import DEC and Import VD
        • Drop Down of the Standard Conceptual Domain Names, the CD ID will automatically populate
    • Import Value Domain
      • Drop Downs for Context, Format, UoM, Datatype, and Type (Enumerated/NonEnumerated)
    • Import Designations - this is used to import the "USED_BY" designation as well as all AI types for Data Elements, Data Element Concepts, Value Meanings and Value Domains
      • Drop Downs for AI type, Alternate Name, Alternate Name Type, Alternate Name Context
      • For Alternate Name, when the Alternate Name Type is "USED_BY" the Alternate Name must be the same as the Alternate Name Context
    • Import PV/VM 
      • Drop Down for VM String Type
      • Drop Down for optional VM Alternate Name Type
    • VM Match
      • No drop downs.

The Batch Owner and Batch User columns can be any text that you provide. The system does not validate these columns. 


The following table lists the latest templates.

  File Modified
Microsoft Word Document DEC Bulk Load Instructions Sprint23 v1.docx Initial instructions for running DEC Import in Sprint 23 Jan 17, 2022 by Warzel, Denise (NIH/NCI) [E]
File S36 VM Match Import.csv S36 VM Match Templates - No changes were made in S36 Sep 21, 2022 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet S40 DEC Import Template-with Data Validation-20221214_FINAL.xls S40 DEC Import Template with Data Validations Pediatric Context Added Jan 17, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet S40 CDE Import Template-with Data Validation-20221214_FINAL.xls S40 CDE Template with Validations, Pediatric Context Added Jan 17, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet S40 CDE Match Template.xlsm S40 CDE Match Template with Macro Jan 17, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet S40 Form Template With Macro-FINAL-20221215.xlsm S40 Form Import Template with MACRO Feb 08, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet CDE Match Data Standardization Template Sprint 41v1-01272023 FINAL.xlsm CDE Match Template Sprint 41 with Macros Feb 16, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet S40 VD Import Template-with Data Validation-20230104_FINAL.xls VD Import Sprint 40-with Validation added in upload code in addition to within the template - Template was not changed in Sprint 41, Sprint 40 include Pediatric Cancer Context, 41 did not. Mar 03, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Powerpoint Presentation Jira-2236-CDEMatchLogicv4.pptx Current CDE Match Logic as of Nov. 2022 Apr 06, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Word Document Jira-2236-CDEMatchInstructionsSprint42.docx CDE Match Instructions, Notes and Tips Apr 06, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet S44 Designation Import Template-with Data Validation-20230525.xls S44 PV VM Import Template-with Data Validation May 26, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet S44 PV VM Import Template-with Data Validation-20230525.xls S44 PV VM Import Template with Validations, Alt Names Updated May 30, 2023 by Warzel, Denise (NIH/NCI) [E]


Tips for All Imports

Use Data ValidationAlways start with a fresh Template with Data Validation to help avoid uncaught errors due to invalid manually entered data. We have added a lot more error handling in this release, but it's not perfect yet, in particular when invalid or retired concepts are used.
Required Columns

All templates require a BATCH USER, BATCH NAME, and SEQ ID on each row.


When reimporting, it is safest to select all the rows and "Delete", then select "Purge", then select all rows and “Purge Records".

For Forms, select "Delete Hierarchies", then select "Purge", then select the rows and "Purge Hierarchies".

Unique Column NamesYou may add additional columns to the template for taking/keeping notes after the last template column, but the column names must be unique.
Invalid or Retired ConceptsInvalid VM Concepts on import are not saved in the system and can create a GAP in the concept drop down. If the invalid or missing concepts are found, the system will display and error. Curators should fix all Concept Errors reported on Import BEFORE Validating.
Concept FormattingConcept String should not contain commas, tabs, or or other hidden characters. Always "Paste Values" in order to avoid issues.
CSV Files!!Template must be saved as CSV for importing. Other file types will cause an error.
Seq ID RulesSeq ID must be a unique integer.
Import ExceptionsMissing Batch User, Batch Name, or Seq ID will cause Exception.
Selecting the "Import" Command

For imports and matching, select the type of import/match item from the "Favorites" list, then select the "Import" command. For most imports there will be a dropdown with either one or two of the following choices: "Conceptual Object Import" and "Data Object Import".  For most imports choose the first choice in the list.

For VM Match both choices are presented, choose the second one, "Data Object Import".

Exception on SEQ/Row NumberMultiple missing required fields may not report the correct row number in the Exception Message. Inspect the template for empty cells.
Save / RevalidateIf making changes through front-end, Save and Revalidate. This will refresh the messages.

Tips for CDE Imports

Duplicates in FileDuplicate rows within the imported file are not caught on Import, but duplicate CDEs will not be created.

Tips for CDE Match

Run TransformCDE Match transforms the Source input into one row per column and permissible value. The command to transform the Source input details is "opt"+"cmd"+"t".
See InstructionsPlease see the CDE Match Instructions document attached to this page. 

Tips for DEC Imports

Validating Lots of RowsSometimes, the system throws an error when attempting to validate more than 4 rows at a time. If that happens, try validating a few rows at a time.

Tips for VD Imports

None at this time.

Tips for PV VM Imports

Import into Existing VDThe PV VM import is designed to import an enumeration into an existing Value Domain. PV/VMs can be created using existing VMs, or you can specify a new VM either with or without concept codes.
Specified VM DefinitionOptional Specified VM Definition is only used if the VM Type is "TEXT". The system will not throw an error but it won't be added to the VM.
Possible TimeoutValidate/Create will timeout if more than 5 minutes. Try batches of 100 until we can redesign the code. Watch.
VM ReuseIn a Text VM is specified and an existing VM with the exact name is found, it will be reused, even if it has concepts. The Validation message will indicate that existing VM is being used (need to fix spacing).

Tips for VM Match

After Importing

The results after importing are displayed one row per Value Meaning. All VM Match user results are kept in one database table, and without filtering results, they are all displayed in the same grid. So, after selecting "Go to Data Manager" enter your batch name and click "Apply Filter" to see just your results.

User Tips

The User Tips column is used to replace the VM Name. Sometimes the VM Name from your form or data is not very representative of the its meaning, or consists of too many terms IF A "User Tip" is entered it is used Instead of the VM Name.

"Run Match" Command

Select the rows to run the match algorithm against and select the "RUN MATCH" Command.

Column Order

The main column in VM Match is the VM Name. The processing is designed to find matching concepts based on the name and insert the Concept Name and Concept Code into the table next to the VM NAME. This order of the columns makes it easier to populate the PV/VM Template With cut and paste. 

You can use "Delivery Options" command to export the VM Match results into and Excel file. This simplifies creating the list of PV/VMs to import into a Value Domain.

Tips for Form Imports

Import from REDCap Data Dictionary FormatForm import uses a csv file format which is one row per Question. The template contains an Excel Macro so when opening the template to paste your content into, select "Enable Macros".
Running the MacroChoose "Shift" + "CMD" + "t"
Filling the template with form dataThe 1st row of headings are the REDCap DD headings. The 2nd row of headings match caDSR headings. Enter a short name in the Variable / Field Name, this can be used for matching to an existing CDE. In future release it will be added as a Question Short Name. 
Form ContextREDCap does not have a column for Context. In Sprint 41 Context for the form is selected after importing the template into caDSR.

Tips for Designation Imports

Consistency CheckingIf AI Long Name does not match the specified AI Public ID Long Name, the system will show an error. We are requesting AI Long Name and AI Type to ensure Alt names are not inadvertently added to the wrong AI. Public IDs are very similar, typos are easy to make. If this becomes onerous, we could add a Validate Step to display the AI Long Name and AI Type for curator to see and visually verify before selecting Create.

  • No labels