NIH | National Cancer Institute | NCI Wiki  
Create content

All the bulk import and matching tools need specially formatted templates that are attached to this wiki page.

  • Click on the three dots in the upper right corner of this page to access "Attachments".  
  • Click on the name of the template and when it opens, select the download icon.
  • Content is entered into the downloaded template and used with the tool on the main page under "Favorites". 

Some of the tools in caDSR II allow you to process content from a CSV, XLS, or XLSX file for bulk/batch importing to create new content. 

  • None of the tools on the main caDSR II or Code Map portals require an account or login for their use. 
    • These tools are for browsing caDSR content or for running "matching" algorithms that do not create content in caDSR>
  • The IMPORT tools create new content in caDSR and therefore require an account and login. 
    • To determine whether a login is required, refer to the "Tips" section for each tool.
    • To view or use tools that require a login, refer to Logging In.
  • Other tools do not require a login.
Contents of This Page

Conventions

Common usage details for the templates are that due to inclusion of the following Data Validations please follow these conventions: 


  • Basics
    • When the provided Template has .xls or .xlsx file extension, always start with a fresh Template for each import
      • This insures that the Data Validation and Formulas that are embedded in the template for drop-downs and auto-populate are correct for each Import and rows/valid choices have not been inadvertently deleted.
        • Templates can contain Data Validation for several columns that are found on several of the templates to help end users avoid errors and streamline completing the spreadsheets. 
          • All templates - Context
            • Drop down of all valid Contexts
    • Use the Save-As command for the completed Template to save as .cvs before importing
      • NOTE: This is a common step to forget and if you try to import a .xsl or .xslx file it will cause an immediate exception "error" when clicking "Import", so check the file extension if you have a problem
    • While editing the Template do not delete entire Rows or Columns
      • This is due to the Data Validation, if you delete rows some of the valid choices will be deleted
      • If you delete an entry in a cell that was auto-populated, you may lose the formula. These formulas are not essential for importing, but are helpful to avoid typos or other errors
      • You can use the Delete Key to clear the contents of individuals cell or cells but do not delete entire rows or column
      • PC users can select "Alt + F8" to open the Macros dialogue box and select the desired command and "run".  Be sure to set the Macros to "This Worksheet" in case you have more than one template with macros open.
  • Templates with Macros
    • Macros are a recorded sequence of actions or commands that automate tasks to reformat the data in the template. 
      • Upon opening templates with Macros the system will prompt to "Enable" or "Disable" Macros. Choose "Enable". 
      • Macros can be executed in several ways.
        • Enter a combination of pre-defined template specific keystrokes 
        • Selecting Tools→Macros→Macro from the Excel toolbar
          • Select the Macro you wish to run and click "Options" to display the correct key strokes.
          • Select the Macro and select "Run". 
    • CDE Match:
      • The CDE Match tool supports multiple spreadsheet layouts for use with this feature. There are several Excel Macros that must be used to convert from the source format into the CDE Match input format required by the CDE Match tool.
        • TransformInput
          • A simple CSV with one column per data element to match, the field name in the first row and any permissible values in the rows below.
          • Keystrokes: "Opt"+"CMD"+"t" (MACOS), "Alt" + F8 (Windows) to transform content into the correct format for CDE Match. After filling in your column headings and permissible values, the macro will take each column heading and permissible values and create a new "Transform" sheet containing one row per heading/unique permissible value combination. For example, if the column has 5 permissible values, there will be 5 rows in the transformed sheet. Please remember to save as .cvs before importing it the CDE Match Import feature.
        • TransformInput-REDCap
          • A REDCap Data Dictionary CSV format.
          • Select "Tools>>Macros" (MACOS), select "Alt" + F8 (Windows) to see the dialog box and select the desired command.
        • Word Document
          • A custom word document with the data element to match in column A,  any permissible values in Column B, one row per permissible value.
          • Select "Tools>>Macros" (MACOS), select "Alt" + F8 (Windows) to see the dialog box and select the desired command.
    • Form Import:

      • The Form Import supports multiple spreadsheet layouts for use with this feature. There are several Excel Macros that must be used to convert from the source format into the form input format required by the Form Import tool.

      • The macros below perform actions on the source input to reformat it into the required caDSR II format.  They are listed in the order they are intended to be executed.

      • CombineZip

        • “Ctrl” + “w” (Windows) to select multiple files in REDCap Data Dictionary format from a folder on your desktop. After importing your files into the workbook, run the Standardize macro described below.
      • Standardize
        • "Shift" + "CMD" + "t" (MacOS) or "Ctrl" + "Shift" + "t" (Windows) to transform content into the correct format for importing.
        • The macro cleans up hidden characters that could raise errors on import.
      • PhenX Standardize and  REDCap Data Dictionary layout :

        • "Shift" + "CMD" + "x" (MacOS) or "Ctrl + "Shift" + "x" (Windows) to transform content into the correct format for importing. If you are using a REDCap data dictionary csv file, cut and paste your content into the template starting on Row 3, Column E "Variable / Field Name".

        • The macro will:
          • Add 4 columns to the worksheet (DO NOT USE, Batch User, Batch Name, and Seq ID). Leave "DO NOT USE" empty. Enter your "Batch User Name" and a "Batch Name" on each row. The Batch Owner and Batch User columns can be any text that you provide. The system does not validate these columns. The combination of "Batch User Name" AND "Batch Name" must be unique for each imported template.
          • Generate a unique "Seq ID" for each row
          • Copy "Form Name" into Column G, Section Header for each row since this is a required field in caDSR. 
          • Remove "PX" from the Section header/Module Name.
          • Remove the extra row for "Record ID" that is found in some PhenX templates.
          • Clear the "Field Note" column as it contains information not imported or used in caDSR Forms. 
    • Import DEC and Import VD
      • Drop Down of the Standard Conceptual Domain Names, the CD ID will automatically populate
    • Import Value Domain
      • Drop Downs for Context, Format, UoM, Datatype, and Type (Enumerated/NonEnumerated)
    • Import Designations - this is used to import the "USED_BY" designation as well as all AI types for Data Elements, Data Element Concepts, Value Meanings and Value Domains
      • Drop Downs for AI type, Alternate Name, Alternate Name Type, Alternate Name Context
      • For Alternate Name, when the Alternate Name Type is "USED_BY" the Alternate Name must be the same as the Alternate Name Context
    • Import PV/VM 
      • Drop Down for VM String Type
      • Drop Down for optional VM Alternate Name Type
    • VM/Concept Match
      • No drop downs.


Templates

The following table lists the latest templates.

  File Modified
Microsoft Excel Spreadsheet CDE Match_Source Input Examples.xlsx CDE Match Source Format Examples Jul 03, 2025 by Warzel, Denise (NIH/NCI) [E]
Microsoft Word Document Jira-2236-CDEMatchInstructionsSprint42.docx CDE Match Instructions, Notes and Tips Apr 06, 2023 by Warzel, Denise (NIH/NCI) [E]
Microsoft Powerpoint Presentation Jira-2236-CDEMatchLogicv5_20250212.pptx Current CDE Match Logic as of February 2025. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Spreadsheet Master_TemplateLOVs_20241113.xlsx Lists of Values from November 2024. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
File S47 DEC Match Template.csv DEC Match Template S47 Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Sheet S47 Form Template With Macro v2-20231002-Clean.xlsm Form Import with Macros S47 Aug 13, 2024 by Warzel, Denise (NIH/NCI) [E]
File S52 VM Match Template-20240209.csv VM Match Template Updated Headings to remove spaces S52 Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Spreadsheet S53 Rep Term Import.xlsx Representation Term Import S53. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Spreadsheet S61 CDE Import-DataValidation Jira-3686-20241114.xlsx CDE Import with Validation S61. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Spreadsheet S61 CDE Update-DataValidation Jira-3686-20241114.xlsx CDE Update with Validation S61 Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Spreadsheet S61 DEC Import Template-with Data Validation Jira-3686-20241114.xlsx DEC Import Template with Validation S61. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Spreadsheet S61 DEC Update Template-DataValidation Jira-3686-20241114.xlsx DEC Update with Validation S61. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Sheet S61 Definition Import Template-with Data Validation_20241120.xls Definitions Import S61 Dec 06, 2024 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet S61 Designation Import Template-with Data Validation Jira-3686-20241113.xls Designation Import with Validation S61. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Spreadsheet S61 Protocol Import Template_with_Validation Jira-3686-20241113.xlsx Protocol Import with Validation S61. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Sheet S61 PV VM Import Template-with Data Validation Jira-3686-20241113.xls PV VM Import with Validation S61. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Sheet S61 ReferDocumentsTemplate-WithDataValidation Jira-3686-20241113.xls Reference Document Import S61. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Sheet S61 VD Import Template-with Data Validation-20241114.xls VD Import with Validation S61 Dec 12, 2024 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Sheet S61 VD Update Template-with Data Validation-20241114.xls VD Update With Validation - S61 Dec 12, 2024 by Warzel, Denise (NIH/NCI) [E]
File S62 NCI Thesaurus Concept Import Template Jira-3820-20241218.csv NCI Thesaurus Concept Import S62. Feb 28, 2025 by Frost, Ruth (NIH/NCI) [C]
File S63 Value Map Import Jira-3622-20250214.csv Value Map Import S63. Feb 21, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Sheet S67 CDE Match Template Jira-4104-20250625.xlsm CDE Match Template S67. Jul 03, 2025 by Warzel, Denise (NIH/NCI) [E]
Microsoft Excel Spreadsheet S67 Model Import v10 Jira-4172-20250622.xlsx Model Import/Update S67 Jun 26, 2025 by Frost, Ruth (NIH/NCI) [C]
Microsoft Excel Spreadsheet S67 Model Map Import-with Data Validation Jira-4154-20250625.xlsx Model Map Import with Validation S67 Jun 26, 2025 by Frost, Ruth (NIH/NCI) [C]

Tips for All Templates

Use Data ValidationAlways start with a fresh Template with Data Validation to help avoid uncaught errors due to invalid manually entered data. We have added a lot more error handling in this release, but it's not perfect yet, in particular when invalid or retired concepts are used.
Required Columns

All templates require a BATCH USER, BATCH NAME, and SEQ ID on each row.

Delete/Purge

When reimporting, it is safest to select all the rows and "Delete", then select "Purge", then select all rows and “Purge Records".

For Forms, select "Delete Hierarchies", then select "Purge", then select the rows and "Purge Hierarchies".

Unique Column NamesYou may add additional columns to the template for taking/keeping notes after the last template column, but the column names must be unique.
Invalid or Retired ConceptsInvalid VM Concepts on import are not saved in the system and can create a GAP in the concept drop down. If the invalid or missing concepts are found, the system will display and error. Curators should fix all Concept Errors reported on Import BEFORE Validating.
Concept FormattingConcept String should not contain commas, tabs, or or other hidden characters. Always "Paste Values" in order to avoid issues.
CSV Files!!Template must be saved as CSV for importing. Other file types will cause an error.
Seq ID RulesSeq ID must be a unique integer.
Import ExceptionsMissing Batch User, Batch Name, or Seq ID will cause Exception.
Selecting the "Import" Command

For imports and matching, select the type of import/match item from the "Favorites" list, then select the "Import" command. For most imports there will be a dropdown with either one or two of the following choices: "Conceptual Object Import" and "Data Object Import".  For most imports choose the first choice in the list.

For VM Match both choices are presented, choose the second one, "Data Object Import".

Exception on SEQ/Row NumberMultiple missing required fields may not report the correct row number in the Exception Message. Inspect the template for empty cells.
Save / RevalidateIf making changes through front-end, Save and Revalidate. This will refresh the messages.


Tips for Import Templates

Tips for CDE Imports

Duplicates in FileDuplicate rows within the imported file are not caught on Import, but duplicate CDEs will not be created.


Tips for Concept Imports

New Concepts

New concepts are automatically loaded once a month when EVS publishes the monthly NCI Thesaurus file on their FTP site. If new concepts have been created for your content before they are published in the NCI Thesaurus, please contact the NCI Registration Authority at caDSR.RA@mail.nih.gov for assistance. The NCI Curation Team can import concepts directly into caDSR for their use before they are published in the NCI Thesaurus.

.

Tips for DEC Import and Update

Validating Lots of RowsSometimes, the system throws an error when attempting to validate more than 4 rows at a time. If that happens, try validating a few rows at a time.
DEC Import Required FieldsDEC Context, OC Concepts, Property Concepts and DEC CD Name/ID are required.
DEC Update Required FieldsThe DEC Context, DEC Public ID and Version are required for an Update. Any other columns that are populated will be treated as a change to the existing DEC, such as DEC CD, OC Concepts, Property Concepts.  Retired DECs cannot be updated. DEC WFS will be set to "DRAFT NEW".  
Known IssueIn release 1.53.4, if a duplicate DEC exists in another Context, you cannot bulk update the DEC. This will be fixed in future release.


Tips for VD Imports

None at this time.

Tips for PV VM Imports

Import into Existing VDThe PV VM import is designed to import an enumeration into an existing Value Domain. PV/VMs can be created using existing VMs, or you can specify a new VM either with or without concept codes.
Specified VM DefinitionOptional Specified VM Definition is only used if the VM Type is "TEXT". The system will not throw an error but it won't be added to the VM.
Possible TimeoutValidate/Create will timeout if more than 5 minutes. Try batches of 100 until we can redesign the code. Watch.
VM ReuseIn a Text VM is specified and an existing VM with the exact name is found, it will be reused, even if it has concepts. The Validation message will indicate that existing VM is being used (need to fix spacing).


Tips for Form Imports

Import from REDCap Data Dictionary Format

The Form Import Template is an xslm filetype and contains several Excel Macros, so please be sure to select "Enable Macros" when opening the template. The QA process tests only on a Windows PC platform, not Mac laptops, however team testing using Mac laptops have not reported issues.

  1. Paste your form content into the template starting on Row 3, column E. At a minimum you must run the Macro "Standardize", see instructions below.
  2. If importing a PhenX REDCap Data Dictionary, you must run "Standardize", then "PhenxStandarize".
  3. "CombineZip" is optional, it automates copying form content into the template from one or more REDCap dictionary csv files. See instructions below. 
  4. Enter a Batch User, Batch Name and ensure that each row has a unique "Seq Id"
  5. Save the Form Template file as csv.
  6. Run Form Import from Manage>>Manage Data (or your bookmarked Favorites)
  7. Select "Import" from the toolbar, then "Conceptual Data Object" from the drop-down
  8. Select the checkbox next to "Form Import RedCAP v1" and select "Import" option from the toolbar (Between "Add New Mapping" and "Go To Data Manager")
  9. Select the "Choose File" and select the Form Import Template from our desktop that you have prepared for loading
  10. Change the last row in the table "Start at Row" from 2 → 3
  11. Select "Import" at the bottom of the Table
  12. After "Success" message, Select "Go to Data Manager" or Manage>>Manage Data>> Form Import (or from your bookmarked Favorites)
  13. Use the filter to search for your form, Click "Apply Filter"
  14. Select "Edit" for your form
  15. Select a "Context" from the dropdown
  16. Save
  17. Select "Parse Form"
  18. If successful, select "Validate Form"
  19. Review the results
  20. After successful Validate, Select "Create Form"
  21. Copy the Public Id and choose "Forms" from Manage>>Manage Data (or your bookmarked Favorites) to view the newly created form in caDSR
Running the Macros

There are 3 macros in the Excel Template. The Macro run key sequence can be viewed by selecting "Tools" from the Excel menu bar, then "Macro". 

Macro NameDescriptionExecute macro
CombineZipThis macro allows the user to import one or more forms in REDCap Data Dictionary format into the Form Template from a desktop folder.  This allows the user to run Form Import on a file containing multiple forms instead of one form import at a time. After choosing the keys as indicated in Execute Macro, select a Folder from your desktop and select the forms that you want to combine and import into the spreadsheet. Then run the "Standardize" macro, instructions below.Choose "Option" + "Command" + "w"
StandardizeEach row must have a Section Header and a few other details. This macro transforms content from REDCap structure into the structure suitable for caDSR Form Import.Choose "Control" + "Shift" + "x"
PhenxStandardizeFiles sent to us by PhenX team contained a row with Question Text "Record ID", and the Protocol ID was inserted into the Field Note Column. This macro removes the extra row for "Record ID", and clears the "Field Note" columnChoose "Control" + "Shift" + "t"
Filling the template with form dataThe 1st row of headings are the REDCap DD headings. The 2nd row of headings match caDSR headings. Enter a short name in the Variable / Field Name, this can be used for matching to an existing CDE. Paste content into the template starting in the 3rd row. Please see the template for the mappings between the REDCap Data Dictionary column names and the caDSR form column names.
Form ContextREDCap does not have a column for Context. Context for all the forms in the templates is selected after importing the template into caDSR.

Tips for Designation Imports

Consistency CheckingIf AI Long Name does not match the specified AI Public ID Long Name, the system will show an error. We are requesting AI Long Name and AI Type to ensure Alt names are not inadvertently added to the wrong AI. Public IDs are very similar, typos are easy to make. If this becomes onerous, we could add a Validate Step to display the AI Long Name and AI Type for curator to see and visually verify before selecting Create.

Tips for Definition Imports

Administered Items

You can update the following attributes for Administered Items Value Meaning, Data Element Concept, Data Element, and Value Domain via the Definitions Import template:

  • Preferred Definitions
  • Alternate Definitions

Tips for Importing Updates to Registered Models

TBD

TBD


Tips for Importing Updates to Registered Model Mappings

TBD

TBD


Tips for Importing Updates to Registered Model Mappings' Value Maps

TBD

TBD


Tips for Match Templates

Tips for CDE Match

Run Transform

CDE Match transforms the Source input into one row per column, permissible value and optionally permissible value meaning depending on the format of the data on the Source tab and which Excel Macro is used. There are 3 Excel Macros for 3 different input formats: TransformInput (shortcut: "opt"+"cmd"+"t"), TransformInput_REDCap (shortcut: "opt"+"cmd"+"r"), and WordDocumentFormat. 

To access the macros from the Excel tool bar select "Tools>>Macro>>Macros". Make sure the "Macros In" option is set to "This Workbook". Select the Macro that matches the format pasted into the Source tab. 

To access examples of the three source formats download the "CDE Match_Source Input Examples" spreadsheet attached to this page.

See InstructionsPlease see the CDE Match Instructions document attached to this page. 
Special FeaturesCDE Match has a command "Run DEC Match". This will run the match algorithm on the imported files using just the CDE names, ignoring the permissible values. Click the "Matched DEC" node to see results.


Tips for DEC Match

DescriptionDEC match will run matching algorithms on the names in the template and display results in the "Matched DEC" node. You can select one of the DECs as "Preferred" and then download the results using the Delivery Options feature. 
Context Column

In the Context column, you can enter one or more contexts to search for matching DECs. Separate each DEC with a comma. Do not use spaces between DECs. For example:

XXX,CTEP,CCR


Tips for VM and Concept Match

After Importing

The results after importing templates are stored in one table for all imported files. After the "success message" select "Go to Data Manager", or if on the main page select "VM/Concept Match" again. The system will display a filter where users can enter their batch user name and/or batch name, then click "Apply Filter" to see just your results. 

To see the details for the matches for each row in the input template, select "Edit" next to the rows and then select "VM Matched Results".

Use Cases

There are several commands available because the feature can be used to match text strings entered into the template to:

  1. Existing caDSR Value Meanings (VM). caDSR VMs are already associated with Concepts
  2. EVS Concepts from NCI Thesaurus or selected external terminologies

By selecting the appropriate command, the user can control the breadth of the matching algorithm.

Terms in the input template can be any text, the Permissible Value, the Permissible Value label, or the name of a variable/data element. 

"User Tips" Column

The User Tips column is used by the system instead of the user supplied name. Sometimes the string from your form or data is not very representative of the its meaning, or consists of too many terms. If a "User Tip" is entered it is used instead of the imported name. User Tips can be entered through the front end or entered into the Template for each row and imported. 

Matched Results

The system indicates how many matches were found, why each row was considered a match, and the type of match that was run. Once a match is selected, it is displayed in the table of summary results by row.

Algorithm

Punctuation and spaces are removed. The "Rule Description" explains why each match was selected. The algorithm does not currently account for spelling errors.

"Run Match" VM Header Command

Select the rows to run the match algorithm against and select the "RUN MATCH" Command. Punctuation and spaces are removed. This command will try to find Exact or Like Matches using the caDSR Value Meaning (VM) preferred name for existing VMs, Concept Preferred name and Synonyms.

Click "VM Matched Results" to review recommended matches. If you do not find a desired match, select "Run VM Match Unrestricted".

"Run Match - VM Only" VM Header Command

This match searches only for existing VM names that are good matches to either the name of synonyms. It includes exact matches to the name or alternate VM names.

"Run VM - Match Unrestricted" 

VM Header Command

This match does not stop after finding exact matches. It extends the search to emphasize matches on the longest term in addition to "like" matches. 

"Run Match - Concepts Only" VM Header Command

This match searches only for Concepts by name and synonyms.

"Run Match Concepts Only - Unrestricted" VM Header Command

This match searches only for Concepts by name and synonyms the longest term in addition to more extensive matches. 

"Run Match - Terminology"

This matches allows the user to select a specific terminology to search for Concepts by name, synonyms or Concept Code.

"VM Matched Results"

If there is only one exact match, the system will automatically associate the matched result with the imported VM Name.

If there are multiple matches the user can select a row from "VM Matched Results" and select the "Set Preferred" command.

Multiple Concepts can be selected one at a time to post coordinate concepts. The system will appended the concept to the preferred concept string and generate the resulting name in the order the concepts are selected.  This can be viewed by selecting the Node with the Concept Name to view the selected items. The concepts and be reordered manually, but be sure to reorder the concept names, or clear the results, "Save" and then select concepts again from the matched results. 

Advancing through the imported VMs

When on the "VM Match Header" you can advance to the next VM in your imported file by using the "Rows x of xxx" arrows.

Column Order

The main column in VM Match is the VM Name. The processing is designed to find matching concepts based on the name and insert the Concept Name and Concept Code into the table next to the VM NAME unless "User Tips" are entered. This order of the columns makes it easier to populate the PV/VM Template using cut and paste. 

Exporting results

Use the "Delivery Options" command to export the VM Match results into and Excel file. This simplifies creating the list of PV/VMs for PV/VM Import to populate and existing Value Domain.