NIH | National Cancer Institute | NCI Wiki  

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migration of unmigrated content due to installation of a new plugin
Scrollbar
iconsfalse

Page info
title
title

Panel
titleContents of this Page
Table of Contents
minLevel2

...

Introduction

This document is a section of the Loader Guide. It was formerly the LexEVS v5.0 Source Mapping Guide.

NCI MetaThesaurus: RRF content (v5.1)

In LexEVS v5.1, loader enhancements for RRF content were made. Loads of the NCI MetaThesaurus RRF formatted data into the LexGrid model now accurately reflect the state of the data as it exists in the current RRF files. In the text that follows, the Problem sections describe LexEVS v5.0 behavior, and the Solution sections describe LexEVS v5.1 behavior.

Data Model Elements

Most data elements will be loaded as either properties or property qualifiers.

[ https:--//wiki.nci.nih.gov-download-attachments-18947057-Property.jpg]

A few will be loaded as qualifiers to associations.

MRREL.RRF File

Problem:

REL and RELA column elements from the RRF source need to be connected.
Currently these are loaded as separate relationships preventing the user from connecting to the REL/RELA combinations that actually occur in the NCI-META (e.g. RELA may be different for same REL value in different sources).

...

User is unable to distinguish individual relationships from one source or another. The same association "entity" exists only once but has two "source" qualifiers.
User is unable to distinguish the AUI1/STYPE1 and AUI2/STYPE2 which gives us the information about what source data structures are actually being connected by MRREL entries. Users also need the ability to associate AUI/STYPE fields with SAB.
Users sole choice for rendering a relationship in terms of the strings on either side is to use preferred concept names.

...

Propose AUI to AUI - the way CUI to CUI are currently handled in the implementation.
Propose entity to entity relationship - will still have to account for CUI to CUI relationships.
Load each unique RUI (would be quite large). They would need to be listed as supported association (this is not traditional how it is used).

...

Do not treat a CUI1 = CUI2 relationships differently than a CUI1 != CUI2 relationship. For API and query purposes, qualify these relationships with a 'selfReferencing=true' Qualifier. In this way, we can still avoid cycles in the API, but maintain all relevant Qualifier information in the relation.

MRSAT.RRF

Problem:

MRSAT.RRF is not loaded but only accessed for given preferred term algorithms. This data should be loaded as concept properties (STYPE=CUI), properties on properties (STYPE=AUI, SAUI, CODE, SCUI, SDUI), qualifiers on associations (STYPE=RUI,SRUI). Some complexity may arise as concept properties can have additional qualifiers, but property-properties cannot and association-qualifiers cannot.

...

If the STYPE is something other than RUI or SRUI, you can load
that row as an entity property. The fields you'd want to capture
are:

  • CUI - We use this as the entityCode and is loaded as such in the table.
  • METAUI - load as a propertyQualifier (name=METAUI, value)
  • STYPE - load as a propertyQualifier (name=STYPE, value)
  • ATUI - load as propertyId
  • ATN - load as property name
  • SAB - load as a propertyQualifier (typeName=source)
  • ATV- load as a propertyValue
  • SUPPRESS - load as propertyQualifier if value != N

MRRANK.RRF

Problem:

SAB specific ranking of representational form in MRRANK is not exposed to the user (used in an underlying ranking and specifying of preferred presentations for a given concept)

...

Available in current LexEVS api

MRSAB.RRF

Problem:

MRSAB.RRF file data is not loaded or is otherwise unavailable to the user.

...

Entire content of each row of MRSAB file is loaded as metadata to an external xml file with tags created from column names and value inserted between tags as is appropriate

MRMAP.RRF, MRSMAP.RRF

Problem:

MRMAP.RRF source load is not supported in current load. Currently this RRF file is not populated in NCI Metathesaurus distributions. Mapping is not explicitly supported in the LexGrid Model.

...

To be evaluated for a load to current model elements or possible new model mapping elements. The general agreement is that this is more appropriately implemented in 6.0.

MRHIER.RRF

Problem:

HCD is loaded as a property on the presentation but the SAB isn't associated with it so we do not know the source of the HCD. (only look at row that has HCD field populated)
Path to Root, (PTR) is also not loaded, but is instead used to determine path to root operations in LexEVS.

...

Load HCD associated field SAB as property qualifier when HCD is present. Load PTR as property.

MRDOC.RRF

Problem:

MRDOC contains metadata unavailable to the user. It is not loaded by LexEVS.

...

MRDOC's column names and content will be processed as tag/value mappings to a metadata file.

MRDEF.RRF

Problem:

Some values from each row are not loaded by LexEVS.

...

AUI should be loaded to connect it with the presentation

ATUI, SUPPRESS, CVF, SATAUI should be loaded and exposed to the user.

ATUI, SUPPRESS, CVF, SATAUI, column values will be loaded as property qualifiers on the Definition type property derived from MRDEF column.

MRCONSO.RRF

Problem:

Some elements from the columns of MRCONSO.RRF are not loaded by LexEVS.

...

All noted values will be loaded as property qualifiers.

Unified Medical Language System

The Unified Medical Language System (UMLS) and Rich Release Format (RRF) files

...

Hierarchies in the UMLS are expressed in the MRREL.RRF file as source, target pairs. However source hierarchies may also be expressed in the MRHEIR.RRF file. These context based hierarchies are realized in LexBIG by accessing the MRHEIR source where the HCD column value is populate. When this is the case, as in MESH, the path of AUI's to root from the code in the HCD column is processed as a hierarchy. LexBIG's behavior is as follows:*

  • Entries in MRHIER that define multiple contexts (HCD field) per CUI will trigger additional tracking within the LexBIG environment.

...

  • Each link is tracked via the corresponding contextual chain(Path To Root field). To do this, we add association qualifiers that tag the association between each participating concept. The qualifier name is 'HCD' and the value will be the HCD field value from the MRHIER file.

...

  • An individual association between two concepts can participate in multiple context chains by assigning additional association qualifiers. A complete flow across the entire chain of links (essentially reconstructing PTR field) can be derived by recursive evaluation of surrounding links that have the same context qualifications. Since each concept can carry multiple text presentations, property qualifiers will be used to track the individual terms used in each context.

...

  • As with associations, multiple qualifiers can be assigned to each text property. Once again, the qualifier name will be 'HCD' and the value will be the HCD field value from the MRHIER file.

...

  • In order to query context-specific relationships, we can first use the API to filter the relationships a concept participates in, then query neighboring nodes to determine the complete context path, and finally map back to specific terms through the registered HCD qualifiers .

OBO Mapping

The OBO each remark in the document header will be combined and put into the coding scheme entityDescription.

...

Code Block
remark: autogenerated-by:     DAG-Edit version 1.320
remark: saved-by:             mariacos
remark: date:                 Fri Jun 27 09:41:28 EDT 2003
remark: version: $Revision: 1.1 $

Protege OWL

DatatypeProperty Representation

Owl:

Code Block
<owl:DatatypeProperty rdf:ID="currency">
          <rdfs:domain rdf:resource="#Money"/>
          <rdfs:range rdf:resource="http://www.w3.org/2001/XMLSchema#string"/>
     </owl:DatatypeProperty>

...

Code Block
<lgRel:association id="hasDomain" forwardName="hasDomain" isReflexive="false" isSymmetric="false" 
isTransitive="true" reverseName="kindIsDomainOf">  
      <lgRel:sourceConcept sourceEntityType="association" sourceId="currency">
        <lgRel:targetConcept targetEntityType="concept" targetId="Money"/>
      </lgRel:sourceConcept>

 <lgRel:association id="currency">
      <associationProperty propertyId="P0007" propertyName="isDatatypeProperty">
        <lgCommon:text>true</lgCommon:text>
      </associationProperty>
      <associationProperty propertyId="P0008" propertyName="isObjectProperty">
        <lgCommon:text>false</lgCommon:text>
      </associationProperty>
    </lgRel:association>

 <lgRel:association id="datatype" forwardName="datatype">
      <lgRel:sourceConcept sourceEntityType="association" sourceId="currency">
        <lgRel:targetDataValue dataId="D0001">
          <lgRel:dataValue>string</lgRel:dataValue>
        </lgRel:targetDataValue>

Equivalent Class Representation

Owl:

Code Block
<owl:Class rdf:ID="Father">
    <owl:equivalentClass>
      <owl:Class>
        <owl:intersectionOf rdf:parseType="Collection">
          <owl:Class rdf:about="#Person"/>
          <owl:Restriction>
            <owl:onProperty>
              <owl:FunctionalProperty rdf:about="#hasSex"/>
            </owl:onProperty>
            <owl:hasValue rdf:resource="#MaleSex"/>
          </owl:Restriction>
          <owl:Restriction>
            <owl:someValuesFrom rdf:resource="#Person"/>
            <owl:onProperty>
              <owl:ObjectProperty rdf:about="#hasChild"/>
            </owl:onProperty>
          </owl:Restriction>
        </owl:intersectionOf>
      </owl:Class>
    </owl:equivalentClass>
  </owl:Class>

...

Code Block
<lgRel:association id="equivalentClass" forwardName="equivalentClass" isReflexive="true" isSymmetric="true" isTransitive="true" reverseName="equivalentClass"> 
  <lgRel:sourceConcept sourceEntityType="concept" sourceId="Father">
        <lgRel:targetConcept targetEntityType="concept" 	targetId="A38"/>
      </lgRel:sourceConcept>

Restriction Representation

Owl:

Code Block
<owl:Class rdf:ID="Large-Format">
          <rdfs:subClassOf rdf:resource="#Camera"/>
          <rdfs:subClassOf>
               <owl:Restriction>
                     <owl:onProperty rdf:resource="#body"/>
                     <owl:allValuesFrom rdf:resource="#BodyWithNonAdjustableShutterSpeed"/>
               </owl:Restriction>
          </rdfs:subClassOf>
     </owl:Class>

...

Code Block
<lgRel:association id="equivalentClass" forwardName="equivalentClass" isReflexive="true" isSymmetric="true" isTransitive="true" reverseName="equivalentClass"> 
  <lgRel:sourceConcept sourceEntityType="concept" sourceId="Father">
        <lgRel:targetConcept targetEntityType="concept" 	targetId="A38"/>
      </lgRel:sourceConcept>


 <lgRel:association codingSchemeId="" id="hasSex" forwardName="hasSex" isFunctional="true" isReverseFunctional="false" isSymmetric="false" isTransitive="false"> 
    <lgRel:sourceConcept sourceEntityType="concept" sourceId="A38">
        <lgRel:targetConcept targetEntityType="concept" targetId="MaleSex">
          <lgRel:associationQualification associationQualifier="owl:hasValue"/>
        </lgRel:targetConcept>

<lgRel:association codingSchemeId="rdfs" id="subClassOf" forwardName="subClassOf" isFunctional="false" isReflexive="true" isSymmetric="false" isTransitive="true" reverseName="hasSubClass"> 
   <lgRel:sourceConcept sourceEntityType="concept" sourceId="A38">
        <lgRel:targetConcept targetEntityType="concept" targetId="Person"/>
      </lgRel:sourceConcept>

 <lgRel:association codingSchemeId="" id="hasChild" forwardName="hasChild" isFunctional="false" isReverseFunctional="false" isSymmetric="false" isTransitive="false"> 
   <lgRel:sourceConcept sourceEntityType="concept" sourceId="A38">
        <lgRel:targetConcept targetEntityType="concept" targetId="Person">
          <lgRel:associationQualification associationQualifier="owl:someValuesFrom"/>
        </lgRel:targetConcept>

<lgCon:concept id="A38" isAnonymous="true">
      <lgCommon:entityDescription>Person and (hasSex has MaleSex) and (hasChild some Person)</lgCommon:entityDescription> 
      <lgCon:presentation propertyId="P0002" propertyName="textualPresentation" isPreferred="true">
        <lgCommon:text>Person and (hasSex has MaleSex) and (hasChild some Person)</lgCommon:text>
      </lgCon:presentation>
      <lgCon:conceptProperty propertyId="P0001" propertyName="type">
        <lgCommon:text>owl:intersectionOf</lgCommon:text>
      </lgCon:conceptProperty>
    </lgCon:concept>

Property Restriction Representation

Anonymous LexGrid concepts are created for property restrictions (UnionOf, hasValue).

...

Code Block
<lgRel:association id="hasTopping" forwardName="hasTopping" isFunctional="false" isNavigable="true" isReverseFunctional="true" isSymmetric="false" isTransitive="false">

    <lgRel:sourceEntity sourceCodingScheme="pizza" sourceEntityType="concept" sourceId="AmericanHot">
        <lgRel:targetEntity targetCodingScheme="pizza" targetEntityType="concept" targetId="A16">
          <lgRel:associationQualification associationQualifier="owl:allValuesFrom"/>
        </lgRel:targetEntity>
      </lgRel:sourceEntity>
  </lgRel:association>


        <rdfs:subClassOf>
            <owl:Restriction>
                <owl:onProperty rdf:resource="#hasTopping"/>
                <owl:allValuesFrom>
                    <owl:Class>
                        <owl:unionOf rdf:parseType="Collection">
                            <owl:Class rdf:about="#MozzarellaTopping"/>
                            <owl:Class rdf:about="#PeperoniSausageTopping"/>
                            <owl:Class rdf:about="#JalapenoPepperTopping"/>
                            <owl:Class rdf:about="#TomatoTopping"/>
                            <owl:Class rdf:about="#HotGreenPepperTopping"/>
                        </owl:unionOf>
                    </owl:Class>
                </owl:allValuesFrom>
            </owl:Restriction>
        </rdfs:subClassOf>


<lgCon:concept id="A16" isActive="true" isAnonymous="true">
      <lgCommon:entityDescription>MozzarellaTopping or PeperoniSausageTopping or JalapenoPepperTopping or TomatoTopping or HotGreenPepperTopping</lgCommon:entityDescription>
      <lgCon:presentation propertyId="P0002" propertyName="textualPresentation" isPreferred="true">
        <lgCommon:text>MozzarellaTopping or PeperoniSausageTopping or JalapenoPepperTopping or TomatoTopping or HotGreenPepperTopping</lgCommon:text>
      </lgCon:presentation>
      <lgCon:conceptProperty propertyId="P0001" propertyName="type">
        <lgCommon:text>owl:unionOf</lgCommon:text>
      </lgCon:conceptProperty>
    </lgCon:concept>

NCI OWL

Top-level containers for relations are created, which separate the association types based on the notion of 'associations' and 'roles' as defined by NCI:

...

If no equivalent class for a concept, it is considered primitive and is indicated by creating a concept property set to 'true.'

Embedded XML

Property text with embedded XML fragments are identified by by the following identifiers:

  • qual-name
  • qual-value
  • qual

If the extracted tag is one of XML Text identifiers:

  • Value
  • term-name
  • def-definition
  • go-term

The text of the property is set to the tag value.

If the extracted tag is one of XML Source Name identifiers:

  • term-source
  • def-source

A property source is created and the tag value identifies the source.

If the property is a presentation and the extracted tag is XML Representational Form:

  • term-group

The representational form of the presentation property is set to the tag value.

If the extracted tag is one of DB XRef Prefix:

  • dbxref.*

A property qualifier is created. The property qualifier id is set to the tag, the value is set to the tag value.

HL7 RIM

To build a single coding scheme from the HL7 MS Access database, implementation is similar to how the NCI MetaThesaurus is stored in LexGrid.

For example, here is how entries MTHU021347 and MTHU033458 in ICPC2ICD10ENG (NCI MethThesaurus C1394796) are structured in LexGrid:

  • Coding Scheme: NCI MetaThesaurus - urn:oid:2.16.840.1.113883.3.26.1.2
  • Concept Code: C1394796
  • Entity Description: decompensation; heart, senile
  • Status: Active
  • Is Active: true
  • Is Anonymous: false
  • Presentation: decompensation; heart, senile
    • Property Name: textualPresentation
    • Property Id: T-1
    • Language: ENG
    • Is Preferred: true
    • Representational Form: PT
    • Source: ICPC2ICD10ENG , Role: null, SubRef: null
    • Property Qualifier Id: source-code , Property Qualifier Content: MTHU021347
  • Presentation: heart; decompensation, senile
    • Property Name: textualPresentation
    • Property Id: T-2
    • Language: ENG
    • Is Preferred: false
    • Representational Form: PT
    • Source: ICPC2ICD10ENG , Role:''' null, SubRef: null
    • Property Qualifier Id: source-code , Property Qualifier Content: MTHU033458
  • ConceptProperty: Mental or Behavioral Dysfunction
    • Property Name: Semantic_Type
    • Property Id: SemType-1

In HL7, code systems, concepts, and designations are in the following tables:

Table: VCS_concept_code_xref

Internal concept identifier

Code system

OID

Concept code

Case difference Status

10011

2.16.840.1.113883.5.55

M

0

A

10011

2.16.840.1.113883.5.55

R

0

A

10013

2.16.840.1.113883.5.55

RQ

0

A

10014

2.16.840.1.113883.5.55

NP

0

A

10015

2.16.840.1.113883.5.55

NR

0

A

10016

2.16.840.1.113883.5.55

RE

0

A

10017

2.16.840.1.113883.5.55

X

0

A

10019

2.16.840.1.113883.5.57

R

0

A

10020

2.16.840.1.113883.5.57

D

0

A

10021

2.16.840.1.113883.5.57

I

0

A

10022

2.16.840.1.113883.5.57

K

0

A

10023

2.16.840.1.113883.5.57

V

0

A

10025

2.16.840.1.113883.5.57

ESA

0

A

10026

2.16.840.1.113883.5.57

ESD

0

A

10027

2.16.840.1.113883.5.57

ESC

0

A

10028

2.16.840.1.113883.5.57

ESAC

0

A

Table: VCS_concept_designation

Internal Id

Designation

seq - for case differences

language

preferredForLanguage

10011

Mandatory

0

en

-1

10011

Required - V2.x

0

en

0

...

For example, the following structure represents both HL7 10011 entries in code system 2.16.840.1.113883.5.55:

  • Coding Scheme: HL7 - urn:oid:2.16.840.1.113883.3.26.1.2
  • Concept Code:

...

  • 10011:M
  • Entity Description:

...

  • The message element must appear every time the message is communicated and its value must not be null. This condition is subject to the rules of multiplicity and conditionality. If a non-null default value is defined for the element, a null value may be communicated.
  • Status: Active
  • Is Active: true
  • Is Anonymous: false
  • Presentation:

...

  • Mandatory
    • Property Name:

...

    • textualPresentation
    • Property Id: T-1
    • Language: ENG
    • Is Preferred: true
    • Representational Form: PT
    • Source:

...

    • HL7ConformanceInclusion, Role: null, SubRef: null
    • Property Qualifier Id:

...

    • source-code, Property Qualifier Content:

...

    • M
  • Presentation:

...

  • Required - V2.

...

  • x
    • Property Name: textualPresentation
    • Property Id: T-2
    • Language: ENG
    • Is Preferred: false
    • Representational Form: PT
    • Source:

...

    • HL7ConformanceInclusion, Role: null, SubRef: null
    • Property Qualifier Id:

...

    • source-code, Property Qualifier Content:

...

    • M
  • Coding Scheme: HL7 - urn:oid:2.16.840.1.113883.3.26.1.2
  • Concept Code:

...

  • 10011:R
  • Entity Description:

...

  • The message element must appear every time the message is communicated and its value must not be null. This condition is subject to the rules of multiplicity and conditionality. If a non-null default value is defined for the element, a null value may be communicated.
  • Status: Active
  • Is Active: true
  • Is Anonymous: false
  • Presentation:

...

  • Mandatory
    • Property Name: textualPresentation
    • Property Id: T-1
    • Language: ENG
    • Is Preferred: true
    • Representational Form: PT
    • Source:

...

    • HL7ConformanceInclusion, Role: null, SubRef: null
    • Property Qualifier Id:

...

    • source-code, Property Qualifier Content:

...

    • R
  • Presentation: Required - V2.x
    • Property Name: textualPresentation
    • Property Id: T-2
    • Language: ENG
    • Is Preferred: false
    • Representational Form: PT
    • Source:

...

    • HL7ConformanceInclusion, Role: null, SubRef: null
    • Property Qualifier

...

    • Id: source-code, Property Qualifier Content:

...

    • R

Loading the HL7 Rim as a monolithic coding scheme1.

  1. Load coding scheme data as HL7 Rim Metadata from the Model table (rather than the coding scheme data for each HL7 coding scheme).

      ...

        1. Mapping of these values will be incomplete:

      ...

        1. Mapping proposal:

          LexGrid

          HL7 RIM

          <codingSchemeName>

          <modelID>

          <formalName>

          <name>

          <registeredName>

          http://www.hl7.org/Library/data-model/RIM

      ...

        1. *

          <defaultLanguage>

          en*

          <representsVersion>

          <versionNumber>

          <isNative>

          0*

          <approximateNumberofConcepts>

          Result of count on concept bearing table?

          <firstRelease>

          MISSING

          <modifiedInRelease>

          MISSING

          <deprecated>

          MISSING

          <entityDescription>

          <description>

          <copyright>

          MISSING

      ...

        1. No URN exists and we may need to consider creating one (see entry for registeredName).

      ...

      1. Locate and load all mappings (such as supportedAssociations and supportedProperties).

          ...

            1. Create a supportedHiearchy with a root node of @ on hasSubtype?

          ...

          1. Iterate through the code system table rows and get each coding scheme.

              ...

                1. Create and persist an "@" node in the database

              ...

                1. Prepare an artificial "top node" for each coding scheme. (Metadata persisted here as concept properties?) This will result in 250 top nodes.

                ...

                    • The artificial top nodes will need to have a concept code created for them.

                ...

                    • Attach to "@" the artificial top nodes as a hasSubtype.

                ...

                    • Locate the actual top nodes of each coding scheme by querying the relations table to see if they exist as a target code, if not, they are top nodes so attach them to the artificial top node via hasSubtype.

                ...

                  1. Translate the RRF source property loads to the EMF world.

                  ...

                      • Load the concepts ensuring that the coding scheme name is loaded as a "source" property

                  ...

                      • Load the relations ensuring that the source and target coding scheme data is loaded with the coding scheme name.

                  ...

                  1. Concurrent to this process create an updated "HL7 RIM to LexGrid for NCI" mapping from the current Excel mapping document.

                  LexGrid Text

                  The text files that can be imported must use the following formats. Items surrounded by <> are required. Items further surrounded by [] are optional. \t represents a tab - the default delimiter - however other delimiters may be used.

                  ...

                  If the same code occurs twice, the names must match. Description rules same as "Format A."

                  Example of Format A

                  No Format
                  #

                  ...

                  Code Block
                  #lines starting with "#" are

                  ...

                  No Format
                  #

                  blank lines are ok

                  No Format
                  #

                  ...

                   comments
                  #blank lines are ok 
                  #the first "real" line of the file must be of the following format:

                  ...

                  No Format
                  #

                  ...

                  
                  #<codingSchemeName>\t<codingSchemeId>\t<defaultLanguage>\t<formalName>[-t_version_][-t_source_][-t_description_][-t_copyright_]

                  colors 1.2.3 en colors coding scheme 1.0 Someone's Head a simple example coding scheme using colors This isn't worth copyrighting

                  ...

                  ...

                  #

                  The rest of the file (for format A) should look like this:

                  Code Block
                  Color	Holder of colors
                  	Red
                  	Green	The color Green
                  		Light Green	foobar
                  		Dark Green	The color dark green
                  	Blue
                  		Red
                  			Green	The color Green

                  Example of Format B

                  No Format
                  #

                  ...

                  Code Block
                  
                  #lines starting with "#" are

                  ...

                  No Format
                  #

                  blank lines are ok

                  No Format
                  #

                  ...

                   comments
                  #blank lines are ok 
                  #the first "real" line of the file must be of the following format:

                  ...

                  No Format
                  #

                  ...

                  
                  #<codingSchemeName>\t<codingSchemeId>\t<defaultLanguage>\t<formalName>[-t_version_][-t_source_][-t_description_][-t_copyright_]

                  colors2 1.2.4 en colors coding scheme 1.1 Someone's Head a simple example coding scheme using colors This isn't worth copyrighting

                  ...

                  The rest of the file (for format B) should look like this:

                  Code Block
                  1	Color	Holder of colors
                  	4	Red
                  	6	Green	The color Green
                  		7	Light Green
                  		8	Dark Green
                  	5	Blue
                  		8	Dark Green	The color dark green
                  	6	Green	A different color of green

                  ...

                  Scrollbar
                  iconsfalse