Page History
Scrollbar | ||
---|---|---|
|
Page info | ||||
---|---|---|---|---|
|
Panel | ||||
---|---|---|---|---|
| ||||
|
...
Introduction
This document is a section of the Loader Guide. It was formerly the LexEVS v5.0 Source Mapping Guide.
NCI MetaThesaurus: RRF content (v5.1)
In LexEVS v5.1, loader enhancements for RRF content were made. Loads of the NCI MetaThesaurus RRF formatted data into the LexGrid model now accurately reflect the state of the data as it exists in the current RRF files. In the text that follows, the Problem sections describe LexEVS v5.0 behavior, and the Solution sections describe LexEVS v5.1 behavior.
Data Model Elements
Most data elements will be loaded as either properties or property qualifiers.
[ https:--//wiki.nci.nih.gov-download-attachments-18947057-Property.jpg]
A few will be loaded as qualifiers to associations.
MRREL.RRF File
Problem:
REL and RELA column elements from the RRF source need to be connected.
Currently these are loaded as separate relationships preventing the user from connecting to the REL/RELA combinations that actually occur in the NCI-META (e.g. RELA may be different for same REL value in different sources).
...
User is unable to distinguish individual relationships from one source or another. The same association "entity" exists only once but has two "source" qualifiers.
User is unable to distinguish the AUI1/STYPE1 and AUI2/STYPE2 which gives us the information about what source data structures are actually being connected by MRREL entries. Users also need the ability to associate AUI/STYPE fields with SAB.
Users sole choice for rendering a relationship in terms of the strings on either side is to use preferred concept names.
...
Propose AUI to AUI - the way CUI to CUI are currently handled in the implementation.
Propose entity to entity relationship - will still have to account for CUI to CUI relationships.
Load each unique RUI (would be quite large). They would need to be listed as supported association (this is not traditional how it is used).
...
Do not treat a CUI1 = CUI2 relationships differently than a CUI1 != CUI2 relationship. For API and query purposes, qualify these relationships with a 'selfReferencing=true' Qualifier. In this way, we can still avoid cycles in the API, but maintain all relevant Qualifier information in the relation.
MRSAT.RRF
Problem:
MRSAT.RRF is not loaded but only accessed for given preferred term algorithms. This data should be loaded as concept properties (STYPE=CUI), properties on properties (STYPE=AUI, SAUI, CODE, SCUI, SDUI), qualifiers on associations (STYPE=RUI,SRUI). Some complexity may arise as concept properties can have additional qualifiers, but property-properties cannot and association-qualifiers cannot.
...
If the STYPE is something other than RUI or SRUI, you can load
that row as an entity property. The fields you'd want to capture
are:
- CUI - We use this as the entityCode and is loaded as such in the table.
- METAUI - load as a propertyQualifier (name=METAUI, value)
- STYPE - load as a propertyQualifier (name=STYPE, value)
- ATUI - load as propertyId
- ATN - load as property name
- SAB - load as a propertyQualifier (typeName=source)
- ATV- load as a propertyValue
- SUPPRESS - load as propertyQualifier if value != N
MRRANK.RRF
Problem:
SAB specific ranking of representational form in MRRANK is not exposed to the user (used in an underlying ranking and specifying of preferred presentations for a given concept)
...
Available in current LexEVS api
MRSAB.RRF
Problem:
MRSAB.RRF file data is not loaded or is otherwise unavailable to the user.
...
Entire content of each row of MRSAB file is loaded as metadata to an external xml file with tags created from column names and value inserted between tags as is appropriate
MRMAP.RRF, MRSMAP.RRF
Problem:
MRMAP.RRF source load is not supported in current load. Currently this RRF file is not populated in NCI Metathesaurus distributions. Mapping is not explicitly supported in the LexGrid Model.
...
To be evaluated for a load to current model elements or possible new model mapping elements. The general agreement is that this is more appropriately implemented in 6.0.
MRHIER.RRF
Problem:
HCD is loaded as a property on the presentation but the SAB isn't associated with it so we do not know the source of the HCD. (only look at row that has HCD field populated)
Path to Root, (PTR) is also not loaded, but is instead used to determine path to root operations in LexEVS.
...
Load HCD associated field SAB as property qualifier when HCD is present. Load PTR as property.
MRDOC.RRF
Problem:
MRDOC contains metadata unavailable to the user. It is not loaded by LexEVS.
...
MRDOC's column names and content will be processed as tag/value mappings to a metadata file.
MRDEF.RRF
Problem:
Some values from each row are not loaded by LexEVS.
...
AUI should be loaded to connect it with the presentation
ATUI, SUPPRESS, CVF, SATAUI should be loaded and exposed to the user.
ATUI, SUPPRESS, CVF, SATAUI, column values will be loaded as property qualifiers on the Definition type property derived from MRDEF column.
MRCONSO.RRF
Problem:
Some elements from the columns of MRCONSO.RRF are not loaded by LexEVS.
...
All noted values will be loaded as property qualifiers.
Unified Medical Language System
The Unified Medical Language System (UMLS) and Rich Release Format (RRF) files
...
Hierarchies in the UMLS are expressed in the MRREL.RRF file as source, target pairs. However source hierarchies may also be expressed in the MRHEIR.RRF file. These context based hierarchies are realized in LexBIG by accessing the MRHEIR source where the HCD column value is populate. When this is the case, as in MESH, the path of AUI's to root from the code in the HCD column is processed as a hierarchy. LexBIG's behavior is as follows:*
- Entries in MRHIER that define multiple contexts (HCD field) per CUI will trigger additional tracking within the LexBIG environment.
...
- Each link is tracked via the corresponding contextual chain(Path To Root field). To do this, we add association qualifiers that tag the association between each participating concept. The qualifier name is 'HCD' and the value will be the HCD field value from the MRHIER file.
...
- An individual association between two concepts can participate in multiple context chains by assigning additional association qualifiers. A complete flow across the entire chain of links (essentially reconstructing PTR field) can be derived by recursive evaluation of surrounding links that have the same context qualifications. Since each concept can carry multiple text presentations, property qualifiers will be used to track the individual terms used in each context.
...
- As with associations, multiple qualifiers can be assigned to each text property. Once again, the qualifier name will be 'HCD' and the value will be the HCD field value from the MRHIER file.
...
- In order to query context-specific relationships, we can first use the API to filter the relationships a concept participates in, then query neighboring nodes to determine the complete context path, and finally map back to specific terms through the registered HCD qualifiers .
OBO Mapping
The OBO each remark in the document header will be combined and put into the coding scheme entityDescription.
...
Code Block |
---|
remark: autogenerated-by: DAG-Edit version 1.320 remark: saved-by: mariacos remark: date: Fri Jun 27 09:41:28 EDT 2003 remark: version: $Revision: 1.1 $ |
Protege OWL
DatatypeProperty Representation
Owl:
Code Block |
---|
<owl:DatatypeProperty rdf:ID="currency"> <rdfs:domain rdf:resource="#Money"/> <rdfs:range rdf:resource="http://www.w3.org/2001/XMLSchema#string"/> </owl:DatatypeProperty> |
...
Code Block |
---|
<lgRel:association id="hasDomain" forwardName="hasDomain" isReflexive="false" isSymmetric="false" isTransitive="true" reverseName="kindIsDomainOf"> <lgRel:sourceConcept sourceEntityType="association" sourceId="currency"> <lgRel:targetConcept targetEntityType="concept" targetId="Money"/> </lgRel:sourceConcept> <lgRel:association id="currency"> <associationProperty propertyId="P0007" propertyName="isDatatypeProperty"> <lgCommon:text>true</lgCommon:text> </associationProperty> <associationProperty propertyId="P0008" propertyName="isObjectProperty"> <lgCommon:text>false</lgCommon:text> </associationProperty> </lgRel:association> <lgRel:association id="datatype" forwardName="datatype"> <lgRel:sourceConcept sourceEntityType="association" sourceId="currency"> <lgRel:targetDataValue dataId="D0001"> <lgRel:dataValue>string</lgRel:dataValue> </lgRel:targetDataValue> |
Equivalent Class Representation
Owl:
Code Block |
---|
<owl:Class rdf:ID="Father"> <owl:equivalentClass> <owl:Class> <owl:intersectionOf rdf:parseType="Collection"> <owl:Class rdf:about="#Person"/> <owl:Restriction> <owl:onProperty> <owl:FunctionalProperty rdf:about="#hasSex"/> </owl:onProperty> <owl:hasValue rdf:resource="#MaleSex"/> </owl:Restriction> <owl:Restriction> <owl:someValuesFrom rdf:resource="#Person"/> <owl:onProperty> <owl:ObjectProperty rdf:about="#hasChild"/> </owl:onProperty> </owl:Restriction> </owl:intersectionOf> </owl:Class> </owl:equivalentClass> </owl:Class> |
...
Code Block |
---|
<lgRel:association id="equivalentClass" forwardName="equivalentClass" isReflexive="true" isSymmetric="true" isTransitive="true" reverseName="equivalentClass"> <lgRel:sourceConcept sourceEntityType="concept" sourceId="Father"> <lgRel:targetConcept targetEntityType="concept" targetId="A38"/> </lgRel:sourceConcept> |
Restriction Representation
Owl:
Code Block |
---|
<owl:Class rdf:ID="Large-Format"> <rdfs:subClassOf rdf:resource="#Camera"/> <rdfs:subClassOf> <owl:Restriction> <owl:onProperty rdf:resource="#body"/> <owl:allValuesFrom rdf:resource="#BodyWithNonAdjustableShutterSpeed"/> </owl:Restriction> </rdfs:subClassOf> </owl:Class> |
...
Code Block |
---|
<lgRel:association id="equivalentClass" forwardName="equivalentClass" isReflexive="true" isSymmetric="true" isTransitive="true" reverseName="equivalentClass"> <lgRel:sourceConcept sourceEntityType="concept" sourceId="Father"> <lgRel:targetConcept targetEntityType="concept" targetId="A38"/> </lgRel:sourceConcept> <lgRel:association codingSchemeId="" id="hasSex" forwardName="hasSex" isFunctional="true" isReverseFunctional="false" isSymmetric="false" isTransitive="false"> <lgRel:sourceConcept sourceEntityType="concept" sourceId="A38"> <lgRel:targetConcept targetEntityType="concept" targetId="MaleSex"> <lgRel:associationQualification associationQualifier="owl:hasValue"/> </lgRel:targetConcept> <lgRel:association codingSchemeId="rdfs" id="subClassOf" forwardName="subClassOf" isFunctional="false" isReflexive="true" isSymmetric="false" isTransitive="true" reverseName="hasSubClass"> <lgRel:sourceConcept sourceEntityType="concept" sourceId="A38"> <lgRel:targetConcept targetEntityType="concept" targetId="Person"/> </lgRel:sourceConcept> <lgRel:association codingSchemeId="" id="hasChild" forwardName="hasChild" isFunctional="false" isReverseFunctional="false" isSymmetric="false" isTransitive="false"> <lgRel:sourceConcept sourceEntityType="concept" sourceId="A38"> <lgRel:targetConcept targetEntityType="concept" targetId="Person"> <lgRel:associationQualification associationQualifier="owl:someValuesFrom"/> </lgRel:targetConcept> <lgCon:concept id="A38" isAnonymous="true"> <lgCommon:entityDescription>Person and (hasSex has MaleSex) and (hasChild some Person)</lgCommon:entityDescription> <lgCon:presentation propertyId="P0002" propertyName="textualPresentation" isPreferred="true"> <lgCommon:text>Person and (hasSex has MaleSex) and (hasChild some Person)</lgCommon:text> </lgCon:presentation> <lgCon:conceptProperty propertyId="P0001" propertyName="type"> <lgCommon:text>owl:intersectionOf</lgCommon:text> </lgCon:conceptProperty> </lgCon:concept> |
Property Restriction Representation
Anonymous LexGrid concepts are created for property restrictions (UnionOf, hasValue).
...
Code Block |
---|
<lgRel:association id="hasTopping" forwardName="hasTopping" isFunctional="false" isNavigable="true" isReverseFunctional="true" isSymmetric="false" isTransitive="false"> <lgRel:sourceEntity sourceCodingScheme="pizza" sourceEntityType="concept" sourceId="AmericanHot"> <lgRel:targetEntity targetCodingScheme="pizza" targetEntityType="concept" targetId="A16"> <lgRel:associationQualification associationQualifier="owl:allValuesFrom"/> </lgRel:targetEntity> </lgRel:sourceEntity> </lgRel:association> <rdfs:subClassOf> <owl:Restriction> <owl:onProperty rdf:resource="#hasTopping"/> <owl:allValuesFrom> <owl:Class> <owl:unionOf rdf:parseType="Collection"> <owl:Class rdf:about="#MozzarellaTopping"/> <owl:Class rdf:about="#PeperoniSausageTopping"/> <owl:Class rdf:about="#JalapenoPepperTopping"/> <owl:Class rdf:about="#TomatoTopping"/> <owl:Class rdf:about="#HotGreenPepperTopping"/> </owl:unionOf> </owl:Class> </owl:allValuesFrom> </owl:Restriction> </rdfs:subClassOf> <lgCon:concept id="A16" isActive="true" isAnonymous="true"> <lgCommon:entityDescription>MozzarellaTopping or PeperoniSausageTopping or JalapenoPepperTopping or TomatoTopping or HotGreenPepperTopping</lgCommon:entityDescription> <lgCon:presentation propertyId="P0002" propertyName="textualPresentation" isPreferred="true"> <lgCommon:text>MozzarellaTopping or PeperoniSausageTopping or JalapenoPepperTopping or TomatoTopping or HotGreenPepperTopping</lgCommon:text> </lgCon:presentation> <lgCon:conceptProperty propertyId="P0001" propertyName="type"> <lgCommon:text>owl:unionOf</lgCommon:text> </lgCon:conceptProperty> </lgCon:concept> |
NCI OWL
Top-level containers for relations are created, which separate the association types based on the notion of 'associations' and 'roles' as defined by NCI:
...
If no equivalent class for a concept, it is considered primitive and is indicated by creating a concept property set to 'true.'
Embedded XML
Property text with embedded XML fragments are identified by by the following identifiers:
- qual-name
- qual-value
- qual
If the extracted tag is one of XML Text identifiers:
- Value
- term-name
- def-definition
- go-term
The text of the property is set to the tag value.
If the extracted tag is one of XML Source Name identifiers:
- term-source
- def-source
A property source is created and the tag value identifies the source.
If the property is a presentation and the extracted tag is XML Representational Form:
- term-group
The representational form of the presentation property is set to the tag value.
If the extracted tag is one of DB XRef Prefix:
- dbxref.*
A property qualifier is created. The property qualifier id is set to the tag, the value is set to the tag value.
HL7 RIM
To build a single coding scheme from the HL7 MS Access database, implementation is similar to how the NCI MetaThesaurus is stored in LexGrid.
For example, here is how entries MTHU021347 and MTHU033458 in ICPC2ICD10ENG (NCI MethThesaurus C1394796) are structured in LexGrid:
- Coding Scheme: NCI MetaThesaurus - urn:oid:2.16.840.1.113883.3.26.1.2
- Concept Code: C1394796
- Entity Description: decompensation; heart, senile
- Status: Active
- Is Active: true
- Is Anonymous: false
- Presentation: decompensation; heart, senile
- Property Name: textualPresentation
- Property Id: T-1
- Language: ENG
- Is Preferred: true
- Representational Form: PT
- Source: ICPC2ICD10ENG , Role: null, SubRef: null
- Property Qualifier Id: source-code , Property Qualifier Content: MTHU021347
- Presentation: heart; decompensation, senile
- Property Name: textualPresentation
- Property Id: T-2
- Language: ENG
- Is Preferred: false
- Representational Form: PT
- Source: ICPC2ICD10ENG , Role:''' null, SubRef: null
- Property Qualifier Id: source-code , Property Qualifier Content: MTHU033458
- ConceptProperty: Mental or Behavioral Dysfunction
- Property Name: Semantic_Type
- Property Id: SemType-1
In HL7, code systems, concepts, and designations are in the following tables:
Table: VCS_concept_code_xref
Internal concept identifier | Code system | OID | Concept code | Case difference Status |
---|---|---|---|---|
10011 | 2.16.840.1.113883.5.55 | M | 0 | A |
10011 | 2.16.840.1.113883.5.55 | R | 0 | A |
10013 | 2.16.840.1.113883.5.55 | RQ | 0 | A |
10014 | 2.16.840.1.113883.5.55 | NP | 0 | A |
10015 | 2.16.840.1.113883.5.55 | NR | 0 | A |
10016 | 2.16.840.1.113883.5.55 | RE | 0 | A |
10017 | 2.16.840.1.113883.5.55 | X | 0 | A |
10019 | 2.16.840.1.113883.5.57 | R | 0 | A |
10020 | 2.16.840.1.113883.5.57 | D | 0 | A |
10021 | 2.16.840.1.113883.5.57 | I | 0 | A |
10022 | 2.16.840.1.113883.5.57 | K | 0 | A |
10023 | 2.16.840.1.113883.5.57 | V | 0 | A |
10025 | 2.16.840.1.113883.5.57 | ESA | 0 | A |
10026 | 2.16.840.1.113883.5.57 | ESD | 0 | A |
10027 | 2.16.840.1.113883.5.57 | ESC | 0 | A |
10028 | 2.16.840.1.113883.5.57 | ESAC | 0 | A |
Table: VCS_concept_designation
Internal Id | Designation | seq - for case differences | language | preferredForLanguage |
---|---|---|---|---|
10011 | Mandatory | 0 | en | -1 |
10011 | Required - V2.x | 0 | en | 0 |
...
For example, the following structure represents both HL7 10011 entries in code system 2.16.840.1.113883.5.55:
- Coding Scheme: HL7 - urn:oid:2.16.840.1.113883.3.26.1.2
- Concept Code:
...
- 10011:M
- Entity Description:
...
- The message element must appear every time the message is communicated and its value must not be null. This condition is subject to the rules of multiplicity and conditionality. If a non-null default value is defined for the element, a null value may be communicated.
- Status: Active
- Is Active: true
- Is Anonymous: false
- Presentation:
...
- Mandatory
- Property Name:
...
- textualPresentation
- Property Id: T-1
- Language: ENG
- Is Preferred: true
- Representational Form: PT
- Source:
...
- HL7ConformanceInclusion, Role: null, SubRef: null
- Property Qualifier Id:
...
- source-code, Property Qualifier Content:
...
- M
- Presentation:
...
- Required - V2.
...
- x
- Property Name: textualPresentation
- Property Id: T-2
- Language: ENG
- Is Preferred: false
- Representational Form: PT
- Source:
...
- HL7ConformanceInclusion, Role: null, SubRef: null
- Property Qualifier Id:
...
- source-code, Property Qualifier Content:
...
- M
- Coding Scheme: HL7 - urn:oid:2.16.840.1.113883.3.26.1.2
- Concept Code:
...
- 10011:R
- Entity Description:
...
- The message element must appear every time the message is communicated and its value must not be null. This condition is subject to the rules of multiplicity and conditionality. If a non-null default value is defined for the element, a null value may be communicated.
- Status: Active
- Is Active: true
- Is Anonymous: false
- Presentation:
...
- Mandatory
- Property Name: textualPresentation
- Property Id: T-1
- Language: ENG
- Is Preferred: true
- Representational Form: PT
- Source:
...
- HL7ConformanceInclusion, Role: null, SubRef: null
- Property Qualifier Id:
...
- source-code, Property Qualifier Content:
...
- R
- Presentation: Required - V2.x
- Property Name: textualPresentation
- Property Id: T-2
- Language: ENG
- Is Preferred: false
- Representational Form: PT
- Source:
...
- HL7ConformanceInclusion, Role: null, SubRef: null
- Property Qualifier
...
- Id: source-code, Property Qualifier Content:
...
- R
Loading the HL7 Rim as a monolithic coding scheme1.
- Load coding scheme data as HL7 Rim Metadata from the Model table (rather than the coding scheme data for each HL7 coding scheme).
...
- Mapping of these values will be incomplete:
...
- Mapping proposal:
LexGrid
HL7 RIM
<codingSchemeName>
<modelID>
<formalName>
<name>
<registeredName>
- Mapping proposal:
...
*
<defaultLanguage>
en*
<representsVersion>
<versionNumber>
<isNative>
0*
<approximateNumberofConcepts>
Result of count on concept bearing table?
<firstRelease>
MISSING
<modifiedInRelease>
MISSING
<deprecated>
MISSING
<entityDescription>
<description>
<copyright>
MISSING
...
- No URN exists and we may need to consider creating one (see entry for registeredName).
...
- Locate and load all mappings (such as supportedAssociations and supportedProperties).
...
- Create a supportedHiearchy with a root node of @ on hasSubtype?
...
- Iterate through the code system table rows and get each coding scheme.
...
- Create and persist an "@" node in the database
...
- Prepare an artificial "top node" for each coding scheme. (Metadata persisted here as concept properties?) This will result in 250 top nodes.
- Prepare an artificial "top node" for each coding scheme. (Metadata persisted here as concept properties?) This will result in 250 top nodes.
...
- The artificial top nodes will need to have a concept code created for them.
...
- Attach to "@" the artificial top nodes as a hasSubtype.
...
- Locate the actual top nodes of each coding scheme by querying the relations table to see if they exist as a target code, if not, they are top nodes so attach them to the artificial top node via hasSubtype.
...
- Translate the RRF source property loads to the EMF world.
- Translate the RRF source property loads to the EMF world.
...
- Load the concepts ensuring that the coding scheme name is loaded as a "source" property
...
- Load the relations ensuring that the source and target coding scheme data is loaded with the coding scheme name.
...
- Concurrent to this process create an updated "HL7 RIM to LexGrid for NCI" mapping from the current Excel mapping document.
LexGrid Text
The text files that can be imported must use the following formats. Items surrounded by <> are required. Items further surrounded by [] are optional. \t represents a tab - the default delimiter - however other delimiters may be used.
...
If the same code occurs twice, the names must match. Description rules same as "Format A."
Example of Format A
No Format |
---|
# |
...
Code Block |
---|
#lines starting with "#" are |
...
No Format |
---|
# |
blank lines are ok
No Format |
---|
# |
...
comments #blank lines are ok #the first "real" line of the file must be of the following format: |
...
No Format |
---|
# |
...
#<codingSchemeName>\t<codingSchemeId>\t<defaultLanguage>\t<formalName>[-t_version_][-t_source_][-t_description_][-t_copyright_] |
colors 1.2.3 en colors coding scheme 1.0 Someone's Head a simple example coding scheme using colors This isn't worth copyrighting
...
...
#
The rest of the file (for format A) should look like this:
Code Block |
---|
Color Holder of colors Red Green The color Green Light Green foobar Dark Green The color dark green Blue Red Green The color Green |
Example of Format B
No Format |
---|
# |
...
Code Block |
---|
#lines starting with "#" are |
...
No Format |
---|
# |
blank lines are ok
No Format |
---|
# |
...
comments #blank lines are ok #the first "real" line of the file must be of the following format: |
...
No Format |
---|
# |
...
#<codingSchemeName>\t<codingSchemeId>\t<defaultLanguage>\t<formalName>[-t_version_][-t_source_][-t_description_][-t_copyright_] |
colors2 1.2.4 en colors coding scheme 1.1 Someone's Head a simple example coding scheme using colors This isn't worth copyrighting
...
The rest of the file (for format B) should look like this:
Code Block |
---|
1 Color Holder of colors 4 Red 6 Green The color Green 7 Light Green 8 Dark Green 5 Blue 8 Dark Green The color dark green 6 Green A different color of green |
...
Scrollbar | ||
---|---|---|
|