Page History

Versions Compared

Old Version 141

changes.mady.by.user Unknown User (wileyal)

Saved on Jan 21, 2012

compared with

New Version 142

changes.mady.by.user Unknown User (wileyal)

Saved on Jan 21, 2012

Key

This line was added.
This line was removed.
Formatting was changed.

...

Information	*listValueDomains(String valueDomainName)*
Description:	Return the URI's for the value domain definition(s) for the supplied domain name. If the name is null, returns everything. If the name is not null, returns the value domain(s) that have the assigned name. Note: plural because there is no guarantee of valueDomain uniqueness. If the name is the empty string "", returns all unnamed valueDomains.
Input:	java.lang.String
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="e3008fb14dc5eb09-a127b33a-4e87437b-9c7e85c0-1fa17c73e904be48e028e9e3"><ac:plain-text-body><![CDATA[	Output:	java.net.URI[]	]]></ac:plain-text-body></ac:structured-macro>
Exception:	org.LexGrid.LexBIG.Exceptions.LBException
Implementation Details:	*Implementation:* Step 1: Call this method on the associated LexEVS Value Domain Service instance to get the list of Value Domain URI that matches the supplied name. *Sample Call:* Step 1 : Using LexBIGService instance, get the LexEVSValueDomainServices interface org.lexgrid.valuedomain.LexEVSValueDomainServices vds = lbs.getValueDomainService(); Step 2 :Call listValueDomains method: <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="bde3ce8b47792775-572f98da-486c4111-ab849cfd-6db5d0969e60c0aeb09dd610"><ac:plain-text-body><![CDATA[URI[] uris = vds.listValueDomains("someValueDomainName");	]]></ac:plain-text-body></ac:structured-macro>

...

Information	*getAllValueDomainsWithNoNames()*
Description:	Return the URI's of all unnamed value domain definition(s).
Input:	none
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="df581097bb1bec25-028ae10c-42eb4f6f-8fabae9b-304e582f55fdf39b5f27eeed"><ac:plain-text-body><![CDATA[	Output:	java.net.URI[]	]]></ac:plain-text-body></ac:structured-macro>
Exception:	org.LexGrid.LexBIG.Exceptions.LBException
Implementation Details:	*Implementation:* Step 1: Call this method on the associated LexEVS Value Domain Service instance to get the list of Value Domain URI that have no names. *Sample Call:* Step 1 : Using LexBIGService instance, get the LexEVSValueDomainServices interface org.lexgrid.valuedomain.LexEVSValueDomainServices vds = lbs.getValueDomainService(); Step 2 :Call getAllValueDomainsWithNoNames method: <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="f20730d15b3e533a-a80781f9-4cd542c7-bae5b195-a788bdda8ff5604942efd379"><ac:plain-text-body><![CDATA[URI[] uris = vds.getAllValueDomainsWithNoNames();	]]></ac:plain-text-body></ac:structured-macro>

...

Information	getPickListDefinitionsForDomain(URI valueDomainURI)
Description:	Returns all the pickList definitions that represents supplied valueDomain URI.
Input:	java.net.URI
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="7e2429ce6043f088-82269508-46634012-ab889ee5-29925b61c83cc62c205e46e4"><ac:plain-text-body><![CDATA[	Output:	org.LexGrid.emf.valueDomains.PickListDefinition[]	]]></ac:plain-text-body></ac:structured-macro>
Exception:	org.LexGrid.LexBIG.Exceptions.LBException
Implementation Details:	*Implementation:* Step 1: Call this method on the associated LexEVS Pick List Service instance to get all the Pick List Definitions that are represented by supplied Value Domain URI. *Sample Call:* Step 1 : Using LexBIGService instance, get the LexEVSPickListServices interface org.lexgrid.valuedomain.LexEVSPickListServices pls = lbs.getPickListService(); Step 2 :Call getPickListDefinitionsForDomain method: <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="589d48c13ec9ebd5-bad48cc8-4fb04c4b-b50382f9-dbb01d3ebd6f3ba2dbb391e1"><ac:plain-text-body><![CDATA[PickListDefinition[] plDefs = pls.getPickListDefinitionsForDomain(valueDomainURI);	]]></ac:plain-text-body></ac:structured-macro>

...

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="10d2162a912ba7f1-10b78fbb-42e84eec-ac599053-d94cbee840604ca4f3327f4f"><ac:plain-text-body><![CDATA[	Information	resolvePickListForTerm(String pickListId, String term, String matchAlgorithm, String language, String[] context, boolean sortByText)	]]></ac:plain-text-body></ac:structured-macro>
Description:	Resolves pickList definition by applying supplied arguments.
Input:	java.lang.String, java.lang.String, java.lang.String, java.lang.String, <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="94f3b8d05cfda7be-8976ca87-48e54f63-9668b4e8-5ee40dd82a70b4a17766a596"><ac:plain-text-body><![CDATA[java.lang.String[], ]]></ac:plain-text-body></ac:structured-macro> boolean
Output:	org.lexgrid.valuedomain.dto.ResolvedPickListEntryList
Exception:	org.LexGrid.LexBIG.Exceptions.LBException
Implementation Details:	*Implementation:* Step 1: Call this method on the associated LexEVS Pick List Service instance to get list of Pick List Entries that matches the term supplied and meets other supplied restrictions. *Sample Call:* Step 1 : Using LexBIGService instance, get the LexEVSPickListServices interface org.lexgrid.valuedomain.LexEVSPickListServices pls = lbs.getPickListService(); Step 2 :Call resolvePickListForTerm method: ResolvedPickListEntryList pleList = pls.resolvePickListForTerm ("AUTO:DomesticAutoMakers","Jaguar", MatchAlgorithms.exactMatch.name(), "en", null, true);

...

Information	removePickList(String pickListId)
Description:	Removes supplied Pick List Definition from the system.
Input:	java.lang.String
Output:	none
Exception:	org.LexGrid.LexBIG.Exceptions.LBException, org.LexGrid.managedobj.RemoveException
Implementation Details:	*Implementation:* Step 1: Call this method on the associated LexEVS Pick List Service instance to remove Pick List Definition from the system that matches the supplied pickListId. *Sample Call:* Step 1 : Using LexBIGService instance, get the LexEVSPickListServices interface org.lexgrid.valuedomain.LexEVSPickListServices pls = lbs.getPickListService(); Step 2 :Call removePickList method: vds.removePickList ("AUTO:AllDomesticANDGM");

Resolved Value Domain Objects

ResolvedValueDomainCodedNodeSet

Contains Coding Scheme Version reference list that was used to resolve the Value Domain and the CodedNodeSet.
The CodedNodeSet is not resolved.

ResolvedValueDomainDefinition

A resolved Value Domain Definition containing the Coding Scheme Version reference list that was used to resolve the Value Domain and an iterator for resolved concepts.

Resolved Pick List Objects

ResolvedPickListEntry

Contains resolved Pick List Entry Nodes

ResolvedPickListEntryList

Contains the list of resolved Pick List Entries. Also provides helpful features to add, remove, enumerate Pick List Entries.

Error Handling

Both LexEVS Value Domain and Pick List services uses org.LexGrid.LexBIG.Impl.loaders.MessageDirector to direct all fatal, error, warning, info messages with appropriate messages to the LexBIG log files in the 'log' folder of LexEVS install directory.

Along with MessageDirector, the services will also make use of org.LexGrid.LexBIG.exception.LBException to throw any fatal and error messages to the log file as well as to console.

Load Scripts

Scripts to load Value Domain and Pick List Definitions into LexEVS system will be located under 'Admin' folder of LexEVS install directory. This loader scripts will only load data in XML file that is in LexGrid format.

Value Domain Loader

LoadValueDomain.bat for Windows environment and LoadValueDomain.sh for Unix environment.
Both these scripts takes take in following parameters :

Parameter	Function
-in	Input <uri> URI or path specifying location of the source file.
-v	Validate <int> Perform validation of the candidate resource without loading data. Supported levels of validation includes : 0 = Verify document is well-formed 1 = Verify document is valid

...

No Format
sh LoadValueDomain.sh \-in "file:///path/to/file.xml"

Pick List Loader

LoadPickList.bat for Windows environment and LoadPickList.sh for Unix environment.
Both these scripts takes take in following parameters:

Parameter	Function
-in	Input <uri> URI or path specifying location of the source file.
-v	Validate <int> Perform validation of the candidate resource without loading data. Supported levels of validation includes : 0 = Verify document is well-formed 1 = Verify document is valid

...

No Format
sh LoadPickList.sh \-in "file:///path/to/file.xml"

Sample XML files

Value Domain Definitions

Below is a sample XML file containing Value Domain Definitions in LexGrid format that can be loaded using LexEVS Value Domain Service.

Code Block

	xml
	xml

<?xml version="1.0" encoding="UTF-8"?>
<systemRelease xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://LexGrid.org/schema/2009/01/LexGrid/versions http://LexGrid.org/schema/2009/01/LexGrid/versions.xsd"
    xmlns="http://LexGrid.org/schema/2009/01/LexGrid/versions" xmlns:lgVer="http://LexGrid.org/schema/2009/01/LexGrid/versions"
    xmlns:lgCommon="http://LexGrid.org/schema/2009/01/LexGrid/commonTypes" xmlns:data="data"
    xmlns:lgVD="http://LexGrid.org/schema/2009/01/LexGrid/valueDomains" xmlns:lgNaming="http://LexGrid.org/schema/2009/01/LexGrid/naming"
    releaseURI="http://testRelease/04" releaseDate="2008-11-07T14:55:51.615-06:00">
    <lgCommon:entityDescription>Sample value domains</lgCommon:entityDescription>
    <lgVer:valueDomains>
        <lgVD:mappings>
            <lgNaming:supportedAssociation localId="hasSubtype" uri="urn:oid:1.3.6.1.4.1.2114.108.1.8.1">hasSubtype</lgNaming:supportedAssociation>
            <lgNaming:supportedCodingScheme localId="Automobiles" uri="urn:oid:11.11.0.1">Automobiles</lgNaming:supportedCodingScheme>
            <lgNaming:supportedDataType localId="testhtml">test/html</lgNaming:supportedDataType>
            <lgNaming:supportedDataType localId="textplain">text/plain</lgNaming:supportedDataType>
            <lgNaming:supportedHierarchy localId="is_a" associationNames="hasSubtype" isForwardNavigable="true" rootCode="@">hasSubtype</lgNaming:supportedHierarchy>
            <lgNaming:supportedLanguage localId="en" uri="www.en.org/orsomething">en</lgNaming:supportedLanguage>
            <lgNaming:supportedNamespace localId="Automobiles" uri="urn:oid:11.11.0.1" equivalentCodingScheme="Automobiles">Automobiles</lgNaming:supportedNamespace>
            <lgNaming:supportedProperty localId="textualPresentation">textualPresentation</lgNaming:supportedProperty>
            <lgNaming:supportedSource localId="lexgrid.org">lexgrid.org</lgNaming:supportedSource>
            <lgNaming:supportedSource localId="_111101">11.11.0.1</lgNaming:supportedSource>
        </lgVD:mappings>
       <lgVD:valueDomainDefinition valueDomainURI="SRITEST:AUTO:DomesticAutoMakers" valueDomainName="Domestic Auto Makers" defaultCodingScheme="Automobiles" effectiveDate="2009-01-01T11:00:00Z" isActive="true" status="ACTIVE">
            <lgVD:properties>
                <lgCommon:property propertyName="textualPresentation">
                    <lgCommon:value> Domestic Auto Makers</lgCommon:value>
                </lgCommon:property>
            </lgVD:properties>
            <lgVD:definitionEntry ruleOrder="1" operator="OR">
                <lgVD:entityReference entityCode="005" referenceAssociation="hasSubtype" transitiveClosure="true"  targetToSource="false" leafOnly="false"/>
            </lgVD:definitionEntry>
        </lgVD:valueDomainDefinition>
       <lgVD:valueDomainDefinition valueDomainURI="SRITEST:AUTO:AllDomesticButGM" valueDomainName="All Domestic Autos But GM" defaultCodingScheme="Automobiles" effectiveDate="2009-01-01T11:00:00Z" isActive="true" status="ACTIVE">
            <lgVD:properties>
                <lgCommon:property propertyName="textualPresentation">
                    <lgCommon:value> Domestic Auto Makers</lgCommon:value>
                </lgCommon:property>
            </lgVD:properties>
            <lgVD:definitionEntry ruleOrder="1" operator="OR">
                <lgVD:entityReference entityCode="005" referenceAssociation="hasSubtype" transitiveClosure="true"  targetToSource="false" leafOnly="false"/>
            </lgVD:definitionEntry>
            <lgVD:definitionEntry ruleOrder="2" operator="SUBTRACT">
                <lgVD:entityReference entityCode="GM" referenceAssociation="hasSubtype" transitiveClosure="true"  targetToSource="false" leafOnly="false"/>
            </lgVD:definitionEntry>
        </lgVD:valueDomainDefinition>
        <lgVD:valueDomainDefinition valueDomainURI="SRITEST:AUTO:AllDomesticANDGM" valueDomainName="All Domestic Autos AND GM" defaultCodingScheme="Automobiles" effectiveDate="2009-01-01T11:00:00Z" isActive="true" status="ACTIVE">
            <lgVD:properties>
                <lgCommon:property propertyName="textualPresentation">
                    <lgCommon:value> Domestic Auto Makers AND GM</lgCommon:value>
                </lgCommon:property>
            </lgVD:properties>
            <lgVD:definitionEntry ruleOrder="1" operator="OR">
                <lgVD:entityReference entityCode="005" referenceAssociation="hasSubtype" transitiveClosure="true"  targetToSource="false" leafOnly="false"/>
            </lgVD:definitionEntry>
            <lgVD:definitionEntry ruleOrder="2" operator="AND">
                <lgVD:entityReference entityCode="GM" referenceAssociation="hasSubtype" transitiveClosure="true"  targetToSource="false" leafOnly="false"/>
            </lgVD:definitionEntry>
        </lgVD:valueDomainDefinition>
         <lgVD:valueDomainDefinition valueDomainURI="SRITEST:AUTO:DomasticLeafOnly" valueDomainName="Domestic Leaf Only" defaultCodingScheme="Automobiles" effectiveDate="2009-01-01T11:00:00Z" isActive="true" status="ACTIVE">
            <lgVD:properties>
                <lgCommon:property propertyName="textualPresentation">
                    <lgCommon:value>Domestic Leaf Only</lgCommon:value>
                </lgCommon:property>
            </lgVD:properties>
            <lgVD:definitionEntry ruleOrder="1" operator="OR">
                <lgVD:entityReference entityCode="005" referenceAssociation="hasSubtype" transitiveClosure="true"  targetToSource="false" leafOnly="true"/>
            </lgVD:definitionEntry>
        </lgVD:valueDomainDefinition>
        </lgVer:valueDomains>
</systemRelease>

Pick List Definitions

Below is a sample XML file containing Pick List Definitions in LexGrid format that can be loaded using LexEVS Pick List Service.

Code Block

	xml
	xml

<?xml version="1.0" encoding="UTF-8"?>
<systemRelease xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://LexGrid.org/schema/2009/01/LexGrid/versions http://LexGrid.org/schema/2009/01/LexGrid/versions.xsd"
    xmlns="http://LexGrid.org/schema/2009/01/LexGrid/versions" xmlns:lgVer="http://LexGrid.org/schema/2009/01/LexGrid/versions"
    xmlns:lgCommon="http://LexGrid.org/schema/2009/01/LexGrid/commonTypes" xmlns:data="data"
    xmlns:lgVD="http://LexGrid.org/schema/2009/01/LexGrid/valueDomains" xmlns:lgNaming="http://LexGrid.org/schema/2009/01/LexGrid/naming"
    releaseURI="http://testRelease/04" releaseDate="2008-11-07T14:55:51.615-06:00">
    <lgCommon:entityDescription>Sample value domains</lgCommon:entityDescription>
    <pickLists>
        <lgVD:mappings>
            <lgNaming:supportedCodingScheme localId="Automobiles" uri="urn:oid:11.11.0.1">Automobiles</lgNaming:supportedCodingScheme>
            <lgNaming:supportedLanguage localId="en" uri="www.en.org/orsomething">en</lgNaming:supportedLanguage>
            <lgNaming:supportedNamespace localId="Automobiles" uri="urn:oid:11.11.0.1" equivalentCodingScheme="Automobiles">Automobiles</lgNaming:supportedNamespace>
            <lgNaming:supportedProperty localId="textualPresentation">textualPresentation</lgNaming:supportedProperty>
            <lgNaming:supportedSource localId="lexgrid.org">lexgrid.org</lgNaming:supportedSource>
            <lgNaming:supportedSource localId="_111101">11.11.0.1</lgNaming:supportedSource>
        </lgVD:mappings>
        <lgVD:pickListDefinition pickListId="SRITEST:AUTO:DomesticAutoMakers" representsValueDomain="SRITEST:AUTO:DomesticAutoMakers" isActive="true" defaultEntityCodeNamespace="Automobiles" defaultLanguage="en" completeDomain="false">
            <lgCommon:owner>Owner for Domestic Auto Makers</lgCommon:owner>
            <lgCommon:entityDescription>DomesticAutoMakers</lgCommon:entityDescription>
            <lgVD:mappings>
                <lgNaming:supportedCodingScheme localId="Automobiles" uri="urn:oid:11.11.0.1">Automobiles</lgNaming:supportedCodingScheme>
                <lgNaming:supportedDataType localId="texthtml">text/html</lgNaming:supportedDataType>
                <lgNaming:supportedDataType localId="textplain">text/plain</lgNaming:supportedDataType>
                <lgNaming:supportedLanguage localId="en" uri="www.en.org/orsomething">en</lgNaming:supportedLanguage>
                <lgNaming:supportedNamespace localId="Automobiles" uri="urn:oid:11.11.0.1" equivalentCodingScheme="Automobiles">Automobiles</lgNaming:supportedNamespace>
                <lgNaming:supportedProperty localId="textualPresentation">textualPresentation</lgNaming:supportedProperty>
                <lgNaming:supportedSource  assemblyRule="rule1" uri="http://informatics.mayo.edu" localId="lexgrid.org">lexgrid.org</lgNaming:supportedSource>
                <lgNaming:supportedSource localId="_111101">11.11.0.1</lgNaming:supportedSource>
            </lgVD:mappings>
           <lgVD:pickListEntryNode pickListEntryId="PLGMp1" isActive="true">
                <lgCommon:owner>Owner for PLGMp1</lgCommon:owner>
                <lgCommon:entryState containingRevision="R001" relativeOrder="1" changeType="NEW" prevRevision="R00A"/>
                <lgVD:inclusionEntry entityCode="GM" entityCodeNamespace="Automobiles" propertyId="p1">
                    <lgVD:pickText>General Motors</lgVD:pickText>
                </lgVD:inclusionEntry>
                <lgVD:properties>
                    <lgCommon:property propertyName="textualPresentation" isActive="true" language="en" propertyId="p1" propertyType="presentation" status="active" effectiveDate="2001-12-17T09:30:47Z" expirationDate="2011-12-17T09:30:47Z">
                        <lgCommon:owner role="role" subRef="subref">General Motors</lgCommon:owner>
                        <lgCommon:entryState containingRevision="R001" relativeOrder="1" changeType="NEW" prevRevision="R00A"/>
                        <lgCommon:source subRef="subref1" role="role1">General Motors</lgCommon:source>
                        <lgCommon:value dataType="textplain">Property for General Motors</lgCommon:value>
                    </lgCommon:property>
                </lgVD:properties>
            </lgVD:pickListEntryNode>
            <lgVD:pickListEntryNode pickListEntryId="PLGMp2" isActive="true">
                <lgCommon:owner>Owner for PLGMp2</lgCommon:owner>
                <lgCommon:entryState containingRevision="R001" relativeOrder="1" changeType="NEW" prevRevision="R00A"/>
                <lgVD:inclusionEntry entityCode="GM" entityCodeNamespace="Automobiles" propertyId="p2">
                    <lgVD:pickText>GM</lgVD:pickText>
                </lgVD:inclusionEntry>
            </lgVD:pickListEntryNode>
            <lgVD:pickListEntryNode pickListEntryId="PLJaguarp1" isActive="true">
                <lgCommon:owner>Owner for PLJaguarp1</lgCommon:owner>
                <lgCommon:entryState containingRevision="R001" relativeOrder="1" changeType="NEW" prevRevision="R00A"/>
                <lgVD:inclusionEntry entityCode="Jaguar" entityCodeNamespace="Automobiles" propertyId="p1">
                    <lgVD:pickText>Jaguar</lgVD:pickText>
                </lgVD:inclusionEntry>
            </lgVD:pickListEntryNode>
            <lgVD:pickListEntryNode pickListEntryId="PLChevroletp1" isActive="true">
                <lgCommon:owner>Owner for PLChevroletp1</lgCommon:owner>
                <lgCommon:entryState containingRevision="R001" relativeOrder="1" changeType="NEW" prevRevision="R00A"/>
                <lgVD:inclusionEntry entityCode="Chevy" entityCodeNamespace="Automobiles" propertyId="p1">
                    <lgVD:pickText>Chevrolet</lgVD:pickText>
                </lgVD:inclusionEntry>
            </lgVD:pickListEntryNode>
        </lgVD:pickListDefinition>
       <lgVD:pickListDefinition pickListId="SRITEST:AUTO:DomasticLeafOnly" representsValueDomain="SRITEST:AUTO:DomasticLeafOnly" completeDomain="true" defaultEntityCodeNamespace="Automobiles" defaultLanguage="en" isActive="true">
            <lgCommon:entityDescription>Leaf Only Nodes of Domastic AutoMakers</lgCommon:entityDescription>
        </lgVD:pickListDefinition>
    </pickLists>
</systemRelease>

Database structure

Value Domain Tables

Image Removed!value_domain_VDTablesDiagram.jpg,alt='value domain VDTablesDiagram"!

Table Name	Description
valueDomain	Will contain Value Domain Definition information
vdEntry	Contains Value Domain Entries information and its rules
entryState	Contains entry state details of every entry
mappings	Contains supported mapping information for a Value Domain Definition
property	Contains Property informations for Value Domain Definition

Pick List Tables

Image Removed!value_domain_PLTablesDiagram.jpg,alt="value domain PLTablesDiagram"!

Table Name	Description
pickList	Will contain Pick List Definition information
plEntry	Contains Pick List Entry Nodes information
entryState	Contains entry state details of every entry
mappings	Contains supported mapping information for a Pick List Definition
property	Contains Property informations for Pick List Definition and its Nodes

Installation / Packaging

Both LexEVS Value Domain and Pick List services are integrated part of core LexEVS API and will be packaged and installed with other LexEVS services.

System Testing

Value Domain Service

The System test case for the LexEVS Value Domain service is performed using the JUnit test suite:

...

This test suite will be run as part of regular LexEVS test suites AllTestsAllConfigs and AllTestsNormalConfigs.

Pick List Service

The System test case for the LexEVS Value Domain service is performed using the JUnit test suite:

...

This test suite will be run as part of regular LexEVS test suites AllTestsAllConfigs and AllTestsNormalConfigs.

Improved Loader Framework Detailed Design

Document Purpose

This document provides the detailed design and implementation of LexBIG Enterprise Vocabulary Service (LexEVS) Loader Framework Extension. It is also the goal of this document to provide enough information to allow those persons wishing to create their own loaders can do so. This document will also assume the reader is already familiar with the LexEVS software.

Implementation Overview

Description

The LexEVS software already provides a set of loaders within an existing legacy framework which served LexEVS developers well over many years. But as LexEVS has gained users, and requests for new loaders has grown , it was decided that a new loader framework should be developed that would: (1) be easier to extend (2) provide improved performance (3) dynamic loading of new loaders (4) take advantage of proven open source components such as Spring Batch and Hibernate.

...

Also, this Framework is completely independent of the current loader code so there is no impact to current loaders.

Scope

The LexEVS Loader Framework will provide a way for LexEVS developers to write new loaders and have them recognized dynamically by the LexEVS code. Also the framework will provide help to loader developers in the form of utility classes and interfaces.

Architecture

The LexEVS Loaders Framework extend the functionality of LexBIG 5.0 . For more information on LexBIG, see https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/LexEVS_Version_5.0

High Level Overview

Figure 1 (below) The following figure shows the major components of the Loader Framework (A) in relation to a hypothetical new loader and what expected API usage would be. Ideally, the new loader can find make most if its API calls through the utilities provided by the Loader Framework API (B). Some work will need to be done with Spring (C) such as configuration of a Spring config file. Also it may or may not be necessary for a loader to use Hibernate (D) or use the LexBIG API (E). However, again, the hope is that much of the work a new loader may need to do can be accomplished by the Loader Framework API.

The Loader Framework utilizes Spring Batch for managing its Java objects to improve performance and Hibernate provides the mapping to the LexGrid database.

Image Removed
Figure 1

!loader_image003.gif,alt="diagram of major components of the Loader Framework"!

Assumptions

None

Dependencies

This Loader Framework requires LexEVS release 5.0 or above.
Development system are required to install the Sun Java Development Kit (SDK) or Java Runtime Environment (JRE) version 1.5.0_11 or above.
Maven 2.1 or greater.
For software and hardware dependencies for the system hosting the LexEVS runtime, refer to https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/LexEVS_5.0_Documentation.

Issues

None

Third Party Tools

Spring: A lightweight open-source application framework.
- Spring see http://www.springsource.com/
- Spring Batch see http://static.springsource.org/spring-batch/
- Sprint Batch Reference see http://static.springsource.org/spring-batch/reference/html/index.html

Hibernate: An open source Java persistence framework. See https://www.hibernate.org/
Maven: Apache build manager for Java projects. See http://maven.apache.org/
Eclipse: An Open Source IDE. See http://www.eclipse.org/

Implementation Contents

Development and Build Environment

The Loader Framework code is available in the NCI Subversion (SVN) repository. It is comprised of three Framework projects. Also at the time of this writing there are three projects in the repository that utilize the Loader Framework. These projects utilize Maven for build and dependency management.

...

The above projects are built and managed by Maven.
Maven plugin for Eclipse: http://m2eclipse.codehaus.org/

How to Use the Loader Framework: A Roadmap

So you want to write a loader and use the Loader Framework. What are the key considerations?
In general the process can be described as:

...

An example may help in understanding the Framework. Our discussion will refer to the following figure 2 below. Lets say we are writing a loader to load the ICD-9-CM codes and their description which are contained in a text file. We know we'll need a data structure to hold the data after we've read it so we have a class:

...

Enter Spring. The Loader Framework uses Spring Batch to manage the reading, processing and writing of data. Spring provides classes and interfaces to help do this work and the Loader Framework also provides utilities to help loader developers. In our example, we will write a class that will use the Spring ItemReader interface. It will take a line of text and return an ICD9SourceObject (1 and 2). Next we'll want to process that data into a LexEVS object such as an Entity object. So we'll write class that implements Spring's ItemProcessor interface. It will take our ICD9SourceObject and output a LexEVS Entity object (3,4). Finally, we'll want to write the data to the database (5). Note that the LexEVS model objects provided in the Loader Framework are generated by Hibernate and utilize Hibernate to write the data to the database. This will free us from having to write SQL.

Image Removed
Figure 2

!loader_image004.jpg,alt="diagram of loader framework"!

Spring
You will need to configure Spring to be aware of your objects and how to manage them. This is done via a XML configuration file. More details on the Spring config file below.

...

From the Maven documentation:Image Removed

!Eclipse_Project.gif,alt="Eclipse project illustration"!

For more information on the Maven project see:
http://maven.refer to the documentation on apache.org/guides/getting-started/maven-in-five-minutes.html

Configure your Spring Config (myLoader.xml)

Spring is a lightweight bean management container and among other things it contains a batch function which is utilized by the Loader Framework. A loader using the framework will need to work closely with Spring Batch and the way it does that is through Spring's configuration file where you configure beans (your loader code) and how the loader code should be utilized by Spring Batch (by configuring a Job, Step and other Spring Batch stuff in the spring config file). What follows is a brief overview of those tags related to the LoaderFramework. For more detail please see refer to the Spring documentation: http://static.springsource.org/spring-batch/reference/html/index.html.

Image Removed
Figure 3

!loader_image005.gif,alt="loader diagram"!

Beans

The 'beans:beans' tag is the all-encompassing tag. You define all your other tags in here. You can also define an import within this tag to import an external Spring config file. Not shown in figure 3.

...

You can do anything you want within a Tasklet such as sending an email or a LexBIG function such as indexing. You're not limited to just database operations. The Spring documentation also has this to say about Tasklets:

Info
The Tasklet is a simple interface that has one method, execute, which will be a called repeatedly by the TaskletStep until it either returns RepeatStatus.FINISHED or throws an exception to signal a failure. Each call to the Tasklet is wrapped in a transaction

Chunk

Spring documentation says it best:

Info
Spring Batch uses a 'Chunk Oriented' processing style within its most common implementation. Chunk oriented processing refers to reading the data one at a time, and creating 'chunks' that will be written out, within a transaction boundary. One item is read in from an ItemReader, handed to an ItemWriter, and aggregated. Once the number of items read equals the commit interval, the entire chunk is written out via the ItemWriter, and then the transaction is committed.

Info

Spring Batch uses a 'Chunk Oriented' processing style within its most common implementation. Chunk oriented processing refers to reading the data one at a time, and creating 'chunks' that will be written out, within a transaction boundary. One item is read in from an ItemReader, handed to an ItemWriter, and aggregated. Once the number of items read equals the commit interval, the entire chunk is written out via the ItemWriter, and then the transaction is committed.

Reader

An attribute of the chunk tag. Here is the class that you defined implementing the Spring ItemReader interface to read data from your data file and create domain-specific objects.

...

Attribute of the chunk tag. This class will implement the Spring interface ItemWriter. In the case of the Loader Framework these classes have been written for you. They are the LexGrid model objects that use Hibernate to write to the database.

Key Directories

Below is an image of the loader-framework-core project in Eclipse which shows the key directories of the Loader Framework. The following is a summary of the contents of those directories.

...

Content

Space Tools