 |
Page History
...
Most data elements will be loaded as either properties or property qualifiers:
A few will be loaded as qualifiers to associations.
...
Here is a UML representation of Value Domain within LexGrid 200901 model.
Value Domain Definition
...
Class Name | Description |
|---|---|
VDEntryTypeServices | Class to handle Entry Type objects to and fro database . |
VDEntryStateServices | Class to handle Entry State objects to and fro database. |
VDPropertyServices | Class to handle Property objects to and fro database. |
VDMappingServices | Class to handle supported Mappings objects to and fro database. |
VDServiceHelper | Helper class containing methods that are commonly used. |
VDBaseSQLServices | Class to handle SQL Services. |
VDBaseService | Base service class to handle all Value Domain and Pick List related objects to and fro from database. |
!value_domain_CommonClassDiagram.jpg.alt\="value domain CommonClassDiagram"!
...
Class Name | Description |
PickListsServices | Class to handle list of Pick List Definitions. |
PickListServices | Class to handle individual Pick List Definition objects to and fro database. |
PLEntryServices | Class to handle Pick List Entry objects to and fro database. |
LexEVSPickListServices | Primary interface for LexEVS Pick List API. |
LexEVSPickListServicesImpl | Implementation of LexEVSPickListServices which is primary interface for LexEVS Pick List API. |
LoadPickList | Imports the Pick List Definitions in the source file, provided in LexGrid canonical format, to the LexBIG repository. |
ResolvedPickListEntyList | Class to hold list of resolved pick list entries. |
ResolvedPickListEntry | Bean for resolved pick list entries. |
LexBIG Services Class Diagram
...
Information | listValueDomains(String valueDomainName) | ||
|---|---|---|---|
Description: | Return the URI's for the value domain definition(s) for the supplied domain name. If the name is null, returns everything. If the name is not null, returns the value domain(s) that have the assigned name. | ||
Input: | java.lang.String | ||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="393159051edc0c7f-ebacfabd-40c3477e-a5caa1e8-31aee7cfea7fe7be78915c42"><ac:plain-text-body><![CDATA[ | Output: | java.net.URI[] | ]]></ac:plain-text-body></ac:structured-macro> |
Exception: | org.LexGrid.LexBIG.Exceptions.LBException | ||
Implementation Details: | Implementation: | ]]></ac:plain-text-body></ac:structured-macro> |
...
Information | getAllValueDomainsWithNoNames() | ||
|---|---|---|---|
Description: | Return the URI's of all unnamed value domain definition(s). | ||
Input: | none | ||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="358a6b9d31dfe548-5ecc349b-401747a4-947bb506-2778892628b7a4d6afa18463"><ac:plain-text-body><![CDATA[ | Output: | java.net.URI[] | ]]></ac:plain-text-body></ac:structured-macro> |
Exception: | org.LexGrid.LexBIG.Exceptions.LBException | ||
Implementation Details: | Implementation: | ]]></ac:plain-text-body></ac:structured-macro> |
...
Information | getPickListDefinitionsForDomain(URI valueDomainURI) | ||
|---|---|---|---|
Description: | Returns all the pickList definitions that represents supplied valueDomain URI. | ||
Input: | java.net.URI | ||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="1386a3f3bb0a4e31-8b36db61-4b454ed4-b8949377-4a074cac6eb7853cf208f92c"><ac:plain-text-body><![CDATA[ | Output: | org.LexGrid.emf.valueDomains.PickListDefinition[] | ]]></ac:plain-text-body></ac:structured-macro> |
Exception: | org.LexGrid.LexBIG.Exceptions.LBException | ||
Implementation Details: | Implementation: | ]]></ac:plain-text-body></ac:structured-macro> |
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="e681ed06642b0f78-d6cccdb7-4e2d4569-ac2b9a3b-a5e67099282d90b3922383dc"><ac:plain-text-body><![CDATA[ | Information | resolvePickListForTerm(String pickListId, String term, String matchAlgorithm, String language, String[] context, boolean sortByText) | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|
Description: | Resolves pickList definition by applying supplied arguments. | ||
Input: | java.lang.String, | ||
Output: | org.lexgrid.valuedomain.dto.ResolvedPickListEntryList | ||
Exception: | org.LexGrid.LexBIG.Exceptions.LBException | ||
Implementation Details: | Implementation: |
...
Database structure
Value Domain Tables
Table Name | Description |
|---|---|
valueDomain | Will contain Value Domain Definition information |
vdEntry | Contains Value Domain Entries information and its rules |
entryState | Contains entry state details of every entry |
mappings | Contains supported mapping information for a Value Domain Definition |
property | Contains Property informations for Value Domain Definition |
...
The LexEVS Loaders Framework extend the functionality of LexBIG 5.0 . For more information on LexBIG, see https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/LexEVS_Version_refer to LexEVS 5.0.
High Level Overview
The following figure shows the major components of the Loader Framework (A) in relation to a hypothetical new loader and what expected API usage would be. Ideally, the new loader can find make most if its API calls through the utilities provided by the Loader Framework API (B). Some work will need to be done with Spring (C) such as configuration of a Spring config file. Also it may or may not be necessary for a loader to use Hibernate (D) or use the LexBIG API (E). However, again, the hope is that much of the work a new loader may need to do can be accomplished by the Loader Framework API.
...
- This Loader Framework requires LexEVS release 5.0 or above.
- Development system are required to install the Sun Java Development Kit (SDK) or Java Runtime Environment (JRE) version 1.5.0_11 or above.
- Maven 2.1 or greater.
- For software and hardware dependencies for the system hosting the LexEVS runtime, refer to https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/LexEVS_the LexEVS 5.0 _ Documentation.
Issues
- None
Third Party Tools
...
Enter Spring. The Loader Framework uses Spring Batch to manage the reading, processing and writing of data. Spring provides classes and interfaces to help do this work and the Loader Framework also provides utilities to help loader developers. In our example, we will write a class that will use the Spring ItemReader interface. It will take a line of text and return an ICD9SourceObject (1 and 2). Next we'll want to process that data into a LexEVS object such as an Entity object. So we'll write class that implements Spring's ItemProcessor interface. It will take our ICD9SourceObject and output a LexEVS Entity object (3,4). Finally, we'll want to write the data to the database (5). Note that the LexEVS model objects provided in the Loader Framework are generated by Hibernate and utilize Hibernate to write the data to the database. This will free us from having to write SQL.
Spring
You will need to configure Spring to be aware of your objects and how to manage them. This is done via a XML configuration file. More details on the Spring config file below.
...
From the Maven documentation:
For more information on the Maven project refer to the documentation on apache.org.
...
Spring is a lightweight bean management container and among other things it contains a batch function which is utilized by the Loader Framework. A loader using the framework will need to work closely with Spring Batch and the way it does that is through Spring's configuration file where you configure beans (your loader code) and how the loader code should be utilized by Spring Batch (by configuring a Job, Step and other Spring Batch stuff in the spring config file). What follows is a brief overview of those tags related to the LoaderFramework. For more detail refer to the Spring documentation.
Beans
The 'beans:beans' tag is the all-encompassing tag. You define all your other tags in here. You can also define an import within this tag to import an external Spring config file. Not shown in figure 3.
...
Below is an image of the loader-framework-core project in Eclipse which shows the key directories of the Loader Framework. The following is a summary of the contents of those directories.
Figure 4
Directory | Summary |
|---|---|
connection | Connect to LexBIG and do LexBIG tasks such as register and activate. |
constants | Assorted constants. |
dao | Access to the LexBIG database. |
data | Directly related to data going into the LexBIG database tables. |
database | Database specific tasks not related to data, such as finding out the database type (MySQL, Oracle) |
fieldsetter | Spring related classes for helping to write to the database. |
lexbigadmin | Common tasks you want LexBIG to do for you such as indexing. |
listener | You can attach listeners to a load so that the code will execute and certain points in the load such as a cleanup listener that runs when the load is finished or a setup listener etc... |
logging | Gives you access to the LexBIG logger. |
processor | Important directory. Contains classes that you can pass your domain specific object to and will return a LexBIG object. |
properties | Code used internally by the Loader Framework. |
reader | Readers and reader-related tools for loader developers. |
rowmapper | Classes for reading from a database. Currently experimental code. |
setup | Classes such as JobRepositoryManager that help Spring do its work. As Spring hums along it keeps tables of its internal workings. Loader developers should not need to dive into this directory. |
staging | If your loader needs to load data to the database temporarlily you can find helper classes in this directory. |
wrappers | Helper classes and data strucutres such as a Code/CodingScheme class. |
writer | Miscellanous Miscellaneous classes that write to the database. These are not the same ones you'd use in your loader, i.e the LexBIG model objects that use Hibernate. Those classes are contained in the PersistanceLayer PersistenceLayer project (next figure 5, below). Its It is by using those classes in the PersistenceLayer that you let the Loader Framework do some of the heavy lifting for you. |
Figure 5
Algorithms
None
Batch Processes
None
Error Handling
Spring Batch gives the Loader Framework some degree of recovery from errors. Like the other features of Spring it is something the Loader developer would need to configure in their Spring config file. Basically, Spring will keep track of the steps it has executed and make note of any step that has failed. Those failed steps can be re-run at a later time. The Spring documentation provides additional information on this function. See http://static.springsource.org/spring-batch/reference/html/configureJob.html and http://static.springsource.org/spring-batch/reference/html/configureStep.htmlRefer to the configure job documentation and configure step documentation.
Database Changes
None
Client
Currently, the LexBIG GUI does not provide a framework to dynamically load extendable GUI components. While not impossible to extend the GUI functionality especially for those working closely with the LexBIG code, loaders written to use the new framework should expect that their loader will be called via the command line or script.
JSP/HTML
None
Servlet
None.
Security Issues
None.
Performance
Spring can accommodate parallel processing to enhance performance. The Spring documentation provides a good discussion of this topic. See http://static.Refer to scalability on springsource.org/spring-batch/reference/html/scalability.html.
Internationalization
Not internationalized.
Installation / Packaging
The Loader Framework is packaged as a LexBIG extension and thus is not included in the LexBIG jar.
Migration
None.
Documentation Considerations
The Loader Framework will also be described in the knowledge centerKnowledge Center.
Testing
Automated test tests are run via Maven. As mentioned earlier the projects containing the Loader Framework code are configured to work with Maven. Figure 6 (below), shows how The following figure shows the PersistenceLayer project and its standard Maven layout. Notice the structure of the test code mirrors the structure of the application code. To run the automated test in our Eclipse environment we select the project, right click, select 'Run As' and select 'Maven test'. Maven will do the rest.
Figure 6
Test Guidelines
The test cases are also integrated into the LexBIG 5.1 build environment and are run with each build.
Test Cases
See System Testing.
Test Results
See System Testing.
Custom Loader Feasibility Report and Recommendation
Persistence Layer Feasibility
...
- Group Readers
- Group Writers
- Writers configurable to skip certain records
- Partitionable readers to break up large source files
- Error checking Readers and Writers
- A Validating framework for inspecting content before it is inserted into the database.
- etc.
BDA Support Detailed Design
LexEVS uses the BDA (Build and Deployment Automation) system to build and deploy artifacts. This build script that produces these artifacts and deploys them is kicked off via a build server (an instance of Anthill pro).
...
Each of these is produced as a separate artifact via a single build call. The local java Java application is built and integrated into the web application. The data service is also integrated into the web application. Separate analytical and data grid services are also created.
...
In this case two separate JBoss server instances are required. In one instance the web enabled application (lexevsapi) is copied to the remote server as a war file, it is accompanied by a grid data service zip file that is unzipped into a typical war file directory structure. In the other instance the LexEVS grid analytical service is deployed. In every case the server is stopped, configured as necessary, receives the new war file or directory and is started again. The BDA process is considered to be successful when all project builds are complete, all artifacts are deployed, and each public interface of these services is verified.
Implementation Plan
This will include the technical environment (HW,OS, Middleware), external dependencies, teams/locations performing development and procedures for development (e.g. lifecycle model,CM), and a detailed schedule.
Technical environment
No new environment requirements exist for the the LexEVS 5.1, with the exception of additional storage to accommodate larger content loads.
Software (Technology Stack)
Operating System
- Linux (though no operating system dependencies currently exist)
...
- caGrid 1.3 / Globus 4.0.3
Server Hardware
Server
- NCI standard hardware.
...
- Minimum required by JBoss.
Storage
Expected file server disk storage (in MB)
...
- 100GB (May increase due to additional RRF content load)
Networking
Application specific port assignments
...
- Grid services can NOT be in the same JBoss container because of naming - they both need to be named /wsrf in the container. This results in the use of 1 JBoss container for LexEVS 5.0 and 1 JBoss container for LexEVS 5.1.
External dependencies
N/A
Team/Location performing development
- Traci St. Martin / Mayo Clinic
- Craig Stancl / Mayo Clinic
- Scott Bauer / Mayo Clinic
- Kevin Peterson / Mayo Clinic
- Sridhar Dwarkanath / Mayo Clinic
- Michael Turk / Mayo Clinic
Procedures for Development
Development will follow procedures as defined by NCI.
Detailed schedule
The LexEVS 5.1 project plan is located in Gforge at: LexEVS 5.1 Project Plan and LexEVS 5.1 Project Plan (PDF)on Gforge in MS Project format and PDF format
The LexEVS 5.1 BDA Project plan is located at : LexEVS 5.1 BDA Project Plan (login required).
Training and documentation requirements
Documentation for LexEVS 5.1 will be located on provided by the Vocabulary Knowledge Center Wiki https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/LexBig_and_LexEVS: LexEVS Documentation.
Download center changes
Downloads for LexEVS 5.1 will be located on the Vocabulary Knowledge Center Wiki https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/LexBig_and_LexEVS LexEVS wiki.
| Scrollbar | ||
|---|---|---|
|
