Overview
LexEVS provides classes to extend and interfaces to implement that help provide the framework for in memory transformations of source files into the LexGrid model. If the target source is large, batch loading may well be a better solution in which case the Loader Framework built on Spring Batch may be a better match. On the other hand users may find the standard LexEVS interface a little easier to implement.
Writing loaders requires both programming skill and content expertise. Mapping source into the LexGrid data model requires knowledge of how the source defines entities, relationships between entities and any qualifiers or properties for these elements. However some study of other presentations of that content, and documentation of the source files, can help most programmers make informed choices as to that mapping. Examples of source mapped into LexGrid are documented here (TODO) link.
Where to Start
The LexEVS API programming environment is currently available in a github repository here: LexEVS Source Code
Cloning this repository produces a large set of eclipse friendly projects that can be imported into eclipse.
Loader interfaces must extend the interface org.LexGrid.LexBIG.Extensions.Load.Loader from the lbInterfaces project and implementations of your new interface must also extend the abstract class org.LexGrid.LexBIG.Impl.loaders.BaseLoader in the lbImpl project. Let's take as an example the Medra loader interfaces and classes as it is a relatively simple implementation:
The extension of Loader is org.LexGrid.LexBIG.Extensions.Load.MedDRA_Loader.java and is found in the lbInterfaces project. This defines a loader and validation method and creates initialized class attributes name and description. The name of this interface will eventually be used by the LexEVS extension function to call the loader into existence. (Loaders are always extensions to LexEVS).
Looking at the implementation of this interface in lbImpl, org.LexGrid.LexBIG.Impl.loaders.MedDRALoaderImpl, notice that implementation is kept relatively clean thanks to much of the mechanism of loading into LexEVS is taken care of under the covers by BaseLoader. It creates a constructor that always calls the BaseLoader constructor, then prevents the use of manifests. If you wish you may choose to allow manifest use, since it allows load time manipulation of coding scheme metadata. Often source files do not provide all that needs to be known about the source. A manifest file allows values that may not be present in the source, such as copyrights, authoring information, version definition and formal coding scheme name to be added to the load.
The buildExtensionDescription method provides a background method for the registration of this loader to take place within LexEVS. It should be created the same way for each loader.
The validate method is not always implemented, but can and should be when a mechanism exists to insure that this is a correctly structured version of the source. When source is XML formatted and has a schema or dtd document commonly available for validation, this is a relatively easy process. However any free text files that do not have any associated java-based validation API, would rely on the developer to create validation functions. This method is not required to create a loader for LexEVS.
In theory the developer could at this point implement only doLoad by mapping content to a coding scheme object and persisting it before turning control over to BaseLoader which will call the doLoad method and set default load options in its load method. The other option is to implement doLoad and override the load method which sets up end user option choices for the loader. Most loaders implement the load method, customizing load options to provide to the end user. In the case of the MedDRA loader, a CUI load option is provided to the end user.
Beyond these methods, where a coding scheme object is passed to LexEVS, the structuring of code is largely and necessarily left up to the developer. However a few common patterns are fairly consistent in this implementation. Generally speaking, there is a central mapping where the coding scheme object is built. Other classes, when necessary, are supportive to this central class.
These classes are generally classified by those that are responsible for either reading a source file or accessing an API that reads the file for you and those that map objects created from that file into LexEVS coding scheme metadata, entity and relationship objects. A third category will include accessing LexEVS persistence mechanisms and passing these values to the database.
In the lbConverter project the edu.mayo.informatics.lexgrid.convert.directConversions.medDRA package contains the classes that do much of the work of the load. edu.mayo.informatics.lexgrid.convert.directConversions.medDRA.MedDRA2LGMain provides central kickoff point with some methods that can be wrapped for load and validation responsibilities down further up the execution chain. This package also contains edu.mayo.informatics.lexgrid.convert.directConversions.medDRA.MedDRAMapToLexGrid which does the active mapping of data objects derived from the MedDRA source and in this case also provides pushes the completed LexGrid correlations to the database. So this contains read, map and store functions all in one class.
Notice the map method in MedDRAMapToLexGrid. It calls three methods that separately load the coding scheme metadata, the entities and the relationships. This level of granularity provides a guideline for what can safely be loaded into LexEVS separately and sometimes as parallel functions.
