Page History

...

...

public class MedDRALoaderImpl extends BaseLoader implements MedDRA_Loader 

This defines a load and validation methods and creates initialized class attributes name and description.  The name of this interface will eventually be used by the LexEVS extension function to call the loader into existence.  (Loaders are always extensions to LexEVS).

...

Looking at the implementation of this interface in lbImpl, org.LexGrid.LexBIG.Impl.loaders.MedDRALoaderImpl, notice that implementation is kept relatively clean thanks the fact that much of the mechanism of loading into LexEVS is taken care of under the covers by the BaseLoader class.  MedDRALoaderImpl creates a constructor that always calls the BaseLoader constructor, then prevents the use of manifests.  (If you wish you may choose to allow manifest use, since it allows load time manipulation of coding scheme metadata.  Often source files do not provide all that needs to be known about the source.  A manifest file allows values that may not be present in the source such as copyrights, authoring information, version definition and formal coding scheme name to be added to the load.)

    public MedDRALoaderImpl() {
        super();
        this.setDoApplyPostLoadManifest(false);
    }

Preventing the use of a manifest is not typical loader behavior.

Define the Extension

The buildExtensionDescription method provides a background method for the registration of this loader to take place within LexEVS. It should be created the same way for each loader.

    @Override
    protected ExtensionDescription buildExtensionDescription(){
        ExtensionDescription temp = new ExtensionDescription();
        temp.setExtensionBaseClass(MedDRALoaderImpl.class.getInterfaces()[0].getName());
        temp.setExtensionClass(MedDRALoaderImpl.class.getName());
        temp.setDescription(description);
        temp.setName(name);
        temp.setVersion( getMedDRAVersion() );
        return temp;
    }

Validate the Source (Optional)

The validate method is not always implemented, but can and should be when a mechanism exists to insure that this is a correctly structured version of the source.  When source is XML formatted and has a schema or dtd document commonly available for validation, this is a relatively easy process.  However any free text files that do not have any associated java-based validation API, would rely on the developer to create validation functions.  The validation method is not required to create a loader for LexEVS. 

 @Override
    public void validate(URI sourceDir, int validationLevel) throws LBParameterException {

Pass in the URI for the path to the source file and, if desired, provide an integer value for a level of validation for this source. (I.e. what level of errors or issues can this load live with.  This assumes a very detailed validation of source)

Implement the Abstract Method doLoad() (and possibly override the load() method)

The developer could at this point implement only doLoad by mapping content to a coding scheme object before pass control over to BaseLoader which will call the doLoad method and set default load options in its load method and kick off the processes to persist a coding scheme object to the database.  The other option is to implement doLoad and override the load method which sets up end user option choices for the loader.   Most loaders implement the load method, customizing load options to provide to the end user.  In the case of the MedDRA loader, a CUI load option is provided to the end user.

    @Override
    protected URNVersionPair[] doLoad() throws Exception{
        LgMessageDirectorIF messages = new CachingMessageDirectorImpl(this.getMessageDirector());
        MedDRA2LGMain mainTxfm = new MedDRA2LGMain();
        //Set the source for the CUI file when using the GUI
        if(UMLSCUISource == null){
            UMLSCUISource = this.getOptions().getURIOption(UMLSCUI_FILE_OPTION).getOptionValue();
        }
        CodingScheme codingScheme = mainTxfm.map(UMLSCUISource, this.getResourceUri(), this.getMessageDirector());
  
        if(codingScheme != null){
            messages.info("Completed mapping.  Now saving to database");
            this.persistCodingSchemeToDatabase(codingScheme);

            messages.info("Saved to database.  Now constructing version pairs");            
            return this.constructVersionPairsFromCodingSchemes(codingScheme);
        }
            
         return null;   
    }

This allows the BaseLoader functions to access the results of the load of the MedDRA source file into a LexEVS coding scheme object.

    @Override
    public void load(URI uri, URI cuiUri, boolean stopOnErrors, boolean async) throws LBParameterException,
            LBInvocationException {
        this.getOptions().getBooleanOption(FAIL_ON_ERROR_OPTION).setOptionValue(stopOnErrors);
        this.getOptions().getBooleanOption(ASYNC_OPTION).setOptionValue(async);
        this.getOptions().getURIOption(UMLSCUI_FILE_OPTION).setOptionValue(cuiUri);
        UMLSCUISource = cuiUri;
        this.load(uri);
    }

This method adds a UMLS CUI source file option for the end user to load as a supplement to the regular MedDRA load.

Less Structure Beyond the Loader, BaseLoader Implementation and Extension

...

These classes are generally classified by those that are responsible for either reading a source file or accessing an API that reads the file for you and those that map objects created from that file into LexEVS coding scheme metadata, entity and relationship objects. 

In summary:

• Read source
• Map to objects related to source structure
• Map from source structure objects to LexEVS coding scheme object

Mapping Entry Point

In the lbConverter project  the edu.mayo.informatics.lexgrid.convert.directConversions.medDRA package contains the classes that do much of the work of the MedDRA load into LexGrid.  edu.mayo.informatics.lexgrid.convert.directConversions.medDRA.MedDRA2LGMain provides a central kickoff point with some methods that can be wrapped for load and validation responsibilities down further up the execution chain.

public class MedDRA2LGMain {

	//providing parameters for the source directory, the UMLS CUI file and a logging object from LexEVS
   	public CodingScheme map(URI cuiUri, URI sourceDir, LgMessageDirectorIF lg_messages)
	
	//Validate choice for the MedDRA source only
	private boolean validateSourceDir(URI sourceDir)

Read the Data to Source Model Objects

This package also contains edu.mayo.informatics.lexgrid.convert.directConversions.medDRA.MedDRAMapToLexGrid with a readMedDRAFile method that reads from a CSV file and persists it to objects defined in package edu.mayo.informatics.lexgrid.convert.directConversions.medDRA.Data.  While this data package defines beans that model the content of lines read from the CSV file, it also organizes them into structures that are easier for the mapping code to use. The individual manner of implementation is left up to the developer but this is a good example of how a third party library was used to process the file and how the resulting objects were stored in a data structure that's easy for the mapping code to consume.

    public void readMedDRAFiles() {
        String input;
        
        for(int i=0; i < meddraMetaData.length; i++){
            input = medDRASourceDir.getPath() + meddraMetaData[i].filename();
            try {
              
                FileReader fileReader = new FileReader(input);
                CSVReader reader = new CSVReader(fileReader, '$');
                ColumnPositionMappingStrategy<DatabaseRecord> strat = new ColumnPositionMappingStrategy<DatabaseRecord>();
                strat.setType(meddraMetaData[i].classname());
                String[] columns = getFields(meddraMetaData[i].classname()); 
            
                strat.setColumnMapping(columns);
    
                CsvToBean<DatabaseRecord> csv = new CsvToBean<DatabaseRecord>();
                List<DatabaseRecord> list = csv.parse(strat, reader);
                meddraDatabase.add(meddraMetaData[i].tablename(), list);
            } catch (FileNotFoundException e) {
                messages_.error("MedDRA input file missing.", e);
            } catch (Exception e) {
                messages_.error("Failed to read MedDRA files.");
            }
        }
    }

CSVReader is a third party CSV reader. 

Map the Model Objects to LexGrid Model Objects

Once the source is read an and persisted to the appropriate model objects the MedDRAMapToLexGrid class can map these data objects derived from the MedDRA source into a complete coding scheme object.

    public void mapToLexGrid(CodingScheme csclass) {
        try {
            loadCodingScheme(csclass);
            loadConcepts(csclass);
            loadRelations(csclass);
        } catch (Exception e) {
            messages_.error("Failed to map MedDRA data to LexEVS.");
        }     
        
        messages_.info("Mapping completed, returning to loader");
    }

Pass Control Back to BaseLoader

The tasks are to read the file into some kind of logical model or bean class object, organize these objects or make them available to be mapped into LexEVS objects, tie all objects together as a coding scheme, and pass this potentially large coding scheme object to the LexEVS BaseLoader to be persisted to the database.  Back up the execution chain in MeDRALoaderImpl the doLoad method first calls the mapping method of the conversion class MedDRA2LGMain to get the reading and mapping done, then passes control of the resulting coding scheme to BaseLoader by calling this.persistCodingSchemeToDatabase method on the BaseLoader super class. 

    if(codingScheme != null){
            messages.info("Completed mapping.  Now saving to database");
            this.persistCodingSchemeToDatabase(codingScheme);

            messages.info("Saved to database.  Now constructing version pairs");            
            return this.constructVersionPairsFromCodingSchemes(codingScheme);
        }

While it's not necessary that things are done this way, many required, most loaders have been written in this fashionway.