Page History

{scrollbar:icons=false}

...

...

...

...

...

LexEVS Architecture Overview

LexEVS software architecture and implementation is designed to facilitate flexibility and future expansion in the caBIG research community. The purpose of LexEVS is to enable individual Cancer Centers and others to use the provided caCORE EVS services and if desired, install local instances of vocabularies.The figure below shows the following components pointing to the caBIG Grid:

• caBIG Nodes
• Partial Online Replica
• Local Replica
• NCI

Image Removed

History and Definition

What is LexGrid?

LexGrid is the model used to store terminologies. The LexGrid Model is Mayo Clinic's proposal for standard storage of controlled vocabularies and ontologies. The LexGrid Model defines how vocabularies should be formatted and represented programmatically, and is intended to be flexible enough to accurately represent a wide variety of vocabularies and other lexically-based resources. The model also defines several different server storage mechanisms (e.g., relational database, LDAP) and a an XML format. This model provides the core representation for all data managed and retrieved through the LexBIG system, and is now rich enough to represent vocabularies provided in numerous source formats including:

• Open Biomedical Ontologies (OBO)
• Web Ontology Language (OWL), e.g., NCI Thesaurus
• Unified Medical Language System (UMLS) Rich Release Format (RRF), e.g., NCI MetaThesaurusMetathesaurus

This common model is a critical component of the LexGrid project. Once disparate vocabulary information can be represented in a standardized model, it becomes possible to build common repositories to store vocabulary content and common programming interfaces and tools to access and manipulate that content. The HL7/OMG Common Terminology Services (CTSCTS2) and LexBIG API as developed for the Cancer Biomedical Informatics Grid (caBIG®) initiative are two examples of APIs able to query information stored in the LexGrid Model.

For more information see the LexGrid Background Information.

What is LexBIG?

LexBIG is the set of services that EVS adapters use to store/retrieve terminology metadata. LexBIG is a more specific project that applies applied LexGrid vision and technologies to requirements of the caBIG® research community. The goal of the project is to build a vocabulary server accessed through a well-structured application programming interface (API) capable of accessing and distributing vocabularies as commodity resources. The server is to be built using standards-based and commodity technologies. Primary objectives for the project include:

• Provide a robust and scalable open source implementation of EVS-compliant vocabulary services. The API specification will be based on but not limited to fulfillment of the caCORE EVS API. The specification will be further refined to accommodate changes and requirements based on prioritized needs of the caBIG® NCI community.
• Provide a flexible implementation for vocabulary storage and persistence, allowing for alternative mechanisms without impacting client applications or end users. Initial development will focus on delivery of open source freely available solutions, though this does not preclude the ability to introduce commercial solutions (e.g. Oracle).
• Provide standard tooling for load and distribution of vocabulary content. This includes but is not limited to support of standardized representations such as UMLS Rich Release Format (RRF), the OWL web ontology language, and Open Biomedical Ontologies (OBO).

...

The LexEVS 6.x infrastructure exhibits an n-tiered architecture with client interfaces, server components, domain objects, data sources, and back-end systems (Figure 1.1). This n-tiered system divides tasks or requests among different servers and data stores. This isolates the client from the details of where and how data is retrieved from different data stores.

...

• Application Service Layer - accepts incoming requests from all public interfaces and translates them, as required, to Java calls in terms of the native LexEVS API. Non-SDK queries Queries are invoked against the Distributed LexEVS API, which handles client authentication and acts as proxy to invoke the equivalent function against the LexEVS core Java API. The caGrid and SDK-generated services are optionally run in an application server separate from the Distributed LexEVS API.
  The LexEVS caCORE SDK services work directly against the database, via Hibernate bindings, to resolve stored objects without intermediate translation of calls in terms of the LexEVS API. However, the LexEVS SDK services do still require access to metadata and security information stored by the Distributed and Core LexEVS API environment to resolve the specific database location for requested objects and to verify access to protected resources, respectively.
  From the client prospective, the LexEVS services will function as "ports" accessible through the caGrid 1.3 service architectural model. LexEVS services will follow the caGrid architecture for analytical and data services. See the caGrid 1.3 documentation for architectural details. 
  
  
• Core API Layer - Local Java API implementation that underpins all LexEVS API requests. Search of pre-populated Lucene index files is used to evaluate query results before incurring cost of database access. Access to the LexGrid database is performed as required to populate returned objects using pooled connections.
  
  
• Data Source Layer - is responsible for storage and access to all data required to represent the objects returned through API invocation.

LexEVS 6.x High-level Design Diagram

The figure below that follows shows the following components of the Architecture Diagram:

• Clients: Local, Distributed, and caGrid clients, and SOAP, REST, QBE Clients.
• Application Service: CTS2 (LexEVS Local Runtime), Distributed LesEVS, LexEVS caCORE API's.
• Core API: LesEVS Model Objects Extended for CTS 2 = > DAO
• Indexes and Data Source: Lucene Index Files and LexGrid DB

...

• Optional Arangodb support for high speed graph resolution of terminology hierarchy and relationship graphs

LexBIG Architecture

Image Added

LexBIG Services

This section describes architectural detail for services provided by the LexBIG system. These services are geared toward the administration, management, and serving of vocabularies defined to the LexGrid/LexBIG information model. A system overview is provided, followed by a description of key subsystems and components. Each subsystem is described in terms of its overall structure, formal model, and specification of key public interfaces.

...

LexBIGServiceManager - The service manager provides a centralized access point for administrative functions, including write and update access for a service's content. For example, the service manager allows new coding schemes to be validated and loaded, existing coding schemes to be retired and removed, and the status of various coding schemes to be updated and changed.

caGRID Hosting

The following figure shows the caGrid Hosting Environment. The Hosting Environment comprises the LexBIG Service which comprises the Service metadata, Query Service, Service Manager, and Extensions. A Service Discovery points to the Service Metadata component.

Image Removed

The LexBIG architecture provides the underpinnings LexBIG services to be made accessible through the caGRID environment in the future, where LexBIG services might optionally be deployed in a caGRID Globus container. caGrid provides a Globus service for service registration and discovery. LexBIG services deployed to the grid would be registered in the NCICB registry and be searchable through the NCICB index service.

Additional specifications related to the registration and discovery of LexBIG services in the caGRID environment will be included later phases of work in concordance with caGRID 1.0. This is will be coordinated with caBIG® Architecture workspace designees.

Service Management Subsystem

...

• IndexersVocabularies may be indexed to provide enhanced performance or query capabilities. Types of indexes incorporated into the LexBIG system include but are not limited to the following:
  • Lexical Match - for example, "begins-with" and "contains"
  • Phonetic - allows for the ability to query based on "sounds-like" entry of search criteria.
  • Stemming - allows for the ability to find lexical variations of search terms.
    Index creation is typically bundled into the load process. Architecturally speaking, however, this capability is decoupled and extensible.
  • Asserted Value Set Indexing – Indexes value sets separately from the source terminology
• Loaders
  Vocabularies may be imported to the system from a variety of accepted formats, including but not limited to:
  • LexGrid XML (LexBIG canonical format)
  • NCI Thesaurus, provided in Web Ontology Language format (OWL)
  • UMLS Rich Release format (RRF)
  • Open Biomedical Ontologies format (OBO)
    As with indexers, the load mechanism is designed to be extensible from an architectural standpoint. Additional loaders can be supported by the introduction of pluggable modules. Each module is implemented in the Java programming language according to a LexBIG-provided interface, and registered to the loader runtime environment.
  • Asserted Value Set Definition Loads – References a database load of an Asserted Value Set Source terminology to interpret and load value set definitions to the data model based LexGrid schema.

Metadata and Discovery Subsystem

...

In addition, each LexBIGService provides a centralized metadata index that allows registration and query of code system metadata without requiring resolution of individual CodingSchemes. This metadata index is optionally populated, typically during the vocabulary load process. The metadata index allows for the metadata of multiple code systems to be cross-indexed and searched as part of the query subsystem.

Finally, the LexBIG architecture provides the underpinnings for LexBIG services to be made accessible through the caGRID environment in the future, where vocabulary services might be deployed and discovered within a caGRID Globus container. However, this portion of the API is preliminary and awaits coordination with caBIG® Architecture WS designees to determine exact recommendations and nature of LexBIG services on the grid.

Query Subsystem

The following figure shows the Query Subsystem.

...

Lexical Set Operations provides methods to return a lists or iterators of coded entries. Supported query criteria include the application of match/filter algorithms, sorting algorithms, and property restrictions. Support is also provided to resolve the union, intersection or difference of two node sets.

...

Graph Operations support the subsetting of concepts according to relationship and distance, identification of relation related source and target concepts, and graph traversal. Additional operations include enumeration and traversal of concepts by relation, walking of directed acyclic graphs (DAGs), enumeration of source and target concepts for a relation, and enumeration of relations for a concept.

...

History provides vocabulary-specific information about concept insertions, modifications, splits, merges, and retirements when supplied by the content provider.

...

LexEVS CTS 2

...

Services

Overview

The CTS2 standard is actually a collection of relatively small standards and a set of assembly rules.

The goal of the CTS2 standard is distribution and federation:

Key is that implementations can be shared – No one organization has to offer all services

Standards

XML Schema

• used for REST, SOAP and JSON formatting
• Includes item / directory / list

Functions

• Read
• Query
• Update
• Import and Export content

Component Standards

• Code System
• Code System Version
• Entity Description
• Association
• Value Set
• Value Set Definition
• Resolved Value Set
• Map
• Map Version
• Map Entry
• Concept Domain
• Concept Domain Binding
• Statement

Assembly Rules

LexEVS CTS2 Documentation

LexEVS 6.x REST CTS2 Service

Structure of the Preliminary CTS 2 Service

The CTS 2 specification defines several functional profiles which are a focused subset of the functionality of a CTS 2 implementation. Functional profiles are defined to subset a group of operations which must be supported in order to claim conformance to the profile.

The following functional profiles are addressed by LexEVS 6.x:

Terminology Query Profile

• Searching and querying terminologies
• Provide access to terminology content and representational structures (description logic) consistent with the terminology author's intent.

Terminology Administration Profile

• Restricting administrative access
• Obtaining and loading terminologies
• Maintaining terminology access
• Control Content Access

Terminology Authoring Profile

• Functional terminology analysis/query
• Direct terminology edits

Each profile specifies the minimal functional coverage required to be provided by LexEVS.

Image Removed

• CTS2 Functional profiles: Query Profile, Administration Profile, and Authoring Profile.
• CTS2 STM Specific Methods: LexEVS Local Runtime, Distributed LexEVS, and LexEVS Analytical Grid.

LexEVS CTS 2 Services

The following figure shows the LexEVS CTS 2 Services.

Image Removed

LexEVS CTS 2 Services can be further categorized from the above profile details to include:

• Administrative Services
• Versioning Services
• Authoring Services
• Searching and Querying Services
• Association and Mapping Services
• Value Set and Pick List Definition Services

LexEVS CTS 2 API Architecture

The LexEVS CTS 2 API provides programmatic access to LexEVS 6.x implementation of the preliminary CTS 2 features and services. (For LexEVS 6.1 Query mechanisms have been implemented in REST based architecture and are fully compatible with the CTS2 Normative specification.  This deprecates the work against the preliminary CTS2 SFM, but useful features for authoring and versioning have been left in place.)

Documentation can be found here LexEVS 6.0 CTS2 API

LexEVS API/Grid Service Interaction

See the LexEVS 4.2 Grid Service Design and Implementation.

...