Page History

...

Background and Objectives

The objective of this project is to provide sufficient information about AIM 4.0 and guidelines for extending AIM foundation model for other imaging domains.

Modern medical images contain vast amounts of information captured in standard DICOM format. While this information may include metadata about the image, such as how or when the image was acquired, the majority of image information remains in pixel data. This data contains rich content that is neither explicit nor easily accessible by computer programs. The information about how images are perceived by human or machine observers is not currently captured in a form that is directly tied to the images in a structured manner. A wealth of data pertaining to image content is thus segregated from the images, limiting the value of radiologic images for use with other non-imaging data, such as cancer clinical trials.

...

The AIM Foundation model was created to respond to the above requirements. The model had been designed for extensibility by other image researchers to capture information not available in the Foundation model. The AIM 4.0 model is an extension model from the Foundation model. It stores lesion observation information for radiology and oncology.The objective of this project is to provide sufficient information about AIM 4.0 and guidelines for extending AIM foundation model for other imaging domains.

AIM Basic Concepts

An image annotation can be explanatory or descriptive information, generated by humans or machines, directly related to the content of a referenced image or images. It describes information about the meaning of pixel information in images. Annotations become a collection of image semantic content that can be used for data mining purposes. An image markup is a graphical symbol or textual description associated with an image. Markups can be used to visually depict textual information and the region-of-interest next to an image.

AIM provides a standard way to store annotation information in a structured manner with standard terminology such as RadLex, SONMED CT, etc. Searching the majority of AIM data can be done using coded terms stored in an AIM instance. AIM data can be stored as AIM DICOM Structured Reporting (SR) objects, AIM XML documents, and AIM HL7 CDA documents.

What is an AIM annotation?

An AIM annotation is a collection of associated image annotations and markups. An AIM annotation may contain one or more image annotation or annotation of annotation instances. An AIM annotation can only be of either type image annotation or annotation of annotation. These are the two kinds of annotations that can be created. ImageAnnotation class annotates images. AnnotationOfAnnotation class annotates other AIM annotations (both image annotation and annotation of annotation) for comparison and reference purposes.

In the AIM 3.0 model, an image annotation or annotation of annotation is stored as a single file, either as an AIM DICOM SR or AIM XML document. When AIM was used in Pathology, hundreds of thousands of AIM files were created for a single study. Managing AIM files from different image studies became very complicated. A collection described in the next section was used to store AIM instances from the same imaging study.
The next section describes changes from AIM version 3.0 to the AIM Foundation model.

From AIM 3.0 to the AIM Foundation Model

Based on the new requirements in the background and objective section, the development team modified and extended the AIM 3.0 model [2] to create the AIM Foundation model. The Foundation model extends the model to support other imaging disciplines. This section provides a real-world example how of the Foundation model was created.

The most significant change from the AIM 3.0 to the AIM Foundation model is the use of ISO 21090 data types and the introduction of AIM Statements. An AIM Statement describes a relationship between the subject and object entities in the AIM Foundation model. As such, this is a generic and very expressive way to capture a broad range of information AIM annotations need to capture. AIM statements are a core aspect of the design of AIM that will permit it to be extended in future to meet the needs of other domains.

Why Use AIM Statements and Other Enhancements?

The AIM 3.0 model reflects relationships between classes using containment of a source class to a target class and inheritance or IS-A relationships. The expressive power of the model is limited by these two types of relationships. No other types of relationships in the AIM 3.0 model are possible and not all necessary relationships are present. For instance, there is no direct relationship between instances of the AnatomicEntity class, e.g. right-upper lobe of lung, and the ImagingObservation class, e.g. mass. Such classes can be indirectly linked to each other only via the Annotation class.

...

As the AIM model is used by an increasing number of users, additional requests will arise that add relationships between classes or create new classes to store other important information related to AIM. Managing and rearranging associations with classes will be too complex to manage without the AIM Statement class and its subclasses.

Fundamental Changes from AIM 3.0 to the Foundation Model

The foundation has seventy new classes. We renamed twelve classes and deleted four classes. The detailed AIM model change logs are at https://wiki.nci.nih.gov/x/N4FSBg. This section describes changes from the AIM 3.0 to AIM Foundation model.

...

The model is used to capture image annotation and markup information relevant to images. It describes explicitly what kind of information the model can collect. An annotation describes information about the meaning of pixel information in images. It is captured as a coded term supplied by medical lexicons (e.g. SNOMED CT®, RadLex, LOINC, etc.) or user-defined terms. Annotations become a collection of image semantic content that can be used for data mining purposes. An AIM annotation is a collection of associated annotations, markups, calculation results that may or may not be directly related to a markup, a well-defined role of an annotation (e.g. baseline or follow-up), workflow activity, simple subject demographics, and creator and tools used to create AIM instances. This information is used to design and create AIM information model.

AIM UML Modeling

The AIM UML model illustrated as a UML class diagram is used to capture information about how images are perceived by human or machine observers. Our design process started with understanding the initial requirements [2] and the information in "From AIM 3.0 to the AIM Foundation Model," on page . We identified a set of information objects that are used to collect information about imaging annotations and markup. Classes are divided into image semantic content, calculation, markup, image reference, and AIM statements. If there are classes that do not pertain to a specific group, we classify them in the general information group. These classes contain information about the workstation used to create the AIM annotations, the user that creates the AIM annotations, patient identification, DICOM segmentation, annotation role, inference, workflow activity, adjudication observation, image annotation, and annotation-of-annotation.
The AIM Foundation Model, shown in figure 1, has evolved through an iterative feedback process since the release of AIM version 3, revision 11. The model has gone through many reviews and recommendation processes. Enterprise Architect can be used to view the AIM UML class diagram file, AIM_Foundation_v4_rv47_load.eap. One can also view this diagram in JPEG format. You can download the model from the NCI Wiki.

...

Note that abstract classes in the AIM schema are AnnotationCollection, AnnotationStatement, ImageAnnotationStatement, AnnotationOfAnnotationStatement, AnnotationEntity, CharacteristicQuantification, CalculationResult, Entity, GeometricShapeEntity, ImageReferenceEntity, TwoDimensionGeometricShapeEntity, ThreeDimensionGeometricShapeEntity, MarkupEntity, SegmentationEntity and SpatialCoordinate. [0..1] denotes an optional occurrence of an attribute.

How to Create and Use AIM Annotations

AIM annotations can only contain either one or more image annotation or annotation-of-annotation type. An image annotation is represented by the ImageAnnotation class. The class is used to annotate images. Annotation-of-annotation is represented by AnnotationOfAnnotation class. The class is used to annotate other AIM annotations for comparison and reference purposes. Each class inherits all properties of the abstract class, AnnotationEntity.
There are two root classes that can contain either ImageAnnotation or AnnotationOfAnnotation; namely ImageAnnotationCollection and AnnotationOfAnnotationCollection respectively. These are the two kinds of annotation collections that can be instantiated. An instance of ImageAnnotationCollection class contains one or more instances of related ImageAnnotation class. An instance of AnnotationOfAnnotation class may contain a collection of instances of ImageAnnotation and/or AnnotationOfAnnotation class.

...

The previous section describes the AIM foundation in detail. This section provides information used to extend the AIM foundation model to store other information that is not currently available in the foundation model. As described in sections 4 and 5, the AIM information model uses UML to depict the type of information it is currently capable of collecting. The model explicitly describes how image annotation semantic contents can be stored. A class name in the model was intentionally created to inform a reader of what information is stored in the class. Each attribute in a class supports the class purpose. A relationship between classes indicates how one class works with other classes to form and capture a larger concept and information.

Guidelines to the AIM Foundation Model

This section provides a set of guidelines to extend the AIM Foundation Model. The foundation model conceptually orders classes in the AIM UML model into six groups: AIM Statements, General Information, Calculation, Image Semantic Content (Finding), Image References, and Markup. You may extend the foundation model to fit your own needs by following the criteria. If you want your additions to be included in the official NCI AIM model, please enter your request directly at https://tracker.nci.nih.gov/browse/AIM.

...

Association name of a target class, figure 12, has the same name as the target class name with the first character being a lower case.

Figure 12. An Association Name of a Target Class

• A source class may have or contain 1-to-0..* (zero-to-many) or 1-to-1..* (one-to-many) associations to a target class. The target association name must append "Collection", figure 13, at the end of the class name.

Figure 13. An Example of Collection

1. It must be possible to map a new class to DICOM SR data element(s). See "AIM DICOM SR templates" for more information. It is important for a user to have knowledge about DICOM SR and how to modify a DICOM SR template properly because new classes added to the model will need to be stored in DICOM format.
2. Use ISO 21090 data type for every attribute in a class. Using ISO 21090 data type provides a good foundation to convert AIM information to HL7 Clinical Document Architecture (CDA).
3. A name of any class extending from the Entity class must end with the word "Entity". A class derived from the Entity class represents an existence of a thing or concept that is not currently captured in the AIM foundation model. A new entity class can also be included if a user wants to explicitly express what information the new model will be able to store.
4. A class that can be used to construct an AIM statement must extend or inherit from the Entity class.
   1. There must be a subject and object class.
   2. A predicate must be selected from a list of existing predicate from section 4.b.4. You may use your own predicates.
   3. A name of an AIM statement shall be a concatenation between the name of a subject class, predicate and the name of object class.
5. AIM markup is modeled after DICOM 2D and 3D spatial coordinate geometries [2]. New additional markups must be able to be stored in DICOM SR format.

Extending AIM Foundation Model to AIM 4.0

AIM 4.0 model, Figure 14, is an official extension of the AIM foundation model. This extension explicitly captures lesion results and measurements derived during image-based clinical trials [7,8]. There are nine new classes that cover lesion annotation needs. Three classes were created for lesion observation as follows.

...

The AIM software library is a collection of C++ application programming [10] interfaces used to construct the AIM information model based on a UML class diagram. ANIVATR is a referenced implementation of the AIM library. The ANIVATR software application validates AIM annotations and transcodes them into different artifacts, namely native AIM XML and DICOM SR. We have developed AIM annotations in DICOM SR such that they can be created and displayed in a variety of medical imaging workstations, notably the AIM on ClearCanvas workstation and eXtensible Imaging Platform (XIP) as well as in clinical imaging devices. ANIVATR reads and transcodes DICOM SR into an AIM XML representation and vice versa.

AIM Library

The AIM library is a C++ [9] module. It consists of two logical parts: implementation of the AIM Schema as an object model and definition of a set of operations, which can be performed on the object model.

...

The object model implementation creates an ANSI C++ class for each class in the AIM Schema. The Class hierarchy closely follows the AIM Schema. Each object's model class provides mutation methods (Set and Get method) for every attribute in the corresponding AIM Schema class. All changes to the class states are done via those mutation methods. The whole AIM Schema is represented by the object model through containment and inheritance.
The set of object model operations supported by the AIM library includes persisting the model in XML and DICOM SR formats. The reverse set of operations of reading XML and DICOM SR instances into the object model is supported as well.

Using AIM Library

To instantiate an AIM model, a software developer should start with creating either an AnnotationOfAnnotationCollection or ImageAnnotationCollection object and populating its related objects' content. All required attributes need to be populated with valid data. Optional attributes are depicted in the AIM schema with [0..1].

ImageAnnotation

The ImageAnnotationCollection object is required to have at least one ImageAnnotation object of type DICOMImageReferenceEntity or WebImageReferenceEntity. The DICOMImageReferenceEntity object must have one imaging study. The ImageStudy object may have one series objects. Each ImageSeries object may have one or more Image objects. It is implied in the model that all images originate from the same study of the same patient.
ImageAnnotationCollection object may have a Person object. An ImageAnnotation object may have DicomSegmentationEntity objects, which contain references to its own instance UID and referenced instance UID of the image to which the segmentation is applied. It also has the segmentation type, DICOM SOP class, UID, and an identification number of the segment. The identification of the segment shall be unique within the segmentation instance in which it is created. ImagingObservationEntity object is being captured as DICOM code sequence with a possible textual comment. An ImagingObservationEntity may have zero or more ImagingObservationEntityCharacteristic objects, which are captured as DICOM code sequences. An ImageAnnotationEntity may store conclusions derived by interpreting an imaging study and/or medical history in a collection of InferenceEntity objects that store the information as code sequence based on a controlled terminology.

...

A CalculationEntity object can be related to a single markup or to a collection of markups and other calculations, which are not related to markup. A calculation may reference other calculations by using CalculationEntityReferencesCalculationEntity and CalculationEntityUsesCalculationEntity objects, which contain a referenced CalculationEntity object UID. A CalculationEntity object may have zero or more CalculationResult objects. It is possible for a Calculation to have no CalculationResult. This means that the information provided in the CalculationEntity object is sufficient to describe the calculation.
A calculation result can be a scalar, vector, matrix, histogram, or array. Dimensionality of calculation results is represented by Dimension objects. A CalculationResult object must have at least one Dimension object. The Index attribute in the Dimension object is a zero based unique index of the dimension. The Size attribute in the Dimension object specifies how many members a dimension has. Label attribute provides textual meaning to a dimension.
A CalculationResult object may have zero or more Data objects. The absence of any Data object means that result is an empty set. Each Coordinate object specifies a dimension index and a position within the dimension. The number of Coordinate objects for each Data object cannot exceed the total number of Dimension objects in a CalculationResult. A Data object cannot have more than one Coordinate object with the same dimensionIndex.

AnnotationOfAnnotation

The AnnotationOfAnnotationCollection object is required to have at least one AnnotationOfAnnotation object. AnnotationOfAnnotation works very much the same way as ImageAnnotation for calculation group and image semantic content group (see section 7.c). The AnnotationOfAnnotation object must have at least one AIM statements that contains a UID of ImageAnnotation or AnnotationOfAnnotation object. AnnotationOfAnnotation may store a conclusions derived by interpreting an imaging study and/or medical history in a collection of Inference object, which stores the information as a code sequence based on a controlled terminology.
AnnotationOfAnnotation may refer to a collection of ImageAnnotation objects that can come from different studies.

The AIM model and DICOM templates do not explicitly address the issue of Study Instance UID, Series Instance UID and SOP Instance UID creation of an AnnotationOfAnnotation object. These three UIDs can be generated by AIM implementers for the purpose of creating an AIM DICOM object. When an AIM DICOM object is transformed to AIM XML or HL7 CDA, these three UIDs are not being used.

AIM Library Objects

AIM library is created using C#. The library objects and operations are in the aim_lib namespace. All object model files are located in the model/ sub-directory. For convenience purposes, there is a model/AimHeaders.h header file that includes all headers of the object model. Below is a sample of how one can populate parts of the object model.
#include "AIMLib/model/AimHeaders.h"
…………
// Calculation and its dependents
aim_lib::Dimension dim;
dim.SetIndex(0);
dim.SetLabel("Centimeters");
dim.SetSize(1);
aim_lib::DimensionVector dimColl;
dimColl.push_back(dim);
aim_lib::CalculationResult calcResult;
calcResult.SetUnitOfMeasure("cm");
calcResult.SetType("CalculationResultTYpe::Scalar");
calcResult.SetNumberOfDimensions(1);
calcResult.SetDimensionCollection(dimColl);
aim_lib::Coordinate coordinate;
coordinate.SetDimensionIndex(0);
coordinate.SetPosition(0);
aim_lib::CoordinateVector coordColl;
coordColl.push_back(coordinate);
aim_lib::Data data;
data.SetValue(150.0);
data.SetCoordinateCollection(coordColl);
aim_lib::DataVector dataColl;
dataColl.push_back(data);
calcResult.SetDataCollection(dataColl);
aim_lib::CalcResultVector calcResults;
calcResults.push_back(calcResult);
aim_lib::ReferencedCalculation refCalc;
refCalc.SetReferencedCalculationUID("1.23.5698.24546.231365.74654");
aim_lib::ReferencedCalcVector refCalcs;
refCalcs.push_back(refCalc);
aim_lib::Calculation calc;
calc.SetUID(AimUidGenerator::GenerateNewUID("33.333"));
calc.SetCodeValue("Value::Length");
calc.SetCodeMeaning("length");
calc.SetCodingSchemeDesignator("CALC_SCHEME");
calc.SetDescription("Description of the Calculation One - Length");
calc.SetMathML("MathML for the Calc One goes here");
calc.SetCalculationResultCollection(calcResults);
calc.SetReferencedCalculationCollection(refCalcs);
calculations.push_back(calc);

AIM Library Operations

Operations provided by the AIM library reside in the operations/ folder of the library. All operations are performed on the object model via DCMModel and XMLModel classes.
AIMLib/operations/DCMModel.h contains operations for DICOM SR.

...

1. WriteAnnotationToFile / WriteAnnotationsToFile will write AIM library object(s) to file.
2. WriteAnnotationToString / WriteAnnotationsToString will write AIM library object(s) to a string buffer.

Environment Configuration

The AIM development environment on Windows systems requires a few configurations. An example of how an environment may be set up follows.
Environment variables:

JAVA_HOME=c:\jdk6
BOOST_ROOT=C:\Program Files\boost\boost_1_34_1

Sample Code

The AIMTestLib project contains AIMLibTest/AIMLibTest.cpp, which shows examples of how to use AIM Library for generating/reading/writing AIM library objects and files.

References

1. Channin, D. S., Mongkolwat, P., Kleper, V., Sepukar, K. Rubin, D. L., The caBIG^TM Annotation and Image Markup Project. Journal of Digital Imaging, Vol 23, No. 2, April, 2010.
2. Annotation and Image Markup Version 3 Project: Requirements, Design, Implementation and Usage; September 2010, URL: https://ncisvn.nci.nih.gov/svn/files/trunk/aim/aim/AIMToolkit3.0.2/AIMToolkit_v3.0.2_rv11.rar Access 2013-09-30.
3. Rubin, D. L., Mongkolwat, P., Kleper, V., Supekar, K., Channin, D. S., Medical Imaging on the Semantic Web: Annotation and Image Markup, Association for the Advancement of Artificial Intelligence, 2008 Spring Symposium Series, Stanford, CA, 2008
4. Channin D. S., Mongkolwat P., Kleper V, Rubin D. L., The Annotation and Image Mark-up Project: Radiology. 2009; 253:590–592
5. Clunie, D. A., DICOM Structured Reporting, PixelMed Publishing, 2000.
6. DICOM 2011. Digital Imaging and Communications in Medicine (DICOM) [online] URL: http://medical.nema.org/standard.html Accessed 2013-09-30.
7. Clunie, D. A., DICOM Structured Reporting and Cancer Clinical Trials Results. Cancer Informatics, pp. 33-56, 2007.
8. Clunie, D. A., Clinical Trials Results Reporting, unpublished DICOM supplemental from Working Group 18, http://www.dclunie.com Access 2013-09-30.
9. C++: URL: {+}http://www.cplusplus.com/+ Access 2013-09-30.
10. AIM Toolkit: URL: https://wiki.nci.nih.gov/display/AIM/Annotation+and+Image+Markup+-+AIM#AnnotationandImageMarkup-AIM-AIMToolkit Access 2013-09-30.