Services specifications developed by NCI and the community have to be testable to ensure that the implementation conforms to the specification. Conformance testing leverages the artifact and service registries along with predefined reasoning systems to validate that an implementation adequately addresses the requirements stated in the service specification. An example of service requirement is the ability to specify a response time in the specification (design time) and validate that this response time is adequate for an implementation of the service. Additional test points include but are not limited to binding to specific terminologies and domain models.
Conformance testing allows both CBIIT and other HL7 SAIF adopters to validate specifications as follows:
Analyze a given artifact for its stated ECCF purpose. Determine if a given artifact satisfies the requirements of the ECCF artifact that it declares itself to be. This analysis should look at such things as datatypes matching the appropriate level (abstract data types in a Platform Specific Model (PSM)).
Analyze a given artifact to verify traceability. Determine if a given artifact provides correct traceability from level to level. The analysis should look at naming conventions and stereotypes to determine correctness along with promotion of data types from different levels of abstraction.
Analysis of accessibility and interoperability. Used to determine if a given service matches its proposed service specification. Also determine if an artifact or specification is complete as it relates to data binding and value set binding.
Conformance Testing Functions include the ability to:
- Analyze an artifact for ECCF Conformance and traceability
- Produce a non-conformance statement
- Interact with governance systems
The requirements listed above are derived from the following use cases:
CBIIT's adoption of ECCF: ECCF requires all specification developers to make conformance statements; the conformance testing framework leverages these conformance statements to generate validation tests.
Other national initiatives: Other national organizations like NIST are adopting a similar approach to conformance testing.
