 |
Page History
...
Challenges are being increasingly viewed as a mechanism to foster advances in a number of fields, including healthcare and medicine. Large quantities of publicly available data, such as that in TCIA, and cultural changes in the openness of science have now made it possible to use these challenges, as well as crowdsourcing (enlisting the services of people via the Internet), to propel the field forward.
...
Over the last few years, “Grand Challenges” have become popular in several imaging-based research communities. A “Grand Challenge” Grand Challenge is designed to validate and compare imaging analysis algorithms. The algorithms are applied to a single dataset and the results for each algorithm are compared against a previously-determined ground-truth dataset.
The Pilot Challenges sub-project of CTIIP will make a set of integrated data from TCIA and TCGA publicly available to researchers who will participate in three complementary "sponsor complementary Pilot Challenge " projects. (this only happened in the first challenge to figure out which image was from which tumor–look at Miccai ) As opposed to a more rigorous "grand" challengeGrand Challenge, each pilot challenge will function as a proof of concept to learn how to scale challenges up in the future.
A team from Massachusetts General Hospital will guide the pilot challenges, using the Medical Imaging Challenge Infrastructure (MedICI), a system that supports medical imaging challenges.
The proposed work fits into the above-described environment by developing and executing infrastructure that can be used to run Challenges on a smaller scale with data sets of reduced size and will demonstrate the infrastructure as capable of running Grand Challenges.
The MedICI system will utilize the CodaLab framework, a newly developed, open-source challenge platform developed by Microsoft Research and others in the medical imaging and machine learning communities. Because CodaLab does not have built-in imaging handling, display or annotation capabilities, we will build on two application packages, ePad and caMicroscope, to provide those features.
Challenges or sometimes also called Grand Challenges are competitions to determine best computational algorithms in our case image analysis algorithms. These challenges are often conducted in conjunction with scientific conferences. There are several platforms to host these challenges. Jayashree Kalpathy-Cramer’s team is setting up Medici for our CTIIP pilot challenges (and hopefully others after that).
This challenge is part of the Computational Brain Tumor Cluster of Event (CBTC) 2015 which will be held on Oct 9 in Munich, Germany, in conjunction with MICCAI 2015. It will consist of a morning workshop and afternoon challenges.
The pilot challenges are as follows:
| MICCAI 2015 Challenges | Description |
|---|---|
| Combined Radiology and Pathology Classification | The datasets for this challenge are Radiology and Pathology images obtained from the same patients. Each case corresponds to a single patient. There is one Radiology image and one whole slide tissue image for each case. The training set contains a total of 32 cases: 16 cases that are classified by pathologists as Oligodendroglioma and 16 cases classified as Astrocytoma. The test set will have 20 cases. Please note that the number of cases in the test set may not be equally partitioned between the two sub-types. |
| Segmentation of Nuclei in Pathology Images | The characteristics of cancer nuclei are central components in many aspects of pathology classification and nuclear features, combined with “omics” have been shown by many research groups to be linked to patient outcome. Although there are many important and predictive features, characteristics of cancer nuclei are overall the most important. Virtually without exception, all commercial systems and academic groups in the digital Pathology area make use of nuclear segmentation algorithms. Given this, ability to segment and then classify nuclei is a key task. The goal of this challenge is to evaluate the performance of algorithms for detection and segmentation of nuclear material in a tissue image. Participants are asked to detect and segment all the nuclei in a given set of image tiles extracted from whole slide tissue images. The algorithm results will be compared with consensus pathologist segmented sub regions. Winners will be ranked based on their nuclei segmentation best matching the reference standards. The reference standard will be pathologist generated nuclear segmentation on select regions of TCGA Glioma whole slide images for the challenge. |
Comparing Algorithms to Ground Truth
...
Pilot Challenge will involve data sets of reduced size and demonstrate the infrastructure as capable of running Grand Challenges in the future.
Challenges are often conducted in conjunction with scientific conferences. The following Pilot Challenges, supported by the CTIIP project and described in the following table, were part of the Computational Brain Tumor Cluster of Event (CBTC) 2015 which was held on October 9 in Munich, Germany, in conjunction with MICCAI 2015.
| MICCAI 2015 Challenges | Description |
|---|---|
| Combined Radiology and Pathology Classification | The datasets for this challenge are Radiology and Pathology images obtained from the same patients. Each case corresponds to a single patient. There is one Radiology image and one whole slide tissue image for each case. The training set contains a total of 32 cases: 16 cases that are classified by pathologists as Oligodendroglioma and 16 cases classified as Astrocytoma. The test set will have 20 cases. Please note that the number of cases in the test set may not be equally partitioned between the two sub-types. The whole slide tissue images are stored in Aperio SVS format. There are open source tools and libraries that can read these images: OpenSlide and Bio-Formats. |
| Segmentation of Nuclei in Pathology Images | The characteristics of cancer nuclei are central components in many aspects of pathology classification. Nuclear features, when combined with “omics,” have been shown by many research groups to be linked to patient outcome. Although there are many important and predictive features, characteristics of cancer nuclei are the most important. Virtually without exception, all commercial systems and academic groups in the digital Pathology area make use of nuclear segmentation algorithms. The ability to segment and then classify nuclei is therefore a key task. The goal of this challenge is to evaluate the performance of algorithms for detection and segmentation of nuclear material in a tissue image. Participants are asked to detect and segment all the nuclei in a given set of image tiles extracted from whole slide tissue images. The algorithm results will be compared with consensus pathologist segmented sub regions. Winners will be ranked based on their nuclei segmentation best matching the reference standards. The reference standard will be pathologist generated nuclear segmentation on select regions of TCGA Glioma whole slide images for the challenge. |
There are several platforms to host these challenges. Jayashree Kalpathy-Cramer’s team is setting up Medici for our CTIIP pilot challenges (and hopefully others after that).
A team from Massachusetts General Hospital will guide the pilot challenges, using the Medical Imaging Challenge Infrastructure (MedICI), a system that supports medical imaging challenges.
The MedICI system will utilize the CodaLab framework, a newly developed, open-source challenge platform developed by Microsoft Research and others in the medical imaging and machine learning communities. Because CodaLab does not have built-in imaging handling, display or annotation capabilities, we will build on two application packages, ePad and caMicroscope, to provide those features.
MedICI
Jaysharee's program: Medical Imaging Challenge Infrastructure: MedICI
...
