Page History
...
Name | Attended |
---|---|
Jayashree Kalpathy-Cramer | Yes (10 min) |
Ulli Wagner | Yes |
Carolyn Klinger | YesNo |
Benjamin Bearce | Yes |
Currently supported challenges
Name of Challenge | Organization / Meeting | Medici instance used | Release | Release Validation data | Release Test data | Number of Participants | Closing Date | Will the challenge remain open? | Comments | SPIE-AAPM-NCI DAIR Digital Breast Tomosynthesis Lesion Detection Challenge (DBTex) |
| 1/15/2021 | 132 registrants 9 submissions | 1/25/2021 | Challenge completed. The validation phase remains open. | Deep-learning for inverse problems: sparse-view X-ray Computed Tomography image reconstruction | Emil Sidky (UChicago) | 3/17/2021 | March 31, 2021 | May 17, 2021 | 123 registrants | 5/31/2021 | 2021/04/01 In validation phase Ben & Emil are testing Data are on an FTP server. Ben set up an Azure container to download the data for two participants. | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Previously supported challenges that remain open
Name of Challenge | Organization / Meeting | Medici instance used | Release | Release Validation data | Release Test data | Number of Participants | Closing Date | Will the challenge remain open? | Comments |
---|---|---|---|---|---|---|---|---|---|
Tasks / Deliverables
Task | Due Date | Deliverable | Status | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Support of Supporting at least new two challengesthree challenges including one federated learning challenge | 8/30/2021 |
| Reopen three previously executed challenges
| 2022 | |||||||
Support organizers and participants (help desk, FAQ, training, presentations) | 8/30/2022 | ||||||||||
8/30/2021 |
| Maintain and update the website | 8/30/20212022 | ||||||||
Capturing and providing metrics on the use of the MedICI website | 8/30/ | 20212022 | |||||||||
Capturing and providing metrics on the challenges | 8/30/ | 2021Investigate the use of MedICI for federated learning challenges |
| Report |
| Investigate the integration of MedICI with Kheops |
| Report | Presentation: NICC The recordings are towards the bottom of the page. Draft: https://docs.google.com/document/d/1c20ny4UNWw4BTXhdFPmc3o0YG00toOndbklXhc40Rqc/edit The report was finalized on | Support the deployment of MedICI in NCI cloud environments | 8/30/2021 |
Discussion items
2022 | ||||
Support at least three previously executed challenges | 8/30/2022 | |||
MedICI on the NCI Cloud Two (GCP) and Firecloud | 8/30/2022 | Task was suspended per discussion on | ||
Operations and Maintenance | 8/30/2022 | |||
Discussion items
Time | Item | Who | Notes | |||||
---|---|---|---|---|---|---|---|---|
Notes | ||||||||
Action items | ||||||||
Responses to question about keeping challenges open | Unknown User (bearceba) - investigate options on how to keep challenges open, add phase or copy challenge, avoid having users add to the previous submissions SPIE-AAPM-NCI DAIR Digital Breast Tomosynthesis Lesion Detection Challenge (DBTex) - Phase 1 - Unknown User (bearceba) verify with submitters that decision of phase 2 applies to phase 1 SPIE-AAPM-NCI-DAIR & SPIE-AAPM-NCI-DAIR Phase 2 - TBD - (Nick Konz, Wed 11/24/2021 11:36 AM) "Maciej and I discussed this and an idea that we’ve been floating is to have some sort of ongoing validation phase, that would allow anyone to get validation results on their model. We would also want to have a periodic, occasional test phase for people to get test results. BMMR2 - close on as previously planned. (David Newitt, Thu 10/28/2021 7:10 PM) - link to the challenge data will be de-activated on 12/1/2021 DL Sparse - keep open (Emil Sidky, Thu 10/28/2021 11:42 PM) MICCAI 2020 - has already an open test phase ISBI 2018 - has already an open test phase AAPM RT-MAC Challenge - is open-ended | |||||||
Upcoming challenges | ||||||||
Information sharing | ||||||||
Monthly status reports | due on the 10th day of the following month | |||||||
Time | Item | Who | Notes | |||||
Notes | Action items | Upcoming challenges |
| Information sharing | Monthly status reports | add Monday NCI cloud email to the monthly status report | NCI MedICI installation | CodaLab on AWS, Codalab's older version is still working, and some developers check-in code. The newer version (Python 3) had the last commit in August 2020. Deploying on a different cloud platform (AWS) and within the NCI environment is difficult.|
Contract / invoice status / SOW | PIV card / GFE laptop / VPN | VPN access |
| |||||
Paper | The draft will be submitted by September 15, 2020. - Not received | |||||||
Videos |
| |||||||
Federated learning challenge - proposal for MICCAI, ACR confirmed that the data can be used Wagner, Ulrike (NIH/NCI) [C] Change SOW, take cloud out, put federated learning modifications for Codalab in, send to Medici for review | Strategy meeting | |||||||