2022-05 Monthly Status Report

Google Analytics Report:

Activities completed this month

MIDRC is up and running and challenge details are being ironed out: https://midrc-medici-competitions.eastus.cloudapp.azure.com/
- Training Phase underway and participants are submitting the provided template to practice submitting docker images.
CodaLab (Open Source Team):
- I drafted an SOW and they have accepted. Our focus is going to be to have me work on Codabench which is the next generation of Codalab.
- I've launched a local version of Codabench and have uploaded some of our old competitions from MedICI (DL-Sparse). It uploads but it looks like submissions fail. Investigating now.
Federated learning challenge:
- Temp site is up to furnish challenge data: https://medici-codalab-main.eastus.cloudapp.azure.com/competitions/24
- Their security scans said python2 and python3 have vulnerabilities. They are asking me to not use these libraries or delete the files from these libraries:
  - ACR requested the scan of MedICI docker containers for use in their system.
- New Infrastructure Diagram:
True-CT Challenge
- Challenge ended this month and organizer have chosen to keep the leaderboard private.
DL-SPARSE Challenge
- Test phase took pace this month and is now over.
- Leaderboard here: https://dl-sparse-view-ct-challenge.eastus.cloudapp.azure.com/competitions/3#results
SIRPRISE Challenge
- Challenge ended this month as well but some participants needed to submit after the deadline as they kept getting failures for malformed submissions.

Planned for next month

MIDRC Challenge 0
- Ends next month. We will want to review what happened and talk about what we want to change for challenge1.
Federated Learning Challenge:
- Still need VPN access to environment.
- Kendall Schmidt (from ACR) and Holger Roth (from NVIDIA) are creating a bash script that kicks off a participant container.
- I will be in charge of orchestrating the start of that script from the MedICI website.
CodaLab (Open Source Team):
- The team wants me to review documentation they have from the original developers of Codabench regarding sample challenges and new functionality and general operation of the site as a whole.
- They want to deploy a dev VM for me with Codabench launched so I can test many competitions and figure out the state of the platform and codebase.

Paused

Continue getting CodaLab running on NCI's network and VMs:
1. UPDATE (01_31_2022): May have a work around with a storage service called MINIO.
2. UPDATE (04_29_2022): The CodaLab (Open Source Team) has updates to the python2 → python3 update branch on github and I think it will work on NCI's network. However I think it is better to spend time working on CodaBench to get the new version working so that we aren't constantly behind when serving challenges and integrating with new software. It's taking lots of time to maintain older challenges while trying to work on new code.

Meeting	Date
Bi-weekly Meeting #1	28 Apr 2022
Bi-weekly Meeting #2

Milestones

Description	Date
MIDRC Challenge Underway (might get pushed to May 5th, but it's live)	30 Apr 2022

Task in Progress

Description
FL Learning Challenge deploy
Working out GPU VM worker for MIDRC

Risks

Description	Mitigation	Rank	Status	Creation Date	Realization	Close Date
Security issues with MedICI at ACR could be a potential issue restricting the use and operation of this challenge	delete or remove these libraries and hope that doesn't break anything fundamental or crucial to MedICI	1	ongoing
Having trouble activating certain azure functionality in ACR's environment. I can't connect to the internet when on their VPN and certain sign in work flows require me to sign in via a browser and so I can't hook MedICI up to it's normal resources yet.