2022-07 Monthly Status Report

Google Analytics Report:

Activities completed this month

MIDRC - Challenge1 planning is starting. We received the new data and are prepping the annotation app for receiving images. This is a COVID detection challenge and we are using chest X-RAYs. Further analysis is needed to determine the images we will use.
CodaLab (Open Source Team):
- https://github.com/codalab/codabench/pull/704

Deployed all Codabench and Codalab sample challenges on test platform. They work and necessary github pull request was generated:
In addition I was able to launch the Dl-Sparse Challege (https://dl-sparse-view-ct-challenge.eastus.cloudapp.azure.com/competitions/3) on the test platform and was able to get results to show (https://codabench-test.lri.fr/competitions/56/).
We are in the process of getting a GPU VM setup to test GPU challenges. Pending…

Federated learning challenge:
- UPDATE: This is done. All is integrated.
- UPDATE: Done

Participants were instructed to use the actual challenge site located here: https://breastdensityfl.acr.org/competitions/1.
Submissions are automated in general but there are hiccups that we deal with manually. I spend about 3 hours a day on this and it eats a lot of time as the platform isn’t exactly stable. The submissions sometimes fail and we don’t know why and I have to manually change their status to pass in the database and created the detailed results that were supposed to be created. I can hack the system to do what I need but it is taking up a lot of time. We need to consider a more stable platform for the future.
Development is ongoing regarding integrating the scoring program and finishing the FL workflow automation.
Finish docker upload email notification.

Planned for next month

Federated Learning Challenge:
- Challenge will end next month and we need to send some emails to participants reminding them that the queue is getting longer by the day and that a week before the challenge ends, there may not be time for new submissions to run. We need to tell them to queue their best submission before Aug 31st.
CodaLab (Open Source Team):
- Continue working on GPU submissions with the compute worker.
- Potentially test other challenges we have run.
- Help get automated tests to pass maybe…stretch goal honestly.
- Check in any code changes as needed to fix issues found.
Discuss Milo Fryling’s request to run another challenge. Direct him to the google form.
- Jayashree is opposed but I think we should consider testing out a challenge on the new codabench platform.
MIDRC:
- Data analysis needs to be done to identify cases we can use for challenge…ongoing.

Meeting	Date
Bi-weekly Meeting #1	07 Jul 2022
Bi-weekly Meeting #2	21 Jul 2022