Challenge ended and we turned off the GPU. The MIDRC challenge organizer are pleased and will use us in the future.
Details about more metrics in the scoring program have been discussed. They want a way to convey how confident we are in the results and performance of the algorithm.
CodaLab (Open Source Team):
The team has created a new dev VM for me and I have deployed Codabench and am in the process of identifying all challenge examples and testing all possible combos of submissions to see what is and is not working.
I'm them going to try some old challenges from MedICI to see how well they work.
We have run the first submission from a participant and are tracking the leaderboard in a spreadsheet. An email account was created for the challenge to correspond with the participants.
Development is ongoing regarding integrating the scoring program and finishing the FL workflow automation.
Planned for next month
Federated Learning Challenge:
Finish automation workflow and get participant submissions to run through MedICI
Finish docker upload email notification
Debug the sign up bug we have where the website hangs
CodaLab (Open Source Team):
Identify all challenges\submissions that don't work as expected
Check in any code changes as needed to fix issues found