MIDRC - Challenge1 planning is starting. We received the new data and are prepping the annotation app for receiving images. This is a COVID detection challenge and we are using chest X-RAYs. Further analysis is needed to determine the images we will use.
Submissions are automated in general but there are hiccups that we deal with manually. I spend about 3 hours a day on this and it eats a lot of time as the platform isn’t exactly stable. The submissions sometimes fail and we don’t know why and I have to manually change their status to pass in the database and created the detailed results that were supposed to be created. I can hack the system to do what I need but it is taking up a lot of time. We need to consider a more stable platform for the future.
Development is ongoing regarding integrating the scoring program and finishing the FL workflow automation.
Finish docker upload email notification.
Planned for next month
Federated Learning Challenge:
Challenge will end next month and we need to send some emails to participants reminding them that the queue is getting longer by the day and that a week before the challenge ends, there may not be time for new submissions to run. We need to tell them to queue their best submission before Aug 31st.
CodaLab (Open Source Team):
Continue working on GPU submissions with the compute worker.
Potentially test other challenges we have run.
Help get automated tests to pass maybe…stretch goal honestly.
Check in any code changes as needed to fix issues found.
Discuss Milo Fryling’s request to run another challenge. Direct him to the google form.
Jayashree is opposed but I think we should consider testing out a challenge on the new codabench platform.
MIDRC:
Data analysis needs to be done to identify cases we can use for challenge…ongoing.