NIH | National Cancer Institute | NCI Wiki  

  • Created by Unknown User (bearceba), last modified on Jul 15, 2022

Google Analytics Report:

Activities completed this month

  • MIDRC is up and running and challenge details are being ironed out: https://midrc-medici-competitions.eastus.cloudapp.azure.com/
    • Training Phase underway and participants are submitting the provided template to practice submitting docker images.
    • Validation phase happens this month as well and we transition to the Test phase at the beginning of June
  • CodaLab (Open Source Team):
    • I drafted an SOW and they have accepted. Our focus is going to be to have me work on Codabench which is the next generation of Codalab.
    • I've launched a local version of Codabench and have uploaded some of our old competitions from MedICI (DL-Sparse). It uploads but it looks like submissions fail. Investigating now.
  • Federated learning challenge:
    • ACR requested the scan of MedICI docker containers for use in their system (Their security scans said python2 and python3 have vulnerabilities. They are asking me to not use these libraries or delete the files from these libraries).
    • Security is setting up a firewall and some other components like an Azure storage account as well as a docker container registry that are needed to launch the MedICI website.
    • I have VPN access now
    • Kendall Schmidt (from ACR) and Holger Roth (from NVIDIA) have created a bash script that kicks off a participant container. 
    • Working on orchestrating the start of the participant submission container from the MedICI website. I need to work through all steps in bash in order to know how to orchestrate the same steps in python.
    • Holger is working on the evaluation script that scores participant predictions and I will also work on integrating that into the website submission.
    • IP whitelisting is now going to be required for participants so that only those IPs can access the challenge site. 
    • I will also spend time tweaking the code we have for docker uploads as we need to manually build and upload the participant images to the azure container registry. This is different than how the code works now as it completely automates all steps from upload, building and pushing to the container registry. Email notifications will be needed as participants upload image archives so that we know to investigate it and build and push manually.
  • True-CT Challenge
    • Challenge ended this month and organizer have chosen to keep the leaderboard private. 
  • DL-SPARSE Challenge
  • SIRPRISE Challenge
    • Challenge ended this month as well but some participants needed to submit after the deadline as they kept getting failures for malformed submissions. 
Planned for next month
  • MIDRC Challenge 0
    • Ends next month. We will want to review what happened and talk about what we want to change for Challenge 1.
  • Federated Learning Challenge: 
    • Once security is ok with the django code in the website container we can proceed with the actual challenge building and workflow.
    • Integrate scoring program
    • Start processing submissions if possible
  • CodaLab (Open Source Team): 
    • The team wants me to review documentation they have from the original developers of Codabench regarding sample challenges and new functionality and general operation of the site as a whole. 
    • They want to deploy a dev VM for me with Codabench launched so I can test many competitions and figure out the state of the platform and codebase. 
MeetingDate
Bi-weekly Meeting #1

 



  • No labels