NIH | National Cancer Institute | NCI Wiki  

Versions Compared

Old Version 4

changes.mady.by.user Unknown User (bearceba)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (bearceba)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Activities completed this month

  • MIDRC is up and running and challenge details are being ironed out: https://midrc-medici-competitions.eastus.cloudapp.azure.com/
    • Challenge ended and we turned off the GPU. The MIDRC challenge organizer are pleased and will use us in the future. (smile)
    • Details about more metrics in the scoring program have been discussed. They want a way to convey how confident we are in the results and performance of the algorithmTraining Phase underway and participants are submitting the provided template to practice submitting docker images.
  • CodaLab (Open Source Team):
    • The team has created a new dev VM for me and I have deployed Codabench and am in the process of identifying all challenge examples and testing all possible combos of submissions to see what is and is not working. 
    • I'm them going to try some old challenges from MedICI to see how well they workWrote some notes for them regarding various demos we get from Tyler Thomas, a contract worker for CodaLab.
  • Federated learning challenge:
    • Temp site is up to furnish challenge dataParticipants continue to register here: https://medici-codalab-main.eastus.cloudapp.azure.com/competitions/24
    • Their security scans said python2 and python3 have vulnerabilities. They are asking me to not use these libraries or delete the files from these libraries:
      • ACR requested the scan of MedICI docker containers for use in their system.
    • Image Removed
    • New Infrastructure Diagram:
    • Image previewImage Removed
    • Image Removed
  • SIRPRISE Challenge
    • Some participants are still making submissions and Diane Alvarez is tending to them. She seems to have it handled.
    • We have run the first submission from a participant and are tracking the leaderboard in a spreadsheet. An email account was created for the challenge to correspond with the participants.
    • Development is ongoing regarding integrating the scoring program and finishing the FL workflow automation.
Planned for next month
  • Federated Learning Challenge: 
    • Finish automation workflow and get participant submissions to run through MedICI
    • Finish docker upload email notification
    • Debug the sign up bug we have where the website hangs
Planned for next month
  • MIDRC Challenge 0
    • UPDATE (3/31/2022): Infrastructure is built and now we are editing the template challenge to be the real challenge. 
    • UPDATE (4/29/2022): Training phase is underway and participants are testing submissions.
  • Federated Learning Challenge: 
    • Still need VPN access to environment.
    • Kendall Schmidt (from ACR) and Holger Roth (from NVIDIA) are creating a bash script that kicks off a participant container. 
    • I will be in charge of orchestrating the start of that script from the MedICI website.
  • CodaLab (Open Source Team): 
    • I need to draft a SOW to work with the team and become a contract worker with them
    • Launch local version of CodaBench and see if we can get some old challenges to run.
Paused
    • Identify all challenges\submissions that don't work as expected
    • Check in any code changes  as needed to fix issues found
Continue getting CodaLab running on NCI's network and VMs:
  • UPDATE (01_31_2022): May have a work around with a storage service called MINIO.
    • UPDATE (04_29_2022): The CodaLab (Open Source Team) has updates to the python2 → python3 update branch on github and I think it will work on NCI's network. However I think it is better to spend time working on CodaBench to get the new version working so that we aren't constantly behind when serving challenges and integrating with new software. It's taking lots of time to maintain older challenges while trying to work on new code. 
    MeetingDate
    Bi-weekly Meeting #1

     

    Bi-weekly Meeting #2


    ...