NIH | National Cancer Institute | NCI Wiki  

Versions Compared

Old Version 3

changes.mady.by.user Unknown User (bearceba)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (bearceba)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Google Analytics Report:

Image RemovedImage Added

Activities completed this month

  • MIDRC is up and running and challenge details are being ironed out: https://midrc-medici-competitions.eastus.cloudapp.azure.com/
    • Training Phase underway and participants are submitting the provided template to practice submitting docker images.
    • Validation phase happens this month as well and we transition to the Test phase at the beginning of June
  • CodaLab (Open Source Team):
    • I drafted an SOW and they have accepted. Our focus is going to be to have me work on Codabench which is the next generation of Codalab.
    • I've launched a local version of Codabench and have uploaded some of our old competitions from MedICI (DL-Sparse). It uploads but it looks like submissions fail. Investigating now
    • Wrote some notes for them regarding various demos we get from Tyler Thomas, a contract worker for CodaLab.
  • Federated learning challenge:
    • Temp site is up to furnish challenge data: https://medici-codalab-main.eastus.cloudapp.azure.com/competitions/24
    • ACR requested the scan of MedICI docker containers for use in their system (Their security scans said python2 and python3 have vulnerabilities. They are asking me to not use these libraries or delete the files from these libraries:ACR requested the scan of MedICI docker containers for use in their system).
      • Image Modified
    • New Infrastructure Diagram:
    • Image previewImage Removed
  • True-CT Challenge
    • There were some upload data issues where participants were getting permission denied when trying to sign into their sftp file shares.
    • Otherwise things are good and we have pushed the phase to submit back to May 31st.
  • DL-SPARSE Challenge
    • We are in the validation phase and data for that phase has been updated:
      • ground truth
      •  test set has been added to sftp server
  • SIRPRISE Challenge
    • There may be an email bug so working with Diane to iron that out. UPDATE: I believe we fixed this. Sometimes though CodaLab is blocked and emails are sent to spam. Not sure how to deal with that yet.
    • Challenge is over on the 23rd due to supply issues of some chemicals. 
Planned for next month
  • MIDRC Challenge 0
    • UPDATE (3/31/2022): Infrastructure is built and now we are editing the template challenge to be the real challenge. 
    • UPDATE (4/29/2022): Training phase is underway and participants are testing submissions.
    • Federated Learning Challenge: 
        • View file
          namemedici_iqserver_high_and_med_summary.csv
          height250
      • Security is setting up a firewall and some other components like an Azure storage account as well as a docker container registry that are needed to launch the MedICI website.
      • I have VPN access now
    Still need VPN access to environment.
      • Kendall Schmidt (from ACR) and Holger Roth (from NVIDIA)
    are creating
      • have created a bash script that kicks off a participant container. 
    I will be in charge of
      • Working on orchestrating the start of
    that script
      • the participant submission container from the MedICI website.
    CodaLab (Open Source Team): 
      • I need to
    draft a SOW to work with the team and become a contract worker with them
  • Launch local version of CodaBench and see if we can get some old challenges to run.
    • Paused
    1. Continue getting CodaLab running on NCI's network and VMs:
      1. UPDATE (01_31_2022): May have a work around with a storage service called MINIO.
      2. UPDATE (04_29_2022): The CodaLab (Open Source Team) has updates to the python2 → python3 update branch on github and I think it will work on NCI's network. However I think it is better to spend time working on CodaBench to get the new version working so that we aren't constantly behind when serving challenges and integrating with new software. It's taking lots of time to maintain older challenges while trying to work on new code. 
    MeetingDateBi-weekly Meeting #1

     

    Bi-weekly Meeting #2

    Milestones

    ...

     

      • work through all steps in bash in order to know how to orchestrate the same steps in python.
      • Holger is working on the evaluation script that scores participant predictions and I will also work on integrating that into the website submission.
      • IP whitelisting is now going to be required for participants so that only those IPs can access the challenge site. 
      • I will also spend time tweaking the code we have for docker uploads as we need to manually build and upload the participant images to the azure container registry. This is different than how the code works now as it completely automates all steps from upload, building and pushing to the container registry. Email notifications will be needed as participants upload image archives so that we know to investigate it and build and push manually.
    • True-CT Challenge
      • Challenge ended this month and organizer have chosen to keep the leaderboard private. 
    • DL-SPARSE Challenge
    • SIRPRISE Challenge
      • Challenge ended this month as well but some participants needed to submit after the deadline as they kept getting failures for malformed submissions. 
    Planned for next month
    • MIDRC Challenge 0
      • Ends next month. We will want to review what happened and talk about what we want to change for Challenge 1.
    • Federated Learning Challenge: 
      • Once security is ok with the django code in the website container we can proceed with the actual challenge building and workflow.
      • Integrate scoring program
      • Start processing submissions if possible
    • CodaLab (Open Source Team): 
      • The team wants me to review documentation they have from the original developers of Codabench regarding sample challenges and new functionality and general operation of the site as a whole. 
      • They want to deploy a dev VM for me with Codabench launched so I can test many competitions and figure out the state of the platform and codebase. 
    MeetingDate
    Bi-weekly Meeting #1

    Task in Progress

    ...

    Risks

    Description

    Mitigation

    Rank

    Status

    Creation Date

    Realization

    Close Date

    Security issues with MedICI at ACR could be a potential issue restricting the use and operation of this challenge

    delete or remove these libraries and hope that doesn't break anything fundamental or crucial to MedICI

    1

    ongoing

    Having trouble activating certain azure functionality in ACR's environment. I can't connect to the internet when on their VPN and certain sign in work flows require me to sign in via a browser and so I can't hook MedICI up to it's normal resources yet.