NIH | National Cancer Institute | NCI Wiki  

Versions Compared

Old Version 1

changes.mady.by.user Unknown User (bearceba)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (bearceba)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Google Analytics Report:

Image RemovedImage Added

Activities completed this month

  • MIDRC is up and running and challenge details are being ironed out: https://midrc-medici-competitions.eastus.cloudapp.azure.com/
    • Challenge ended and we turned off the GPU. The MIDRC challenge organizer are pleased and will use us in the future. (smile)
    • Details about more metrics in the scoring program have been discussed. They want a way to convey how confident we are in the results and performance of the algorithmTraining Phase underway and participants are submitting the provided template to practice submitting docker images.
  • CodaLab (Open Source Team):
    • The team has created a new dev VM for me and I have deployed Codabench and am in the process of identifying all challenge examples and testing all possible combos of submissions to see what is and is not working. 
    • I'm them going to try some old challenges from MedICI to see how well they workWrote some notes for them regarding various demos we get from Tyler Thomas, a contract worker for CodaLab.
  • Federated learning challenge:
    • Temp site is up to furnish challenge dataParticipants continue to register here: https://medici-codalab-main.eastus.cloudapp.azure.com/competitions/24
    • Their security scans said python2 and python3 have vulnerabilities. They are asking me to not use these libraries or delete the files from these libraries:
      • ACR requested the scan of MedICI docker containers for use in their system.
    • Image Removed
    • New Infrastructure Diagram:
    • Image previewImage Removed
    • Image Removed
  • True-CT Challenge
    • There were some upload data issues where participants were getting permission denied when trying to sign into their sftp file shares.
    • Otherwise things are good and we have pushed the phase to submit back to May 31st.
  • DL-SPARSE Challenge
    • We are in the validation phase and data for that phase has been updated:
      • ground truth
      •  test set has been added to sftp server
    • We have run the first submission from a participant and are tracking the leaderboard in a spreadsheet. An email account was created for the challenge to correspond with the participants.
    • Development is ongoing regarding integrating the scoring program and finishing the FL workflow automation.
    SIRPRISE Challenge
    • There may be an email bug so working with Diane to iron that out. UPDATE: I believe we fixed this. Sometimes though CodaLab is blocked and emails are sent to spam. Not sure how to deal with that yet.
    • Challenge is over on the 23rd due to supply issues of some chemicals. 
Planned for next month
  • Federated Learning Challenge: 
    • Finish automation workflow and get participant submissions to run through MedICI
    • Finish docker upload email notification
    • Debug the sign up bug we have where the website hangs
  • MIDRC Challenge 0
    • UPDATE (3/31/2022): Infrastructure is built and now we are editing the template challenge to be the real challenge. 
    • UPDATE (4/29/2022): Training phase is underway and participants are testing submissions.
  • Federated Learning Challenge: 
    • Still need VPN access to environment.
    • Kendall Schmidt (from ACR) and Holger Roth (from NVIDIA) are creating a bash script that kicks off a participant container. 
    • I will be in charge of orchestrating the start of that script from the MedICI website.
  • CodaLab (Open Source Team): 
    • I need to draft a SOW to work with the team and become a contract worker with them
    • Launch local version of CodaBench and see if we can get some old challenges to run.
Paused
    • Identify all challenges\submissions that don't work as expected
    • Check in any code changes  as needed to fix issues found
Continue getting CodaLab running on NCI's network and VMs:
  • UPDATE (01_31_2022): May have a work around with a storage service called MINIO.
    • UPDATE (04_29_2022): The CodaLab (Open Source Team) has updates to the python2 → python3 update branch on github and I think it will work on NCI's network. However I think it is better to spend time working on CodaBench to get the new version working so that we aren't constantly behind when serving challenges and integrating with new software. It's taking lots of time to maintain older challenges while trying to work on new code. 
    MeetingDate
    Bi-weekly Meeting #1

    28 Apr  

    Bi-weekly Meeting #2

    Milestones

    ...

     

    Task in Progress

    ...

    Risks

    Description

    Mitigation

    Rank

    Status

    Creation Date

    Realization

    Close Date

    Security issues with MedICI at ACR could be a potential issue restricting the use and operation of this challenge

    delete or remove these libraries and hope that doesn't break anything fundamental or crucial to MedICI

    1

    ongoing

    Having trouble activating certain azure functionality in ACR's environment. I can't connect to the internet when on their VPN and certain sign in work flows require me to sign in via a browser and so I can't hook MedICI up to it's normal resources yet.