NIH | National Cancer Institute | NCI Wiki  

  • Created by Unknown User (bearceba), last modified on May 14, 2021
Activities completed this month
  1. I have a ported version of codalab to python3 working with custom changes to python and jQuery to update those libraries
    1. Kevin Olexa is working hard to get me an AA account and has succeeded. At this point they are adding my username to the server (development) that I will be working on.
  2. Opened these Challenges:
    1. SPIE AAPM (http://spie-aapm-nci-dair.westus2.cloudapp.azure.com/competitions/4#participate-submit_results) - submissions working, but we need to change open validation phase to open ended phase after test. Will require a code update from main codalab branch
      1. Need a code change and it non-trivial at the moment. Will investigate.
  3. Demo docker submissions to Emily Townley who is designing a MIDRC challenge. I should have a non-gpu and a gpu submission ready soon:
    1. I did in fact get a cpu submission working and am fine tuning it. (http://medici-codalab-main.eastus.cloudapp.azure.com/)
    2. I testing briefly with a gpu machine and it works too. 
    3. We are using the forum to go back and forth as people test my tutorial (https://github.com/QTIM-Lab/MedICI-Challenge-Bundles/tree/main/aks_MedNIST_bundle/README). There some interesting bugs that we are working out:
      1. Windows uploaded zip files are not behaving the same as ubuntu or mac based ones.
      2. While not an issue on my system, some parts of the training algorithm require some adjustments and the organizers (MDRC folks) are requesting that I make code changes to the tutorial so that users don't have to figure that out. Sounds reasonable.
  4. With Carolyn's help the Kheops document has been revised to have a more fluid and professional language consistent with reports of this style. Jayashree has commented on the report and her comments are as follows:
    1. what is the gap that you are trying to fill
    2. add 1/2 page about what medici is and why you are looking for something like kheops
    3. also the tone of the writing (especially closing thoughts) is pretty conversational - perhaps a more formal tone is preferred for  reports of this sort
    4. i would be careful about “Where we are with challenges, using KHEOPS solves no issue and provides a solution to a problem we don’t really have.”
Planned for next month
  1. Continue opening older challenges listed on https://medici-challenges.org so that people can still make submissions. (wrap up SPIE AAPM from above)
    1. I've been busy with MDRC project to fine tune the docker submission main platform and haven't had a chance to change code to add a phase. To elaborate, whatever code base I used for this codalab doesn't have an "add phase" button and so the only way to keep it open is to have the validation phase open indefinitely. This is related to 2 (a) (i ) from above under Completed This Month.
  2. Investigate Federated Learning Challenges (https://wiki.nci.nih.gov/display/NICC) presentation from Daniel Rubin, July 6th (under Presentations and Recordings from Previous Calls)
    1. Jayashree and I met with Daniel and they have a plan. The plan is to first spin up some VMs and perform the basic federated learning concepts Daniel has already built a framework for. 
    2. Soon he will be reaching out and we will perform this.
    3. Begin the Federated Learning Document by integrating MedICI and Federated Challenges tech and some theoretical ideas for implementation.
  3. Add indication on website (medici-challenges.org) which challenges accept submissions.
    1. Pending completion. It's been a very busy month and adding this felt like a lower priority. Based on a discussion with Ulli and Carolyn, I will establish an appropriate priority.
  4. Ping Dr. Mazurowski on new 2nd part of his challenge.
MeetingDate
Bi-weekly Meeting #1

 

Bi-weekly Meeting #2

Bi-weekly Meeting #3

 

Milestones

DescriptionDate
  • Python3 and jQuery Update works on my local computer. I am in the process of helping NCI deploy this so we can do an app scan.

 

  • Got VPN access, now just need machine access

 

  • Received an AA account (administrator) for sshing into NCI machines.

 

Task in Progress

DescriptionStatusCreation DateClose Date

Investigated Kheops and am finishing the report. I met with Joël Spaltenstein from Kheops and he helped me setup a fully secured and working Kheops. I will be collecting more notes from this and talking more about it in the report.

In Progress

 

 

Continued ascertaining the validity of v2 codalab (codabench). I honestly haven't had too much time because of the covid challenge platform. I did get it to launch but there are some issues. I plan to get it all ready and deployed with the MICCAI challenge from last year. Once I have that we can scan it on the NCI platform and run challenges there. Deploy "develop" branch of codabench code to see if it is farther along and more bug free.

UPDATE: we will use old codalab ported to python3 with jQuery update.

Closed

 

 

Finishing setting up MedNIST and Basic Template sample challenges within the kubernetes deployment. This project is a collaboration with Microsoft that has been ongoing since July\August 2020. This should allow us to scale CodaLab up for more users to use and allow a robust docker submission platform. In Progress

 


Risks

DescriptionMitigationRankStatusCreation DateRealizationClose Date
Codabench could prove to be hard to setup and may be buggyNothing we can do but try and see what happens1In Progress

 

We will go with python3 port of codalab for now.

 

  • No labels