NIH | National Cancer Institute | NCI Wiki  

Versions Compared

Old Version 7

changes.mady.by.user Klinger, Carolyn (NIH/NCI) [C]

Saved on

compared with

New Version 8

changes.mady.by.user Klinger, Carolyn (NIH/NCI) [C]

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The following documentation is available for the MedICI challenge management system.

CodaLab Competitions is a powerful open source framework for running competitions that for running competitions that involve result or code submission. You can either participate in either participate in an existing competition or host a or host a new competition. Most competitions hosted on Codalab are machine learning are machine learning (data science) competitions, but Codalab is NOT limited to this application domain. It can accommodate any problem for which a solution can be provided in the form of a zip archive containing a number of files to be evaluated quantitatively by a scoring program (provided by the organizers). The scoring program must return a numeric score, which is displayed on a leaderboard where the performances of participants are compared.

Note

Release notes v1.5:

  • Participants: can join competition-specific teams.
  • Organizers: can enable/monitor competition-specific teams; can run competitions in their own docker; can launch their own compute workers (to process participants' submissions) without needing to run their own Codalab instance; can provide an ingestion program to receive/parse participants' submissions; can dump their competition bundle (to save changes made with the editor); can run their own instance in "single competition mode".
  • Developers: can run Codalab in dockers.
  • Default docker: if not special docker is specified, submissions are run in codalab/codalab-legacy the default docker, which can be found on Github.
Panel

Children Display