Page History

...

CodaLab competitions can be set up in several different ways. We provide several templates, which differ in the type of data and complexity of the organization. These three templates are the most basic:

...

• If the submission contains no "metadata" file: treat it as a "result" submission and forward if it directly to the "scoring program".
• If the submission contains a "metadata" file: execute the command found in the metadata file.

...

• You want participants to submit libraries of functions rather than executables (the ingestion program will then run them with a the main function, which is the same for all participants). This allows you to use, for example, the same program to read data for everybody or to call the library functions in cross-validation loops.
• You organize active learning, query learning, time series prediction, or reinforcement learning competitions in which the code of the participants get progressively exposed to data in an interactive way.

...

Creating a Starting Kit

The starting kit is a "free stylefreestyle" bundle created by the organizers, containing everything the participants need to get quickly started. A typical starting kit contains:

• A README file (use e.g. README.md or README.ipynb to facilitate sharing the starting kit on Github).
• A sample result submission.
• A sample code submission. The readme file contains instruction and/or sample code to read/display data, and prepare an a sample submission. We provide a template starting kit -- also downloadable as zip -- for the Iris challenge.

...

Once your competition is up and running, you can manage it from your CodaLab Dashboard. For more details, see Running a Competition.