 |
Page History
...
Challenge Post has been used to organize hackathons, online challenges and other software collaborative activities. In person hackathons are free while the online challenges cost $1500/month (plus other optional charges). and separate comment: "Could the suitability of each platform for imaging challenges be discussed?"
Open Source
Synapse is both an open source platform and a hosted solution for challenges and collaborative activities created by Sage bionetworks. It has been used for a number of challenges including the DREAM challenge. Synapse allows the sharing of code as well as data. However, the code typically is in R, Python and similar languages. Synapse also has a nice programmatic interface and methods to upload/download data, submit results, create annotations and provenance through R, Python, command line and Java. These options can be configured for the different challenges. Content in Synapse is referenced by unique Synapse IDs. The three basic types of Synapse objects include projects, folders and files. These can be accessed through the web interface or through programmatic APIs. Experience and support for running image analysis code within Synapse is limited.
...
Matrix of Features and Frameworks (1 -5)
...
Below is a table that rates the relative merits of the most relevant frameworks that we evaluated. The scale is 1-5 where 1 indicates excellent support for the feature while 5 indicates that that feature is not currently part of the system or there is limited support. We have included Kaggle as a representative "paid" framework for comparison. The other five frameworks are the open-source frameworks that were seriously considered in this comparison.
| Kaggle | Synapse | HubZero (challenges/projects) | COMIC | VISCERAL | CodaLab | |
|---|---|---|---|---|---|---|
| Ease of setting up new challenge | 2/4 (if new metrics need to be used) | 2 | 2/5 | 2 | 3 | 1 |
Cost (own server/hosting options) | $10-$25k/challenge | Free/hosted | Free/hosted | Free/hosted | Free/Azure costs | Free/hosted |
License | Commercial | OS | OS | OS | OS | OS |
Ease of extensibility | 5 | 4 | 4 | 2 | 3 | 2 |
Cloud support for algorithms | 4 | 3 | 3 | 4 | 1 | 3 |
Maturity | 1 | 1 | 1/5 | 3 | 4 | 3 |
Flexibility |
|
|
|
|
|
|
Number of users | 1 | 1 | 1/5 | 3 | 3 | 3 |
Types of challenges | 1 | 1 | 1 | 3 | 1 | 1 |
Native imaging support | No | No | No | Yes | Limited | No |
API to access data, code | 5 | 1 | 3 | 4 | 4 | 4 |
...
