NIH | National Cancer Institute | NCI Wiki  

WIKI MAINTENANCE NOTICE

Please be advised that NCI Wiki will be will be undergoing maintenance on Monday, June 24th between 1000 ET and 1100 ET.
Wiki will remain available, but users may experience screen refreshes or HTTP 502 errors during the maintenance period. If you encounter these errors, wait 1-2 minutes, then refresh your page.

If you have any questions or concerns, please contact the CBIIT Atlassian Management Team.

Versions Compared

Old Version 17

changes.mady.by.user Klinger, Carolyn (NIH/NCI) [C]

Saved on

compared with

New Version 18

changes.mady.by.user Klinger, Carolyn (NIH/NCI) [C]

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Use the following command to start the CodaLab server locally.

    python manage.py runserver

  2. Open a browser and navigate to http://127.0.0.1:8000

    Multiexcerpt include
    nopaneltrue
    MultiExcerptNameExitDisclaimer
    PageWithExcerptwikicontent:Exit Disclaimer to Include
    to preview the site.

When your next coding session comes along, remember to work in the virtual environment you created:

Windows

Code Block
 venv\Scripts\activate

Linux

Code Block
source venv/bin/activate
Info

If you experience database errors try deleting the database file (\codalab\codalab\dev_db.*) and run syncdb again. After creating a new database be sure to run initialize.py in the scripts folder in order to insert initial data required by the app.

Start the Worker Roles

In order to test competitions and bundles locally, you'll need to run the compute worker roles. There are two worker.py scripts that you will need to run.

  1. Open a command prompt and activate your CodaLab virtual environment.

  2. Start the first compute worker as shown here:

    Code Block
    cd codalab
python worker.py

  3. Open a second command prompt and activate your CodaLab virtual environment.

  4. Start the second compute worker as shown here:

    Code Block
    cd codalabtools\compute
python worker.py

  5. If you plan to test competitions locally, open a third command prompt and activate the virtual environment for the CodaLab CLI, then start the bundle server:

    Code Block
    cl server

 

Execution Using Docker

Every execution on CodaLab (should ideally) happen in a docker

Multiexcerpt include
nopaneltrue
MultiExcerptNameExitDisclaimer
PageWithExcerptwikicontent:Exit Disclaimer to Include
container, which provides a standardized Linux environment that is lighter-weight than a full virtual machine.

The current official docker image is codalab/ubuntu, which consists of Ubuntu 14.04 plus some standard packages. See the CodaLab docker registry.

To install docker on your local machine (either if you want see what's actually in the environment or to run your own local CodaLab instance), follow these instructions

Multiexcerpt include
nopaneltrue
MultiExcerptNameExitDisclaimer
PageWithExcerptwikicontent:Exit Disclaimer to Include
:

Code Block
sudo sh -c "echo deb https://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
sudo apt-get update
sudo apt-get install lxc-docker
sudo useradd $USER docker
 
Then, to test out your environment, open a shell (the first time you do this, it will take some time to download the image):
 Code Block
 docker run -t -i codalab/ubuntu:1.8
Now, let us integrate docker into CodaLab. First, we need to setup a job scheduling system (that manages the deployment of runs on machines). Note that CodaLab itself doesn't do this, so that it can be easily integrated into different systems. An easy way to set this up is to use q from Percy Liang's fig package:
 Code Block
git clone https://github.com/percyliang/fig
# Add fig/bin/q to your $PATH
q -mode master   # Run in a different terminal
q -mode worker   # Run in a different terminal
Now, let us tell CodaLab to use q and run things in docker (these two things are orthogonal choices). Edit the .codalab/config.json as follows:
 Code Block
 "workers": {
    "q": {
        "verbose": 1,
        "docker_image": "codalab/ubuntu:1.8"
        "dispatch_command": "python $CODALAB_CLI/scripts/dispatch-q.py"
    }
}
 
To test it out:
 Code Block
cl work-manager -t q                 # Run in a different terminal
cl run 'cat /proc/self/cgroup' -t    # Should eventually print out lines containing the string `docker`