Page History

...

1. Log in to the Microsoft Azure Portal

   Multiexcerpt include
   MultiExcerptName MultiExcerptName ExitDisclaimer nopanel true ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   .
2. Go to VM Screen, select "+ NEW" from the lower left corner of the screen.

3. On the pane that appears select, Compute > Virtual Machine > Quick Create.

   Code Block
   DNS Name: pick something unique and memorable Image: Ubuntu Server 13.04 New Password/Confirm: provide a good password for the azureuser Region/Affinity Group: West US

4. Click Create a virtual machine.

5. Once your virtual machine is created, select it from the VM screen.

6. Go to the Endpoints menu.

   (+ Add) an Endpoint at the bottom of the screen.

   Code Block
   Provide the following information Name: HTTP Public Port: 80 Private Port: 8000

7. Restart the VM from the management console.

   Troubleshooting note: You may get a message that Endpoint was successfully created but restart failed. In this case, go back to Dashboard (Management Console) and click Restart. In some cases, multiple attempts may be required.

   At this point, it appears that a restart is not mandatory to continue with configuration.

8. Now you can login to it and start configuring things.

9. Login to your VM via ssh as: azureuser <password provided at vm build - Step 3)

10. Things to do once you're in Ubuntu.

    1. sudo apt-get update

       1. enter password (azureuser password from Step 3)

    2. sudo apt-get upgrade -y

    3. sudo apt-get install git python-pip -y

    4. sudo pip install virtualenv

Installation Prerequisites

1. Install Python 2.7. For current Debian-based Linux distributions (such as Ubuntu), BSD and Mac Python 2.7 is usually installed. However Redhat-based Linux distributions, such as RHEL and CentOS, are sometimes behind the curve and do not have Python 2.7. As of this writing, CentOS 6.4 is at Python 2.6, which is well on its way to EOL. Python 2.6 may work, but code will be written with 2.7 and 3.3+ in mind. Below are instructions for Ubuntu Linux 13.04+.

   Code Block
   sudo apt-get install python2.7 python2.7-dev python-virtualenv

2. Install PIP

   Multiexcerpt include
   nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   .

   Code Block
   sudo apt-get install python-pip

3. Install virtualenv.

   Code Block
   sudo apt-get install python-virtualenv

4. Install Git

   Multiexcerpt include
   nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   .

   Code Block
   sudo apt-get install git

5. Install the prerequisites for MySQL-Python

   Multiexcerpt include
   nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   .

   Code Block
   sudo apt-get install build-essential python-dev libmysqlclient-dev

Create Storage Containers

Create Storage Containers

In order to test uploading and running bundles in CodaLab, you will need to have a In order to test uploading and running bundles in CodaLab, you will need to have a Windows Azure storage account

1. Log on to the Azure Portal

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   .
2. In the left pane, click Storage.
3. Select your storage account.
4. At the bottom of the dashboard, click Manage Access Keys. Copy your access keys. You'll need them later.
5. At the top of the dashboard page, click Containers.
6. At the bottom of the Containers page click Add.
7. Create a new container named "bundles". Set the Access to "Private".
8. Add another container named "public". Set the Access to "Public Blob".

...

1. Install azure-cli

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   .

   In Ubuntu it can be installed using the following command:

   Code Block
   sudo apt-get install nodejs-legacy sudo apt-get install npm sudo npm install -g azure-cli

   To login run the following command:

   Code Block
   azure login

   Copy the code offered to you, above, and open a browser to http://aka.ms/devicelogin

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   . Enter the code, and then you are prompted to enter the username and password for the identity you want to use. When that process completes, the command shell completes the log in process.

2. From command line azure sb namespace create <name> <location> where <location> can be "East US"

3. Log on to the Azure Portal

   Multiexcerpt include
   MultiExcerptName MultiExcerptName ExitDisclaimer nopanel true ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   .
4. In the left pane, click Service Bus.
5. Select the service bus you just created.
6. At the top of the screen click Queues.
7. Click Create a new queue.
8. Click Quick Create and create a new queue named "compute".
9. Click Create A New Queue.
10. At the bottom of the screen, click New, and create another queue named "response".
11. In the left pane, click Service Bus.
12. At the bottom of the page, click Connection Information.
13. Copy the following connection information:
    • Namespace name
    • Default issuer
    • Default key

...

Install CodaLab

1. Fork

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   the CodaLab repo

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   from GitHub.

2. Clone your fork:

...

For more details and recommended practices, see Developer Guidelines

...

1. Run the dev_setup script.

   Windows

   Code Block
   cd codalab .\dev_setup.bat

   If you are going to use SQL Server as a database, you will need to install the Python PyODBC library

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   . Before running dev_setup, you can download the installer, then run the setup script as follows (assuming the installer was downloaded at the root of a D drive):

   Code Block
   cd codalab dev_setup.bat D:\pyodbc-3.0.7.win-amd64-py2.7.exe

   Linux 

   
   

   Code Block
   cd codalab source ./dev_setup.sh

2. Activate the virtual environment.

...

...

1. Make sure you have a valid management certificate to connect to the Service Management endpoint. This tutorial

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   explains how to create a certificate and upload it to the Azure management portal.

   Open codalab/codalabtools/compute/enableCORS.py in a text editor, and update account_name and account_key with the account name and key for your blob storage account:

   Code Block

   import sys import yaml from os.path import dirname, abspath # Add codalabtools to the module search path sys.path.append(dirname(dirname(abspath(__file__)))) from codalabtools.azure_extensions import (Cors,CorsRule,set_storage_service_cors_properties) account_name = "<your blob storage account name>" account_key = "<your blob storage account key>" cors_rule = CorsRule() cors_rule.allowed_origins = '*' # this is fine for dev setup cors_rule.allowed_methods = 'PUT' cors_rule.exposed_headers = '*' cors_rule.allowed_headers = '*' cors_rule.max_age_in_seconds = 1800 cors_rules = Cors() cors_rules.cors_rule.append(cors_rule) set_storage_service_cors_properties(account_name, account_key, cors_rules)

2. Save your changes, activate your virtual environment and run the script:

...

To initialize the database, you will need to run a few standard Django commands

...

• You can already run the Django web site on your local machine as described on this page

  Multiexcerpt include
  MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

  .

• You have also forked the codalab-cli

  Multiexcerpt include
  MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

  project and have gone through the steps listed in the Readme

  Multiexcerpt include
  MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

  .

...

1. Use the following command to start the CodaLab server locally.

   python manage.py runserver
   

2. Open a browser and navigate to http://127.0.0.1:8000

   Multiexcerpt include
   MultiExcerptName ExitDisclaimer nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include

   to preview the site.

...

sudo sh -c "echo deb https://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
sudo apt-get update
sudo apt-get install lxc-docker
sudo useradd $USER docker

 docker run -t -i codalab/ubuntu:1.8

git clone https://github.com/percyliang/fig
# Add fig/bin/q to your $PATH
q -mode master   # Run in a different terminal
q -mode worker   # Run in a different terminal

 "workers": {
    "q": {
        "verbose": 1,
        "docker_image": "codalab/ubuntu:1.8"
        "dispatch_command": "python $CODALAB_CLI/scripts/dispatch-q.py"
    }
}

cl work-manager -t q                 # Run in a different terminal
cl run 'cat /proc/self/cgroup' -t    # Should eventually print out lines containing the string `docker`