Page History
...
Make sure you have the dependencies (Python 2.7 and virtualenv). If you're running Ubuntu:
Code Block sudo apt-get install python2.7 python2.7-dev python-virtualenv
Clone the CodaLab repository:
Code Block git clone https://github.com/codalab/codalab-cli cd codalab-cli
Run the setup script (will install things into a Python virtual environment):
Code Block ./setup.sh
Set your path to include CodaLab (add this line to your
.bashrc
):Code Block export PATH=$PATH:<path to codalab-cli>/codalab/bin
Optional: include some handy macros (add this line to your .bashrc
):
Code Block |
---|
. <path to codalab-cli>/rc |
Install CodaLab
the CodaLab repoMultiexcerpt include nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include
from GitHub.Multiexcerpt include nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include Clone your fork:
Code Block |
---|
git clone https://github.com/<username>/codalab.git |
For more details and recommended practices, see Developer Guidelines
Multiexcerpt include | ||||||
---|---|---|---|---|---|---|
|
...
Run the dev_setup script.
Windows
Code Block cd codalab .\dev_setup.bat
If you are going to use SQL Server as a database, you will need to install the Python PyODBC library
. Before runningMultiexcerpt include nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include dev_setup
, you can download the installer, then run the setup script as follows (assuming the installer was downloaded at the root of a D drive):Code Block cd codalab dev_setup.bat D:\pyodbc-3.0.7.win-amd64-py2.7.exe
Linux
Code Block cd codalab source ./dev_setup.sh
Activate the virtual environment.
Windows
Code Block |
---|
...
venv\Scripts\activate |
...
Linux
Code Block |
---|
|
...
source venv/bin/activate |
...
Install App Schema and Default Data
Now you are ready to install the application schema and default data into the database.
...
- Open a Windows command prompt.
Navigate to the virtual environment (venv) for CodaLab and use the following command to install MySql-Python.
easy_install mysql-python
Launch the MySQL Command Line Client.
Use the following command to create a new database:
Code Block create database if not exists MySQL_DevDB;
Linux
- Open a terminal window.
Run the following command to install MySQL:sudo
Code Block sudo apt-get install mysql-server
Login to MySQL as root by typing the following command:
mysql -u root -p
Enter your root password when prompted.Use the following command to create a new database:
createCode Block create database if not exists MySQL_DevDB;
Type
exit
to return to the terminal prompt.
...
Open a terminal window.
Login to MySQL as root by typing the following command:
mysql -u root -p
Enter your root password when prompted.Use the following command to create a new database:
Code Block create database if not exists MySQL_DevDB;
Type
exit
to return to the terminal prompt.Finish up installation for Django
Code Block export DYLD_LIBRARY_PATH=/usr/local/mysql/lib:$DYLD_LIBRARY_PATH
Create a Local Configuration File
- Open your local configuration file (
local.py
). If there is nolocal.py
, save a copy oflocal_sample.py
namedlocal.py
in the same directory. In the
Azure storage
section, enter your Azure account details:Code Block DEFAULT_FILE_STORAGE = 'codalab.azure_storage.AzureStorage' AZURE_ACCOUNT_NAME = "<enter name>" AZURE_ACCOUNT_KEY = '<enter key>' AZURE_CONTAINER = '<enter container name>' PRIVATE_FILE_STORAGE = 'codalab.azure_storage.AzureStorage' PRIVATE_AZURE_ACCOUNT_NAME = "<enter name>" PRIVATE_AZURE_ACCOUNT_KEY = "<enter key>" PRIVATE_AZURE_CONTAINER = "<enter container name>" BUNDLE_AZURE_CONTAINER = "<enter the name of your bundle container>" BUNDLE_AZURE_ACCOUNT_NAME = PRIVATE_AZURE_ACCOUNT_NAME BUNDLE_AZURE_ACCOUNT_KEY = PRIVATE_AZURE_ACCOUNT_KEY
In the
Service Bus
section, enter your service bus connection information:Code Block SBS_NAMESPACE = '<enter the name of your service bus>' SBS_ISSUER = 'owner' SBS_ACCOUNT_KEY = '<enter value for 'default key'>'
Note title Important Do not change the values for
DEFAULT_FILE_STORAGE
andPRIVATE_FILE_STORAGE
, as these parameters contain the name of the Python class which implements the Azure storage back-end for Django.In the
DATABASES
section, enter the configuration settings for the database you want to use.SQL Server*
Code Block DATABASES = { 'default': { 'ENGINE': 'sql_server.pyodbc', 'NAME': 'somename', # Leaver user and password blank to use integrated security 'USER': '', 'PASSWORD': '', 'HOST': '(localdb)\\v11.0', 'PORT': '', 'OPTIONS': { 'driver': 'SQL Server Native Client 11.0', } }
MySQL
Code Block DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'MySQL_DevDB', 'USER': 'someuser', 'PASSWORD': 'somepassword', 'HOST': 'someserver', # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP. 'PORT': '', # Set to empty string for default. }
...
- Open
codalab/codalab/codalabtools/compute/sample.config
. - Save a copy of
sample.config
named.codalabconfig
in the same directory. - Open
.codalabconfig
.
In the compute-worker
section, enter the configuration settings for the storage account and the compute queue.
Code Block |
---|
compute-worker:
azure-storage:
account-name: "your account name"
account-key: "your account key"
azure-service-bus:
namespace: "your namespace"
key: "your secret key"
issuer: "owner"
listen-to: "name of queue"
local-root: "D:\\Temp" |
Enable CORS (Cross-Origin Resource Sharing) on Blob Storage
In order to work with competitions and bundles in your local development environment, you will need to manually enable Cross-origin resource sharing (CORS). CORS is a mechanism that allows many resources on a web page to be requested from another domain outside the domain the resource originated from. Web Browsers commonly apply same origin restriction policy to network requests. CORS relaxes these restrictions allowing domains to give each other permissions for accessing each other's resources. CORS is supported for Blob, Table and Queue services and can be enabled for each service using Azure SDK for python.
Make sure you have a valid management certificate to connect to the Service Management endpoint. This tutorial explains how to create a certificate and upload it to the Azure management portal: http://azure.microsoft.com/en-us/documentation/articles/cloud-services-python-how-to-use-service-management/
.Multiexcerpt include nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include Open
codalab/codalabtools/compute/enableCORS.py
in a text editor, and updateaccount_name
andaccount_key
with the account name and key for your blob storage account:Code Block import sys import yaml from os.path import dirname, abspath # Add codalabtools to the module search path sys.path.append(dirname(dirname(abspath(__file__)))) from codalabtools.azure_extensions import (Cors,CorsRule,set_storage_service_cors_properties) account_name = "<your blob storage account name>" account_key = "<your blob storage account key>" cors_rule = CorsRule() cors_rule.allowed_origins = '*' # this is fine for dev setup cors_rule.allowed_methods = 'PUT' cors_rule.exposed_headers = '*' cors_rule.allowed_headers = '*' cors_rule.max_age_in_seconds = 1800 cors_rules = Cors() cors_rules.cors_rule.append(cors_rule) set_storage_service_cors_properties(account_name, account_key, cors_rules)
- Save your changes, Save your changes, activate your virtual environment and run the script:
Windows
Code Block |
---|
python |
...
scripts\cors-enable.py |
...
Linux
Code Block |
---|
python |
...
scripts/cors-enable.py |
...
Initialize the Database
To initialize the database, you will need to run a few standard Django commands
Multiexcerpt include | ||||||
---|---|---|---|---|---|---|
|
...
Run the following commands to initialize the database.
Windows
Code Block cd codalab python manage.py validate python manage.py syncdb --migrate python scripts\initialize.py
Linux
Code Block cd codalab python manage.py validate python manage.py syncdb --migrate python scripts/initialize.py
Run tests to verify that everything is working.
python manage.py test
Optional: Populate the site with some sample data.
Windows
Code Block |
---|
python scripts\users.py
python scripts\competitions.py |
Linux
Code Block |
---|
python scripts/users.py
python scripts/competitions.py |
Configure the Bundle Service to Run Locally
...
Use the following command to start the CodaLab server locally.
python manage.py runserver
Open a browser and navigate to http://127.0.0.1:8000
to preview the site.Multiexcerpt include nopanel true MultiExcerptName ExitDisclaimer PageWithExcerpt wikicontent:Exit Disclaimer to Include
When your next coding session comes along, remember to work in the virtual environment you created:
Windows
...
Code Block |
---|
venv\Scripts\activate |
...
Linux
...
Code Block |
---|
source venv/bin/activate |
...
Info |
---|
If you experience database errors try deleting the database file (\codalab\codalab\dev_db.*) and run syncdb again. After creating a new database be sure to run |
Start the Worker Roles
In order to test competitions and bundles locally, you'll need to run the compute worker roles. There are two worker.py
scripts that you will need to run.
- Open a command prompt and activate your CodaLab virtual environment.
Start the first compute worker as shown here:
Code Block cd codalab python worker.py
Open a second command prompt and activate your CodaLab virtual environment.
Start the second compute worker as shown here:
Code Block cd codalabtools\compute python worker.py
If you plan to test competitions locally, open a third command prompt and activate the virtual environment for the CodaLab CLI, then start the bundle server:
Code Block cl server