This document is intended to show how to install CodaLab from start to finish in a linear fashion. CodaLab is under active development, so the procedures here are subject to change.
Your first step is to decide which virtual machine or computer you want to install MedICI.
This tutorial will focus on Ubuntu based installations on Azure although there are instructions on how to do this on Microsoft and Apple OSes and in AWS as well.
First create a VM in Azure or get yourself a computer with Ubuntu Server 18.04 installed. Once you have an Azure acount go to the home page here (https://portal.azure.com/#home). You should see a side-bar with many options or categories. You have the option to create either a resource or a resource group and then add resources into it. This tutorial will demo with a resource group to demonstrate good organizational practices. First click Resource groups and then add.
Name the resource group and then decide how you will pay for this service. After then click Review + create. Once that is complete, go to the resource group and add a VM:
I clicked on +Add and then was redirected to a bunch of choices of which Ubuntu Server 18.04 LTS was one of the choices. When I chose that I was brought to the VM creation page and I chose these settings:
Now click Review + create. After reviewing the options you click create and Azure starts creating the VM. Azure starts creating the VM and then finishes:
Now navigate to the VM by clicking Go to resource. Below is the resources home page and we will need the public IP address so we can access the machine.
One thing we should do before continuing is to set the IP address setting from Dynamic to Static. Grab the IP and also click Configure to edit DNS settings. Change the Dynamic to Static option and click save.
We need to configure the machine to be ssh accessible (port 22) and open http (port 80) (https://medium.com/techinpieces/practical-azure-how-to-enable-ssh-on-azure-vm-84d8fba8103e). Go to the VM and click Networking. On the right find the button Add inbound port rule. Change Destination port ranges to 22, Protocol to TCP, Priority to 330, and Name to anything you’d like:
Once you have that let’s open a terminal and get inside the machine by typing $ ssh <user>@<public IP address>.
If you get a message regarding encryption keys, enter y\yes as this is the first time signing into this machine.
On the right find the button Add inbound port rule. Change Destination port ranges to 80, Protocol to TCP, Priority to 300, and Name to anything you’d like:
This will be important later when we deploy the site. Without this you cannot access the site.
CodaLab has documentation regarding their preferred installation configuration. The two best sources are here:
The reason this is confusing is because the base code for codalab can be setup on different cloud providers (Google Cloud, AWS, Azure). They have made docker images for AWS and Google cloud that in theory should just run once some settings are established. If you are familiar with docker this seems reasonable. For the purposes of MedICI documentation we will go down the “Configure CodaLab from scratch (harder documentation)” found in the first of the above links.
Before we do any github cloning we need to install docker and docker-compose. First docker:
This link details installing docker for Ubuntu: https://docs.docker.com/install/linux/docker-ce/ubuntu/. For someone not familiar with linux there is a lot going on on that page. So for a basic installation this is what I ran in order:
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
Now if we run ‘docker’, we should get the options list telling us how to use this command verifying it is installed and we can use it.
Next docker-compose. In the terminal on your new VM check to see if docker-compose is installed. If not which it is not for me then we need to install it. While it seems a ‘sudo apt install docker-compose’ should work, the CodaLab documentation from link “Configure CodaLab from scratch (harder documentation)” says to go here and install from dockers website. The command I’m actually running is this: sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
Now we have to change the permissions in /usr/local/bin/docker-compose to use it: sudo chmod +x /usr/local/bin/docker-compose.
Now if we run ‘docker-compose’, we should get the options list telling us how to use this command verifying it is installed and we can use it.
The latest and greatest code is located here: https://github.com/codalab/codalab-competitions. Both sets of instructions want you to clone this repo locally to the VM. Before we get started I want to say since we will be making our own customizations down the road we will fork the repo CodaLab has so generously provided and do two things:
Clone the MedICI project. This follows CodaLabs instructions but with our github repo instead. Below is a picture of the commands I’ve executed from “Configure CodaLab from scratch (harder documentation)”. The green circles represent once inside the “src” folder, the changes that are not in the raw CodaLab github repo. This is the start of our customizations.
We can now push these changes into our branch.
At this point the app should be ready to deploy so we run ‘$ docker compose up -d’. However I have an error:
Turns out I needed to run with sudo or add myself to the docker group:
Or
○ $ sudo groupadd docker (this may be done already)
○ $ sudo usermod -aG docker $USER (now restart the VM)
○ Now you should be able to run $ docker-compose up -d
By the way the flag ‘-d’ means to run in detached mode. If you don’t use it you will notice your command prompt is in a live feed mode telling you everything that is going on with this docker.
Using this documentation( [2] from earlier): https://codalab-competitions.readthedocs.io/en/latest/
Below is the blob storage section for Azure:
You may sign up for an Azure account here, then follow the directions below.
This is the BLOB settings:
*Note: there is one final step that the CodaLab documentation leaves out. We NEED to enable CORS. Click CORS in the above photo and add these settings (https://stackoverflow.com/questions/50785255/azure-storage-got-403-url-cors-not-enabled-or-no-matching-rule-found-for-thi):
If you don’t, uploading a competition will give you CORS errors in the browser:
Once applied uploading works:
Codalab Instructions here: https://github.com/codalab/codalab-competitions/wiki/Customizing-your-self-hosted-instance
The only issue is that you can’t log into the customizer without being the super user. To make a superuser you need to log into the database as superuser. Django doesn’t have one by default and you need to create it. To do so ssh into your VM that hosts the site and run this command:
$ docker exec -it django python manage.py createsuperuser
Now use CodaLabs instructions and navigate to http://your-instance.com/customizer
Login with the newly made credentials. Choose a new file to be your logo.