Skip to content

Launching notebooks from a cluster

Jump to bottom
Grant Wallace edited this page May 14, 2019 · 13 revisions

Running notebooks from a server/cluster

Make sure you have a userid on the server/cluster and that it is activated.

If you are working off-campus, you will need to setup a secure remote access VPN. Information on how to configure secure remote access can be found on your university/organization IT webpage.

Windows Users Follow the instructions in this video: https://github.com/NEU480/organization/blob/master/auditors/putty_jupyter.mp4

Mac Users Try logging in for the first time. Open a terminal window and type:

ssh [email protected] #Replace servername, university to what is relevant for your cluster

You will be prompted for a password. Enter you password and hit Enter. If it is your first time logging into the cluster, you will see the following message. Don't be scared!

The authenticity of host 'servername.university.edu (xxx.xxx.xxx.xxx)' can't be established. RSA key fingerprint is xxxxxxxxxxx. Are you sure you want to continue connecting (yes/no)?

Type 'yes', press Enter, and continue to enter your password. You are successfully connected to your cluster. Congrats!

Make sure you have the tutorials installed and scripts configured. If you have already completed this task, proceed to Jupyter Tunnel

  1. Clone the BrainIAK Tutorials repo to your current directory, type git clone https://github.com/brainiak/brainiak-tutorials.git

  2. Edit the run_jupyter_remote_server.sh using a text editor (e.g. vim) and enter the modules you need to run BrainIAK on your cluster via the module load statements.

Instructions to set up a jupyter tunnel

Jupyter tunnels are wonderful things that allow you to run code on a cluster without the sluggishness of an interactive session.

  1. Start tunnel: To initiate a tunnel, run the script: ./run_jupyter_remote_server.sh. Note: this script will not run if you do not have execute permissions (execute the following command in the folder where you have the script stored: chmod 711 run_jupyter_remote_server.sh). It will print out a bunch of information needed to open the jupyter tunnel (see screen shot below).

Reference the above information for the following steps.

  1. Initiate local connection: In a second terminal window, copy and paste the ssh-tunnel command (labeled as 'ssh key') that this command outputs. This terminal may ask for your password to login to the cluster (if you have saved your password then it will not react, but that is fine).

  2. Point browser to notebook: In your browser copy the url (beginning with localhost) and paste it in your browser, along with the numbers after it. If prompted for a password or login token, go back to your first terminal window. You should now see a prompt that says "Copy/paste this URL into your browser...". At the end of the url, it should say "token=" followed by a long alphanumeric code. Copy the alphanumeric code only and paste into the login token field in your browser window. This will open you in a notebook directory tree.

Voilà! You are good to go. Click on any notebook to begin.

Clone this wiki locally