Running jupyter notebook on a keeling compute node

This uses a technique adapted from the ipynbhpc github project by Andrea Zonca (https://github.com/pyHPC/ipynbhpc) for use on the DAS cluster using the SLURM package manager.  You could use the same technique on other high performance computing resources.

Step 1: Copy the following tar file to your home directory on keeling:

cd ~
cp /data/keeling/a/snesbitt/ipynbhpc.tar.gz .
tar xvf ipynbhpc.tar.gz

This will create a software directory in your home directory that will contain the script.

Step 2: Modify your .bashrc to contain the following: (xxxx is the port number you use for keeling jupyter notebook sessions)

export IPYNB_PORT=xxxx

export IPYNB_MINUTES="120"  #modify for the number of minutes you want the session to be active

export IPYNB_QSUB_TEMPLATE="qlogin -n 1 -mem 16G -time %d:00" # modify n to the number of processors you want, and mem to be the number of gigabytes of RAM you need

export IPYNB_NOTEBOOK_TEMPLATE='bash -c "source /data/keeling/a/snesbitt/.bashrc; source activate py3k; jupyter notebook --port=%s --no-browser --ip=127.0.0.1 "' #this is set to implement your py3k (python 3) environment.

#export IPYNB_NOTEBOOK_TEMPLATE='bash -c "source /data/keeling/a/snesbitt/.bashrc ; jupyter notebook --port=%s --no-browser --ip=127.0.0.1 "'

export PATH=$PATH:/data/keeling/a/$USER/software/bin

Save the file, and now you’re set up.  To run, while in the bash shell,

 source .bashrc

Now setup is done.


To invoke the script, type the following in bash:

ipynbhpc

You will see a bunch of outputs.  It will ask you for your password and may ask you to respond yes to a new ssh key.  Eventually, it will say:

Succesfully opened notebook!
Kill this process to end your notebook connection.

Now you should be able to point your web browser to the port as you have done before.

To quit, type Control-C in the terminal where it is launched.  The session will end when it times out (based on your .bashrc setting).  I would recommend not leaving the session idle for a long time, as it wastes computer resources.