Using iPython notebook on keeling

I usually develop code for python using the iPython Notebook, which is a browser-based python interactive environment.  I am assuming you already have anaconda python installed following the directions here.

I created a notebooks subdirectory in my $HOME/python directory,

mkdir -p $HOME/python/notebooks

and launch ipython from there to save the files in a convenient, organized location.  To start iPython notebook, do the following.  If you’re working on your local machine (mac/linux), this will launch a web browser window/tab, however if you’re working on keeling (if you have an ability to display X11 graphics on your local machine or in a remote desktop session), and have the VPN running if necessary to connect, this will launch a firefox window from keeling.

ipython notebook --pylab==inline

You should see a browser window or tab pop up with the iPython notebook within it.  You can create a new notebook, and work interactively on the data.  You can read more about the power of iPython notebook at http://ipython.org/notebook.html.  This may be really slow if you are off campus and don’t have a remote desktop connection, so see below for faster performance in remote display.

Running a remote iPython session from keeling on your local machine

If you have installed iPython on keeling, you have the option to run iPython remotely on keeling (using its computing resources) while displaying iPython on your local machine’s browser.  This is a machine dependent operation…I will demonstrate it from an OS X machine.

First, you need to launch iPython on keeling, with the option to not launch a browser.  You’re also going to set up a port for iPython to connect to.  This number is reserved for your session, and if another keeling user or process uses the same port, then it could get confused.  So coordinate with Steve Nesbitt to get a port number that won’t conflict with anyone else.  Hopefully we can work out a solution to this issue soon, but for now follow these instructions.

On keeling, start the iPython notebook server, where the port number is your assigned port:

ipython notebook –no-browser –port=70xx

Now, on your local machine, set up an ssh tunnel to keeling using this port you run (with your netid).  If you are using OS X or linux, use a terminal window.  If you are using windows, follow directions here using the source and destination ports described below.

ssh -N -f -L localhost:8080:localhost:70xx netid@keeling.earth.illinois.edu

This will set up an ssh tunnel from port 8080 (which is accessible from your web browser) to the iPython server on keeling.  You will need to kill this ssh session (with Control-C) when you are finished, as you can’t connect again to this port without killing it explicitly.  If you are having trouble killing it, do the following in another window, typing in the process number of the hanging ssh session (be careful to kill the correct job, otherwise results my vary):

[snesbitt@chuva:~/Downloads]$ ps -u snesbitt -f | grep ssh
snesbitt   850   251   0 31Jan14 ??         0:01.20 /usr/bin/ssh-agent -l
snesbitt 11900  4163   0 Fri03PM ttys000    0:00.07 ssh -Y keeling.earth.illinois.edu
snesbitt 12848 12839   0  6:47PM ttys001    0:00.01 ssh -N -L localhost:8080:localhost:7000 snesbitt@keeling.earth.illinois.edu
snesbitt 12559  4456   0  5:43PM ttys003    0:00.12 ssh -Y keeling.earth.illinois.edu
snesbitt 12868 12862   0  6:49PM ttys004    0:00.00 grep ssh
[snesbitt@chuva:~/Downloads]$ kill -9 12848

Note that in this case, process 12848 (second column) was the ssh port command I used for forwarding.

Now, point a web browser to http://localhost:8080.  You should see an iPython notebook that displays locally, but executes code on keeling, and has access to keeling file systems.  Note that it will not have access to your local file systems.

Important notes:

  • Closing your web browser does not close your ssh tunnel or end your python session on keeling.  Remember to close each of these when you are done with them to save resources on keeling.
  • You should be able to reconnect to a keeling iPython session if your local computer goes to sleep, or disconnects from the network.  You may need to re-establish your ssh tunnel, and kill it locally (either with Control-C or using the kill command) as described above if it doesn’t respond.

Here are some screen shots of iPython notebook from keeling displaying on my OS X laptop (from home).  I can execute code as if I were connected on keeling.  Note that some things (like complicated graphics or lots of text output in my iPython notebook) may significantly tax my network connnection between my local computer and keeling, but this can be typically avoided.

Opening up an iPython notebook:

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>