Installing anaconda python and Py-ART on keeling

Updated 8 February 2014

Py-ART is an incredibly powerful interactive tool for visualizing, performing quality control (Doppler dealiasing, data clean up, calculating derived quantities like specific differential phase), gridding, and file conversion.  In this tutorial, we will install Py-ART using anaconda python on DAS’s cluster, keeling.  In theory, you should be able to use many python packages, including the system defaults, but having your own python distribution allows you to install and have control over your own packages without root permissions.  I’m assuming you have a keeling account.  If not, contact SESE computing help.

Prerequisites

We will first install anaconda python, which is an easy to use python distribution created and maintained by Continuum Analytics.  Install anaconda python by downloading the from from Continuum Analytics. http://continuum.io/downloads

Copy the link to the download for your system type (linux, mac, etc.) in the web browser, and then paste it into your keeling terminal window.  For example, for 64-bit linux (keeing), copy the appropriate download link and paste it into your terminal window.

cd ~
wget [URL you get from copy and pasting from the download page, should look like http://09c8d0b2229f813c1b93-c95ac804525aac4b6dba79b00b39d1d3.r79.cf1.rackcdn.com/Anaconda-1.8.0-Linux-x86_64.sh]

Once downloaded, install it by

bash Anaconda-1.8.0-Linux-x86_64.sh

This will install anaconda python into a directory ~/anaconda.  Anaconda prefers to work in the bash shell (and tcsh is the default on keeling), so you will have to invoke the bash shell when you want to use anaconda python (instead of the default python installed on keeling, which lacks some of the packages we need).

To enter bash, type

bash -l

You’ll do this every time you run anaconda.  Now let’s set up the environment to work with python.  You can edit .bash_profile (which sets environment variables for python) to look like ~snesbitt/.bash_profile, or just copy it.  Then activate the changes by sourcing this file.  Now create an anaconda environment using conda, which is one of anaconda’s python package managers.  Finally, update anaconda python with its latest packages using conda.  Note that you may get an error message when you do the source command the first time.

cp ~snesbitt/.bash_profile ~
source ~/.bash_profile
conda create -n py27 anaconda
source ~/.bash_profile
conda update conda

Test python

If you’re not already, invoke bash shell by typing

bash

Start python up to test it.

[snesbitt@keeling ~]$ bash
bash-4.1$ python
Python 2.7.6 |Anaconda 1.8.0 (64-bit)| (default, Jan 17 2014, 10:13:17) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Type Control-D to exit python.

Install some software dependencies for Py-ART

Make a local software repository.  I usually make a software directory in my keeling home directory.  Then, enter it to install some software prerequisites.

mkdir ~/software
cd ~/software

Install TRMM RSL libraries into ~/software.  Radar Software Library is a NASA software package that stands on its own, but it allows Py-ART to read some radar formats.  See http://trmm-fc.gsfc.nasa.gov/trmm_gv/software/rsl/.

wget ftp://trmm-fc.gsfc.nasa.gov/software/rsl-v1.44.tar.gz
tar xvf rsl-v1.44.tar.gz
cd rsl-v1.44
./configure --prefix=$HOME/software
make install

If you have compile troubles with RSL, see the ‘Frequently Asked Questions’ here.

Install CBC libraries into into ~/software.  CBC (Coin-or branch and cut) is an open-source mixed integer programming solver written in C++.  It allows Py-ART to calculate specific differential phase.

wget http://www.coin-or.org/download/source/Cbc/Cbc-2.8.9.zip
unzip Cbc-2.8.9.zip
cd Cbc-2.8.9
./configure --prefix=$HOME/software 
make install

Install pyart

There are a few python dependencies to install before installing Py-ART.  Install CyLP, which is the python package wrapper for CBC, which we just installed.  Were going to use pip at the command line, which is another package manager for python.

export COIN_INSTALL_DIR=$HOME/software
pip install cylp

Now we’ll install basemap as a dependency:

conda install basemap

Finally, install Py-ART

Now, we will install pyart.  I create a python directory in my home directory, and enter it:

mkdir python
cd python

Now, I am going to use github (an open source package distribution site), where the latest Py-ART distribution is available.  Go to http://github.com and sign up for an account (or sign in if you have one already).  The developers recommend setting up your keeling ssh keys so you can be on the development chain for pyart.  See instructions here on how to do this for linux. [This section needs some help, ultimately generate keys on keeling, copy the contents of ~/.ssh/id_rsa.pub and paste them into the github account settings, and you should be good to go]

You also need to set your github username on keeling, substituting your username for the github username you just set up.

git config --global user.name "githubusername"

Now, clone the repository to your computer

git clone github.com:ARM-DOE/pyart.git

This will download Py-ART into a pyart subdirectory in your python directory.

Now, we’ll install Py-ART (almost)

cd github
python setup.py install

This will install the Py-ART package into your anaconda python distribution.

Unfortunately, as of writing, there is a bug that some of the packages don’t quite work, so run the following from the pyart directory (one time only):

pyart/io/rebuild_rsl_interface.sh
pyart/io/rebuild_sigmetfile.sh

Now you’re ready to run Py-ART.  To test it, you should see the following without error messages:

bash-4.1$ python
Python 2.7.6 |Anaconda 1.8.0 (64-bit)| (default, Jan 17 2014, 10:13:17) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyart
>>>

Now what?

You now have a powerful tool to interactively process and display radar data.  You may want to also check out installing Radx (see accompanying page) for additional file conversion, gridding, and stratiform-convective separation tools.  These two tools together provide an excellent toolkit to work with radar data.

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>