1. Introduction
  2. Versions
  3. Adding Python to Your Environment
  4. Installing Python Packages (in user specified locations)
  5. Viewing Installed Python Packages

Introduction

Python is an interpreted, high-level, general-purpose programming language..

Python and Python packages are available via the “The Python Package Index” (PyPI), which hosts thousands of third-party modules for Python. Both Python’s standard library and the community-contributed modules allow for endless possibilities.

Anaconda also provides a Python environment with python packages

Versions

The table below list the versions of Python installed on the Campus Cluster.

Version
Python 2 2.7.15
Python 3 3.7.0
Anaconda 5.2.0 Python 2.7.15
Python 3.6.5
Anaconda 2018.12 Python 2.7.15
Python 3.6.8
Anaconda 2019.10 Python 2.7.16
Python 3.7.4
Anaconda 2020.07 Python 3.8.3

Adding Python To Your Environment

Each Python installation on the Campus Cluster has a corresponding modulefile that can be used to load a specific version of Python into your user environment. You can see the available versions of Python by typing module avail python and/or module avail anacondaon the command line. See the section Managing Your Environment in the User Guide for more information about modules.

Installing Python Packages (in user specified locations)

Generally, any Python package not available in the system installation (see here for information on viewing the list) can be installed from the Python Package Index (PyPI) in a user specified location that you have write access to with just a few easy steps.

Generally when installing any software there are a few things to consider

  • Home directory disk usage quotas.
  • Will the software be used by multiple users in my group?
    (Project space is the **recommended** location to install software.)
  • Does the desired software have dependencies/requirements?
  • Is the prerequisite software already installed?

Campus Cluster users must install software/libraries into user write-able locations like their home directory, their group’s project space or their scratch space.

The example instructions below use scratch space. Please note that software installed in scratch space is not permanent and may be removed at any time. We recommend that you install in your group’s project space.

[So users would replace the path to their scratch space with the path to their
project space in the instructions below]

Note: There is a 5GB quota (disk usage) on all user’s home directories, so we do not recommend that users install software/libraries in their home directories since multiple Python packages (or an Anaconda installation) can push the disk usage over quota very quickly.

Installation Command Syntax

Under Python:

To Install a specific python package into a user location use the “pip install” command with the “-t” option.

  module load python/3

  mkdir -p /home/$USER/scratch/mypython3

  pip install -t /home/$USER/scratch/mypython3  phonopy

  export PYTHONPATH=/home/$USER/scratch/mypython3:${PYTHONPATH}

Users will have to run the above “export…” command after loading python into their user environment with the module command on future login sessions. This allows the Campus Cluster’s python installation to see the user’s specific python packages.

Alternatively users can add two commands

  module load python/3
   
  export PYTHONPATH=/home/$USER/scratch/mypython3:${PYTHONPATH}

to their $HOME/.bashrc file so that Python is setup for their user environment every time they login to the Campus Cluster.

Note: You can run the “pip install -t …” command for each python package/library you want to install.

Under Anaconda:

The following commands will create an anaconda environment in your scratch directory (which is temporary) We recommend that you use your group’s project space to install software/libraries specific to your needs.

To create a minimal clone anaconda environment type the following:

  mkdir -p /scratch/users/$USER/my.conda.dir

  ln -s /scratch/users/$USER/my.conda.dir .conda

  module load anaconda/2019-Oct/3

  conda create -n my.anaconda python

  conda info -e

  source activate my.anaconda

  conda info -e

  conda install pytorch

  conda list

To deactivate the anaconda environment just type

  conda deactivate

To create a complete clone anaconda environment replace

  conda create -n my.anaconda python

with

  conda create -n my.anaconda anaconda

Viewing Installed Python Packages

After enabling Python in your user environmnet by loading a Python or Anaconda modulefile, one can view a list of the Python packages installed under Python/Anaconda installation (including your own installed packages) by typing pip list if you have loaded a Python modulfile or by typing conda list if you have loaded an Anaconda modulefile.