- Introduction
- Versions
- Adding Python to Your Environment
- Installing Python Packages (in user specified locations)
- Viewing Installed Python Packages
Introduction
Python is an interpreted, high-level, general-purpose programming language..
Python and Python packages are available via the “The Python Package Index” (PyPI), which hosts thousands of third-party modules for Python. Both Python’s standard library and the community-contributed modules allow for endless possibilities.
Anaconda also provides a Python environment with python packages
Versions
The table below list the versions of Python installed on the Campus Cluster.
| Version | modulefile | |
|---|---|---|
| Python 2 | 2.7.15 | python/2 |
| Python 3 | 3.7.0 3.9.16 |
python/3 python/3.9.16 |
| Anaconda 5.2.0 | Python 2.7.15 Python 3.6.5 |
anaconda/2 anaconda/3 |
| Anaconda 2018.12 | Python 2.7.15 Python 3.6.8 |
anaconda/2018-Dec/2 anaconda/2018-Dec/3 |
| Anaconda 2019.10 | Python 2.7.16 Python 3.7.4 |
anaconda/2019-Oct/2 anaconda/2019-Oct/3 |
| Anaconda 2020.07 | Python 3.8.3 | anaconda/2020-Jul/3 |
| Anaconda 2021.05 | Python 3.8.8 | anaconda/2021-May/3 |
| Anaconda 2022.05 | Python 3.9.12 | anaconda/2022-May/3 |
| Anaconda 2023.03 | Python 3.10.9 | anaconda/2023-Mar/3 |
Adding Python To Your Environment
Each Python installation on the Campus Cluster has a corresponding modulefile that can be used to load a specific version of Python into your user environment. You can see the available versions of Python by typing module avail python and/or module avail anacondaon the command line. See the section Managing Your Environment in the User Guide for more information about modules.
Installing Python Packages (in user specified locations)
Generally, any Python package not available in the system installation (see here for information on viewing the list) can be installed from the Python Package Index (PyPI) in a user specified location that you have write access to with just a few easy steps.
Generally, when installing any software there are a few things to consider
- Home directory disk usage quotas.
- Will the software be used by multiple users in my group?
(Project space is the **recommended** location to install software.) - Does the desired software have dependencies/requirements?
- Is the prerequisite software already installed?
Campus Cluster users must install software/libraries into user write-able locations like their home directory, their group’s project space or their scratch space.
The example instructions below use scratch space. Please note that software installed in scratch space is not permanent and may be removed at any time. We recommend that you install in your group’s project space. [So for a persistent installation, users should replace the path to their scratch space with the path to their project space in the instructions below]
Note: There is a 5GB quota (disk usage) on all user’s home directories, so we do not recommend that users install software/libraries in their home directories since multiple Python packages (or an Anaconda installation) can push the disk usage over quota very quickly. Additionally, users should use one of the installation methods listed below (Python or Anaconda). Combining installation methods can cause unexpected behavior/errors.
Installation Command Syntax
Under Python:
To Install a specific python package into a user location, use the “pip install” command with the “-t” option.
module load python/3
mkdir -p /home/${USER}/scratch/mypython3
pip install -t /home/${USER}/scratch/mypython3 phonopy
export PYTHONPATH=/home/${USER}/scratch/mypython3:${PYTHONPATH}
Users will have to run the above “export…” command after loading python into their user environment with the module command on future login sessions. This allows the Campus Cluster’s python installation to see the user’s specific python packages.
Alternatively users can add two commands
module load python/3
export PYTHONPATH=/home/${USER}/scratch/mypython3:${PYTHONPATH}
to their ${HOME}/.bashrc file so that Python is setup for their user environment every time they login to the Campus Cluster.
Note: You can run the “pip install -t …” command for each python package/library you want to install.
Under Anaconda:
The following commands will create an anaconda environment in your scratch directory (which is temporary), we recommend that you use your group’s project space to install software/libraries specific to your needs.
To create a minimal anaconda environment with python and any dependencies, type the following:
(other packages can be used in place of “python” if you want to base your environment around that specific package Ex. “tensorflow”)
mkdir -p /scratch/users/$USER/my.conda.dir
cd ${HOME}
ln -s /scratch/users/${USER}/my.conda.dir .conda
module load anaconda/2023-Mar/3
conda create -n my.anaconda python
conda info -e
source activate my.anaconda
conda info -e
conda install pytorch
conda list
Note: Use the command source activate environment_name instead of conda init or conda activate environment_name to activate your anaconda environment.
To deactivate the anaconda environment just type
conda deactivate
To create a complete clone anaconda environment replace
conda create -n my.anaconda python
with
conda create -n my.anaconda anaconda
Viewing Installed Python Packages
After enabling Python in your user environment by loading a Python or Anaconda modulefile, one can view a list of the Python packages installed under the Python/Anaconda installation (including your own installed packages) by typing pip list if you have loaded a Python modulefile or by typing conda list if you have loaded an Anaconda modulefile.