Using Conda and Python

Installing and Using Conda

Conda is a cross-platform package manager that installs and manages software packages. We also advise you don't initialise the shell with conda init to avoid complications associated with modifying shell startup scripts. Instead we advise the use of the activate script provided with Miniforge (see below) to perform shell initialisation as needed.

conda-forge

Our recommended installation method is to install Conda Miniforge. This is due to the need of a license to use the mainline anaconda channel.

To install the latest version of Conda:

$ cd $HOME
$ curl --location --remote-name "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
$ bash Miniforge3-$(uname)-$(uname -m).sh
$ rm Miniforge3-$(uname)-$(uname -m).sh

then to activate:

$ source ~/miniforge3/bin/activate

You should not install packages in your base environment, instead create separate environments to manage your software packages.

E.g. to create an environment named test with an installation of Python 3.10:

(base) $ conda create -n test python=3.10 
(base) $ conda activate test

You can then install further packages in your test environment using conda install. E.g. to install the Python package scipy:

(test) $ conda install scipy

It can be useful to specify the packages in a Conda environment in a Conda environment YAML file. This allows creation of a Conda environment using a single command:

$ conda env create -f environment.yml

You can list the installed packages in your environment using conda list and you can deactivate your environment using conda deactivate.

Finding aarch64 compatible conda packages

Since Isambard clusters are mainly based on Linux Arm64 architecture (aarch64), it is important to find packages built for this architecture. To find these packages, please go to the anaconda organisation website.

Installing Python Packages

We recommended you use Conda environments to install and manage your Python packages as above.

Alternatively, Cray Python is available as a pre-installed module. It can be accessed as follows:

$ module avail # list available modules
...
$ module load cray-python
$ which python3
/opt/cray/pe/python/3.11.5/bin/python3
$ python3
Python 3.11.5 (main, Nov 29 2023, 20:19:53) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

When using Cray Python we recommend working inside virtual environments using the Python module venv. Virtual environments isolate your pip installed dependencies for each unique project.

Let's create a venv:

$ mkdir ~/.virtualenvs/ # Create folder for virtual environments
$ python3 -m venv --upgrade-deps ~/.virtualenvs/test2

To activate our test2 environment and install the package scipy:

$ source ~/.virtualenvs/test2/bin/activate
(test2) $ which python3
$HOME/.virtualenvs/test2/bin/python3
(test2) $ python3 -m pip install scipy

To list the installed packages in your environment:

(test2) $ python3 -m pip list

Note the use of the module approach when using pip, i.e. python3 -m pip, rather than pip. This helps avoid confusion around which Python installation/virtual environment pip is acting on. You can exit your environment using deactivate.