Anaconda

Anaconda provides Python and a long list of packages as well as Jupyter and environment and package manager conda and pip.

Many bioinformatics software is distributed via Anaconda.

To use anaconda on Spartan, load the required anaconda module. You can then set the conda environment path, create a conda environment, install packages, and use it in a job.

Note

[scrosby@spartan-login1 ~]$ module load Anaconda3/2022.10

By default, conda creates environments in your home directory, which only has a 50GB limit. With many conda environments, this space can be easily filled.
 
You can change this path using the environment variable CONDA_ENVS_PATH.

To change the conda environments location to a project directory (e.g. /data/gpfs/projects/punim0001), add this to your ~/.bash_profile file

export CONDA_ENVS_PATH=/data/gpfs/projects/punim0001/anaconda3/envs

To create a conda env called myenv, load the anaconda module, and then do

[scrosby@spartan-login1 ~]$ conda create --name myenv
Collecting package metadata (current_repodata.json): done
Solving environment: done

To activate a created environment myenv, load the anaconda module, and then do

[scrosby@spartan-login1 ~]$ eval "$(conda shell.bash hook)"
[scrosby@spartan-login1 ~]$ conda activate myenv
(myenv) [scrosby@spartan-login1 ~]$ 

Warning

Do not use conda init as this adds unwanted and problematic code to your .bashrc file

To install a package in your conda env, activate your conda environment, and then do

(myenv) [scrosby@spartan-login1 ~]$ conda install numpy
Collecting package metadata (current_repodata.json): done
Solving environment: done

To use your conda environment in a job, add the following to your submit script

module load Anaconda3/2022.10
eval "$(conda shell.bash hook)"
conda activate myenv

Warning

Do not try to use modules from our module system as well as packages installed in a conda environment in the same job.
If you use a conda environment, install everything you require for that job inside of your conda environment (e.g. don't use Tensorflow from our module system inside a job where you are using a conda environment). Mixing modules from our software system with packages installed in conda environments will fail or cause major issues.