Containers

Container frameworks such as Docker allow an application, its dependencies and operating system to be encapsulated in a single file that can then be run in isolation from the host system. This allows your software to be more portable and consistent if it needs to be shared with others or other environments.
 
Containers should be considered if a software stack:

  • Has complex or out-of-date dependencies that aren't easily installed on Spartan.
  • Assumes a particular operating system, e.g. requires Ubuntu, but Spartan runs on Red Hat.
  • Is highly-modified or a legacy version that isn't likely to be of use to other researchers.
  • Is being run across heterogeneous infrastructure (e.g. Spartan, your laptop, other HPC systems and/or cloud), and containers make it easier to maintain consistency.

It should be noted that:

  • The images themselves can be very large, consuming your storage quota, and being slow to transfer to/from Spartan.
  • The container might be optimized for a particular processor architecture, running slowly (or not at all) on systems that differ.
  • Their makeup and integrity can be opaque (although many common software packages will have officially supported container images).

In general Docker isn't appropriate for HPC environments like Spartan in which regular users don't have administrator (root) access. We use Apptainer (or previously Singularity) as it is designed for HPC security.

Note

An example job script for Apptainer is available at /apps/examples/Apptainer, which is also mirrored on GitLab.

To use GPUs in your Apptainer container, add the --nv option to apptainer

e.g. apptainer exec --nv mycontainer.img bash

This adds the necessary libraries and binaries (e.g. nvidia-smi, nvcc) into the container at runtime

By default, only your home directory is added to the container when you run it. If your container tries to access your project or scratch directories, it won't have access.

To add project and scratch access, set the APPTAINER_BINDPATH environment variable. This bind mounts paths into the container at runtime. e.g.

export APPTAINER_BINDPATH=/data:/data will bind mount /data outside the container to /data inside the container