Building Containers

Overview

Teaching: 20 min
Exercises: 10 min

Questions

• How to build containers with my requirements?

Objectives

• Download and assemble containers from available images in the repositories.

Running containers from the available public images is not the only option. In many cases, it is required to modify an image or even to create a new one from scratch. For such purposes, Apptainer provides the command build, defined in the documentation as the Swiss army knife of container creation.

The usual workflow is to prepare and test a container in a build environment (like your laptop), either with an interactive session or from a definition file, and then to deploy the container into a production environment for execution (as your institutional cluster). Interactive sessions are great to experiment and test your new container. If you want to distribute the container or use it in production, then we recommend to build it from a definition file, as described in the next episode. This ensures the greatest possibility of reproducibility and transparency.

Build a container in an interactive session

While images contained in the .sif files are more compact and immutable objects, ideal for reproducibility, for building and testing images is more convenient the use of a sandbox, which can be easily modified.

The command build provides a flag --sandbox that will create a writable directory, myAlma9, in your work directory:

apptainer build --sandbox myAlma9 docker://almalinux:9

Notes on shared file systems like AFS

Avoid using the AFS (Andrew File System) and possibly other shared file systems as sandbox directory, as these systems can lead to permission issues. Symptoms could be warnings like “harmless EPERM on setxattr “security.capability” when building the sandbox and IO errors during commands execution, e.g. failure to install RPM packages. In particular, this applies to your home directory on lxplus and to the home and nocache directories on cmslpc. Instead, make sure to use the local file system by creating a folder in /tmp/: mkdir /tmp/$USER. Then, replace myAlma9 in the previous (and next) command with /tmp/$USER/myAlma9.

The container name is myAlma9, and it has been initialized from the official Docker image of AlmaLinux9. To initialize an interactive session use the shell command. And to write files within the sandbox directory use the --writable option. The --cleanenv option is added to make sure that the host environment is not affecting the container. Finally, the installation of new components will require superuser access inside the container, so use also the --fakeroot option, unless you are already root also outside.

apptainer shell --writable --cleanenv --fakeroot myAlma9
Apptainer> whoami

root

--cleanenv clears the environment. It has been added to make sure that the eventual setting of variables on the host is not affecting the container. Variables like PYTHONPATH or PYTHONHOME are affecting the Python execution inside the container. A corrupted Python environment could cause errors like “ModuleNotFoundError: No module named ‘encodings’”

Apptainer environment

Environment variables in Linux are dynamic values that can affect programs and containers. You can use the environment to pass variables to a container. Apptainer by default preserves most of the outside environment inside the container but has many options to control that. You can clear the environment with the --cleanenv option and you can set variables with --env. See the Apptianer manual for more options and details.

PYTHONPATH and PYTHONHOME affect the Python execution, but other variables could affect other programs so, if you don’t care about the outside environment, you can add --cleanenv all the times you start a container (apptainer shell, exec and instance start commands).

Depending on the Apptainer/Singularity installation (privileged or unprivileged) and the version, you may have some requirements, like the fakeroot utility or newuidmap and newgidmap. If you get an error when using --fakeroot have a look at the fakeroot documentation.

--fakeroot is not root

ATTENTION! --fakeroot allows you to be root inside a container that you own but is not changing who you are outside. All the outside actions and the writing on bound files and directories will happen as your outside user, even if in the container is done by root.

As an example, let’s create a container with Pythia8 available using the myAlma9 sandbox. First, we need to enable the CRB (Code Ready Builder/PowerTools) and EPEL (Extra Packages for Enterprise Linux) repositories and to install the development tools (remember that in this interactive session we are superuser):

Apptainer> dnf install 'dnf-command(config-manager)'  # this installs dnf-plugins-core
Apptainer> dnf config-manager --set-enabled crb
Apptainer> dnf install epel-release
Apptainer> dnf groupinstall 'Development Tools'
Apptainer> dnf install python3-devel

Where dnf is the package manager used in RHEL distributions (like AlmaLinux).

Now you can follow all the installation steps described in the Pythia website. Here is a summary of the commands you will need (you may need to adjust link and commands if there is a new Pythia version):

Apptainer> mkdir /opt/pythia && cd /opt/pythia
Apptainer> curl -o pythia8310.tgz https://pythia.org/download/pythia83/pythia8310.tgz
Apptainer> tar xvfz pythia8310.tgz
Apptainer> ln -s pythia8310 pythia8

To enable the Python interface, we add the flag --with-python-include during the configuration, pointing to the location of the Python headers:

Apptainer> cd pythia8310
Apptainer> ./configure --with-python-include=/usr/include/python3.9
Apptainer> make

Apptainer> exit

Installing the Pythia binary package

Pythia is now distributed as RPM in EPEL, so you can use this easier alternative to install it:

Apptainer> dnf install pythia8
Apptainer> dnf install python3-pythia8
Apptainer> dnf install pythia8-devel  # optional, if you need also the development libraries to compile

Now, open an interactive session with your user (no --fakeroot). You can use now the container with Pythia8 ready in a few steps. Let’s use the Python interface:

apptainer shell myAlma9

# These first 2 lines are necessary only if Pythia was installed from source, not if you used the binary package
Apptainer> export PYTHONPATH=/opt/pythia/pythia8/lib:$PYTHONPATH
Apptainer> export LD_LIBRARY_PATH=/opt/pythia/pythia8/lib:$LD_LIBRARY_PATH
Apptainer> python3

>>> import pythia8
>>> pythia = pythia8.Pythia()

 *------------------------------------------------------------------------------------*
 |                                                                                    |
 |  *------------------------------------------------------------------------------*  |
 |  |                                                                              |  |
 |  |                                                                              |  |
 |  |   PPP   Y   Y  TTTTT  H   H  III    A      Welcome to the Lund Monte Carlo!  |  |
 |  |   P  P   Y Y     T    H   H   I    A A     This is PYTHIA version 8.310      |  |
 |  |   PPP     Y      T    HHHHH   I   AAAAA    Last date of change: 25 Jul 2023  |  |
 |  |   P       Y      T    H   H   I   A   A                                      |  |
 |  |   P       Y      T    H   H  III  A   A    Now is 24 Feb 2024 at 22:04:44    |  |

...
 |  |   Copyright (C) 2022 Torbjorn Sjostrand                                      |  |
 |  |                                                                              |  |
 |  *------------------------------------------------------------------------------*  |
 |                                                                                    |
 *------------------------------------------------------------------------------------*

Notice that when installing from source we need to define the environment variables PYTHONPATH and LD_LIBRARY_PATH in order to use Pythia on Python. We will automate this in the next section.

Execute Python with PyROOT available

Build a container to use Uproot, a library for reading and writing ROOT files in pure Python and NumPy, in Python 3.9.

Solution

Start from the Python 3.9 Docker image and create the myPython sandbox:

apptainer build --sandbox myPython docker://python:3.9
apptainer shell myPython

Once inside the container, you can install Uproot.

Apptainer> python3 -m pip install --upgrade pip
Apptainer> python3 -m pip install uproot awkward

Exit the container and use it as you like:

apptainer exec myPython python -c "import uproot; print(uproot.__doc__)"

Uproot: ROOT I/O in pure Python and NumPy.
...

Notice how we did not need neither --writable nor --fakeroot for the installation, but everything worked fine since pip installs user packages in the user $HOME directory. In addition, Apptainer/Singularity by default mounts the user home directory as read+write, even if the container is read-only. This is why a sandbox is great to test and experiment locally, but should not be used for containers that will be shared or deployed. Manual changes and local directories are difficult to reproduce and control. Once you are happy with the content, you should use definition files, described in the next episode.

Key Points

• The command build is the basic tool for the creation of containers.

• A sandbox is a writable directory where containers can be built interactively.

• Superuser permissions are required to build containers if you need to install packages or manipulate the operating system.

• Use interactive builds only for development and tests, use definition files for production or publicly distributed containers.