Skip to content

R in an Apptainer Container

When to Use This Approach

Use Apptainer when your workflow depends on software that cannot be installed via spack or the module system — for example a Docker image that bundles a specific R version together with external tools or system libraries your code requires.

If you only need R and standard R packages, use the spack-based workflow in Section 3 instead. Apptainer adds complexity.

The rslurm workflow from Section 3 stays largely the same. The two things that change are: how R is started (via the container instead of directly), and the job template.

Building the Container Image

Apptainer runs containers from a single image file (.sif). The most common starting point is a Docker image from Docker Hub or a similar registry.

Important for SCC users: building images is possible, but not advised on the login node as it is compute-intensive and slows down the login node for all users. Therefore, start an interactive compute job first:

srun -p scc-cpu -c 8 -N 1 -t 30 --pty bash # interactive session, 8 CPUs, 1 Node, timeout after 30 mins
module load apptainer

Store container images in a personal subdirectory of $PROJECT. Use your first name or a project name (e.g. $PROJECT/sebastian) rather than $USER — the HPC account names (u12345) are not recognisable without looking them up. Create the folder if it doesn't exist yet:

mkdir -p $PROJECT/<name>

Then build the image:

apptainer build $PROJECT/<name>/mycontainer.sif docker://organisation/imagename:tag

The file can be several GB — $PROJECT has more quota than $HOME.

To inspect the container interactively:

apptainer shell --bind $HOME,$PROJECT $PROJECT/<name>/mycontainer.sif

The --bind flag makes HPC filesystems accessible inside the container. Without it, the container only sees its own internal filesystem.

If your simulation writes temporary files intensively, you can also bind the job-specific temporary directories for better I/O performance:

apptainer shell --bind $HOME,$PROJECT,$LOCAL_TMPDIR $PROJECT/<name>/mycontainer.sif

$LOCAL_TMPDIR (local SSD, fastest) and $SHM_TMPDIR (RAM, fastest but tiny) are only available inside a running SLURM job, not on the login node. See Section 2 and the GWDG temporary storage docs for details.

Adding R Packages to the Container

If you need R packages beyond what the container already provides, add them to the Docker image and rebuild the .sif file — do not install packages into your personal library from inside the container. Ad-hoc installs into $HOME create a dependency on the container version: if the image is updated, those installs may break silently.

Update the Dockerfile (or equivalent) upstream, push a new image, then rebuild:

srun -p scc-cpu -c 8 -N 1 -t 30 --pty bash
module load apptainer
apptainer build mycontainer.sif docker://organisation/imagename:tag

Running Parallel Jobs with rslurm

1. Create a job template

The template for container-based jobs extends the one from Section 3 with a module load apptainer line:

#!/bin/bash
#SBATCH --array=0-{{{max_node}}}{{{job_array_task_limit}}}
#SBATCH --cpus-per-task={{{cpus_per_node}}}
#SBATCH --job-name={{{jobname}}}
#SBATCH --output=slurm_%a.out
{{#flags}}
#SBATCH --{{{name}}}
{{/flags}}
{{#options}}
#SBATCH --{{{name}}}={{{value}}}
{{/options}}

module load apptainer

export OMP_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1

{{{rscript}}} --vanilla slurm_run.R

Save this as submit_template_apptainer.sh locally (e.g. on your laptop).

2. Write the rslurm script locally (e.g. on your laptop)

The key difference from Section 3 is rscript_path: instead of a plain path to Rscript, it becomes an apptainer exec command that runs Rscript inside the container. rslurm appends --vanilla slurm_run.R to whatever you put here.

library(rslurm)

sim_fun <- function(param_a, param_b) {
  result <- param_a * param_b   # replace with your simulation
  return(result)
}

params <- data.frame(
  param_a = 1:10,
  param_b = seq(0.1, 1.0, by = 0.1)
)

sjob <- slurm_apply(
  f             = sim_fun,
  params        = params,
  jobname       = "my_container_sim",
  nodes         = nrow(params),
  cpus_per_node = 1,
  rscript_path  = "apptainer exec --bind $HOME,$PROJECT $PROJECT/<name>/mycontainer.sif Rscript",
  sh_template   = "submit_template_apptainer.sh",
  slurm_options = list(
    "partition"   = "scc-cpu",
    "time"        = "01:00:00",
    "mem-per-cpu" = "2G",
    "mail-type"   = "END,FAIL",
    "mail-user"   = "your@email.de"
  ),
  submit = FALSE
)

3–5. Transfer, submit, and retrieve results

These steps are identical to Section 3 — transfer the _rslurm_* folder to the HPC, sbatch submit.sh from Emmy-p3, and rsync results back when done.

See Section 3 — steps 3–5.

Sources

Open Questions