Skip to content

R on the HPC

Loading R

R is not available as a plain module — it is installed through spack. After connecting to Emmy-p3:

module load spack
source $SPACK_ROOT/share/spack/setup-env.sh
spack find r                  # list available R versions
spack load r@4.5.2            # load the recommended version

Installing R Packages

Start R on the HPC frontend and install packages as usual. On first install, R will ask whether to create a personal library in $HOME — confirm with yes.

install.packages("rslurm")

To install from GitHub, use withr to ensure the package goes to your personal library:

install.packages(c("devtools", "withr"))
withr::with_libpaths(
  new = .libPaths()[1],
  code = devtools::install_github("user/repo")
)

All packages used in your simulation function must also be installed on the HPC under the same R version. If you update R via spack, packages need to be reinstalled for the new version.

Running Parallel Jobs with rslurm

Our standard workflow for running many simulation runs in parallel uses the rslurm package. Define your simulation as an R function, pass a data frame of parameter combinations, and let rslurm generate the SLURM job scripts.

Following the parallelization principle from Section 2: we submit one job per parameter combination, each requesting one CPU.

1. Create a job template

rslurm generates a shell script that SLURM runs on each compute node. By default this script does not load any modules or set environment variables, so we provide a custom template. Create a file called submit_template.sh locally (e.g. on your laptop) in your project folder:

#!/bin/bash
#SBATCH --array=0-{{{max_node}}}{{{job_array_task_limit}}}
#SBATCH --cpus-per-task={{{cpus_per_node}}}
#SBATCH --job-name={{{jobname}}}
#SBATCH --output=slurm_%a.out
{{#flags}}
#SBATCH --{{{name}}}
{{/flags}}
{{#options}}
#SBATCH --{{{name}}}={{{value}}}
{{/options}}

module load spack
source $SPACK_ROOT/share/spack/setup-env.sh
spack load r@4.5.2   # must match the version used to install your R packages on the HPC

export OMP_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1

{{{rscript}}} --vanilla slurm_run.R

The {{{ }}} placeholders are filled in by rslurm. The available variables are:

Placeholder Value
{{{max_node}}} Highest job array index (number of nodes minus one)
{{{job_array_task_limit}}} Optional %N throttle appended after the array range
{{{cpus_per_node}}} CPUs requested per node
{{{jobname}}} Job name passed to slurm_apply
{{{rscript}}} Full Rscript invocation, including rscript_path and any container wrapper
{{#flags}} … {{/flags}} Boolean #SBATCH options (no value)
{{#options}} … {{/options}} Key=value #SBATCH options from slurm_options

2. Write the rslurm script locally (e.g. on your laptop)

library(rslurm)

# The function to run on each node — receives one row of params per call
sim_fun <- function(param_a, param_b) {
  # Limit parallelization inside the function if your code or packages
  # have their own settings — e.g. data.table uses getDTthreads()
  # data.table::setDTthreads(1)

  result <- param_a * param_b   # replace with your simulation
  return(result)
}

# One row per simulation run
params <- data.frame(
  param_a = 1:10,
  param_b = seq(0.1, 1.0, by = 0.1)
)

sjob <- slurm_apply(
  f             = sim_fun,
  params        = params,
  jobname       = "my_sim",
  nodes         = nrow(params),     # one node per parameter combination
  cpus_per_node = 1,
  sh_template   = "submit_template.sh",
  slurm_options = list(
    "partition"   = "scc-cpu",
    "time"        = "01:00:00",
    "mem-per-cpu" = "2G",
    "mail-type"   = "END,FAIL",
    "mail-user"   = "your@email.de"
  ),
  submit = FALSE   # creates job folder locally without submitting
)

Running this creates a _rslurm_my_sim/ folder containing the generated job scripts. Keep submit = FALSE when working locally — submission happens on the HPC in step 4.

3. Transfer to the HPC

Transfer the job folder to the HPC (from your laptop). The template is only needed locally — rslurm already used it to generate submit.sh inside the job folder.

rsync -av _rslurm_my_sim/ Emmy-p3:~/_rslurm_my_sim/

4. Submit and monitor

ssh Emmy-p3
cd _rslurm_my_sim
sbatch submit.sh

Monitor progress (or use the jobs_own alias from Section 2):

squeue --me

Job output and any errors are written to slurm_*.out files in the job folder, one file per job array task.

5. Retrieve results

Once all jobs have finished:

rsync -avP Emmy-p3:~/_rslurm_my_sim/ ./_rslurm_my_sim/

Read results back in R on your laptop:

results <- get_slurm_out(sjob, outtype = "table")

Debugging

If jobs fail, check the slurm_*.out log files in the _rslurm_* folder on the HPC for error messages.

If jobs are stuck in PENDING, check your fair-share score:

sshare -U $USER

If stray R processes are running on the login node from a failed submission, kill them:

htop -u $USER   # press F9 to kill a selected process, F10 to exit

Sources

Open Questions