Interactive Slurm Sessions

Introduction

The interactive helper is a small wrapper around Slurm that makes it easier to start interactive jobs on Merlin7. It supports two different modes of operation:

• a default non-X11 mode, which creates a persistent interactive allocation that can be re-used later
• an X11 mode, intended for graphical programs, which starts a direct salloc --x11 session

Although both modes are useful, they behave differently and should be used with different expectations. The most important difference is simple:

• X11-based sessions are not detachable
• non-X11 sessions are detachable and remain alive until you cancel them

Because of that, users should choose the mode that best matches their workflow.

Default behaviour and recommended resources

By default, the script submits jobs to the interactive partition and uses the interactive reservation. This is the recommended way to use it.

The reason is that both are configured specifically for interactive work:

• the interactive partition is meant for shared-CPU interactive allocations
• the interactive reservation provides direct access to reserved interactive resources

In practice, this means that users normally get fast access to an interactive session without having to wait in queue.

Tip

The recommended default is simply to let the script use its built-in defaults:

Bash

interactive          # default, detachable interactive session
interactive --x11    # non-detachable x11-based interactive session

This uses: --partition=interactive --reservation=interactive

If users decide to use another partition, or use the interactive partition without the reservation, that is possible, but then the allocation behaves like a normal Slurm request and may have to wait until resources become available.

Warning

The interactive reservation is what provides immediate access to reserved resources.

If users start noticing delays when using the default interactive setup, please inform the Merlin Admins so the reservation can be reviewed or extended.

Interactive partition, QoS and reservation

The default interactive setup is backed by a dedicated partition, QoS and reservation.

• The interactive partition currently provides shared CPU resources on interactive nodes and is configured with oversubscription enabled. The default time limit is 12 hours, and jobs run under the cpu_interactive QoS.
• The cpu_interactive QoS is designed for modest interactive work. In particular, it limits a single job to up to 16 CPUs, 30 GB of memory, and 1 node. Per user, it allows up to 128 CPUs, 240 GB of memory, in total.
• The interactive reservation currently reserves interactive CPU capacity on the interactive partition for the merlin and admin accounts. This is what makes immediate interactive access possible in normal conditions.

Note

The exact Slurm configuration may evolve over time. The values above reflect the current configuration of the interactive partition, the cpu_interactive QoS, and the interactive reservation at the time this page was written.

You can check any changes by using the following commands:

Bash

# Partition information
scontrol show partition interactive

# QoS information
sacctmgr show qos cpu_interactive format=Name%20,Priority,MaxTRES%25,MaxTRESPU%25

# Reservation information
scontrol show reservation interactive

Use cases

Use case 1: Standard interactive sessions (default, detachable)

This is the default and recommended mode for most users.

In this mode, the script creates a persistent Slurm allocation by submitting a small holder job that simply runs sleep infinity. After that, it opens a shell inside the allocation with srun --jobid.

This means the allocation continues to exist even after the user exits the shell. You can therefore disconnect and later attach to the same allocation again.

A minimal example is:

Bash

interactive

The first time, this creates the interactive allocation and attaches a shell. If a matching allocation already exists, the script reuses it and attaches to it again.

When you leave the shell with exit, the shell ends, but the Slurm allocation keeps running. That is why this mode is considered detachable.

Warning

In the default mode, leaving the shell does not stop the interactive allocation. The job remains allocated and continues to consume resources until it is cancelled.

If you are done with the session, explicitly cancel it:

Bash

interactive --cancel

This behaviour is useful for workflows such as:

• Keeping a long-lived shell on a compute node
• Reconnecting later to the same interactive environment
• Using terminal-based tools, editors, debugging sessions, or text-mode applications
• Preparing a session and then connecting to the allocated node through SSH or VS Code

The script also provides helpers to inspect or reuse the current allocation:

Bash

interactive --status
interactive --print-node
interactive --print-ssh
interactive --print-vscode

--status shows both normal and X11 interactive jobs. --print-node prints the allocated node name. The SSH and VS Code helpers print connection snippets for the current normal interactive allocation.

Cancelling detachable sessions

A detachable session should be cancelled when no longer needed:

Bash

interactive --cancel

If local state was lost, the script can still find the job by its stable job name and cancel it.

If you want to remove only the local state file without cancelling the job, use:

Bash

interactive --reset

Warning

--reset only removes the local bookkeeping file. It does not stop the running Slurm job.

Use case 2: X11 interactive sessions (non-detachable)

The X11 mode is intended for graphical interactive programs. In this case the script does not create a persistent holder job. Instead, it starts a direct salloc --x11 allocation.

A basic example is:

Bash

interactive --x11

This starts a normal interactive X11 allocation and drops you into the interactive session. Unlike the default mode, this session is not persistent and cannot be re-attached.

When you leave the X11 session, the allocation ends as well.

Warning

X11-based sessions are not detachable. Once the shell exits, the allocation ends. If the connection is lost, the session cannot be resumed.

This mode is appropriate for workflows such as:

• Launching simple graphical applications
• Running interactive plotting tools that need X11 forwarding
• Short debugging sessions that require a GUI

Because this is a live salloc --x11 session, users should treat it as an active foreground session, not as something that can be parked and resumed later.

If an X11 session is still active and you want to cancel it explicitly, use:

Bash

interactive --x11 --cancel

To cancel both normal and X11 allocations at once, use:

Bash

interactive --cancel-all

Choosing resources

By default, the script requests:

Text Only

--nodes=1
--cpus-per-task=4
--time=12:00:00
--partition=interactive
--reservation=interactive

These defaults can be overridden through INTERACTIVE_* environment variables (run interactive --help for detailed information) or by passing extra Slurm options after --. For example:

Bash

interactive -- --cpus-per-task=8 --mem=16G
interactive --x11 -- --cpus-per-task=8 --mem=16G

In general, users should request interactive resources with --cpus-per-task. This is the normal and recommended option for most interactive workflows.

The script also accepts --ntasks, but this is mainly intended for cases where users really need to run MPI-based processes interactively. That should be relatively uncommon.

Tip

For most interactive work, think in terms of:

• One shell
• One process or one application
• Several CPUs for that process

In that case, --cpus-per-task is the correct option.

Warning

Do not use --ntasks unless you actually need multiple Slurm tasks, typically for MPI. For ordinary shells, editors, compilers, notebooks, debugging, or GUI tools, --cpus-per-task is the correct choice.

Using other partitions or running outside the reservation

The script does not force users to stay on the default interactive setup. It is possible to pass other Slurm options and request a different partition or omit the reservation.

However, users should understand the consequence: outside the default interactive reservation, there is no guarantee that resources will be immediately available. If the requested resources are busy, the job will wait in queue.

Also, interactive jobs that occupy normal compute resources but sit idle may be cancelled if they are considered misuse of cluster resources.

Warning

The recommended usage is the default one:

Bash

interactive

Using other partitions is supported, but interactive sessions on regular compute resources should only be kept alive while they are actively needed.

Command examples

A few common examples are shown below.

Start a normal detachable session:

Bash

interactive

Reconnect to the same normal session later:

Bash

interactive

Check currently active normal and X11 interactive jobs:

Bash

interactive --status

Cancel the normal detachable session:

Bash

interactive --cancel

Start an X11 session:

Bash

interactive --x11

Cancel an active X11 session:

Bash

interactive --x11 --cancel

Request more CPUs and memory in the default mode:

Bash

interactive -- --cpus-per-task=8 --mem=16G

Request more CPUs and memory in X11 mode:

Bash

interactive --x11 -- --cpus-per-task=8 --mem=16G

Final recommendations

For almost all users, the best approach is:

1. Use the default interactive partition and interactive reservation.
2. Use the normal non-X11 mode unless graphical forwarding is really needed.
3. Request CPUs with --cpus-per-task, always considering QoS limits.
4. Remember that default interactive sessions are persistent and must be cancelled explicitly when finished.

If a session behaves unexpectedly, or if the interactive reservation no longer provides immediate access, please contact the Merlin admins.