2. Linux container primitives for HPC

Running a CAE solver such as OpenFOAM, CalculiX, or Ansys Fluent inside a container on an HPC cluster is not the same as running a web service in Docker. A CFD job may need 64 MPI ranks, 512 GB of RAM, direct access to an InfiniBand HCA, and a shared scratch filesystem mounted at /scratch. At the same time, the cluster operator wants to guarantee that one user's solver cannot starve another's, cannot read another job's memory, and cannot accidentally issue privileged syscalls. The Linux kernel primitives that make this possible are namespaces, cgroups v2, capabilities, and seccomp. Understanding what each one isolates — and, more importantly, what it does not isolate — is the foundation for building a solver container that is both performant and safe.

Namespaces: what a solver process can see

Namespaces partition global kernel resources so that a process inside a container sees its own view of the system. For CAE workloads, four namespaces matter most:

• PID namespace — the solver's MPI ranks get their own PID tree. Rank 0 can be PID 1 inside the container, which simplifies signal handling: SIGTERM sent to PID 1 propagates to all ranks. Without a PID namespace, a stray mpirun cleanup can kill unrelated processes on the node.
• Mount namespace — the container sees only the solver installation, the case directory, and the scratch mount. This prevents a solver from walking into /home/otheruser or /etc on the host.
• Network namespace — by default this gives the container an isolated loopback. For MPI over InfiniBand you typically use host networking instead, because RDMA verbs need direct access to the physical HCA. This is a deliberate trade-off: you lose network isolation to gain wire-speed interconnect.
• UTS and IPC namespaces — separate hostname and System V/POSIX shared memory. IPC isolation matters because some solvers use shared-memory segments for intra-node rank communication; a private IPC namespace prevents collisions between concurrent jobs.

A critical HPC caveat: namespaces isolate visibility, not resources. Two containers with separate PID namespaces still compete for the same physical cores unless cgroups constrain them.

cgroups v2: bounding CPU, memory, and I/O

cgroups v2 provides a single unified hierarchy. For a CAE solver, the relevant controllers are cpu, cpuset, memory, and io. A typical allocation for a 32-rank job pinned to cores 0–31 with a 128 GB memory ceiling looks like this:

# Create a cgroup for the job
mkdir /sys/fs/cgroup/cae_job_1042

# Pin to physical cores 0-31 (no hyperthread siblings)
echo "0-31" > /sys/fs/cgroup/cae_job_1042/cpuset.cpus
echo "0"    > /sys/fs/cgroup/cae_job_1042/cpuset.mems

# Hard memory limit and swap disabled
echo "128G" > /sys/fs/cgroup/cae_job_1042/memory.max
echo "0"    > /sys/fs/cgroup/cae_job_1042/memory.swap.max

# CPU bandwidth: 32 cores worth of quota per 100ms period
echo "3200000 100000" > /sys/fs/cgroup/cae_job_1042/cpu.max

# Attach the solver process
echo $SOLVER_PID > /sys/fs/cgroup/cae_job_1042/cgroup.procs

Two settings deserve emphasis. First, cpuset.cpus is what actually delivers predictable performance — it prevents the scheduler from migrating MPI ranks across NUMA nodes, which is a common cause of 20–30% slowdowns in memory-bandwidth-bound solvers. Second, memory.max is a hard wall: when a solver exceeds it, the kernel invokes the OOM killer and terminates a rank. For a linear solver that allocates a large sparse matrix, you should size memory.max above the peak estimated footprint, not at the average. A useful rule of thumb is to monitor memory.peak during a short representative run and set the limit 15–20% higher.

Capabilities: dropping root without losing what you need

Traditional Unix gives root all-or-nothing power. Capabilities decompose that into ~40 distinct privileges. A CAE solver container should run as a non-root UID and hold almost none of them. The exceptions in HPC are narrow:

In practice you start from a minimal set and add back only what a failing job proves it needs. A common pattern is to run the container with --cap-drop=ALL --cap-add=SYS_NICE,IPC_LOCK and verify with capsh --print inside the container that the effective set matches expectations.

Seccomp: filtering syscalls

Seccomp-BPF lets you attach a filter that inspects every syscall and decides to allow, deny, or trap it. The default Docker profile blocks roughly 44 syscalls; for HPC you often need to relax it slightly because MPI and RDMA libraries use syscalls that a web container never touches. The most frequently encountered are io_uring_setup (used by some modern I/O paths), process_vm_readv (used by certain MPI implementations for intra-node transfer), and mbind/set_mempolicy (NUMA policy control).

A practical approach is to start from the default profile, run a representative solver case, and collect denials from the audit log:

# Observe seccomp denials for the job's cgroup
grep -i seccomp /var/log/audit/audit.log | grep cae_job_1042

Each denial line names the syscall and the offending PID. Add only those syscalls to an allow-list, then re-run. Do not disable seccomp entirely — it is one of the few mechanisms that stops a compromised or buggy solver from calling ptrace or mount.

Putting the primitives together

The four mechanisms are complementary, not interchangeable. Namespaces decide what the solver can see; cgroups decide how much it can consume; capabilities decide which privileged operations it can perform; seccomp decides which syscalls it can invoke. A well-built CAE container uses all four: a PID/mount/IPC namespace for process and filesystem isolation, a cpuset-and-memory cgroup for deterministic performance, a near-empty capability set, and a seccomp allow-list tuned to the MPI and RDMA libraries actually linked by the solver.

When a job misbehaves, the diagnostic order follows the same layering. If ranks are slow, check cpuset.cpus and cpu.max first. If a rank dies with SIGKILL and no solver error, check memory.max and memory.events. If MPI initialization fails with a permission error, check capabilities and seccomp denials in the audit log. If the solver cannot find its case files, check the mount namespace. This layered mental model turns a vague "the container is broken" into a specific, testable hypothesis.