3. Create a minimal Slurm batch script for LS-DYNA

Why a minimal script is the right starting point

An LS-DYNA SMP job occupies a single node and scales by threads, not by MPI ranks. That single fact drives almost every line of the batch script: you request one node, you ask for a number of tasks equal to the SMP thread count, and you let the solver's own threading model consume those cores. A minimal, correct script is therefore not a stripped-down version of an MPI script — it is a different resource request shape. Getting this shape right before adding features (arrays, dependencies, mail notifications) prevents the most common class of wasted allocations: jobs that reserve eight nodes when they only ever use one.

This case builds a complete, submittable sbatch script for an SMP run, explains each directive against the LS-DYNA execution model, and defines how to confirm that the allocation matched the solver's actual behaviour.

Resource request logic for SMP

Slurm distinguishes between tasks (processes launched by the job step) and CPUs per task. For an SMP LS-DYNA executable, the solver starts as a single process and spawns worker threads internally. The cleanest mapping is:

• --nodes=1 — SMP cannot span nodes.
• --ntasks=1 — one solver process; do not request one task per thread.
• --cpus-per-task=N — the thread count, which must equal the value passed to the solver's thread control.

Requesting --ntasks=N with --cpus-per-task=1 also reserves N cores, but it misrepresents the job to the scheduler and to anyone reading squeue; more importantly, it invites accidental use of srun with N ranks, which would launch N independent SMP solvers competing for the same input deck. Keep the one-task model.

Memory and walltime sizing

LS-DYNA explicit models are memory-hungry but their footprint is predictable from the model: solid element count, contact segment count, and the number of output states retained. A practical starting estimate for a shell/solid crash model is roughly 1–2 GB per million solid elements for the solution phase, plus storage for d3plot states. Request memory explicitly rather than relying on a partition default, because a default that is too low produces an out-of-memory kill mid-run, and a default that is too high wastes the node's capacity for other jobs.

Walltime should be set from a measured or estimated cycle count, not guessed. Run a short restart-free trial (for example, 5 ms of simulated time) and extrapolate: if 5 ms takes 40 minutes, a 40 ms termination time needs roughly 5.3 hours plus output overhead. Add 20–30% margin for I/O and queue-induced variability, then round up to the partition's granularity.

The script

#!/bin/bash
#SBATCH --job-name=dyna_smp_rail
#SBATCH --partition=short
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=64G
#SBATCH --time=06:00:00
#SBATCH --output=dyna_smp_rail.%j.out
#SBATCH --error=dyna_smp_rail.%j.err
#SBATCH --mail-type=END,FAIL
#SBATCH --mail-user=analyst@example.org

set -euo pipefail

# --- Site-specific settings (edit these) ---
export DYNA_ROOT=/opt/lsdyna/R13.1
export DYNA_EXE=${DYNA_ROOT}/ls-dyna_smp_d_R13.1_linux_x86_64_ifort160
export DYNA_THREADS=${SLURM_CPUS_PER_TASK}
export OMP_NUM_THREADS=${DYNA_THREADS}

WORKDIR=${SLURM_SUBMIT_DIR}
cd "${WORKDIR}"

echo "Job ${SLURM_JOB_ID} starting on $(hostname) at $(date)"
echo "Allocated cores: ${SLURM_CPUS_PER_TASK}  Threads: ${DYNA_THREADS}"

# LS-DYNA reads the deck name from stdin; 'i=' names the input file.
# 'memory=' requests the solver memory pool in words (here ~200M words).
${DYNA_EXE} i=rail_impact.k memory=200000000 \
    ncpu=${DYNA_THREADS} \
    d3plot=rail_impact_d3plot \
    glstat=rail_impact_glstat \
    matsum=rail_impact_matsum

echo "Job ${SLURM_JOB_ID} finished at $(date) with exit code $?"

Several details in this script are deliberate. set -euo pipefail makes the job fail loudly if the executable is missing or the working directory cannot be entered, instead of silently producing an empty output file. OMP_NUM_THREADS is exported because some LS-DYNA SMP builds honour OpenMP environment variables in addition to the ncpu= command-line argument; setting both keeps them consistent. The memory= value is expressed in solver words (typically 8 bytes each on 64-bit builds), so 200,000,000 words is approximately 1.6 GB of solver pool — adjust to your model, and note that this is separate from the Slurm --mem request, which must cover the pool plus the OS, libraries, and I/O buffers.

The %j token in the output and error filenames inserts the job ID, so repeated submissions never overwrite each other's logs. Capturing stdout and stderr separately matters for LS-DYNA: the solver writes its banner, licence checkout, and termination summary to stdout, while warnings about negative volumes, contact penetrations, and element distortions frequently appear on stderr. Reviewing only one stream hides half the diagnostic picture.

Allocation-to-solver mapping

Submitting and verifying the allocation

Submit from the directory containing the input deck so that SLURM_SUBMIT_DIR resolves correctly:

sbatch submit_dyna_smp.sbatch
squeue -u $USER -o "%.10i %.20j %.6t %.4C %.10m %.11l %R"

The %C column reports allocated CPUs and %m reports memory per node. Confirm that %C equals your thread count and that the node shown in %R is a single hostname, not a range. If %C shows 16 but the solver banner reports a different number of CPUs, the ncpu= argument is not reaching the executable — check for a wrapper script or a stale alias.

After completion, verify three things in order. First, the .out file should contain the LS-DYNA banner, a licence checkout line, and a normal termination message; a missing termination message with a non-zero exit code usually indicates a solver error rather than a scheduler problem. Second, the .err file should be scanned for warnings even on successful runs — negative volume and contact energy warnings are the leading indicators of a model that will fail later. Third, confirm that the expected output files (d3plot, glstat, matsum) exist and that glstat shows a plausible energy balance.

Common failure symptoms and their causes

Reusable submission checklist

• Input deck validated and present in SLURM_SUBMIT_DIR.
• --nodes=1, --ntasks=1, --cpus-per-task equal to the intended thread count.
• ncpu= and OMP_NUM_THREADS both set to ${SLURM_CPUS_PER_TASK}.
• --mem set from a measured or estimated peak, not a partition default.
• --time derived from a short trial run plus margin.
• Separate --output and --error files with %j in the names.
• Post-run check of sacct for MaxRSS, Elapsed, and final state.

With this script in place, the next step — replacing the direct executable call with an MPI-aware launch — becomes a controlled change to a known-good baseline rather than a rewrite.