1. LS-DYNA HPC Job Submission and Queue Management 100 Cases

What "submitting a job" actually means on an HPC cluster

On a workstation you run ls-dyna i=model.k and wait. On a cluster, the compute nodes are not yours to log into directly; a batch scheduler owns them. Your job is therefore a two-part contract: a resource request (how many cores, how much memory, how long) and a payload (the LS-DYNA command plus its input deck). The scheduler places that contract on a node when resources free up, runs it, and records the outcome. "Successful submission" is not the same as "successful analysis" — you must verify both the queue lifecycle and the solver's own termination status.

This first case walks through the smallest complete cycle: stage a deck, submit it through Slurm, watch it move through the queue states, and confirm that LS-DYNA finished normally by reading its output files rather than trusting the scheduler's exit code alone.

The objects in play

• Login node — where you edit files and run sbatch. Never run the solver here.
• Shared filesystem — the directory visible from both login and compute nodes (e.g. /shared/projects/dyna). If your deck lives only on the login node's local disk, the compute node will not see it.
• Batch script — a shell script with #SBATCH directives at the top.
• LS-DYNA executable — the site's SMP or MPP build, invoked with the standard i=, o=, memory= keywords.
• Output files — d3hsp (echo of input and run summary), messag (progress and errors), d3plot (results), and the job's slurm-.out.

Step 1 — Stage a minimal deck on shared storage

For a first submission, use a deliberately tiny explicit model so the run finishes in seconds and any failure is unambiguous. A single solid element under an initial velocity is enough to exercise the explicit solver end to end.

mkdir -p /shared/projects/dyna/case01
cd /shared/projects/dyna/case01

cat > tiny.k <<'EOF'
*KEYWORD
*TITLE
Case 01 - single element explicit smoke test
*CONTROL_TERMINATION
$#  endtim    endcyc     dtmin    endeng    endmas
     1.0E-3         0       0.0       0.0       0.0
*CONTROL_TIMESTEP
$#  dtinit    tssfac      isdo    tslimt     dt2ms
     0.0       0.9         0       0.0       0.0
*DATABASE_BINARY_D3PLOT
$#      dt      lcdt     beam     npltc
    2.5E-4         0         0         0
*PART
$#                                                                         title
single solid
$#     pid     secid       mid     eosid      hgid      grav    adpopt      tmid
         1         1         1
*SECTION_SOLID
$#   secid    elform       aet
         1         1
*MAT_ELASTIC
$#     mid        ro         e        pr        da        db
         1   7.85E-9   2.10E5       0.3
*NODE
       1       0.0       0.0       0.0
       2       1.0       0.0       0.0
       3       1.0       1.0       0.0
       4       0.0       1.0       0.0
       5       0.0       0.0       1.0
       6       1.0       0.0       1.0
       7       1.0       1.0       1.0
       8       0.0       1.0       1.0
*ELEMENT_SOLID
       1       1       1       2       3       4       5       6       7       8
*INITIAL_VELOCITY
$#     nid        vx        vy        vz       vxr       vyr       vzr
         0       0.0       0.0     100.0
*END
EOF

The units here are mm–ms–kg–MPa (consistent set), so density is 7.85E-9 kg/mm³ and Young's modulus 2.10E5 MPa. Mixing unit systems is the single most common reason a "successful" job produces nonsense, so fix the set before submitting.

Step 2 — Write the batch script

Keep the resource request honest and small. One node, four cores, a short walltime, and an explicit output directory.

cat > run_tiny.sbatch <<'EOF'
#!/bin/bash
#SBATCH --job-name=dyna_case01
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --mem=2G
#SBATCH --time=00:10:00
#SBATCH --output=slurm-%j.out
#SBATCH --error=slurm-%j.err

set -euo pipefail
cd "$SLURM_SUBMIT_DIR"

module purge
module load ls-dyna/2023R1-smp

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

ls-dyna i=tiny.k o=tiny.out memory=50m ncpu=$SLURM_CPUS_PER_TASK
EOF

sbatch run_tiny.sbatch

Two details matter. First, cd "$SLURM_SUBMIT_DIR" guarantees the job runs in the directory you submitted from, not the scheduler's default working directory. Second, OMP_NUM_THREADS and ncpu must agree with --cpus-per-task; if you request four cores but let LS-DYNA default to one, you waste the allocation, and if you oversubscribe, the run slows down and may be killed.

Step 3 — Read the queue lifecycle

Immediately after sbatch returns a job ID, the job is usually PENDING (waiting for resources), then RUNNING, then COMPLETED or FAILED. Watch it with:

squeue -u $USER
squeue -j <jobid> -o "%.18i %.9P %.20j %.8T %.10M %.6D %R"

The %R field is the most informative: (Resources) means the scheduler is waiting for free cores, (Priority) means other jobs rank higher, and (QOSMaxWallDurationPerJobLimit) means your requested walltime exceeds the queue's limit. Once the job leaves the queue, sacct gives the historical record:

sacct -j <jobid> --format=JobID,JobName,State,ExitCode,Elapsed,MaxRSS,AllocCPUS

A State of COMPLETED with ExitCode 0:0 means the shell script exited cleanly. That is necessary but not sufficient — LS-DYNA can exit zero while reporting a fatal error in its own files.

Step 4 — Verify the solver, not just the scheduler

Check the three independent signals below. All three must agree before you call the run good.

grep -i "termination" messag d3hsp
ls -lh d3plot tiny.out slurm-*.out
tail -n 20 slurm-<jobid>.out

For the deck above, expect d3hsp to echo the keyword input and report the initial timestep, and messag to end with a normal-termination line after roughly 1 ms of simulated time. The d3plot should contain several states because the plot interval (2.5E-4) divides the termination time (1.0E-3) four times.

When it does not work

Three failure modes account for most first-submission problems, and each has a distinct signature:

• Job stuck in PENDING with (Resources) — the cluster has no free cores matching your request. Reduce --cpus-per-task or --mem, or check sinfo for partition availability. This is not an error; it is the scheduler doing its job.
• Job FAILED almost instantly, slurm-*.out shows "No such file or directory" — the deck is not on shared storage, or cd "$SLURM_SUBMIT_DIR" is missing and the job ran from the wrong directory.
• Scheduler says COMPLETED but messag reports "Error termination" — the solver rejected the input (bad keyword, inconsistent units, negative volume). The shell exited zero because LS-DYNA returned zero; only the solver's own files reveal the truth. This is exactly why the three-signal check exists.

Reusable submission checklist

The following record captures everything needed to reproduce this run and to diagnose it if it fails. Fill it in for each new case.

With this cycle working, you have a known-good baseline: a deck that runs, a script that requests the right resources, and a verification procedure you trust. Every later case — larger decks, multi-node MPP runs, restart handling — builds on exactly this loop.