4. Benchmarking methodology for crash simulation

Crash simulation benchmarking is not a single measurement — it is a controlled experiment whose credibility depends on four pillars: a representative model set, a well-designed scaling study, demonstrated repeatability, and a comparison protocol that isolates the variable under test. A benchmark that ignores any of these produces numbers that look precise but cannot support procurement, tuning, or regression decisions. This section defines the methodology and shows how to execute it on a real LS-DYNA workload.

Representative Model Selection

A benchmark suite must span the axes that actually drive cost in explicit crash analysis: element count and formulation, contact intensity, material complexity, and solver mix. A single model — even a large one — will mislead, because a 20-million-element shell model with modest contact behaves very differently from a 5-million-element model dominated by self-contact and spot welds.

Build the suite around three tiers, each with a defined purpose:

• Tier A — Micro-kernel (10k–200k elements). A single shell-impact patch or a simplified B-pillar section. Purpose: fast iteration, MPI/OpenMP thread scaling, compiler and flag sensitivity. Runtime target: 30–120 s.
• Tier B — Subsystem (1M–5M elements). A full door, front-end module, or seat structure with realistic materials and contact. Purpose: node scaling, memory bandwidth sensitivity, I/O behavior. Runtime target: 10–40 min.
• Tier C — Full vehicle (10M–50M elements). Complete body-in-white with barrier, dummy, and occupant restraints. Purpose: production-representative throughput, checkpoint/restart cost, licensing behavior. Runtime target: 4–24 h.

Each tier should be frozen as a versioned artifact: identical deck, identical include files, identical initial conditions. Record the model hash (e.g., SHA-256 of the input deck) alongside every result. If the deck changes, the benchmark run is invalid and must be repeated.

Scaling Study Design

The scaling study answers two questions: how does runtime respond to added compute resources, and where does the curve flatten? For explicit crash simulation, the relevant axes are MPI ranks (domain decomposition), OpenMP threads per rank, and GPU offload where supported.

Define the measurement protocol before running anything:

1. Fix the problem size. Use one Tier B model for the entire scaling series. Changing model size between runs confounds scaling with problem size.
2. Fix the time-step control. Keep DTINIT, TSSFAC, and mass-scaling settings identical. A run that terminates earlier because of a larger time step is not a faster run — it is a different simulation.
3. Vary one axis at a time. Run 1, 2, 4, 8, 16, 32, 64 ranks with one thread each; then repeat with 2 and 4 threads per rank. Do not change both rank count and thread count in the same step.
4. Measure wall-clock to a fixed simulated time. Use *CONTROL_TERMINATION with a fixed ENDTIM, and record the wall-clock at that termination, not at job exit (which includes post-processing and license release).
5. Repeat each configuration at least three times. Report the median and the spread. A single run cannot distinguish scaling from noise.

Compute two derived metrics for each configuration:

Speedup(N)        = T(1) / T(N)
Parallel efficiency = Speedup(N) / N

For a well-behaved explicit crash model, efficiency typically stays above 0.80 up to the point where the smallest domain drops below roughly 50k–100k elements per rank. Beyond that, communication and load imbalance dominate. The scaling curve is the deliverable — not a single "best" number.

Repeatability and Noise Control

Repeatability is the precondition for fair comparison. Without it, a 5% difference between two configurations is indistinguishable from run-to-run variation. Establish repeatability by running the identical configuration five times and computing the coefficient of variation (CV) of wall-clock time:

CV = standard_deviation / mean

Target CV below 2% for Tier A and B, below 3% for Tier C. If CV exceeds these thresholds, investigate before proceeding:

• Node contention. Other jobs on the same nodes steal memory bandwidth. Use exclusive node allocation or cgroup isolation.
• Filesystem variability. Shared parallel filesystems introduce I/O jitter. Stage input decks to local NVMe where possible.
• Thermal throttling. Sustained runs on air-cooled nodes may downclock. Log CPU frequency during the run.
• Dynamic load balancing. Some solvers repartition mid-run; this is legitimate but adds variance. Report it explicitly.

Fair Comparison Protocol

Two configurations are comparable only when everything except the variable under test is held constant. The comparison protocol below enforces that discipline.

When comparing solver builds or hardware generations, run both configurations on the same day, on the same nodes, interleaved (A, B, A, B, A, B) to cancel drift. Report the median of each and the ratio, with a confidence interval derived from the repeats.

Worked Example: Tier B Scaling on a Front-End Module

Consider a 3.2M-element front-end module with 180k contact segments and 12 material models, terminated at 80 ms simulated time. The table below shows illustrative results from a scaling series on a dual-socket node with 64 cores per node and InfiniBand interconnect. These are representative example values, not measured production data.

Interpretation: efficiency holds above 0.80 through 16 ranks, then degrades as domains shrink below ~200k elements. The 16-rank × 4-thread configuration achieves the best efficiency (0.92) because it keeps domains large while exploiting shared-memory parallelism. The 64-rank run is faster in absolute terms but wastes more than half its capacity — a poor choice if node-hours are the cost metric.

The deliverable from this exercise is not the fastest configuration but the efficiency curve, which tells you where to stop adding ranks for this model class. That curve, combined with the repeatability CV, is what makes the benchmark defensible.

The workflow above is the minimum viable methodology. Skipping the repeatability check or the interleaved comparison turns a benchmark into an anecdote. When the efficiency curve and the CV are both reported, a reader can judge whether a claimed improvement is real — and that is the entire purpose of benchmarking.