4. Automation of CAE iteration runs

Multi-iteration CAE comparison only becomes practical when the iteration loop is automated. A design study that compares dozens of variants across several load cases and solver settings cannot be driven reliably by hand: manual job submission invites naming drift, missed cases, and inconsistent solver settings, and it makes the run matrix impossible to audit later. This section covers the four layers that turn a one-off simulation into a reproducible iteration campaign: a scripting layer that generates inputs, a submission layer that dispatches jobs to the solver or scheduler, a parameter-sweep layer that defines the design space, and a run-matrix layer that tracks every combination and its status.

Separating the campaign into four layers

The most robust automation architecture keeps these layers independent. The generator reads a base model and a parameter set and writes a complete, self-contained solver input deck per variant. The dispatcher takes a list of run directories and submits each one, either locally or through a job scheduler. The sweep definition declares which parameters vary, over what range, and in what combination. The run matrix is the single source of truth: a table with one row per run, holding the run ID, parameter values, input path, status, solver exit code, and result path. Keeping the matrix separate from the generator means a failed run can be resubmitted without regenerating the whole campaign, and a completed run can be re-analysed without re-solving.

Scripting the input generator

The generator should be deterministic and idempotent: given the same base model and parameter set, it must produce byte-identical input decks. A practical pattern is a template with explicit placeholders plus a small driver script. For a solver that reads a keyword-style input file, the template might contain lines such as *SHELL SECTION, ELSET=RIB, OFFSET=0.0 followed by a thickness token. The driver substitutes tokens, writes the deck into a per-run directory, and records the parameter values in a manifest. Avoid editing the base model in place; always copy into runs/<run_id>/ so that every run is self-contained and can be archived or re-run independently.

# generate_runs.py  (illustrative)
import itertools, json, pathlib, shutil

BASE = pathlib.Path("base/model.inp")
TEMPLATE = BASE.read_text()
OUT = pathlib.Path("runs")

sweep = {
    "thickness": [2.0, 2.5, 3.0],
    "rib_height": [8, 12],
    "load_case": ["LC1", "LC2"],
}

keys = list(sweep)
matrix = []
for i, combo in enumerate(itertools.product(*(sweep[k] for k in keys)), start=1):
    run_id = f"R{i:04d}"
    params = dict(zip(keys, combo))
    run_dir = OUT / run_id
    run_dir.mkdir(parents=True, exist_ok=True)
    deck = (TEMPLATE
            .replace("{{THICKNESS}}", f"{params['thickness']:.2f}")
            .replace("{{RIB_HEIGHT}}", str(params["rib_height"]))
            .replace("{{LOAD_CASE}}", params["load_case"]))
    (run_dir / "input.inp").write_text(deck)
    matrix.append({"run_id": run_id, "params": params,
                   "input": str(run_dir / "input.inp"),
                   "status": "pending", "exit_code": None, "result": None})

(OUT / "matrix.json").write_text(json.dumps(matrix, indent=2))
print(f"generated {len(matrix)} runs")

With three thickness values, two rib heights, and two load cases, the full factorial sweep yields 12 runs. The manifest written to runs/matrix.json is the seed of the run matrix. Note that the run ID is assigned by enumeration order, not by parameter values; this keeps IDs stable when the sweep is extended, but it means the parameter values must always be read from the manifest rather than inferred from the ID.

Job submission and scheduler discipline

For local runs, the dispatcher is a loop that invokes the solver CLI in each run directory and captures the exit code. For cluster runs, the dispatcher renders a scheduler script per run and submits it. Two disciplines matter most. First, concurrency control: submitting 200 jobs at once will either exhaust the license pool or overload the queue, so cap in-flight jobs (for example, eight) and submit the next job only when one completes. Second, exit-code handling: a solver may return zero even when the analysis diverged, so the dispatcher should also scan the solver log for known failure signatures such as *** ERROR or TOO MANY ATTEMPTS and mark the run as failed rather than done.

# submit.sh  (illustrative, Slurm)
#!/bin/bash
#SBATCH --job-name=cae_sweep
#SBATCH --array=1-12%8
#SBATCH --time=02:00:00
#SBATCH --output=runs/R%04a/solver.log

RUN_ID=$(printf "R%04d" "$SLURM_ARRAY_TASK_ID")
cd "runs/${RUN_ID}" || exit 2
solver -i input.inp -o output.odb > solver.log 2>&1
echo "exit=$?" >> status.txt

The array index maps directly to the run ID, and the %8 suffix enforces the concurrency cap. The dispatcher then polls status.txt in each run directory and writes the observed state back into the matrix. This write-back step is what makes the matrix trustworthy: status is never assumed, only recorded from evidence on disk.

Parameter sweeps: full factorial versus targeted designs

A full factorial sweep is the simplest to reason about and the easiest to audit, but its cost grows multiplicatively. Five parameters at four levels each produce 1,024 runs, which is rarely justified in early design. Two alternatives are common. A one-factor-at-a-time (OFAT) sweep varies one parameter while holding the rest at a baseline; it is cheap and easy to interpret but cannot reveal interactions. A Latin hypercube or fractional factorial design samples the space more efficiently and is preferable when the goal is to screen many parameters before committing to a refined study. The choice should be recorded in the sweep definition, not left implicit in the script, so that a reviewer can tell whether the campaign was exhaustive or sampled.

Run matrix management and failure recovery

The run matrix is only useful if it is updated by the automation itself. A practical matrix schema holds: run_id, parameter values, input path, submission timestamp, scheduler job ID, status (pending, running, done, failed), solver exit code, wall-clock time, and result path. Status transitions should be one-way except for explicit resubmission: a failed run is reset to pending only after the cause is understood, and the resubmission count is incremented so that a run that fails repeatedly is flagged rather than retried indefinitely.

Typical failure symptoms and their handling: a run that exits with code 1 within seconds usually indicates a malformed input deck or a missing include file, so check the generator substitution first; a run that terminates after a long wall-clock time with a divergence message points to an unstable load step or an ill-conditioned contact definition, which is a modelling issue rather than an automation issue; a run that never leaves pending usually means the scheduler rejected the submission, so inspect the scheduler's stderr rather than the solver log. In each case the matrix row should record the observed symptom, not just the final status, so that the campaign review can distinguish automation faults from modelling faults.

Once the matrix is complete and every row is either done or explicitly archived as failed, the campaign is ready for the comparison stage: KPI extraction across all completed runs, normalization, and ranking. The discipline established here — deterministic generation, capped and monitored submission, an explicit sweep design, and a matrix that records evidence rather than assumptions — is what makes that downstream comparison defensible.