1. Automotive simulation landscape
Automotive simulation is not a single discipline but a federation of workflows that share geometry, materials, load cases, and — increasingly — a common product timeline. A crash analyst running an explicit LS-DYNA model, a thermal engineer solving a conjugate heat-transfer problem in STAR-CCM+, a durability specialist driving an Abaqus modal transient job, and a systems engineer co-simulating a hybrid powertrain in Simulink with a GT-SUITE plant model all face the same operational reality: jobs are long, licenses are scarce, data is large, and a single corrupted restart file or diverged solver can consume a day of engineering time. Failure recovery and risk management in this context means designing the simulation pipeline so that failures are anticipated, contained, diagnosed, and recovered without invalidating the engineering conclusion.
Why the Four Workflow Families Behave Differently
The recovery strategy must match the numerical character of the workflow. CAE preprocessing and meshing failures are usually deterministic and reproducible: a degenerate shell element or a negative Jacobian will fail every run until the geometry is repaired. CFD failures are frequently nonlinear and iteration-dependent — a Courant number excursion, a turbulence-model stiffness, or a poor initial field can cause divergence at step 4,000 but not at step 3,900. FEA implicit solutions fail through singular stiffness matrices, rigid-body modes, or contact chattering, while explicit crash models fail through negative volume, mass scaling blow-up, or element deletion cascades. System simulation failures are different again: algebraic loops, stiff ODE integration, and event chattering in state machines produce errors that are logical rather than numerical.
This taxonomy matters because it determines what "recovery" means. For a meshing defect, recovery is a corrected input deck. For a diverged CFD run, recovery is a modified numerical control — smaller time step, relaxed under-relaxation factor, or a ramped boundary condition — followed by a restart from the last converged state. For a system simulation, recovery may be a solver setting change or a model restructuring to break an algebraic loop.
Scope of Failure Recovery: From Job-Level to Program-Level
Failure recovery operates at three nested scopes, and confusing them is a common source of wasted effort. Job-level recovery deals with a single solver run: detecting divergence, capturing the last converged state, and restarting with modified controls. Workflow-level recovery deals with the chain of dependent jobs — mesh → solve → post-process → report — where a failure in one link must not silently propagate stale results downstream. Program-level recovery deals with the simulation campaign across a vehicle program: if a crash model must be re-run because a material card was wrong, which downstream durability and NVH conclusions are invalidated?
A practical example clarifies the distinction. Consider a transient CFD run for underhood thermal management using STAR-CCM+, 12 million cells, 8-second physical time, time step 2×10⁻⁴ s. At iteration 3,200 the residuals for energy plateau and then rise. Job-level recovery: the engineer inspects the residual plot, identifies that the outlet boundary condition switched from pressure-outlet to a recirculating state, and restarts from iteration 3,000 with a ramped outlet pressure and a reduced under-relaxation factor for energy from 0.9 to 0.7. Workflow-level recovery: the post-processing script must be prevented from consuming the partial .sim file and producing a temperature map that looks plausible but is physically wrong. Program-level recovery: if the corrected run changes the peak component temperature by 15 °C, the downstream thermal-fatigue FEA that used the old map must be flagged for re-run.
Risk Management: Anticipating Failure Before It Happens
Risk management in simulation is not a bureaucratic overlay; it is the set of engineering decisions that make failures cheap. Four practices carry most of the weight.
Checkpoint cadence. For a 20-hour explicit crash run, writing a restart file every 5 ms of physical time costs storage but bounds the loss from a node failure to minutes. For an implicit FEA job, checkpointing at each converged increment is natural; the risk is storage growth, so a retention policy (keep last N, archive every M) is needed. For system simulation, the solver's internal step control usually provides implicit checkpointing, but the model state must be saved explicitly if the run is to be resumed after a host reboot.
Input validation gates. Before a job enters the queue, automated checks should verify unit consistency, material card completeness, boundary-condition coverage, and mesh quality thresholds. A negative-Jacobian element or a missing density value should fail at submission, not after six hours of queue time.
Version and provenance tracking. Solver version, license feature, input deck hash, and post-processing script version must be recorded with every result. Without this, a "recovered" run may be numerically different from the original in ways that invalidate comparison.
Failure taxonomy and escalation. Not every failure deserves an engineer's attention. A taxonomy — input error, numerical divergence, resource exhaustion, infrastructure fault, software defect — allows routine failures to be handled by automation and escalates only the genuinely novel ones.
A Minimal Risk Register for a Simulation Campaign
The following table is a compact, fillable example for a single vehicle program. It is illustrative, not a claim of measured outcomes.
| Risk | Workflow | Likelihood | Impact | Mitigation | Recovery trigger |
|---|---|---|---|---|---|
| License exhaustion during peak week | CFD / FEA | High | Schedule slip | Reserve tokens; stagger submissions | Queue wait > 4 h |
| Divergence at boundary-condition switch | CFD | Medium | Re-run cost | Ramp BCs; monitor residuals | Residual rise > 1 order |
| Singular stiffness from unconstrained part | FEA implicit | Medium | Job abort | Pre-solve constraint check | Solver error 1005 |
| Algebraic loop in controller model | System sim | Medium | Slow / failed solve | Insert unit delay; restructure | Step size collapse |
| Stale post-processing after re-run | All | High | Wrong conclusion | Hash-linked result manifests | Input hash mismatch |
What "Good" Looks Like
A mature simulation organization does not aim for zero failures — that would require over-conservative models and idle capacity. It aims for failures that are bounded (loss limited to a known checkpoint interval), diagnosable (the failure signature maps to a known cause), and recoverable (a defined procedure restores progress without invalidating prior conclusions). The four workflow families share this goal but differ in their failure signatures and recovery levers, as summarized in the diagram above. The remaining chapters of this course build the specific procedures — checkpoint policies, restart protocols, divergence diagnostics, and cross-workflow invalidation tracking — that turn this goal into daily practice.