1. Regulatory landscape for simulation-base

Regulatory landscape for simulation-based evidence 学习卡片

Simulation-based evidence has moved from a research curiosity to a recognized input in safety-critical decision-making. Regulators across aviation, medical devices, automotive, nuclear, and industrial automation now accept — and in some cases require — modeling and simulation (M&S) results as part of design justification, verification, and certification. The central regulatory question is not whether simulation is useful, but whether a given simulation result is credible enough for the specific decision it supports. This chapter maps the regulatory landscape: which provisions govern simulation evidence, how their scope is bounded, and how applicability differs across safety-critical industries.

The credibility principle underlying all provisions

Despite different vocabularies, the major regulatory frameworks converge on one principle: credibility is context-dependent. A simulation's trustworthiness is judged relative to the decision it informs, the risk of that decision, and the consequence of being wrong. This is the logic behind the ASME V&V 10 and 20 standards, NASA-STD-7009, the FDA's reporting guidance on computational modeling, and the model-based safety assessment provisions in ISO 26262 and IEC 61508. Each asks the same three questions in different words:

  • What is the intended use of the model, and what decision does it support?
  • What is the model risk — the combination of decision consequence and model influence?
  • What verification, validation, and uncertainty quantification evidence is proportionate to that risk?

Regulators rarely prescribe a single method. Instead they require a documented, defensible argument — often called a credibility assessment or V&V plan — that ties evidence to intended use. The practical consequence for engineers is that simulation evidence must be traceable: every claim in a safety case should point to a model, and every model should point to its verification and validation record.

Scope: what counts as simulation-based evidence

Regulatory scope is broader than many teams assume. Simulation-based evidence typically includes:

  • Physics-based models — finite element, computational fluid dynamics, thermal, structural, and multibody simulations used to predict behavior that is expensive or unsafe to test physically.
  • System and control models — Simulink/Stateflow or similar models used for model-in-the-loop and software-in-the-loop verification.
  • Probabilistic risk and reliability models — fault trees, event trees, Markov models, and Monte Carlo analyses supporting safety integrity claims.
  • Human-in-the-loop and training simulators — flight simulators, driving simulators, and medical procedure simulators used for qualification or competency evidence.
  • Digital twins — continuously updated models used for operational decision support, where the model itself becomes part of the safety argument.

What falls outside scope is equally important. Purely illustrative animations, marketing visualizations, and exploratory models that do not feed a safety decision are generally not subject to the same evidentiary burden. The boundary is drawn by influence: if a model output changes a design choice, a test plan, an operating limit, or an acceptance decision, it is in scope.

A useful scoping test: ask "If this model were wrong by 20%, would a safety decision change?" If yes, the model is decision-influencing and requires a credibility argument proportionate to that influence.

Applicability across safety-critical industries

The table below summarizes how the major frameworks treat simulation evidence. Note that the column "Primary instrument" names the governing standard or guidance, while "Evidence expectation" describes what regulators typically look for.

Industry Primary instrument Scope emphasis Evidence expectation
Aviation (aircraft systems) SAE ARP4754A / ARP4761, DO-178C/DO-331 Model-based development and safety assessment Model validation against requirements; tool qualification where models generate code
Medical devices FDA guidance on computational modeling; IEC 62304 Device design, physiological modeling, digital evidence Credibility assessment; question-of-interest framing; sensitivity analysis
Automotive ISO 26262, ISO 21448 (SOTIF) Model-based safety analysis; scenario-based validation Confidence in simulation for ASIL decomposition; scenario coverage evidence
Nuclear NRC RG 1.203; 10 CFR 50 Appendix B Thermal-hydraulic and severe accident codes Code verification, validation against experimental data, uncertainty analysis
Industrial automation IEC 61508, IEC 61511 Safety instrumented system verification Model-based SIL verification; documented V&V per safety lifecycle phase

Two patterns stand out. First, industries with mature physical-test traditions (nuclear, aviation) tend to demand validation against experimental benchmarks, because simulation is replacing or supplementing expensive physical tests. Second, industries with rapid design cycles (automotive, medical devices) emphasize scenario coverage and sensitivity analysis, because the dominant risk is not a single wrong prediction but an unexamined operating condition.

How the provisions connect: a credibility workflow

The figure below shows how the key regulatory concepts relate. The intended use and decision consequence set the model risk, which in turn determines the depth of verification, validation, and uncertainty quantification required. The resulting credibility evidence feeds the safety case, and the safety case is what the regulator actually reviews.

Credibility workflow linking intended use, model risk, V&V evidence, and the safety case A left-to-right diagram showing intended use and decision consequence feeding model risk, which determines V&V and uncertainty quantification depth, producing credibility evidence that supports the safety case reviewed by the regulator. Intended Use Decision supported Question of interest Operating envelope Decision Consequence Severity if wrong Model influence Regulatory class Model Risk Low / Medium / High Sets evidence depth Drives V&V plan Verification Code & solution check Benchmark cases Numerical convergence Validation & UQ Experimental comparison Sensitivity analysis Uncertainty bounds Credibility Evidence Traceable Defensible Safety Case / Regulatory Submission Claims → Arguments → Evidence (model outputs, V&V records, UQ results) risk informs evidence depth

From provisions to practice: a worked scoping example

Consider a medical device team using a lumped-parameter physiological model to set an alarm threshold for a patient monitor. The intended use is to support a design decision on the alarm limit; the decision consequence is high because a missed alarm is a serious harm. Under the FDA's credibility framework, this places the model in a high-risk category, which requires a documented question of interest, a validation dataset independent of the calibration data, and a sensitivity analysis showing how the alarm threshold responds to uncertain model parameters.

A practical way to execute this is to build a small parameter-sensitivity table before running the full validation. For example, if the model has three dominant parameters — cardiac output, systemic vascular resistance, and sensor gain — the team can run a one-at-a-time sweep and record the resulting threshold shift:

Parameter            Baseline   Range tested     Threshold shift
cardiac_output       5.0 L/min  3.5 – 7.0 L/min  ±4.2 mmHg
systemic_resistance  1100 dyn·s/cm⁵  800 – 1500  ±6.8 mmHg
sensor_gain          1.00       0.95 – 1.05      ±1.1 mmHg

These values are illustrative. The point is that the sensitivity table tells the team which parameters must be tightly characterized before the model can support the alarm decision. If systemic resistance dominates, the validation dataset must cover a wide range of vascular states, or the credibility argument must explicitly bound that uncertainty. This is exactly the kind of traceable reasoning regulators expect: the provision does not say "run a sensitivity analysis"; it says "demonstrate credibility proportionate to risk," and the sensitivity table is the evidence that the demonstration is real.

A common failure mode is treating V&V as a one-time gate. Under most frameworks, credibility evidence must be maintained as the model, its inputs, or its intended use changes. A model validated for a nominal operating point does not automatically remain credible when the operating envelope is extended.

Key takeaways for regulatory applicability

  • Scope follows influence. Any model whose output changes a safety decision is in scope, regardless of whether it is called a "simulation," a "digital twin," or a "model."
  • Risk sets depth. The same standard permits a lightweight credibility argument for a low-consequence decision and demands experimental validation and UQ for a high-consequence one.
  • Traceability is the deliverable. Regulators review arguments, not models. Every model claim must connect to verification, validation, and uncertainty evidence.
  • Industry instruments differ in vocabulary, not in logic. ARP4754A, ISO 26262, IEC 61508, RG 1.203, and FDA guidance all operationalize the same credibility principle.

With this landscape established, subsequent chapters can examine the specific artifacts — V&V plans, credibility assessment reports, and uncertainty quantification records — that turn these provisions into reviewable evidence.