4. Evidence hierarchy and admissibility

In a simulation-based evidence regime, the central question is rarely "do we have simulation results?" It is "how much weight should those results carry when they conflict with, or must substitute for, physical test data and operational records?" Regulators, assessors, and internal safety cases all need a defensible answer. This chapter sets out a practical evidence hierarchy, the admissibility criteria that determine whether a simulation artifact can enter the record at all, and a weighting method for combining simulation, physical test, and operational data into a single conclusion.

Why a hierarchy is necessary

Simulation evidence is not a single class of evidence. A high-fidelity, validated finite-element model of a pressure vessel under a defined load case is fundamentally different from a coarse system-level model used to explore a design space. Treating both as "simulation" and assigning them equal weight is a common and serious error in safety cases. A hierarchy makes the differences explicit and forces the analyst to justify the weight assigned to each artifact.

The hierarchy used here has four tiers, ordered by evidential strength for a specific claim:

• Tier 1 — Direct physical test evidence: measurements from the actual article, or a qualified representative, under the actual or bounding condition. Highest weight; the reference against which everything else is judged.
• Tier 2 — Operational data: recorded performance, inspection, and incident data from service. High weight for conditions actually observed, but limited coverage and often uncontrolled.
• Tier 3 — Validated simulation: a model whose predictive accuracy has been demonstrated against Tier 1 or Tier 2 data for the same physics and regime. Weight depends on the validation evidence.
• Tier 4 — Unvalidated or extrapolative simulation: models used outside their validated envelope, or with no comparison to physical data. Lowest weight; admissible only as supporting or exploratory evidence.

The hierarchy is claim-specific, not artifact-specific. The same model may be Tier 3 for a nominal load case and Tier 4 for a beyond-design-basis event. The analyst must state the claim and the tier together.

Admissibility criteria

Before weight is assigned, an artifact must be admissible. Admissibility is a gate, not a score. A simulation artifact is admissible as evidence when all of the following hold:

1. Traceability: the model version, input deck, solver version, and boundary conditions are recorded and reproducible.
2. Verification: the numerical solution has been checked against analytical or benchmark solutions for the relevant physics (mesh convergence, time-step independence, conservation checks).
3. Validation: the model has been compared to physical or operational data for the same regime, with quantified error.
4. Uncertainty statement: input, model-form, and numerical uncertainty are estimated and reported, not omitted.
5. Competence and independence: the analyst is qualified, and the review is independent of the model builder.

Failure of any one criterion does not necessarily exclude the artifact, but it caps its tier. For example, a model that is verified but not validated cannot exceed Tier 4 regardless of how sophisticated it appears.

Weighting simulation against physical and operational data

When evidence of different tiers addresses the same claim, the assessor must combine them rather than pick a winner. A practical method is a structured weighting table. Each artifact receives a weight based on its tier, its validation error relative to the decision margin, and its relevance to the specific claim. The weights are then used to form a weighted judgment, and any residual disagreement is documented as a finding.

Consider a worked example. A claim is made that a valve body survives a specified overpressure event with a margin of 1.5 on yield. The evidence set is:

The weights sum to 1.0 and are justified in the safety case narrative. The Tier 3 model carries substantial weight because its validation error (+6%) is small relative to the 1.5 margin, and its relevance is high. The Tier 4 model is retained only to show sensitivity, not to drive the conclusion. If the Tier 3 model predicted failure while the Tier 1 test passed, the assessor would not simply average them; the discrepancy would trigger a model review, and the Tier 1 result would dominate until the model error was explained.

Reading the evidence chain

The figure below shows how the four tiers feed a single claim, how admissibility gates each artifact, and where disagreement is resolved. The key structural point is that physical test and operational data anchor the top of the chain, while simulation enters only through the validation gate and carries weight proportional to its demonstrated accuracy.

Handling conflict between tiers

Conflict is the normal case, not the exception. When a Tier 3 model disagrees with a Tier 1 test, the correct response is not to average the two or to prefer the more convenient result. The assessor should apply the following decision logic:

• If the Tier 1 result is bounding and the model is non-conservative, the Tier 1 result governs until the model error is explained and corrected.
• If the model is conservative relative to the test, the model may be retained as a bounding tool, but the conservatism must be quantified.
• If the disagreement exceeds the stated validation error, the model is outside its validated envelope for this claim and drops to Tier 4.
• Any unresolved conflict is recorded as an open finding in the safety case, with an owner and a closure action.

This logic prevents the most common regulatory criticism of simulation-based evidence: that the model was tuned until it agreed with the test, then presented as independent confirmation. Independence between the validation data and the claim data is essential; a model validated on the same test it is used to support adds little evidential weight.

Practical weighting guidance

Three rules keep the weighting defensible in review. First, state the claim before assigning any weight; a weight without a claim is meaningless. Second, quantify validation error in the same units as the decision margin, so the reader can see whether the error is material. Third, document the sensitivity of the conclusion to the weights: if moving the Tier 4 weight from 0.10 to 0.20 changes the outcome, the conclusion is fragile and must be flagged.

The hierarchy and weighting method described here give assessors a consistent, auditable way to combine simulation, physical test, and operational data. The next step in building a defensible safety case is to apply this method consistently across every claim, so that the evidence record shows not only what was concluded but why each artifact was trusted to the degree it was.