4. Establishing comparison baselines

Multi-iteration CAE studies only produce meaningful rankings when every candidate is measured against the same, explicitly defined yardstick. That yardstick is the comparison baseline: a fixed set of reference geometry, nominal simulation results, and acceptance bands that remain unchanged across all iterations. Without it, a 3% drop in peak stress between iteration 2 and iteration 5 may reflect a genuine design improvement, a change in mesh density, or simply a different solver tolerance — and the ranking becomes noise. This section defines what a baseline must contain, how to construct it from a reference design and a nominal run, and how to derive tolerance bands that separate real design signal from numerical and manufacturing scatter.

What a Baseline Must Pin Down

A baseline is not a single number. It is a bundle of frozen artifacts and settings that any later comparison can be traced back to. At minimum it should capture:

• Reference design identity — the CAD revision, part numbers, and a hash or checksum of the geometry file actually meshed, not just the file name.
• Nominal run definition — solver version, element type and order, mesh size and quality metrics, material card revision, boundary conditions, and load case IDs.
• Extraction definitions — the exact probe locations, path definitions, surface sets, and averaging windows used to pull scalar metrics (peak von Mises stress, max deflection, first natural frequency, etc.).
• Tolerance bands — the acceptable deviation for each metric, split into numerical and physical contributions.

The critical discipline is that the reference design and the nominal run are established once, before the iteration loop opens, and are then treated as read-only. If the mesh must change because a candidate geometry cannot be meshed at the baseline size, that is a baseline deviation and must be logged, not silently absorbed.

Constructing the Nominal Run

The nominal run is the simulation of the reference design under the same load cases that every candidate will later be subjected to. Its purpose is twofold: it produces the denominator for all relative comparisons, and it exposes the numerical scatter inherent in the model so that tolerance bands can be sized realistically.

A practical way to size numerical scatter is a repeatability sweep on the reference design: run the nominal case at the baseline mesh, then at one refinement level finer and one coarser, and optionally with a tightened convergence tolerance. Record each metric at every level. The spread across these runs is the numerical uncertainty floor — no candidate difference smaller than this spread should be treated as a design effect.

# Example repeatability sweep (illustrative values, not measured)
case            mesh_size   peak_vm_stress   max_deflection   f1
nominal         4.0 mm      182.4 MPa        1.87 mm          64.2 Hz
refined         3.2 mm      179.1 MPa        1.85 mm          64.0 Hz
coarse          5.0 mm      188.6 MPa        1.92 mm          63.5 Hz
tightened tol   4.0 mm      182.1 MPa        1.87 mm          64.2 Hz

numerical spread (max-min)/nominal:
  stress      (188.6-179.1)/182.4 = 5.2%
  deflection  (1.92-1.85)/1.87    = 3.7%
  f1          (64.2-63.5)/64.2    = 1.1%

These spreads tell you that a 2% stress difference between two candidates is inside the mesh-induced noise and cannot be ranked confidently, whereas a 7% difference is likely real. Note that the coarse mesh biases stress upward here — a classic stress-concentration artifact — which is exactly why the baseline mesh must be fixed rather than re-chosen per candidate.

Separating Numerical from Physical Tolerance

The total tolerance band for a metric should combine two independent sources. The numerical band comes from the repeatability sweep above. The physical band comes from manufacturing and material variability: wall thickness tolerance, modulus scatter, load application uncertainty. If the nominal run is deterministic, the physical band must be estimated from process data or engineering judgment and applied as an explicit allowance.

A defensible combination rule for independent contributions is root-sum-square:

band_total = sqrt(band_numerical^2 + band_physical^2)

stress:     sqrt(5.2^2 + 4.0^2) = 6.6%   -> band = +/-6.6% of 182.4 MPa
deflection: sqrt(3.7^2 + 2.5^2) = 4.5%   -> band = +/-4.5% of 1.87 mm
f1:         sqrt(1.1^2 + 1.5^2) = 1.9%   -> band = +/-1.9% of 64.2 Hz

Metrics whose candidate-to-candidate differences fall inside the band should be marked indistinguishable rather than ranked. This is the single most important guard against over-ranking near-ties, which is the dominant failure mode in multi-iteration comparison studies.

Baseline Record and Governance

The baseline should be captured as a versioned record that travels with the study. A compact, fillable form keeps it auditable:

Any candidate that requires a deviation from this record — a different mesh, a changed load application, a new probe — must be flagged as non-comparable until the deviation is either eliminated or the baseline is formally revised and all prior candidates re-run. This rule prevents the slow drift that otherwise corrupts long iteration chains.

How the Baseline Drives Ranking

With the baseline fixed, each candidate metric is expressed as a signed deviation from nominal, and the tolerance band becomes the decision threshold. The figure below shows the relationship between the nominal run, the numerical and physical bands, and how three illustrative candidates are classified: one clearly better, one clearly worse, and one indistinguishable from nominal.

Candidate A at −7.1% lies outside the ±6.6% total band and can be ranked as a genuine improvement. Candidate B at +2.3% sits well inside the band and must be reported as indistinguishable from nominal — ranking it above or below the reference would be over-interpretation. Candidate C at +9.4% is a real regression. This three-way classification, not a raw sort of numbers, is what makes the ranking defensible.

Practical Checklist Before Opening the Iteration Loop

1. Freeze the reference geometry and record its hash.
2. Run the nominal case and the repeatability sweep; compute numerical bands per metric.
3. Estimate physical bands from manufacturing and material data; combine via RSS.
4. Fix probe and path definitions; verify they resolve identically on the reference mesh.
5. Publish the baseline record (BL-ID) and mark it read-only.
6. Define the deviation protocol: any non-baseline mesh or load application flags the candidate as non-comparable until resolved.

Once these six items are in place, every subsequent iteration inherits a stable denominator and a pre-agreed decision threshold. The ranking that emerges is then a statement about design, not about meshing luck or solver drift — which is precisely the goal of a multi-iteration comparison study.