4. Gauge repeatability and reproducibility (GR&R) for CAE validation experiments

In a CAE validation experiment, the "gauge" is rarely a single instrument. It is the entire measurement chain: the data acquisition system, the sensor installation, the test fixture, the signal conditioning, and the analyst who selects channels, filters, and processing parameters. GR&R analysis quantifies how much of the observed variation in a validation response comes from this measurement chain rather than from the physical behavior being studied. If GR&R is large relative to the part-to-part or configuration-to-configuration variation you intend to detect, the experiment cannot reliably distinguish a real design effect from measurement noise, and any correlation with simulation results becomes questionable.

What Repeatability and Reproducibility Mean Here

Repeatability is the variation observed when the same appraiser measures the same item with the same equipment under identical conditions — for example, one test engineer re-running the same channel configuration on the same instrumented specimen within a short time window. It captures short-term equipment noise, sensor drift, mounting repeatability, and processing-parameter sensitivity.

Reproducibility is the variation observed when different appraisers, or different setups, measure the same item under nominally identical conditions — for example, two engineers independently installing strain gauges at the same nominal locations, or two labs running the same test protocol on nominally identical specimens. In CAE validation, reproducibility frequently dominates because sensor placement, fixture torque, and channel zeroing are operator-dependent.

The combined GR&R is usually expressed as a percentage of the total observed variation or of the tolerance band. A common rule of thumb: below 10% is acceptable, 10–30% is conditionally acceptable depending on the application, and above 30% requires investigation before the data are used for validation.

Designing the GR&R Study

A defensible GR&R study for a validation experiment needs three ingredients: a set of items that span the expected variation, multiple appraisers or setups, and repeated trials. The classic crossed design uses a appraisers × b items × n trials. For CAE validation, a practical starting point is 3 appraisers (or 3 independent setups), 10 items (specimens, sensor locations, or operating conditions), and 2–3 trials each, giving 60–90 measurements per response.

The items must be chosen deliberately. If all specimens are nominally identical, part-to-part variation is near zero and GR&R will appear artificially large as a percentage of total variation. Either select items that genuinely differ (different build configurations, different load levels, different sensor locations) or report GR&R against the tolerance or against the simulation-to-test discrepancy you care about, not against total variation alone.

Randomize the run order. Appraiser A should not measure all ten items before Appraiser B begins, because thermal drift, fixture settling, and fatigue will be confounded with appraiser effects. A randomized or fully crossed order, recorded in a run sheet, is essential. Each trial should include the full setup cycle — re-mounting the sensor, re-zeroing the channel, re-running the acquisition — not just re-reading a stored file, otherwise you measure only software repeatability.

Before committing to a full GR&R study, run a quick 2-appraiser × 3-item × 2-trial pilot. If the range between trials already exceeds the effect size you hope to detect, the measurement chain needs improvement before the main study is worth running.

Executing the Study and Computing the Components

The standard calculation is the ANOVA method, which separates variance into appraiser, item, interaction, and repeatability (error) components. For a balanced crossed design, the mean squares from a two-way ANOVA with interaction give:

Equipment variation (repeatability):   EV = sqrt(MSE)
Appraiser variation (reproducibility): AV = sqrt((MSA - MSAB) / (b * n))
Interaction:                           INT = sqrt((MSAB - MSE) / n)
GR&R:                                 GRR = sqrt(EV^2 + AV^2 + INT^2)
Part variation:                        PV = sqrt((MSB - MSAB) / (a * n))
Total variation:                       TV = sqrt(GRR^2 + PV^2)
%GR&R = 100 * GRR / TV        (or GRR / tolerance, if a tolerance is defined)

As a worked example with illustrative (not measured) numbers: suppose a strain-response validation study uses 3 appraisers, 10 sensor locations, and 2 trials, and the ANOVA returns MSE = 4.0, MSAB = 6.0, MSA = 18.0, MSB = 120.0 (units of microstrain squared). Then EV = 2.0, AV = sqrt((18 − 6)/20) = 0.77, INT = sqrt((6 − 4)/2) = 1.0, GRR = sqrt(4 + 0.59 + 1) ≈ 2.37, PV = sqrt((120 − 6)/6) ≈ 4.36, TV ≈ 4.96, and %GR&R ≈ 48%. That result would flag the measurement chain as inadequate: repeatability dominates, and the interaction term suggests appraisers rank the locations differently, which usually points to inconsistent sensor placement or channel zeroing.

When the design is unbalanced or the interaction is negligible, a pooled ANOVA or the average-and-range (X̄–R) method may be used, but the ANOVA approach is preferred because it estimates the interaction and is robust to missing runs.

Interpreting the Numbers

Read the components, not just the headline percentage. A high EV with low AV points to equipment, sensor mounting, or signal processing. A high AV with low EV points to training, procedure, or fixture handling. A significant interaction means the appraisers do not agree on the relative ranking of items — often the most actionable finding, because it invalidates simple averaging across operators.

Also check the number of distinct categories (ndc = 1.41 × PV / GRR). An ndc below 5 means the measurement system cannot resolve enough distinct levels to support the discrimination the validation requires, even if %GR&R looks borderline acceptable.

GR&R variance decomposition and decision path for a CAE validation experiment Diagram showing measurement chain inputs feeding an ANOVA decomposition into repeatability, reproducibility, interaction and part variation, with a decision path based on percent GR&R and number of distinct categories. GR&R Study: Measurement Chain → Variance Components → Decision Measurement chain Sensor & mounting Fixture & torque DAQ & conditioning Channel zeroing Analyst processing 3 appraisers × 10 items × 2 trials ANOVA variance components EV = √MSE (repeatability) example: 2.0 µε AV = √((MSA−MSAB)/bn) example: 0.77 µε INT = √((MSAB−MSE)/n) example: 1.0 µε PV = √((MSB−MSAB)/an) example: 4.36 µε GRR = √(EV²+AV²+INT²) example: 2.37 µε → %GR&R ≈ 48% Interpretation & action %GR&R < 10% and ndc ≥ 5 Acceptable — proceed to test–simulation correlation 10% ≤ %GR&R ≤ 30% Conditional — check EV vs AV, tighten procedure %GR&R > 30% or ndc < 5 Inadequate — investigate dominant component High EV → sensor mount, DAQ noise, filtering High AV → training, fixture handling, zeroing High INT → inconsistent placement or ranking

Practical Failure Modes and How to Diagnose Them

Repeatability dominates. Re-run a single appraiser on one item five times and inspect the raw time histories. Look for 50/60 Hz pickup, cable movement, loose connectors, or a filter cutoff that sits inside the signal band. Re-torque the fixture to the specified value and confirm the sensor is not creeping.

Reproducibility dominates. Compare the appraisers' setup sheets. Common causes are different gauge lengths, different adhesive cure times, different zeroing references, and different channel gain settings. Standardize the setup procedure and re-run the study.

Interaction is significant. Plot item means by appraiser. If the lines cross, one appraiser is systematically mis-ranking items — often because a sensor location was interpreted differently. This is a procedure defect, not a statistical one.

%GR&R looks fine but ndc is low. The items were too similar. Widen the item selection or report GR&R against the tolerance band rather than total variation.

Do not compute GR&R on data that have already been averaged, smoothed, or post-processed with operator-chosen parameters unless those parameters are part of the measurement system definition. Otherwise the study measures the processing pipeline, not the physical measurement, and the result will not transfer to future validation campaigns.

Reporting the Result

A complete GR&R report for a CAE validation experiment should state the response variable and units, the design (appraisers × items × trials), the randomization scheme, the ANOVA table with mean squares, the variance components EV, AV, INT, PV, GRR, TV, the %GR&R against both total variation and tolerance, the ndc, and the action taken. Keep the raw run sheet with the report so the study can be audited or repeated after any change to the measurement chain. When the measurement system is accepted, the GR&R value becomes the noise floor against which simulation-to-test discrepancies are judged: a correlation error smaller than the GR&R is not evidence of a modeling defect.

← Previous 📖 Contents Next →