2. Data quality dimensions for CAE validation

In CAE validation, a simulation result is only as trustworthy as the data that produced it. Measurement data feeding a validation activity is rarely "just numbers"; it carries a chain of assumptions, instrument characteristics, processing steps, and provenance that determine whether a correlation study, model calibration, or pass/fail decision is defensible. This chapter defines five data quality dimensions — accuracy, precision, completeness, consistency, and traceability — and shows how each is assessed and controlled in practice. These dimensions are not independent: a dataset can be highly precise yet inaccurate, complete yet inconsistent, or traceable yet imprecise. Treating them as separate, measurable attributes is what allows a validation engineer to state, with evidence, how much confidence a comparison between test and simulation deserves.

Accuracy: closeness to the true value

Accuracy describes how close a measured value is to the reference (true) value. In CAE validation, the reference is often another measurement — a calibrated reference sensor, a certified load cell, or a national metrology standard — because the physical quantity itself is inaccessible. Accuracy is therefore expressed as a bias or systematic error, and it is corrected through calibration rather than averaging. For example, a strain gauge bridge reading 2.03 mV/V at a known 1000 µε reference load has a bias of +0.3 % FS; that offset must be removed before the data is used to validate a finite element strain prediction, otherwise the correlation error will be misattributed to the model.

Accuracy is quantified with a reference uncertainty budget. A typical budget for a dynamic pressure channel might combine sensor nonlinearity (±0.5 % FS), amplifier gain error (±0.2 %), and reference standard uncertainty (±0.1 %), giving a combined standard uncertainty via root-sum-square. The key discipline is that accuracy claims must be stated with their reference and conditions: "±0.5 % FS at 25 °C, referenced to a deadweight tester" is meaningful; "accurate to 0.5 %" is not.

Precision: repeatability under identical conditions

Precision is the closeness of repeated measurements to each other, independent of their closeness to truth. It is characterized by repeatability (same operator, same instrument, short interval) and reproducibility (different operators, setups, or laboratories). In validation work, precision directly limits the smallest model discrepancy that can be detected: if a test channel has a repeatability standard deviation of 1.5 % of reading, a 2 % difference between simulation and test is not statistically distinguishable from noise.

A practical assessment uses repeated static or dynamic runs. Suppose an accelerometer is subjected to five identical shock pulses and the peak values are 48.2, 48.9, 47.8, 48.5, and 48.4 g. The sample standard deviation is approximately 0.41 g, or 0.85 % of the mean — a repeatability figure that should be recorded alongside the channel metadata. Precision is a property of the measurement system, not the sensor alone, so fixturing, cabling, and digitizer resolution all contribute.

Completeness: coverage of the required evidence

Completeness asks whether the dataset contains everything the validation activity requires — not merely whether files exist. For a modal test validating an FE model, completeness means all instrumented degrees of freedom, all excitation directions, the full frequency band of interest, and the associated metadata (sample rate, window, averaging count). A dataset missing the torsional excitation channel is incomplete even if every recorded channel is accurate and precise, because the validation claim cannot be extended to torsional modes.

Completeness is best managed as a requirements-to-data matrix. Define the validation quantities of interest, map each to the required channels and conditions, and mark coverage. Gaps should be explicit and justified, not silently absent. In practice, a completeness check is a scripted comparison of expected channel lists, time windows, and operating conditions against the delivered dataset — a simple automated gate that prevents incomplete data from entering correlation studies.

Consistency: coherence across sources and processing

Consistency concerns whether data agrees with itself and with related data: units, coordinate systems, sign conventions, time bases, sampling rates, and processing history. A classic failure is a test dataset in g while the simulation output is in m/s², producing a factor-of-9.81 discrepancy that is mistaken for a modeling error. Another is a sign convention mismatch: test acceleration positive upward versus FE acceleration positive downward, which flips phase and corrupts frequency-domain correlation.

Consistency checks are largely mechanical and should be automated: verify units against a declared schema, confirm coordinate frames and sign conventions in metadata, align time-zero between test and simulation, and resample to a common rate before comparison. Where processing differs — for example, test data low-pass filtered at 500 Hz but simulation output unfiltered — the processing must be harmonized or explicitly documented, because an unstated filter difference will appear as a model discrepancy.

Traceability: provenance from measurement to decision

Traceability is the ability to reconstruct, for any data point, where it came from, how it was transformed, and which calibration and configuration applied. It links the raw signal to the sensor serial number, calibration certificate and date, acquisition settings, processing scripts and versions, and the analyst who approved the result. Without traceability, a validation conclusion cannot be audited or reproduced, and a later anomaly cannot be investigated.

Traceability is implemented through immutable raw data, version-controlled processing, and metadata that travels with the dataset. A minimal record for each channel includes sensor ID, calibration reference and uncertainty, acquisition system and settings, processing chain with parameters, and a hash or version identifier for the processing code. When a correlation result is challenged, traceability is what allows the engineer to re-run the exact processing and confirm or refute the finding.

Data Quality Dimensions → Validation Controls → Failure Symptoms Dimension Control / Assessment Typical Failure Symptom Accuracy bias vs. reference Calibration against certified reference; uncertainty budget (RSS of contributions) Constant offset in correlation; bias blamed on FE model Precision repeatability / reproducibility Repeated identical runs; sample std dev; noise floor sets detectable discrepancy Scatter masks small model errors; false "good agreement" Completeness coverage of required evidence Requirements-to-data matrix; automated channel/condition gate Validation claim overreaches; missing DOF or band Consistency units, frames, time base Schema unit checks; sign/frame metadata; time-zero alignment; filter harmonization Factor-of-9.81 or sign flip; phase error in correlation Traceability provenance to decision Immutable raw data; versioned processing; sensor ID + calibration + code hash Result not reproducible; anomaly cannot be audited

Working the dimensions together: a validation data quality gate

The dimensions become actionable when embedded in a gate that data must pass before it enters a correlation or calibration study. Consider a road-load validation dataset intended to compare measured and simulated spindle accelerations. A practical gate proceeds as follows. First, traceability is checked: every channel resolves to a sensor serial number, a calibration certificate within its validity window, and a recorded acquisition configuration. Second, consistency is verified: units are g for both test and simulation, the vertical axis is positive upward in both, time-zero is aligned to the first wheel impact, and both signals are filtered with the same 200 Hz low-pass. Third, completeness is confirmed against the requirements matrix: all four spindles, three axes, and the full 0–150 Hz band are present. Fourth, precision is estimated from repeat runs, yielding a per-channel noise floor. Fifth, accuracy is corrected using the calibration bias, and the residual uncertainty is propagated into the comparison tolerance.

Only after these checks does the correlation metric — for example, a frequency-domain magnitude ratio or a time-domain peak error — get interpreted. The precision and accuracy results define the decision threshold: a discrepancy smaller than the combined measurement uncertainty cannot be attributed to the model. This is the central practical insight of data quality for CAE validation: the measurement system's quality dimensions set the resolution of the validation itself. A model cannot be validated more finely than the data can resolve.

Record the five dimensions as explicit metadata fields for every validation dataset. A short quality statement — bias and uncertainty, repeatability, coverage, unit/frame conventions, and provenance identifiers — turns an informal "the data looks fine" into an auditable claim that reviewers and correlation engineers can rely on.
Do not conflate precision with accuracy. Averaging repeated runs improves the estimate of the mean but does not remove a calibration bias. Correcting bias requires a reference; reducing scatter requires controlling the measurement system. Applying the wrong remedy leaves a systematic error in place while creating false confidence from a small standard deviation.

In summary, accuracy, precision, completeness, consistency, and traceability each answer a distinct question about the data: is it close to truth, is it repeatable, is it sufficient, is it coherent, and can it be reconstructed. Assessing all five before a validation comparison, and carrying their quantified results into the interpretation of correlation metrics, is what separates a defensible CAE validation from a numerical coincidence.

← Previous 📖 Contents Next →