4. Statistical plausibility methods

Once posttest data has been collected and channelized, the next question is whether the numbers themselves make sense. A channel can be electrically healthy, correctly wired, and free of gross protocol errors, yet still produce values that are statistically implausible — a temperature that jumps 40 °C between consecutive samples, a pressure reading pinned at exactly the same value for hours, or a batch of units whose mean drifts steadily upward across the test sequence. Statistical plausibility review is the layer that catches these cases. It does not replace limit checking; it complements it by asking whether the shape of the data is consistent with the physical process and the test setup that produced it.

Three families of methods cover most practical needs: outlier detection, distribution checks, and trend analysis. Each answers a different question, and in a mature review workflow they are applied in that order, because outliers contaminate distribution estimates and trends distort them further.

Outlier Detection: Flagging Individual Suspicious Samples

Outlier detection operates on individual samples or short windows and asks whether a value is far enough from its neighbors to warrant suspicion. The simplest and most transparent method is the modified Z-score based on the median absolute deviation (MAD), which is robust to the very outliers it is trying to find:

median  = median(x)
MAD     = median(|x_i - median|)
M_i     = 0.6745 * (x_i - median) / MAD
flag if |M_i| > 3.5

The constant 0.6745 scales the MAD to be comparable with a standard deviation under normality, and 3.5 is the conventional threshold from Iglewicz and Hoaglin. For a channel with 200 samples, a single sample at |M| = 6.2 is a strong candidate for quarantine, while a cluster of samples at |M| ≈ 4 suggests a distribution shift rather than isolated glitches — a distinction that matters when deciding whether to quarantine one record or the whole channel.

For time-ordered data, a differencing check is often more diagnostic than a magnitude check. Compute successive differences Δx_i = x_i − x_i−1 and compare each against a physically justified maximum slew rate. A thermocouple on a thermal mass cannot legitimately move 40 °C in one 100 ms sample; a step of that size is almost always a loose connection, an ADC glitch, or a multiplexer settling error. The slew-rate limit should come from the physics of the sensor and the sampling interval, not from the observed data — deriving it from the data itself lets a faulty channel define its own acceptance criterion.

Distribution Checks: Is the Shape Consistent With Expectation?

Distribution checks ask whether the overall spread and shape of a channel's data match what the process should produce. For a stable manufacturing test, a channel measuring a controlled parameter should be roughly normal with a spread consistent with the specification and the measurement uncertainty. Two practical checks dominate:

• Spread ratio. Compare the observed robust spread (for example, the interquartile range) against the expected spread derived from tolerance and instrument accuracy. An IQR three times larger than expected points to excessive noise, intermittent contact, or a channel picking up interference; an IQR far smaller than expected often indicates a stuck or heavily filtered channel reporting artificially smooth values.
• Shape indicators. Skewness and kurtosis, or a simple histogram overlay against the expected distribution, reveal bimodality (two populations mixed, such as two fixture positions), truncation (clipping at an ADC rail), and heavy tails (occasional large excursions). A histogram with a sharp edge at exactly the ADC full-scale code is a classic clipping signature and should quarantine the channel regardless of how plausible the individual values look.

Distribution checks are best run on a channel-by-channel basis over a complete test run, and the expected distribution should be documented alongside the test procedure. Without a stated expectation, "the distribution looks wrong" is not an actionable finding.

Trend Analysis: Detecting Drift and Order Dependence

Trend analysis examines whether a channel's values depend on sample order or elapsed time in a way the process does not explain. Three patterns are worth screening for:

1. Monotonic drift. Fit a least-squares line to the channel versus sample index and compare the slope against a physically allowable drift rate. A slow upward drift in a voltage reference channel across a 30-minute run may indicate thermal settling or self-heating; the same drift in a channel that should be constant is a plausibility failure.
2. Step changes. A cumulative-sum (CUSUM) or simple change-point scan localizes abrupt level shifts that a global trend fit would smear out. A step at sample 412 that coincides with a fixture relay actuation is explainable; a step with no corresponding event in the test log is not.
3. Autocorrelation. Compute lag-1 autocorrelation on the residuals after removing any fitted trend. Strong positive autocorrelation in a channel that should be white noise suggests the channel is reporting a filtered or duplicated signal — for example, two channels accidentally mapped to the same source, or a moving-average filter left enabled in the acquisition path.

The figure below shows how the three method families map onto a single channel's data and how their findings combine into a quarantine decision.

Applying the Methods Together: A Worked Screening Example

Consider a 12-channel analog acquisition run with 500 samples per channel at 10 ms intervals, reviewing a channel that measures a nominally constant 2.500 V reference through a 16-bit ADC. The expected spread from instrument accuracy and noise is an IQR of about 0.8 mV, and the physically allowable slew rate is 5 mV per sample.

Running the modified Z-score first yields three samples with |M| between 4.1 and 5.8, all within a 30-sample window near index 210. Because they cluster rather than appear in isolation, the finding is not treated as three independent glitches. The differencing check shows Δx of 22 mV and 19 mV at the cluster boundaries, well above the 5 mV limit. The distribution check reports an IQR of 2.6 mV — roughly three times the expected value — and the histogram shows a secondary lobe. Trend analysis on the residuals finds no significant slope but a lag-1 autocorrelation of 0.62, indicating order dependence.

Taken together, the evidence points to a transient disturbance with a settling tail rather than random noise: a likely relay or multiplexer switching event coupling into the channel. The correct action is to quarantine the affected window, cross-check the test log for a switching event near sample 210, and if none is logged, quarantine the channel and investigate the acquisition path. Note that no single method would have produced this conclusion — the outlier cluster alone could be dismissed as noise, and the elevated IQR alone could suggest a calibration issue.

Recording and Acting on Findings

Each finding should be recorded with the channel identifier, the method applied, the computed statistic, the threshold, the affected sample range, and the disposition. A compact finding record might read: CH07 | MAD | max|M|=5.8 | thr 3.5 | samples 198–227 | quarantined window | log check pending. This level of traceability lets a reviewer reconstruct the decision later and supports root-cause analysis when the same channel fails plausibility review across multiple runs.

The output of statistical plausibility review is therefore not a pass/fail verdict on the whole test but a set of scoped, evidence-backed findings, each resolved as accepted (with justification), quarantined, or escalated to hardware investigation. That discipline is what separates a review that genuinely protects data integrity from one that merely produces a report.