3. Decision margin formulation

A prediction interval is only half of a pass-fail decision. The other half is a threshold, and the decision rule that compares the two. This chapter formalizes that comparison: how to define the pass-fail threshold on the quantity of interest, how to express the decision margin as a signed distance from that threshold, and how the two interval bounds interact with the threshold to produce a decision outcome that is sometimes certain and sometimes indeterminate.

Defining the threshold on the decision quantity

Every pass-fail decision must name a single scalar decision quantity y and a threshold T on that quantity. The threshold is not the same as a specification limit in general. A specification limit is a requirement imposed by a customer or standard; a threshold is the value the decision rule actually compares against, and it may be the specification limit, a derated version of it, or a value shifted to account for measurement bias. Making this distinction explicit prevents a common failure mode in which the model is validated against one number and deployed against another.

Two conventions must be fixed before any margin is computed. First, the direction of the requirement: does a larger y mean better (a lower-bound requirement, pass when y ≥ T) or does a smaller y mean better (an upper-bound requirement, pass when y ≤ T)? Second, the sign convention for the margin. Throughout this course we define the margin as a signed distance that is positive when the point prediction is on the passing side of the threshold. For a lower-bound requirement this gives m = ŷ − T; for an upper-bound requirement it gives m = T − ŷ. Under this convention a positive margin always means "the point estimate passes," regardless of requirement direction, which keeps downstream logic uniform.

Thresholds should be recorded with their units, their direction, and their provenance. A threshold that arrives as "0.85" without units or direction is not a usable decision parameter. In practice the threshold is often derived: for example, a minimum pull strength of 42.0 N after a 3 % safety derating applied to a 43.3 N specification limit, or a maximum surface roughness of 1.6 µm taken directly from a drawing callout. The derivation belongs in the decision record because it is the first place an audit will look.

Margin as distance, and why the interval matters

With the point prediction ŷ and threshold T fixed, the margin m is a single number. On its own it answers only "how far is the estimate from the line?" It does not answer "how confident are we that the true value is on the passing side?" That second question requires the prediction interval [L, U], which carries the model's uncertainty about the true response at the given input.

The decision outcome is determined by where the threshold falls relative to the interval, not relative to the point estimate. Three cases arise for a lower-bound requirement:

• Certain pass: L ≥ T. The entire interval lies on the passing side, so the decision is pass regardless of where the true value sits inside the interval.
• Certain fail: U < T. The entire interval lies on the failing side; the decision is fail.
• Indeterminate: L < T ≤ U. The interval straddles the threshold, so the outcome depends on the unknown true value. The point estimate may be comfortably positive while the decision is still indeterminate.

This is the coupling the chapter title refers to. The margin m and the half-width h = (U − L)/2 together determine the outcome: a certain pass requires m ≥ h, a certain fail requires m ≤ −h, and anything in between is indeterminate. The ratio m/h is therefore a useful normalized quantity — it says how many half-widths of uncertainty separate the estimate from the threshold. A margin of 2.0 with a half-width of 0.5 gives a ratio of 4 and a certain pass; the same margin with a half-width of 2.5 gives a ratio of 0.8 and an indeterminate outcome. Identical margins, opposite decision confidence.

Worked example: three parts, one threshold

Consider a tensile test on a molded polymer coupon with a lower-bound requirement of 42.0 N. A regression model predicts peak load from process and geometry features and reports a 90 % prediction interval. Three coupons are evaluated:

Coupon A passes with confidence because its lower bound clears the threshold. Coupon B has a positive margin of 0.9 N, yet its interval reaches down to 40.2 N, so the decision cannot be made from the model alone — it needs either a tighter interval or a confirmatory measurement. Coupon C fails with confidence. Note that B and A have similar point predictions but very different interval widths; the width, not the point estimate, drives the outcome. These values are illustrative and constructed for teaching, not measured results.

Choosing the decision rule

Once the three cases are recognized, the decision rule is a policy choice. Three common policies exist, and they differ in how they treat the indeterminate band:

1. Conservative (lower-bound) rule: pass only when L ≥ T. Indeterminate cases are treated as fail. This minimizes false passes at the cost of rejecting some good parts.
2. Optimistic (upper-bound) rule: pass when U ≥ T. Indeterminate cases are treated as pass. This minimizes false fails but admits parts whose true value may be below the threshold.
3. Three-way rule: pass, fail, or escalate to a confirmatory test. This is the only policy that preserves the information the interval provides, and it is usually the right choice when a confirmatory measurement is cheap relative to the cost of a wrong decision.

The policy must be chosen before the model is deployed and recorded alongside the threshold. A common error is to compute margins with a conservative rule during validation and then apply an optimistic rule in production, which silently changes the false-pass rate. The interval coverage level (for example 90 % versus 95 %) is part of the same policy: a wider interval at higher coverage moves more cases into the indeterminate band, so coverage and decision rule must be tuned together.

Failure symptoms and troubleshooting

Two symptoms point to a formulation problem rather than a model problem. First, a high rate of indeterminate outcomes across the population usually means the interval is too wide relative to the margin distribution — either the coverage level is too high for the application, the model's uncertainty is genuinely large, or the threshold sits in a region where the model is poorly constrained. Second, a decision rate that flips between validation and production with no change in the model usually means the threshold, direction, or decision rule was recorded inconsistently. In both cases, inspect the threshold provenance and the interval coverage before retraining anything.

For an upper-bound requirement the same machinery applies with the sign convention flipped: m = T − ŷ, a certain pass requires U ≤ T, and a certain fail requires L > T. The three-case structure and the m/h ratio carry over unchanged, which is why fixing the sign convention at the outset matters: it lets one decision engine serve both requirement directions without special-casing.