1. Course overview

Many engineering systems do not output a continuous measurement for a human to interpret; they output a binary verdict. A battery management system declares a cell pack safe or unsafe. An automated optical inspection station passes or rejects a solder joint. A structural health monitor clears an aircraft spar for the next flight cycle or flags it for inspection. In each case the underlying quantity — internal resistance, joint void area, crack length — is continuous and uncertain, but the consequence is discrete. This course is about making those discrete decisions in a way that is defensible, quantifiable, and tied to an explicit risk budget.

The central difficulty is that a pass-fail decision is a comparison between an estimate and a threshold, and both sides carry uncertainty. The estimate comes from a model, a sensor chain, or a regression fit, and it is never exact. The threshold is often a specification limit, a regulatory boundary, or an internally derived safety line, and it too may be uncertain or conservatively chosen. If you compare a point estimate directly against a threshold, you silently assume the estimate is correct. That assumption is the source of most field failures in automated decision systems: the model was right on average, but the individual decision was made at a point where the estimate and the truth diverged.

Why a Point Estimate Is Not a Decision

Consider a concrete example used throughout this course. A regression model predicts the remaining capacity of a lithium-ion cell from impedance spectroscopy features. The acceptance rule is: if predicted capacity is at least 80% of nominal, the cell passes. A particular cell returns a point prediction of 81.2%. Naively, it passes. But suppose the model's 90% prediction interval for that cell is [76.5%, 85.9%]. The interval straddles the 80% limit. The honest statement is not "this cell passes" but "this cell passes if we are willing to accept that the true capacity may be below the limit with non-trivial probability." A decision rule that ignores the interval width is not risk-controlled; it is a coin flip dressed as a measurement.

This is the distinction between a confidence interval and a prediction interval, and it matters enormously here. A confidence interval describes uncertainty in an estimated parameter — for example, the mean capacity of a production batch. A prediction interval describes uncertainty in a single future observation — the capacity of this cell, right now. Pass-fail decisions are almost always about individual items, so prediction intervals, not confidence intervals, are the correct uncertainty object. They are also wider, because they must absorb both the model's parameter uncertainty and the irreducible item-to-item and measurement noise.

Decision Margins as a Risk Budget

A decision margin is the buffer you insert between the decision boundary and the threshold so that uncertainty cannot silently flip the verdict. Formally, if the threshold is T, the point estimate is ŷ, and the prediction interval half-width at coverage level 1−α is w, a conservative pass rule is:

pass  ⇔  ŷ − w ≥ T          (lower-bound rule, "must be provably above")
fail  ⇔  ŷ + w ≤ T          (upper-bound rule, "must be provably below")
otherwise: indeterminate / escalate to review

The margin here is w, and it is not a fixed number — it depends on the coverage level, the local noise, and how far the input sits from the training distribution. That dependence is the whole point. A cell measured under conditions the model has seen thousands of times earns a narrow interval and therefore a small margin; a cell measured at an unusual temperature or state of charge earns a wide interval and a large margin, pushing borderline cases into the indeterminate band where a human or a second test resolves them.

How the Pieces Fit Together

The diagram below shows the decision architecture this course builds toward. A model produces a point prediction and a prediction interval; the interval width, together with the threshold, defines the pass region, the fail region, and the indeterminate band. The width of that band is set by the coverage target and by the local uncertainty, and it is the primary lever for trading false accepts against false rejects.

What "Risk-Controlled" Actually Means

A decision system is risk-controlled when you can state, before deployment, the rate at which it will make each kind of error, and when that stated rate holds under the operating conditions you care about. There are two error types and they are not symmetric. A false accept passes a bad item — a safety escape, usually the expensive and reputationally fatal error. A false reject fails a good item — a yield loss, expensive but recoverable. The prediction interval lets you set the margin so that the false-accept rate is bounded at a chosen level, for example "no more than 1% of truly out-of-spec items are passed," while reporting the resulting false-reject rate as a cost you accept.

Two properties of prediction intervals make this credible rather than aspirational. First, empirical coverage is measurable: on a held-out calibration set, you can count how often the true value falls inside the interval and compare to the nominal level. If you asked for 90% and observe 84%, the interval is too narrow and the margin is unsafe. Second, coverage can be made distribution-free using conformal prediction, which calibrates interval width from observed residuals without assuming Gaussian errors. That matters in engineering because sensor residuals are frequently heavy-tailed or heteroscedastic — variance grows with load, temperature, or age — and a Gaussian interval will under-cover exactly where the decisions are hardest.

What This Course Covers

The course proceeds from foundations to deployment. Early chapters establish the decision-theoretic framing: loss functions for asymmetric errors, the relationship between threshold placement and error rates, and why the indeterminate band is the correct home for high-uncertainty cases. The middle section builds the uncertainty machinery — residual-based intervals, conformal calibration, quantile regression, and the diagnostics that tell you when an interval is untrustworthy (heteroscedasticity, distribution shift, sparse support). Later chapters turn to operations: setting margins from a stated risk budget, monitoring coverage in production, handling the indeterminate queue, and auditing decisions after the fact.

By the end you should be able to take a pass-fail problem, identify the correct uncertainty object, calibrate a prediction interval with verified coverage, derive a decision margin from an explicit false-accept target, and defend the resulting rule with numbers rather than intuition. The recurring discipline is simple: never let a point estimate make a decision that a distribution should make.