2. Explainable AI fundamentals for restraint design
Restraint design decisions—whether they concern occupant classification thresholds, airbag deployment timing, seat-belt load limiters, or child-seat detection logic—are made under uncertainty and carry direct safety consequences. When a machine-learning model participates in those decisions, the engineering team must be able to answer a deceptively simple question: why did the system produce this output? Explainable AI (XAI) is the discipline that makes this question answerable in a rigorous, auditable way. This section establishes the vocabulary and conceptual foundations that the rest of the course builds on.
What "Explainable" Actually Means
An explanation is not a summary of what a model did; it is a causal account of which inputs, features, or internal representations drove a specific output, and by how much. In restraint engineering, that account must be specific enough to support a design decision. Saying "the classifier detected an adult occupant" is a prediction. Saying "the classifier assigned 0.91 probability to the adult class primarily because the seat-rail pressure sensor read 42 kg and the belt-pretensioner payout was 180 mm, while the occupant's estimated stature of 158 cm pushed the score downward" is an explanation.
Three properties distinguish a usable explanation in this domain:
- Fidelity — the explanation must faithfully reflect the model's actual decision function, not a plausible post-hoc story. A surrogate that approximates the model poorly is worse than no explanation, because it creates false confidence.
- Stability — small, physically irrelevant perturbations to the input (sensor noise, a 1 mm seat-position change) should not produce radically different explanations. Unstable attributions signal that the model has learned spurious correlations.
- Actionability — the explanation must map onto quantities the restraint engineer can change or verify: sensor placement, calibration, feature engineering, or the decision threshold itself.
Interpretability versus Explainability
These terms are often used interchangeably, but they describe different relationships between the engineer and the model.
Interpretability is a property of the model itself: the degree to which a human can understand the mapping from input to output by inspecting the model's structure. A shallow decision tree over three occupant features, or a logistic regression with signed coefficients, is intrinsically interpretable—you can read the decision rule directly. Explainability is a broader property of the system: the degree to which an external observer can obtain a faithful account of a decision, whether or not the underlying model is transparent. A gradient-boosted ensemble or a deep neural network is not interpretable in the structural sense, but it can be made explainable through attribution methods, counterfactual analysis, or surrogate models.
| Aspect | Interpretability | Explainability |
|---|---|---|
| Locus | Intrinsic to model structure | System-level, often post-hoc |
| Typical methods | Linear models, shallow trees, rule lists | SHAP, LIME, counterfactuals, saliency |
| Cost | May sacrifice accuracy or expressiveness | Adds computation; fidelity must be validated |
| Restraint example | Threshold rule on belt-payout + weight | Per-decision attribution for a fusion model |
The practical consequence is a design trade-off. If a restraint decision can be captured by an interpretable rule with acceptable accuracy, prefer it—regulators and auditors can review the rule directly. When accuracy demands a complex model, explainability becomes a mandatory engineering deliverable rather than an optional analysis.
Why Transparency Matters in Safety-Critical Decisions
Restraint systems are safety-critical: a false "child present" classification can suppress deployment, and a false "adult" classification can cause an over-aggressive deployment. Transparency matters for four concrete reasons.
Verification and validation. ISO 26262 and comparable functional-safety frameworks require that the behavior of safety-related functions be specified, verified, and traceable. An opaque model whose decisions cannot be decomposed makes it impossible to demonstrate that the function meets its safety goals across the operational design domain.
Failure diagnosis. When a restraint system misbehaves in the field, engineers must distinguish a sensor fault, a calibration drift, a distribution shift, and a model error. Attribution evidence narrows this quickly: if the explanation shows the decision was dominated by a sensor channel known to be noisy at low temperatures, the investigation has a direction.
Accountability and regulation. Type-approval and incident investigation demand a defensible record of why a deployment decision was made. "The model said so" is not a defensible record.
Trust calibration. Engineers and safety reviewers must know when to trust the model and when to override it. Explanations that reveal the model's reliance on physically meaningful features build justified trust; explanations that reveal reliance on spurious correlations (for example, seat-fabric color correlating with occupant class in the training set) expose risks that accuracy metrics alone hide.
How the Explanation Pipeline Fits Together
The diagram below shows the relationship between the physical restraint decision, the model, and the explanation layer. Note that the explanation is generated from the same inputs and outputs that produced the decision—this shared provenance is what makes the explanation auditable.
Reading the Pipeline in Practice
Consider a concrete example. A fusion model receives four features for a given seat: rail pressure 42 kg, belt payout 180 mm, seat-position normalized value 0.62, and estimated stature 158 cm. The model outputs P(adult) = 0.91 and the restraint controller selects deployment stage 2. The explanation layer reports that pressure and payout contributed +0.34 and +0.28 to the adult score, while the stature estimate contributed −0.12. A counterfactual query shows that reducing payout below 120 mm would drop P(adult) to 0.44, crossing the decision boundary.
This is actionable. The engineer can now ask: is the model's reliance on belt payout physically justified, or is it exploiting a correlation in the training data? Is the stature estimate reliable enough to carry negative weight? Does the decision boundary at 0.44 correspond to a validated occupant-classification threshold? Each question is answerable because the explanation exposes the decision's internal structure.
The remainder of this course applies these foundations to specific restraint-design workflows: selecting interpretable versus explainable model architectures, validating attribution fidelity, constructing counterfactual test cases, and documenting explanation evidence for safety review. The vocabulary introduced here—fidelity, stability, actionability, and the interpretability/explainability distinction—recurs throughout.