4. Model-agnostic explanation methods

Restraint design decision models rarely arrive as clean, interpretable rule sets. In practice, a restraint-system classifier or regressor is often a gradient-boosted tree ensemble, a random forest, or a neural network trained on crash-test simulations, occupant-loading measurements, and design-parameter sweeps. These models can predict occupant injury risk or restraint performance with useful accuracy, yet their internal logic is opaque. Model-agnostic explanation methods solve this by treating the model as a black box: they probe the mapping from input features to output predictions and reconstruct a locally faithful account of why a specific decision was made. For restraint design, this matters because engineers must justify why a particular belt-force limiter setting, anchor pre-tensioner threshold, or airbag vent size was recommended — and they must be able to challenge that recommendation when it conflicts with physical intuition or regulatory constraints.

Two families dominate practice: LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations). Both answer the question "which input features drove this prediction?", but they answer it through different mathematical constructions, and that difference has real consequences for how their outputs should be read in a restraint engineering workflow.

LIME: Local Surrogate Fitting Around a Single Decision

LIME explains one prediction at a time. Given a query instance x, it generates a set of perturbed samples in the neighborhood of x, obtains the black-box model's predictions on those samples, and fits a simple, interpretable surrogate model — typically a weighted linear regression — to approximate the black box locally. The weights of that surrogate become the explanation: a positive weight means the feature pushed the prediction up, a negative weight means it pushed it down, and the magnitude indicates influence within the local region.

For a restraint decision model, a concrete LIME run might proceed as follows. Suppose the model predicts a normalized occupant injury criterion (a continuous risk score, 0–1) from features including belt_force_limiter_kN, pretensioner_threshold_ms, airbag_vent_mm2, seat_pan_angle_deg, and occupant_mass_kg. To explain a prediction of 0.72 for a specific design point, LIME samples perturbations around that point — for continuous features, by drawing from a normal distribution centered on the query value; for categorical features such as restraint configuration type, by sampling alternative categories. Each perturbed sample is scored by the model, and a weighted ridge regression is fitted, with sample weights decaying with distance from the query. The resulting coefficients, for example belt_force_limiter_kN: −0.18, airbag_vent_mm2: +0.11, occupant_mass_kg: +0.09, indicate that within this local neighborhood, a higher force-limiter setting reduced predicted risk while a larger vent area increased it.

The practical weakness of LIME is that the surrogate is a local approximation, not a decomposition of the model itself. If the neighborhood is too wide, the linear fit averages over nonlinear behavior and the weights become misleading; if too narrow, the fit is dominated by noise. For restraint models with sharp thresholds — for example, a pretensioner firing decision that flips abruptly at a specific crash pulse intensity — LIME can produce explanations that look confident but describe a region the model does not actually behave linearly in. LIME is therefore best used for rapid, per-decision sanity checks and for communicating a single recommendation to a design reviewer, not for auditing global feature importance.

SHAP: Additive Attribution Grounded in Cooperative Game Theory

SHAP takes a different route. It assigns each feature an attribution value based on Shapley values from cooperative game theory: the contribution of a feature is its average marginal effect on the prediction across all possible orderings in which features could be "added" to the model. The attributions are additive — they sum exactly to the difference between the prediction and a baseline (the expected model output over a background dataset). This additivity is what makes SHAP attractive for engineering documentation: the explanation is a complete accounting, not an approximation.

Exact Shapley computation is exponential in the number of features, so practical implementations use approximations. TreeSHAP computes exact values efficiently for tree ensembles, which covers most restraint decision models built on gradient boosting. KernelSHAP is the model-agnostic fallback and works for any black box, at higher computational cost. A typical restraint workflow uses TreeSHAP when the model is a tree ensemble and reserves KernelSHAP for neural or surrogate models.

import shap
import numpy as np

# model: trained gradient-boosted regressor predicting injury risk score
# X_background: representative design points (e.g., 200 sampled configurations)
explainer = shap.TreeExplainer(model, data=X_background)
shap_values = explainer(X_query)          # X_query: one or more design points

# Per-decision attribution for a single design point
for name, value in zip(X_query.columns, shap_values.values[0]):
    print(f"{name:28s} {value:+.4f}")

# Expected output: attributions summing to (prediction - base_value)
print("base_value:", explainer.expected_value)
print("prediction:", model.predict(X_query)[0])

Reading the output requires care about the baseline. If expected_value is 0.55 and the prediction is 0.72, the attributions must sum to +0.17. A feature with attribution +0.08 contributed roughly half of the excess risk relative to the background population. This is a stronger statement than LIME's local weight, because it is consistent across coalitions of features rather than dependent on one sampling neighborhood.

Choosing Between Them for Restraint Decisions

The two methods answer subtly different questions, and the choice should follow the decision context rather than habit. The diagram below maps the decision contexts typical in restraint design onto the appropriate method and the interpretation each supports.

In a restraint design review, the practical distinction is this: LIME tells you what the model was sensitive to near one design point, while SHAP tells you how the prediction decomposes relative to a defined baseline. If the question is "why did the model flag this configuration as high-risk?", LIME's local weights are usually sufficient and fast. If the question is "which design parameters consistently drive risk across our candidate set, and by how much?", SHAP's additive attributions are the better instrument, because they can be aggregated across many design points into a global importance ranking without re-fitting a surrogate per instance.

Applicability Constraints Specific to Restraint Models

Three constraints deserve attention before either method is trusted in a design decision. First, feature correlation. Restraint parameters are often physically coupled — a higher force-limiter setting may co-vary with a larger vent area across the training sweep. LIME's perturbation scheme can generate physically implausible combinations (e.g., a force limiter paired with a vent size that never co-occurs in the dataset), producing attributions for configurations the model never saw. SHAP's marginal-coalition construction handles correlation more gracefully but still attributes credit under a specific background distribution; changing the background dataset changes the attributions. Always state which background set was used.

Second, baseline choice. SHAP attributions are relative to expected_value, which depends entirely on the background sample. A baseline drawn from the full design space yields different attributions than one drawn from a narrow production configuration. For restraint decisions, define the baseline explicitly — for example, "the current production restraint configuration" — and keep it fixed across comparisons so that attribution changes reflect model or design changes, not baseline drift.

Third, categorical and threshold features. Restraint models frequently include categorical choices (belt type, airbag module variant) and near-discrete thresholds (pretensioner firing time). LIME's perturbation of categorical features by random resampling can distort the local fit; SHAP handles them as coalitions but the resulting attributions can be dominated by the category split rather than by continuous design parameters. Inspect per-feature attributions rather than relying on a single aggregate importance score.

A defensible workflow therefore pairs the two methods rather than choosing one permanently. Use SHAP to produce a global attribution summary across the candidate design set and to document the dominant drivers for review; use LIME to interrogate individual decisions that look anomalous or that conflict with expected restraint behavior. When the two disagree sharply on a single design point, that disagreement is itself diagnostic: it usually signals a nonlinear threshold, a correlated feature pair, or a background-distribution mismatch — each of which is worth resolving before the recommendation is accepted. Record the method, the background dataset, the perturbation or sampling settings, and the random seed alongside every explanation that informs a design decision, so the reasoning can be reproduced and audited later.