3. Taxonomy of explainable AI methods

Explainability is not a single technique but a design space defined by several orthogonal axes. For high-stakes engineering decisions — structural safety margins, turbine blade inspection, power grid fault isolation, or autonomous braking arbitration — the choice of explanation method is itself an engineering decision with safety consequences. This chapter organizes the field along four axes: intrinsic vs. post-hoc, model-specific vs. model-agnostic, local vs. global, and the resulting suitability for engineering decisions. Understanding where a method sits on each axis lets you predict its cost, its fidelity, and its failure modes before you deploy it.

Intrinsic vs. Post-hoc Explanations

Intrinsic (also called self-explaining or ante-hoc) methods build interpretability into the model structure. A shallow decision tree, a sparse linear regression, a generalized additive model (GAM), or a rule list produces an explanation that is the model — there is no separate approximation step. For a linear model predicting fatigue life from stress amplitude, the coefficient on stress amplitude directly quantifies the sensitivity, and its sign tells you the direction of the effect. The advantage is fidelity: the explanation is exact by construction. The cost is capacity. Intrinsically interpretable models often cannot capture the high-order interactions that make gradient-boosted trees or deep networks effective on sensor fusion or image-based defect detection.

Post-hoc methods explain a trained model after the fact. SHAP, LIME, integrated gradients, and permutation importance all fall here. They can be applied to any model, including a black-box ensemble, but they approximate. A SHAP value is an estimate of a Shapley value under a specific background distribution; change the background and the numbers change. This distinction matters operationally: when an intrinsic model says "stress amplitude contributes +0.42 mm to predicted crack growth," that is a model parameter. When a post-hoc method says the same thing, it is an estimate with sampling variance and assumptions you must validate.

Rule of thumb: if the decision is safety-critical and the model can be constrained without unacceptable accuracy loss, prefer intrinsic. If accuracy demands a complex model, use post-hoc but treat the explanation as a measurement with uncertainty, not as ground truth.

Model-Specific vs. Model-Agnostic

Model-specific methods exploit the internal structure of a particular architecture. Grad-CAM uses convolutional feature maps; attention rollout uses transformer attention weights; tree SHAP uses the tree ensemble's path structure. These methods are typically faster and more faithful because they read the model's actual computation. Their limitation is portability: a Grad-CAM implementation does not transfer to a gradient-boosted model on tabular sensor data.

Model-agnostic methods treat the model as a function f: X → Y and only require query access. LIME perturbs inputs and fits a local surrogate; permutation importance shuffles features and measures the drop in a chosen metric; KernelSHAP samples coalitions and solves a weighted least-squares problem. Agnosticism buys flexibility at the cost of query budget and approximation error. For a model with 200 sensor features, KernelSHAP's cost grows with the number of features and the number of samples; you must budget for it.

Local vs. Global Explanations

A local explanation answers "why this prediction, for this input?" — a single SHAP force plot for one bearing vibration window, or a LIME surrogate around one operating point. A global explanation answers "how does the model behave overall?" — a SHAP summary plot, a partial dependence curve, or a surrogate decision tree distilled from the full model.

The engineering trap is assuming local explanations aggregate into a global picture. They do not. A feature can be important locally for one regime and irrelevant globally. In a wind turbine gearbox model, oil temperature may dominate predictions during cold starts but be negligible across the annual operating distribution. If your maintenance policy is global but your explanations are local, you may optimize for the wrong regime. Conversely, a global explanation can hide a dangerous local anomaly: the model may be globally well-behaved yet produce an unsafe prediction for a rare but physically plausible input.

Taxonomy of XAI Methods for Engineering Decisions Intrinsic vs. Post-hoc Intrinsic Linear, GAM, tree, rule list — exact Post-hoc SHAP, LIME, IG — approximate Fidelity vs. capacity trade-off Post-hoc = measurement with variance Validate background distribution Model-specific vs. Agnostic Model-specific Grad-CAM, attention, tree SHAP — fast Model-agnostic LIME, KernelSHAP, permutation — flexible Query budget scales with features Agnostic = portable but costly Specific = faithful but architecture-bound Local vs. Global Local One prediction, one operating point Global Full input distribution, overall behavior Local ≠ aggregated global Regime-dependent importance Global can mask local anomalies Engineering Suitability • Safety-critical → intrinsic or validated post-hoc • High-dim sensors → budget-aware agnostic • Maintenance policy → global + local drill-down • Certification → traceable, reproducible method • Real-time control → low-latency specific methods • Rare-event safety → local stress testing Match method to decision consequence

Choosing a Method for an Engineering Decision

The four axes combine into a practical selection procedure. Start from the decision consequence: what happens if the explanation is wrong or misleading? For a non-critical recommendation (e.g., suggesting a maintenance window), a fast local LIME explanation may suffice. For a safety interlock or a regulatory submission, you need either an intrinsic model or a post-hoc method whose approximation error you have quantified against a held-out set.

Next, consider the model constraint. If you can train an intrinsically interpretable model without dropping below the required accuracy threshold, do so — it eliminates an entire class of explanation-validation work. If not, select a post-hoc method that matches your architecture (tree SHAP for ensembles, Grad-CAM for CNNs) to minimize approximation error, and fall back to agnostic methods only when portability across model versions is required.

Finally, decide the scope. Global explanations support policy, design review, and operator training. Local explanations support incident investigation, real-time operator trust, and edge-case debugging. A robust engineering workflow uses both: a global summary to establish baseline behavior, then local drill-down when a specific prediction is questioned or when an anomaly is detected.

Worked Example: Bearing Fault Classifier

Consider a vibration-based bearing fault classifier trained on 12 spectral features to output one of four classes: healthy, inner-race defect, outer-race defect, or ball defect. The model is a gradient-boosted tree with 300 estimators, achieving 96% test accuracy. You must explain its decisions to a maintenance engineer.

Step 1 — Intrinsic check. Train a depth-4 decision tree on the same features. If accuracy stays above 92%, the tree is a candidate intrinsic surrogate. Suppose it drops to 88% — below threshold. Proceed to post-hoc.

Step 2 — Model-specific post-hoc. Apply TreeSHAP to the gradient-boosted model. For a single vibration window classified as outer-race defect, TreeSHAP returns feature contributions. Example output (illustrative values):

feature_contributions = {
  "BPFO_amplitude":   +0.38,
  "BPFI_amplitude":   -0.12,
  "shaft_speed":      +0.05,
  "RMS_vibration":    +0.09,
  "kurtosis":         +0.04,
  ...
}
base_value = 0.25  # expected model output (log-odds)
prediction = 0.25 + sum(contributions) = 0.69  # outer-race defect

The dominant positive contributor is BPFO (ball-pass frequency outer-race) amplitude, which is physically consistent with an outer-race defect. This is a local, model-specific, post-hoc explanation.

Step 3 — Global validation. Compute mean absolute SHAP values across the full test set to rank global feature importance. If BPFO amplitude ranks first globally and BPFI amplitude ranks second, the local explanation aligns with global behavior. If a feature like shaft_speed dominates globally but is negligible locally, investigate whether the model is relying on a spurious correlation.

Step 4 — Failure symptom check. A common failure mode is that SHAP values are computed with a background distribution that does not match the operating regime. If the background is dominated by healthy samples but the query sample is from a cold-start regime, the explanation may attribute importance to temperature-related features that are irrelevant in steady state. Mitigation: stratify the background by operating regime, or use a conditional expectation background.

Do not treat post-hoc explanations as causal. A high SHAP value means the feature influenced the model's output, not that it caused the physical fault. Always cross-check against domain physics — in the bearing example, BPFO amplitude is meaningful because it corresponds to a known fault frequency, not merely because the model weighted it.

Summary of Axes and Engineering Fit

Axis Option A Option B Engineering consideration
Timing Intrinsic Post-hoc Intrinsic = exact but capacity-limited; post-hoc = flexible but approximate
Portability Model-specific Model-agnostic Specific = faster, faithful; agnostic = portable, query-expensive
Scope Local Global Local ≠ aggregated global; use both for policy + incident response
Validation Built-in Requires held-out check Post-hoc needs approximation-error quantification

The taxonomy is not a hierarchy of quality — no axis is inherently better. The correct choice depends on the decision's consequence, the model's architecture, the available query budget, and the operating regime. In the next chapter, we apply these axes to specific method families, beginning with feature-attribution techniques and their validation protocols.

← Previous 📖 Contents Next →