3. Surrogate modeling foundations

An expensive automotive simulation — a full-vehicle crash model with 5–10 million elements, a CFD run over a 40-million-cell mesh, or a transient thermal solve over a 3-hour drive cycle — can take hours to days per evaluation. Active learning only pays off if we can replace that simulator with a cheap surrogate model (also called an emulator or metamodel) that is accurate enough to guide where the next expensive run should go. This chapter establishes the three surrogate families that dominate automotive emulation: Gaussian processes (GPs), polynomial chaos expansions (PCE), and neural network (NN) surrogates. The goal is not to derive each method, but to build the engineering intuition needed to choose, fit, and diagnose them inside an active-learning loop.

What a surrogate must deliver

Formally, we approximate a simulator f: X → Y with a surrogate f̂ trained on a design of experiments D = {(xᵢ, yᵢ)}. For automotive work, three properties matter more than raw accuracy:

• Sample efficiency. Crash and CFD runs are the budget. A surrogate that needs 500 points is useless when the budget is 60.
• Uncertainty quantification. Active learning needs to know where the surrogate is unsure, not just its prediction. This is the single most important differentiator among the three families.
• Smoothness and physical plausibility. Responses such as intrusion, HIC, or drag coefficient are continuous in the design variables; a surrogate that oscillates between training points will mislead the acquisition function.

Gaussian processes: the uncertainty-native baseline

A GP places a prior over functions: f(x) ~ GP(m(x), k(x, x')). The mean m(x) is usually zero after normalization; the kernel k encodes smoothness and length-scale. Given training data, the posterior at a new point x* is Gaussian with closed-form mean and variance:

μ(x*) = k*ᵀ (K + σₙ²I)⁻¹ y
σ²(x*) = k(x*, x*) − k*ᵀ (K + σₙ²I)⁻¹ k*

Here K is the Gram matrix of kernel evaluations at training inputs, k* the vector of kernel values between x* and the training set, and σₙ² the observation noise. The predictive variance σ²(x*) shrinks to near zero at training points and grows in unexplored regions — exactly the signal an active learner needs.

For automotive responses, the Matérn 5/2 kernel is a common default because it assumes only twice-differentiable functions, which is more realistic than the infinitely smooth squared-exponential kernel for responses with kinks (e.g., a structure that switches from elastic to plastic behavior). A practical starting configuration for a 5-variable crash surrogate:

kernel   = Matérn(nu=2.5, length_scale=[1.0]*5, length_scale_bounds=(1e-2, 1e2))
alpha    = 1e-6      # jitter for numerical stability
normalize_y = True
n_restarts_optimizer = 5

The main limitation is cost: fitting scales as O(n³) and prediction as O(n²) per point. Beyond roughly 2,000–5,000 samples, a vanilla GP becomes impractical, which is precisely why sparse GP approximations and the other two families exist.

Polynomial chaos: spectral accuracy for smooth responses

Polynomial chaos expansion represents the response as a sum of orthogonal polynomials in the input random variables:

f(x) ≈ Σ_{α∈A} c_α Ψ_α(ξ),   ξ = standardized inputs

For independent inputs, Ψ_α is a tensor product of univariate polynomials chosen to match the input distribution — Hermite for Gaussian, Legendre for uniform, Laguerre for exponential. The multi-index set A is typically truncated by total degree p or by a hyperbolic (sparsity) rule. The coefficients c_α can be computed by projection (quadrature) or by regression (least squares / LASSO).

PCE is exceptionally efficient when the response is smooth in the inputs and the input dimension is moderate (roughly d ≤ 10–15). Once fitted, the mean and variance are read directly from the coefficients: E[f] = c₀ and Var[f] = Σ_{α≠0} c_α². This makes PCE a natural fit for NVH, aerodynamic drag, and thermal responses where the underlying physics is smooth. Its weakness is that it provides no per-point predictive variance in the standard regression form, and it degrades sharply for discontinuous or highly nonlinear responses such as crash pulses with contact switching.

Neural network surrogates: scalability and flexibility

A feed-forward network f̂(x; θ) with 2–4 hidden layers and 64–256 units per layer can approximate high-dimensional, non-smooth responses that defeat both GPs and PCE. Training minimizes a loss such as MSE with Adam, typically for a few thousand epochs on a few hundred to a few thousand samples. For automotive use, two design choices matter:

• Input/output normalization. Standardize each design variable to zero mean and unit variance; scale outputs to O(1). Without this, gradient descent stalls.
• Uncertainty. A plain NN gives a point prediction only. To recover uncertainty for active learning, use an ensemble of 5–10 networks with different initializations; the ensemble mean and variance serve as μ(x*) and σ²(x*). Deep ensembles are the pragmatic default in crash and CFD emulation.

NNs scale gracefully to thousands of samples and dozens of inputs, but they are data-hungry and can extrapolate wildly outside the training domain — a serious risk when the active learner proposes a design near the boundary.

Choosing among the three

The decision is driven by sample budget, input dimension, response smoothness, and whether uncertainty is required. The table below summarizes the practical trade-offs for automotive simulation emulation.

A useful rule of thumb: start with a GP when the budget is under ~200 runs and the dimension is under ~10; switch to PCE when the response is known to be smooth and you want analytic moments; move to an NN ensemble when the dimension exceeds ~15 or the response has contact/switch discontinuities. In practice, many teams fit two surrogates and compare their cross-validated error before committing.

Diagnosing a fitted surrogate

Before a surrogate is allowed to drive active learning, validate it with leave-one-out or k-fold cross-validation. Two metrics are standard:

• Normalized RMSE (NRMSE): √(Σ(yᵢ − ŷᵢ)² / Σ(yᵢ − ȳ)²). Values below 0.05 indicate a usable surrogate for most automotive responses; above 0.15 suggests the model is not yet trustworthy.
• Predictive uncertainty calibration: the fraction of held-out points falling within the 95% predictive interval should be close to 0.95. If it is much lower, the surrogate is overconfident and the acquisition function will exploit a false optimum.

How the three families fit into the active-learning loop

The surrogate is not a one-shot artifact; it is refit after every new expensive simulation. The diagram below shows how the three families plug into the same loop, with their distinguishing outputs feeding the acquisition function.

The loop is deliberately family-agnostic: the acquisition function only needs a predictive mean and a predictive uncertainty. GPs supply both analytically; PCE supplies the mean and, with bootstrap or sparse-PCE variants, an approximate variance; NN ensembles supply both through ensemble spread. This is why the choice of surrogate is not merely a modeling preference — it determines how the active learner explores.

With these three foundations in place, the next step is to connect the surrogate's predictive uncertainty to a concrete acquisition rule — the mechanism that turns a fitted emulator into an active learner.