2. Surrogate modeling fundamentals

A surrogate model (also called a metamodel, response surface, or emulator) is a computationally cheap mathematical approximation of an expensive simulation code. In CAE-driven design optimization, the high-fidelity solver — a nonlinear finite element analysis, a CFD run, or a transient thermal simulation — typically costs minutes to hours per evaluation. A gradient-free optimizer may require hundreds or thousands of such evaluations, which makes direct optimization impractical. The surrogate replaces the solver with a function f̂(x) ≈ f(x) that can be evaluated in microseconds, enabling design space exploration, sensitivity analysis, and optimization at negligible cost.

The essential trade-off is accuracy versus cost. A surrogate is never exact; it is trained on a finite set of high-fidelity samples and interpolates or extrapolates between them. Its usefulness depends on whether the approximation error is small enough to support the design decisions being made. This is why surrogate modeling is fundamentally a statistical discipline: it must quantify its own uncertainty, not merely produce point predictions.

Key Terminology

Before describing the workflow, it is worth fixing the vocabulary that recurs throughout this course.

  • Design variables (inputs, factors) — the parameters the designer controls, e.g., plate thickness t = 2.0–6.0 mm, rib height h = 10–40 mm, fillet radius r = 1–8 mm. They form the design vector x ∈ ℝd.
  • Design of Experiments (DoE) — the sampling plan that selects which x values to simulate. Latin Hypercube Sampling (LHS), full factorial, and Sobol sequences are common choices.
  • Responses (outputs, quantities of interest) — the simulated results, e.g., maximum von Mises stress σ_max, mass m, first natural frequency f₁, or drag coefficient C_d.
  • Training set / test set — samples used to fit the surrogate versus held-out samples used to estimate generalization error.
  • Hyperparameters — model-internal settings tuned during fitting (kernel length-scales in a Gaussian process, tree depth in a random forest, network width in a neural surrogate).
  • Prediction uncertainty — the surrogate's own estimate of how wrong it might be at a given x; Gaussian processes provide this natively, most other models require separate treatment.

The Surrogate Modeling Workflow

A defensible surrogate workflow has six stages, and skipping any of them is a common source of silent failure.

  1. Problem definition. Fix the design variables, their bounds, and the responses. Decide the accuracy target — for example, "stress prediction within 5% of FEA over the design space."
  2. Sampling (DoE). Generate n design points. For a 3-variable problem, n = 30–60 LHS samples is a reasonable starting budget; space-filling matters more than raw count for smooth responses.
  3. High-fidelity evaluation. Run the CAE solver at each sample. This is the dominant cost and where parallel job submission pays off. Log every input–output pair with solver version and mesh settings.
  4. Model fitting. Train one or more candidate surrogates on the sample set, tuning hyperparameters by cross-validation.
  5. Validation. Evaluate on held-out samples and compute error metrics: R², RMSE, and maximum relative error. A surrogate with R² = 0.98 but 40% error at one corner of the space is not ready for optimization.
  6. Deployment and adaptive refinement. Use the surrogate inside the optimizer. When the optimizer proposes a promising design, optionally verify it with a high-fidelity run and feed the result back to improve the model (adaptive sampling / infill).
A practical rule of thumb: if the surrogate's validation error is comparable to the manufacturing tolerance or the safety factor margin, the surrogate is accurate enough for the decision at hand. Accuracy requirements should be set by the engineering decision, not by an abstract target like "R² > 0.99."

Taxonomy of Machine Learning Surrogates

Surrogate models fall into several families, each with distinct inductive biases. The choice depends on sample size, dimensionality, smoothness of the response, and whether uncertainty estimates are needed.

FamilyRepresentative methodsStrengthsLimitations
Polynomial / response surfaceQuadratic RSM, polynomial chaosTransparent, cheap, few samplesPoor for highly nonlinear or high-dimensional responses
Kernel / BayesianGaussian process regression, KrigingNative uncertainty, excellent for smooth low-dimensional problemsO(n³) fitting cost; struggles above ~20 dimensions
Tree ensemblesRandom forest, gradient boostingHandles discontinuities and mixed variables; robustPoor extrapolation; uncertainty requires extra machinery
Neural networksMLP, CNN surrogates for field outputsScales to high dimensions and large datasets; can predict full fieldsData-hungry; unstable training; calibration needed
Reduced-order modelsPOD, proper orthogonal decomposition; ROMCompresses high-dimensional field data efficientlyLinear subspace assumption; needs full-field snapshots

For a typical structural optimization with 3–10 design variables and 50–200 FEA runs, a Gaussian process is usually the strongest default: it provides both a prediction and a variance, which the optimizer can exploit through expected improvement. When the response contains sharp transitions — contact activation, buckling mode switching — tree ensembles or neural networks often outperform kernel methods. When the output is a full stress field rather than a scalar, a POD-based ROM or a CNN surrogate is the appropriate choice.

Surrogate Workflow Define variables x, bounds, responses DoE sampling LHS, n = 30–60 CAE evaluation FEA / CFD runs Fit surrogate GP / RF / NN Validate R², RMSE, max err Optimize + infill adaptive refinement (infill) Surrogate Families and Their Fit Polynomial RSM few samples smooth, low-dim Gaussian Process uncertainty native d ≤ ~20 Tree Ensembles discontinuities mixed variables Neural Networks high-dim, large n field outputs POD / ROM full-field data linear subspace Selection guidance • 3–10 variables, 50–200 runs, smooth response → Gaussian process (uncertainty drives expected-improvement infill) • Contact activation, buckling mode switching → tree ensemble or neural network • Full stress/displacement field as output → POD-based ROM or CNN surrogate

What "Good" Looks Like in Practice

Consider a bracket optimization with three design variables (thickness, rib height, fillet radius) and two responses (mass, maximum stress). A team runs 40 LHS samples through a nonlinear FEA, fits a Gaussian process with a Matérn 5/2 kernel, and validates on 10 held-out points. Suppose the validation yields R² = 0.97 for mass and R² = 0.91 for stress, with a maximum relative error of 12% at the smallest fillet radius. The mass surrogate is ready for optimization; the stress surrogate is usable for screening but should not be trusted as a hard constraint without infill verification. The team then runs the optimizer on the surrogate, takes the top five candidate designs, and re-runs FEA on each. If the FEA stress exceeds the surrogate prediction by more than the safety margin, those points are added to the training set and the model is refit — this is the adaptive refinement loop shown in the diagram.

Two failure modes deserve explicit attention. First, extrapolation: a surrogate evaluated outside its training bounds can produce confident nonsense. Always constrain the optimizer to the sampled design space. Second, validation leakage: if the test set was used during hyperparameter tuning, the reported error is optimistic. Keep a strictly held-out set, or use nested cross-validation.

The taxonomy above is not a ranking. A quadratic response surface with 15 samples can outperform a neural network with 15 samples on a smooth low-dimensional problem, because the network has far more parameters than data. Matching model complexity to sample size and response smoothness is the central judgment in surrogate modeling, and it is the skill this course develops through the remaining chapters.

← Previous 📖 Contents Next →