📚 Course ContentsChapter 3 Design of experiments for CAE

3. Design of experiments for CAE

A surrogate model can only be as good as the data it is trained on. In a CAE-driven design workflow, every training sample corresponds to a full simulation run — often minutes to hours of solver time — so the sampling plan is where computational budget is either spent wisely or wasted. Design of Experiments (DoE) is the discipline that decides where in the design space to place those expensive evaluations. This chapter covers the three dominant sampling families used in CAE surrogate workflows (Latin hypercube, Sobol/low-discrepancy sequences, and factorial designs), the geometric criteria that quantify how well a plan covers the space, and how to reason about sample size before committing solver hours.

Why the Sampling Plan Matters More Than in Physical Experimentation

In a CAE context the "experiment" is deterministic: rerunning the same input file yields the same output. There is no measurement noise to average out, so the classical statistical rationale for replication does not apply. What does matter is coverage of the input domain and the ability to resolve the response's structure — gradients, curvature, and interaction effects. A poor plan (for example, all samples clustered near the nominal design) produces a surrogate that interpolates well locally but diverges badly in the region the optimizer will actually explore. The sampling plan therefore directly determines the surrogate's extrapolation risk.

Formally, let the design space be a box defined by lower and upper bounds on d input variables, x ∈ [l, u] ⊂ ℝ^d. A DoE is a set of n points X = {x₁, …, x_n}. The choice of n and the placement of those points are the two decisions this chapter addresses.

Latin Hypercube Sampling

Latin hypercube sampling (LHS) stratifies each of the d dimensions into n equally probable intervals and places exactly one sample in each interval per dimension, with the dimension-wise assignments randomly permuted. This guarantees that the one-dimensional projections of the design are perfectly uniform — no dimension is left with a gap or a cluster. For a 5-variable crashworthiness study with n = 60 runs, LHS ensures that each variable's 60 values are spread one-per-stratum across its range, which is a strong property when the response is dominated by main effects.

The weakness of plain LHS is that it says nothing about joint coverage: two variables can be stratified individually yet still produce correlated or clustered pairs. The standard remedy is to generate many candidate LHS matrices and keep the one that optimizes a space-filling criterion (see below) — a practice called optimized LHS or maximin LHS. In practice, generating 1,000–10,000 candidate designs and selecting the best by maximin distance is cheap relative to a single CFD run and reliably improves coverage.

Sobol and Other Low-Discrepancy Sequences

Sobol sequences are deterministic low-discrepancy sequences: successive points are added such that the discrepancy — a measure of how far the empirical point distribution deviates from uniform — stays as small as possible at every prefix length. The practical consequence is that a Sobol design of n points is nested: the first 32 points are themselves a well-distributed design, and adding points 33–64 refines coverage rather than re-randomizing it. This is extremely useful in CAE when you want to run an initial batch, inspect surrogate error, and then extend the campaign without discarding earlier runs.

Sobol designs are also naturally space-filling in low-to-moderate dimensions and are the default choice in many surrogate toolkits (for example, the sampling modules in Dakota, UQLab, and SciPy's qmc module). Their weakness appears in high dimensions: the discrepancy bound degrades, and the sequence can exhibit visible structure (correlations between certain coordinate pairs) unless scrambled. Scrambled Sobol (Owen or Matoušek scrambling) randomizes the sequence while preserving its low-discrepancy property, and is the recommended variant when you need multiple independent replicates or want to avoid aliasing with a periodic response.

Factorial and Response-Surface Designs

Full factorial designs place samples at every combination of discrete levels — for d variables at k levels each, that is k^d runs. A two-level full factorial in 6 variables is 64 runs; in 10 variables it is 1,024, which is already impractical for expensive CAE. Factorials remain valuable in two situations: (1) screening, where a fractional factorial (e.g., a 2^7-3 resolution IV design) identifies the few variables that matter before a larger space-filling campaign; and (2) response-surface fitting, where a central composite or Box–Behnken design provides the structured points needed to estimate quadratic terms with a small number of runs. The trade-off is explicit: factorial designs give excellent estimates of main effects and interactions at prescribed levels, but they sample only the corners and center of the domain, leaving large interior regions unobserved — a poor fit for surrogates that must interpolate a nonlinear CAE response across the whole box.

Space-Filling Criteria

To compare candidate plans objectively, three geometric criteria are standard:

Maximin distance — maximize the minimum pairwise Euclidean distance between samples. This directly penalizes clustering and is the most widely used criterion for LHS optimization. A maximin-optimized plan guarantees no two runs are redundant.
Minimax distance — minimize the maximum distance from any point in the domain to its nearest sample. This bounds the worst-case interpolation error and is the natural criterion when the surrogate will be evaluated everywhere (e.g., for global optimization).
Centered L₂ discrepancy — a scalar measure of uniformity; lower is better. It is the criterion implicitly optimized by Sobol sequences and is convenient for comparing designs of different sizes.

In practice, maximin and minimax are computed on the normalized design space (each variable scaled to [0, 1]) so that variables with different physical units — say, a 2 mm fillet radius and a 400 MPa yield stress — contribute equally. Failing to normalize is a common and silent error: the criterion then measures distance almost entirely in the variable with the largest numerical range.

When comparing an LHS plan against a Sobol plan of the same size, compute the maximin distance and the centered L₂ discrepancy for both on the normalized box. A maximin-optimized LHS typically wins on minimum spacing; a Sobol sequence typically wins on discrepancy. Which matters more depends on whether your surrogate is an interpolator (favor maximin) or a global fitter such as a polynomial or Gaussian process with a smooth prior (favor low discrepancy).

Choosing the Sample Size

Sample size is a budget decision, and the right answer depends on the surrogate type and the dimensionality. Three practical rules of thumb, each with a clear rationale:

Linear/quadratic response surfaces: n ≈ (d+1)(d+2)/2 for a full quadratic, plus 20–30% margin for lack-of-fit checking. For d = 5 this is 21 terms, so plan 26–28 runs.
Gaussian process / Kriging: a common starting point is n ≈ 10d to 15d, refined by cross-validation. For d = 8, that is 80–120 runs — often the dominant cost of the project.
Neural-network surrogates: data-hungry; n in the hundreds to thousands, and the marginal value of each additional run should be checked against a learning curve.

Rather than guess, run a convergence study: build surrogates on nested subsets of the design (e.g., 20, 40, 60, 80 runs from a Sobol sequence), and plot validation error — typically leave-one-out RMSE or a held-out test set — against n. The curve usually flattens; the knee is your sample size. This is the single most reliable way to avoid both under-sampling (a surrogate that fails in the optimizer) and over-sampling (solver hours spent for a 2% error reduction).

Putting It Together: A Practical Sampling Workflow

A defensible CAE sampling plan for a surrogate project typically proceeds as follows. First, screen with a fractional factorial or a small Sobol batch to eliminate variables with negligible effect, reducing d before the expensive campaign. Second, normalize the remaining variables to [0, 1] and record the physical bounds in a manifest. Third, generate a candidate design — optimized LHS if the surrogate is an interpolator, scrambled Sobol if you anticipate extending the campaign or need low discrepancy. Fourth, evaluate the plan against maximin and discrepancy criteria and, if possible, against a cheap analytical proxy of the response to confirm no region is starved. Fifth, run the solver and archive inputs and outputs with a hash of the input file so runs are reproducible. Sixth, validate with leave-one-out or a held-out set, and if error is unacceptable, extend the design (Sobol nesting makes this trivial) rather than re-randomizing.

A frequent failure mode: the design space bounds are set from CAD tolerances rather than from the range the optimizer will actually explore. The surrogate is then trained on a narrow box and evaluated outside it, producing silent extrapolation error. Always set DoE bounds to encompass the optimizer's feasible region, and flag any surrogate evaluation that falls outside the training box.

The sampling plan is the contract between computational budget and surrogate fidelity. Choosing the strategy by surrogate type, quantifying coverage with normalized geometric criteria, and sizing the campaign with a convergence study turns an expensive guessing game into a controlled engineering decision.