1. Build a Baseline Surrogate Model for a Cantilever Beam Tip Deflection

Why a Baseline Before Diagnostics

Error diagnostics only become meaningful when there is a reference point to diagnose against. Before investigating why a surrogate model fails on certain regions of the design space, we need a defensible answer to a simpler question: how well does a straightforward model perform on this dataset? This case builds that reference. The object is a cantilever beam under a tip point load, the surrogate predicts tip deflection, and the deliverable is a trained regression model plus a holdout-metric baseline that later cases will compare against.

The beam is a clean choice for a first baseline because the underlying physics is closed-form and monotonic in the dominant parameters, so any large validation error is attributable to modeling or data-handling mistakes rather than to genuinely chaotic behavior. That property makes the baseline trustworthy as an anchor.

Defining the Simulation Dataset

We treat the finite element solver as a black box that maps four input parameters to one scalar output. The parameters and their sampling ranges are chosen to keep the beam in the small-deflection, linear-elastic regime so the closed-form check remains valid:

The load P is held fixed at 500 N for this baseline so the output stays one-dimensional; varying P is deferred to later cases. The target is tip deflection δ. For a cantilever with a rectangular section, the reference relation is δ = P·L³ / (3·E·I) with I = b·h³/12. We use this closed form to generate the "simulated" dataset, which is legitimate here because the goal is to establish a validation pipeline, not to validate the solver itself. Every number below is a reproducible synthetic example, not a measured result.

Generating Data and Splitting It

We draw 400 samples with Latin hypercube sampling over the four ranges, compute δ for each, and hold out 25% as a test set. The split is done once with a fixed seed so the baseline is reproducible. Note that the target spans several orders of magnitude because δ scales with L³ and inversely with h³; we therefore model log(δ) rather than δ, which stabilizes the regression and makes relative error the natural metric.

import numpy as np
from scipy.stats import qmc
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline
from sklearn.metrics import mean_absolute_error, r2_score

rng = np.random.default_rng(42)
N = 400
P = 500.0  # N, fixed load

# Latin hypercube over the four parameter ranges
sampler = qmc.LatinHypercube(d=4, seed=42)
u = sampler.random(N)
L = 0.5 + u[:, 0] * (2.0 - 0.5)
b = 0.02 + u[:, 1] * (0.10 - 0.02)
h = 0.02 + u[:, 2] * (0.10 - 0.02)
E = (60.0 + u[:, 3] * (210.0 - 60.0)) * 1e9  # Pa

I = b * h**3 / 12.0
delta = P * L**3 / (3.0 * E * I)  # m

X = np.column_stack([L, b, h, E / 1e9])  # E in GPa for conditioning
y = np.log(delta)

# Single reproducible holdout split
idx = rng.permutation(N)
n_test = int(0.25 * N)
test_idx, train_idx = idx[:n_test], idx[n_test:]
X_train, X_test = X[train_idx], X[test_idx]
y_train, y_test = y[train_idx], y[test_idx]

Two details matter for a baseline that later diagnostics can trust. First, the split is performed on the raw parameter rows, not on derived features, so no information leaks across the boundary. Second, we keep the test set untouched until the very end; tuning on it would inflate the baseline and defeat its purpose as a reference.

Fitting the Surrogate and Reading the Metrics

A quadratic polynomial in the four inputs is a deliberately modest model. It captures the L³ and h⁻³ curvature through interaction and squared terms without pretending to be a tuned black box. If a simple model already reaches low error, later cases can attribute residual error to specific regions rather than to model capacity.

model = make_pipeline(PolynomialFeatures(degree=2, include_bias=False),
                      LinearRegression())
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
mae_log = mean_absolute_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

# Convert log-space error to relative deflection error
rel_err = np.abs(np.exp(y_pred) - np.exp(y_test)) / np.exp(y_test)
print(f"MAE(log delta) = {mae_log:.4f}")
print(f"R2 (log space) = {r2:.4f}")
print(f"Median relative error = {np.median(rel_err)*100:.2f}%")
print(f"95th pct relative error = {np.percentile(rel_err,95)*100:.2f}%")

Report the metrics as a fixed record. On this synthetic dataset a quadratic fit typically lands at R² above 0.99 in log space with a median relative error well under one percent, because the closed form is smooth and the sampling is dense. Treat those figures as expected behavior for this construction, not as a claim about any real solver run. The important artifact is the record: the exact split seed, the model specification, and the four numbers above. Later cases will re-run the same split and compare their error maps against this row.

Where the Baseline Is Weak

A single global metric hides structure. The quadratic model will fit the mid-range well but can drift at the extremes of L and h, where the cubic terms dominate and the polynomial approximation is least accurate. The figure below shows the intended validation workflow and the two regions where the baseline is expected to lose accuracy, so the next diagnostic case has a concrete place to start looking.

Reproducibility Checklist

• Fixed seed: the LHS sampler and the permutation split both use seed 42; changing either invalidates comparison with later cases.
• Frozen test set: the 100 test rows are never used for feature selection, scaling, or hyperparameter choice.
• Log target: modeling log(δ) keeps relative error meaningful across the three-order-of-magnitude output range.
• Recorded metrics: MAE(log δ), R² in log space, median and 95th-percentile relative error, plus the dataset hash and split seed.
• Known limitation: the quadratic surrogate is expected to drift at large L and small h; this is documented, not hidden.

With this baseline recorded, the next diagnostic step has a fixed reference: any later model or data-handling change can be judged against these four numbers on the identical split, and the weak-region callouts give the first place to inspect when the error map is drawn.