3. Sequence-to-Sequence LSTM for Crash Response Curve Generation

Why a Sequence Model for the Whole Pulse

A crash pulse is not a single number; it is a deceleration time history a(t) sampled at a fixed rate (for example 10 kHz) over a window of roughly 100–150 ms. The baseline feedforward network from the previous case compresses this into a fixed output vector, which forces the model to memorize a fixed length and gives it no explicit notion of temporal order. An encoder–decoder LSTM instead treats the curve as a sequence: the encoder consumes the scalar crash parameters, and the decoder emits the pulse one time step at a time, conditioning each sample on everything it has already produced. This is the same architecture family used in neural machine translation, applied here to a physical signal.

The practical payoff is twofold. First, the decoder is a genuine autoregressive generator, so the model learns the conditional distribution p(a_t | a_<t, x) rather than an independent per-sample mapping. Second, the same trained network can be run in two modes: teacher forcing during training (fast, stable, parallelizable) and free-running autoregressive decoding at inference (slow, but produces a self-consistent curve). Getting both modes right, and understanding why they differ, is the core engineering skill of this case.

Data Contract and Normalization

Assume the cleaned dataset from Case 1 is available as a NumPy array X_params of shape (N, 3) holding impact velocity (km/h), impact angle (deg), and a barrier-type one-hot collapsed to an integer code, plus Y_curves of shape (N, T) with T = 1200 samples per pulse. Two normalizations matter and must be applied consistently:

• Input scaling: standardize the three scalar parameters with training-set mean and standard deviation. Keep the scaler object; inference must reuse it.
• Target scaling: divide each curve by a single global peak deceleration g_max (in g) computed over the training set, so targets live in roughly [-1, 1]. Do not normalize per curve — that would destroy the amplitude information the model must predict.

The decoder needs a start token. Prepend a zero column to every target curve so the decoder input at step 0 is a learned <SOS> embedding, and the target at step 0 is the true a_0. The sequence length therefore becomes T+1 for inputs and T for targets.

Model Definition

The encoder is a single-layer LSTM that reads the parameter vector repeated or projected once; a compact and effective choice is to project the three scalars through a linear layer to the hidden size and feed it as a single encoder step. The decoder is a second LSTM whose input at each step is the previous (scaled) acceleration value, and whose output is projected back to one scalar.

import torch
import torch.nn as nn

class CrashPulseSeq2Seq(nn.Module):
    def __init__(self, n_params=3, hidden=64, n_layers=1):
        super().__init__()
        self.hidden = hidden
        self.n_layers = n_layers
        self.param_proj = nn.Linear(n_params, hidden)
        self.encoder = nn.LSTM(hidden, hidden, n_layers, batch_first=True)
        self.decoder = nn.LSTM(1, hidden, n_layers, batch_first=True)
        self.out = nn.Linear(hidden, 1)

    def encode(self, params):
        # params: (B, n_params) -> single encoder step
        h = torch.tanh(self.param_proj(params)).unsqueeze(1)   # (B,1,H)
        _, (hn, cn) = self.encoder(h)
        return hn, cn

    def forward(self, params, dec_in, teacher_forcing=True, sos_val=0.0):
        # dec_in: (B, T) true scaled curves (used only when teacher_forcing)
        B, T = dec_in.shape
        hn, cn = self.encode(params)
        inp = torch.full((B, 1, 1), sos_val, device=params.device)
        outputs = []
        for t in range(T):
            out, (hn, cn) = self.decoder(inp, (hn, cn))
            y = self.out(out)                       # (B,1,1)
            outputs.append(y)
            if teacher_forcing:
                inp = dec_in[:, t:t+1].unsqueeze(-1)  # ground truth
            else:
                inp = y.detach()                      # own prediction
        return torch.cat(outputs, dim=1).squeeze(-1)  # (B, T)

Note the deliberate asymmetry: the encoder state (hn, cn) is the only channel through which the scalar parameters influence the generated curve. If the model ignores the parameters, the first thing to inspect is whether param_proj gradients are vanishing — a symptom of an encoder that has collapsed to a constant state.

Teacher Forcing vs. Autoregressive Decoding

During training, feeding the ground-truth previous sample keeps the decoder on the data manifold and lets gradients flow through every step without compounding error. At inference there is no ground truth, so the decoder must consume its own output. The two regimes are shown below: the same recurrent cell, but the input source switches from the true curve to the predicted sample.

Training Loop with Scheduled Teacher Forcing

Pure teacher forcing trains quickly but creates exposure bias: the model never sees its own mistakes, so at inference small errors feed back and the curve drifts. A standard remedy is to anneal the teacher-forcing ratio from 1.0 down to about 0.5 over training, so the decoder gradually learns to recover from its own predictions.

import numpy as np
from torch.utils.data import TensorDataset, DataLoader

# X_params: (N,3) standardized; Y_curves: (N,T) divided by g_max
params_t = torch.tensor(X_params, dtype=torch.float32)
curves_t = torch.tensor(Y_curves, dtype=torch.float32)
loader = DataLoader(TensorDataset(params_t, curves_t), batch_size=32, shuffle=True)

model = CrashPulseSeq2Seq(hidden=64)
opt = torch.optim.Adam(model.parameters(), lr=1e-3)
loss_fn = nn.MSELoss()

EPOCHS = 60
for epoch in range(EPOCHS):
    tf_ratio = max(0.5, 1.0 - epoch / (EPOCHS * 0.8))  # 1.0 -> 0.5
    model.train()
    running = 0.0
    for p, y in loader:
        opt.zero_grad()
        use_tf = torch.rand(1).item() < tf_ratio
        pred = model(p, y, teacher_forcing=use_tf)
        loss = loss_fn(pred, y)
        loss.backward()
        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        opt.step()
        running += loss.item() * p.size(0)
    print(f"epoch {epoch:02d}  tf={tf_ratio:.2f}  mse={running/len(params_t):.5f}")

Gradient clipping at norm 1.0 is not optional here: recurrent backpropagation through 1200 steps is prone to exploding gradients, and the symptom is a loss that jumps to nan within a few batches. If that happens, lower the learning rate to 3e-4 and confirm the target scaling is actually applied — unscaled curves in the tens of g will saturate the decoder.

Inference and Curve-Level Validation

At inference, run the decoder with teacher_forcing=False, then multiply the output by g_max to recover physical units. Evaluate with metrics that respect the curve, not just pointwise error:

These thresholds are illustrative targets for a 1200-sample, 10 kHz pulse; set them from your own validation split rather than treating them as universal. A model that passes pointwise RMSE but fails peak error is typically under-fitting the high-amplitude region — check whether the loss is dominated by the long low-amplitude tail, and consider weighting samples near the peak.

Reproducibility Checklist

• Fixed seed for torch and NumPy; record the seed alongside the checkpoint.
• Persist the parameter scaler and g_max with the model weights — inference without them produces unit-inconsistent curves.
• Log the teacher-forcing ratio per epoch so a regression in validation peak error can be traced to the annealing schedule.
• Validate on held-out impact conditions (e.g., unseen barrier type) to confirm the encoder generalizes beyond interpolation of velocity and angle.