1. Crash Pulse Data Acquisition and Cleaning for Neural Network Training

Why raw crash pulses cannot be fed to a network directly

A crash acceleration time-history from a vehicle barrier or sled test is a sampled signal, not a tidy feature vector. Before any neural network can learn a mapping from impact parameters to a deceleration curve, the raw channels must be brought to a common, trustworthy state. Three defects dominate real acquisition logs: sensor drift (a slow baseline wander that biases the integrated velocity), dropouts (missing samples from lost packets, buffer overruns, or a saturated DAQ channel), and outliers (single-sample spikes from EMI, connector intermittency, or a loose accelerometer mount). Each defect corrupts the curve differently, and each needs a different correction. The deliverable of this case is a single, uniformly sampled, physically consistent dataset: one fixed sample rate, one fixed time window, zero-mean pre-impact baseline, and no residual spikes.

Acquisition model and the cleaning pipeline

Assume each test is stored as a CSV with columns time_s, accel_g, plus a metadata row carrying test_id, impact_velocity_kph, impact_angle_deg, and barrier_type. The raw rate varies between tests (e.g., 10 kHz and 20 kHz), and the trigger point is not aligned across files. The cleaning pipeline below is what the SVG illustrates: resample to a common grid, remove drift, repair dropouts, reject outliers, then validate against physical invariants.

Step 1 — Resample and align to a common grid

Neural networks require fixed-length inputs, so every test must land on the same time axis. Choose a target rate of 10 kHz and a window of 0–150 ms (1500 samples), which covers the primary deceleration pulse for most passenger-car barrier tests. If the raw rate is higher, apply an anti-alias low-pass filter before decimation; skipping this step folds high-frequency content into the pulse and creates false peaks. Align t = 0 to the trigger sample (the first sample where the pre-impact baseline is exceeded by a defined threshold), then crop or zero-pad to exactly 1500 samples.

Step 2 — Remove sensor drift

Drift shows up as a non-zero mean in the pre-impact region. Estimate the baseline as the mean of all samples with t < 0 and subtract it. For longer records, a first-order high-pass at 0.5 Hz removes slow wander without touching the pulse. The physical check is decisive: integrate the corrected acceleration and compare the resulting velocity change to the recorded impact velocity. A drift error of 0.05 g sustained over 100 ms already contributes about 0.05 m/s of spurious Δv, which is enough to bias the network's learned pulse amplitude.

Step 3 — Repair dropouts and reject outliers

Short gaps (≤ 5 ms) are repaired by cubic interpolation across the missing indices; longer gaps indicate a lost acquisition segment and the test should be flagged rather than silently patched. For outliers, use a robust detector: compute the median absolute deviation (MAD) of the signal and replace any sample whose deviation from the local median exceeds 6·MAD with that local median. A fixed threshold in g is fragile because pulse amplitudes vary by an order of magnitude across tests; MAD scales with the signal. Log every edit with its sample index so the cleaning is auditable.

Reference implementation

import numpy as np
import pandas as pd

TARGET_FS = 10_000
N_SAMPLES = 1500
GAP_MAX_MS = 5.0
MAD_K = 6.0

def resample_and_align(t, a, fs=TARGET_FS, n=N_SAMPLES):
    # t in seconds, a in g; align t=0 to first sample above 0.5 g
    trig = np.argmax(a > 0.5)
    t0 = t[trig]
    grid = np.arange(n) / fs
    a_aligned = np.interp(grid + t0, t, a, left=a[0], right=a[-1])
    return grid, a_aligned

def remove_drift(a, pre_mask):
    return a - np.mean(a[pre_mask])

def repair_and_reject(a, fs=TARGET_FS):
    a = a.copy()
    # dropouts: detect exact-zero runs longer than one sample
    zero_runs = np.flatnonzero(np.diff(np.r_[0, (a == 0).astype(int), 0]))
    for s, e in zip(zero_runs[::2], zero_runs[1::2]):
        gap_ms = (e - s) / fs * 1000
        if gap_ms <= GAP_MAX_MS and s > 0 and e < len(a):
            a[s:e] = np.linspace(a[s-1], a[e], e - s + 2)[1:-1]
        elif gap_ms > GAP_MAX_MS:
            return None, f"unrepaired_gap_{gap_ms:.1f}ms"
    # outliers via MAD
    med = np.median(a)
    mad = np.median(np.abs(a - med)) + 1e-9
    bad = np.abs(a - med) > MAD_K * mad
    if bad.any():
        idx = np.flatnonzero(bad)
        for i in idx:
            lo, hi = max(0, i-3), min(len(a), i+4)
            a[i] = np.median(a[lo:hi])
    return a, "ok"

def clean_test(df, meta):
    t = df["time_s"].to_numpy()
    a = df["accel_g"].to_numpy()
    grid, a = resample_and_align(t, a)
    pre = grid < 0
    a = remove_drift(a, pre)
    a, status = repair_and_reject(a)
    if a is None:
        return None, status
    dv = np.trapz(a, grid) * 9.81  # m/s
    v_ref = meta["impact_velocity_kph"] / 3.6
    if abs(dv - v_ref) / v_ref > 0.05:
        return None, f"dv_mismatch_{dv:.2f}_vs_{v_ref:.2f}"
    return a.astype(np.float32), "ok"

What to check before the data reaches the network

Run the four validation gates on every test and record the outcome in a cleaning report. A test that fails any gate is excluded from training and listed with its reason, so the dataset size is honest and the exclusion is reproducible. The expected observable is a set of curves that all start at zero, share the same length, and integrate to the recorded impact velocity within tolerance. Typical failure symptoms and their causes: a persistent non-zero baseline after drift removal usually means the pre-impact window was too short or the trigger was misdetected; a Δv mismatch points to drift or a clipped pulse; residual spikes after MAD rejection suggest the local median window is too wide and is absorbing the spike itself.

Cleaning report fields

With this pipeline in place, the output is a tensor of shape (N, 1500) in g, a matching target tensor, and a metadata table of impact parameters — exactly the input contract the baseline network in the next case expects.