4. Data profiling and exploratory analysis for crash simulation results

Crash simulation campaigns generate large volumes of structured and semi-structured output: nodal displacements, element stresses, section forces, acceleration time histories, intrusion measurements, and occupant injury criteria. Before any anomaly detection model can be trusted, the analyst must understand the statistical shape of this data. Data profiling and exploratory data analysis (EDA) provide that foundation. They answer three practical questions: what does a normal result look like, how much variation exists between runs, and which channels or entities behave inconsistently enough to warrant investigation.

Why Profiling Precedes Detection

Anomaly detection algorithms are sensitive to distributional assumptions. If a channel is heavily skewed, contains a bimodal response from two legitimate design variants, or has a variance that scales with the mean, a naive z-score threshold will flag normal runs as anomalies. Profiling exposes these properties early. In crash CAE, three data families dominate:

  • Scalar summary metrics — peak acceleration, maximum intrusion, HIC, section force at a defined time, mass, energy balance error.
  • Time-history signals — acceleration, velocity, displacement, force curves sampled at fixed intervals (commonly 0.1–1.0 ms).
  • Field and entity data — per-element stress, per-node displacement, per-part energy absorption.

Each family requires different profiling statistics. Scalars are profiled with moments and quantiles; time histories with functional summaries and correlation; field data with spatial aggregation and outlier maps.

Statistical Summaries for Scalar Metrics

For a scalar channel such as peak_accel_g across a fleet of runs, compute the following in a reproducible script rather than by hand:

import pandas as pd
import numpy as np

df = pd.read_parquet("runs_metrics.parquet")
ch = df["peak_accel_g"]

summary = {
    "n":        ch.count(),
    "mean":     ch.mean(),
    "std":      ch.std(ddof=1),
    "cv":       ch.std(ddof=1) / ch.mean(),
    "min":      ch.min(),
    "q25":      ch.quantile(0.25),
    "median":   ch.median(),
    "q75":      ch.quantile(0.75),
    "max":      ch.max(),
    "skew":     ch.skew(),
    "kurtosis": ch.kurtosis(),
    "iqr":      ch.quantile(0.75) - ch.quantile(0.25),
}
print(pd.Series(summary).round(3))

Interpretation matters more than computation. A coefficient of variation (CV) below roughly 0.05 on a controlled DOE suggests a stable channel. A CV above 0.20 usually indicates either genuine design spread or a data quality problem such as unit inconsistency or a mis-mapped channel. Skewness above 1.5 warns that a symmetric threshold will misfire. Kurtosis above 3 signals heavy tails and potential outliers that deserve manual review before automated flagging.

Always compute robust statistics (median, IQR, MAD) alongside classical ones. In crash data, a single failed run with a numerical instability can inflate the mean and standard deviation enough to hide real anomalies.

Profiling Time-History Signals

Time histories cannot be summarized by a single number without losing the shape that carries physical meaning. A practical profiling workflow reduces each signal to a feature vector while preserving temporal structure:

  1. Resample all signals to a common time base (e.g., 0.5 ms) using linear interpolation, so runs of different solver output intervals are comparable.
  2. Extract functional features: peak value and time of peak, pulse duration above a threshold, rise time, area under the curve, zero-crossing count, and energy (integral of squared signal).
  3. Compute pairwise correlation between channels to detect redundant or coupled responses.
  4. Build a correlation heatmap and a parallel-coordinates plot of the feature matrix.

For example, if B-pillar_intrusion and door_inner_velocity correlate at r = 0.94 across 200 runs, they carry nearly the same information. Keeping both in a multivariate anomaly detector adds noise without adding signal. Conversely, a channel that correlates weakly with everything else is either an independent physical response or a mislabeled channel — both worth investigating.

Field and Entity-Level Profiling

Per-element and per-node data are high-dimensional. Profiling them requires aggregation. Common reductions include:

ReductionFormula / MethodTypical Use
Max over timemaxt |σe(t)|Peak stress per element
Time of maxargmaxt |σe(t)|Detect delayed or early failure
Energy absorbed∫ σe ε̇e dtPer-part crash energy
Spatial percentileP95 over element setRobust hotspot metric

After reduction, each run yields a vector of entity-level metrics that can be profiled with the same scalar statistics described earlier. A useful diagnostic is the entity stability index: for each element, compute the CV of its peak stress across runs. Elements with CV > 0.5 are either in a highly nonlinear region or are subject to mesh sensitivity — both are legitimate anomaly candidates.

Visual Diagnostics That Drive Decisions

Profiling is only actionable when the results are visible. Four plots cover most crash CAE needs:

  • Box-and-whisker plots per channel, grouped by load case or variant, to compare distributions at a glance.
  • Histograms with KDE overlay to reveal bimodality that summary statistics hide.
  • Scatter matrices of key metric pairs to expose clusters, gaps, and nonlinear relationships.
  • Time-history overlay plots with a shaded ±2σ envelope to show which runs deviate from the fleet.

The SVG below illustrates a compact profiling dashboard for a frontal impact campaign. It shows the flow from raw solver output through scalar and functional feature extraction to the diagnostic views that inform threshold selection and anomaly candidate ranking.

Crash Simulation Data Profiling Pipeline Raw Solver Output d3plot / nodout / elout accel, intrusion, stress 200 runs × 45 channels Feature Extraction Scalar: peak, HIC, CV Functional: rise time, AUC Entity: P95 stress, energy resample → 0.5 ms grid Statistical Profiling mean, std, CV, IQR skew, kurtosis, MAD correlation matrix robust + classical Visual Diagnostics box plots histograms scatter matrix Threshold Selection & Anomaly Candidate Ranking CV > 0.20 → unstable channel · skew > 1.5 → asymmetric threshold kurtosis > 3 → heavy-tail review · r > 0.90 → redundant channel Example: peak_accel_g (n=200) mean = 42.3 g std = 3.1 g CV = 0.073 → stable skew = 0.42 → symmetric kurtosis = 2.8 → near-normal → z-score threshold valid Example: door_intrusion_mm mean = 118 mm std = 27 mm CV = 0.229 → unstable skew = 1.82 → right-skewed kurtosis = 4.1 → heavy tail → use IQR / MAD threshold Channel Correlation B-pillar_intrusion ↔ door_inner_vel: r = 0.94 seat_track_force ↔ floor_accel: r = 0.31 → drop redundant channel → keep independent channel

From Profiling to Actionable Thresholds

The output of profiling is not a report — it is a set of decisions. For each channel, the analyst records:

  • Distribution class: normal, skewed, bimodal, or heavy-tailed.
  • Recommended threshold method: z-score for normal channels, IQR multiplier (typically 1.5× or 3×) for skewed channels, MAD-based for heavy tails.
  • Redundancy flag: channels with pairwise |r| > 0.90 are candidates for removal from multivariate models.
  • Stability rating: CV-based classification (stable < 0.10, moderate 0.10–0.20, unstable > 0.20).
Do not apply a single global threshold across all channels. A threshold tuned for a stable acceleration channel will either miss anomalies in a high-variance intrusion channel or flood the review queue with false positives. Profiling exists precisely to prevent this.

Reproducibility and Data Hygiene

Profiling results are only useful if they can be regenerated when the run set changes. Store the profiling script alongside the data, pin the input run list by hash, and log the software versions used for feature extraction. A minimal manifest might look like:

{
  "run_set": "frontal_odb_2024Q3",
  "n_runs": 200,
  "run_list_sha256": "a3f1...9c2e",
  "resample_dt_ms": 0.5,
  "channels": ["peak_accel_g", "door_intrusion_mm", "bpillar_force_kN"],
  "profiling_version": "1.4.2",
  "generated_utc": "2024-09-12T08:30:00Z"
}

With this manifest, any reviewer can reproduce the exact statistics and plots that justified a threshold. When a new batch of runs arrives, re-run the same script and diff the summary tables. A shift in mean or CV between batches is itself an anomaly signal — often the earliest indication of a modeling change, unit error, or solver version drift.

In practice, profiling and EDA are not a one-time gate. They are a recurring checkpoint that keeps the anomaly detection system calibrated to the evolving simulation campaign. The statistics and visualizations described here form the evidence base for every threshold, every channel selection, and every alert that follows.

← Previous 📖 Contents Next →