3. Loading and parsing crash test data formats

Crash test data rarely arrives in a single, convenient container. A typical frontal impact test produces accelerometer channels sampled at 10–20 kHz, recorded by a data acquisition system that writes to MDF4 or TDMS, plus a filtered ISO 6487-compliant channel set exported as CSV for the safety report. Before any pulse analysis can begin, you must load each format correctly, reconcile channel names, and confirm that metadata such as sample rate, units, and filter class survived the round trip. This section covers the four formats you will encounter most often and the Python libraries that handle them.

ISO 6487: The Filtering and Channel Convention Standard

ISO 6487 is not a file format. It is the standard that defines how crash test acceleration channels must be filtered and how their passbands are designated. The standard specifies four channel classes — CFC 60, CFC 180, CFC 600, and CFC 1000 — where CFC stands for Channel Frequency Class. Each class defines a low-pass filter with a specific −3 dB cutoff and tolerance envelope. CFC 180, for example, has a cutoff near 300 Hz and is the default for vehicle body acceleration; CFC 60 is used for occupant chest acceleration; CFC 1000 is reserved for high-frequency events such as toe pan or steering column impacts.

When you load an ISO 6487 channel, the critical metadata is the CFC class, not just the sample rate. A 10 kHz sampled signal filtered at CFC 60 carries far less information than the same signal at CFC 1000. If the CFC class is missing from the file header, you cannot assume a default — you must either recover it from the test log or re-filter from the raw unfiltered channel. The standard also defines the sign convention: positive acceleration is in the direction of vehicle travel for longitudinal channels, and the coordinate system must be documented per channel.

MDF4: The ASAM Measurement Data Format

MDF4 (Measurement Data Format version 4) is the dominant binary format in automotive testing, standardized by ASAM. It stores channel groups, time masters, and metadata in a hierarchical block structure. The asammdf library is the standard Python reader:

from asammdf import MDF

mdf = MDF("crash_run_042.mf4")
print(mdf.channels_db)          # dict of channel name -> list of (group, index)
print(mdf.header.start_time)    # test start timestamp

# Extract a single channel by name
sig = mdf.get("VehSpeed_X")
print(sig.unit, sig.samples[:5], sig.timestamps[:5])

# Convert to a pandas DataFrame for all channels in one group
df = mdf.to_dataframe()
mdf.close()

MDF4 files often contain multiple channel groups with different time bases. A common pitfall is calling to_dataframe() across groups that have different sample rates — the library will resample or raise, depending on version. For crash pulse work, extract channels individually with get() and align them on a common time vector yourself. Also check sig.unit: MDF4 stores units as strings, and a channel labeled g versus m/s2 changes your integration by a factor of 9.81.

TDMS: NI's Hierarchical Time-Domain Format

TDMS (Technical Data Management Streaming) is National Instruments' format, common when the crash lab uses NI hardware. The nptdms library reads it:

from nptdms import TdmsFile

tdms = TdmsFile.read("dummy_impact.tdms")
for group in tdms.groups():
    print(group.name, [ch.name for ch in group.channels()])

ch = tdms["Accel"]["Chest_Y"]
print(ch.properties["unit_string"], ch.properties["wf_increment"])
data = ch[:]   # numpy array

TDMS organizes data into groups and channels, and metadata lives in properties. The sample interval is stored as wf_increment (seconds between samples), not as a sample rate. A frequent error is reading wf_increment as a frequency — it is a period. Convert with fs = 1.0 / wf_increment. TDMS also supports raw and scaled data; if NI_Number_Of_Scales is greater than 1, the channel has a scaling polynomial applied, and you should read the scaled values unless you specifically need raw counts.

CSV: The Interchange Format

CSV is the least standardized and the most error-prone. Crash lab CSVs vary wildly: some use a single header row, others embed metadata in comment lines prefixed with # or ;, and some use semicolons as delimiters because the locale uses commas as decimal separators. A robust loader inspects the first few lines before committing:

import pandas as pd

def load_crash_csv(path):
    with open(path) as f:
        head = [next(f) for _ in range(5)]
    sep = ";" if head[0].count(";") > head[0].count(",") else ","
    skip = sum(1 for line in head if line.lstrip().startswith(("#", ";")))
    df = pd.read_csv(path, sep=sep, skiprows=skip, comment="#")
    df.columns = [c.strip().lower().replace(" ", "_") for c in df.columns]
    return df

After loading, verify that the time column is monotonic and that its spacing matches the expected sample rate. A CSV exported from a 20 kHz system should show a time step of 0.00005 s; if you see 0.001 s, the export was decimated and the CFC class may no longer be valid.

Channel Naming Conventions and Metadata Reconciliation

The hardest part of multi-format loading is not parsing — it is matching channels across sources. The same physical sensor may appear as VehSpeed_X in MDF4, Vehicle_Speed_X in TDMS, and veh_spd_x in CSV. Build a normalization function and a mapping table rather than hard-coding names:

import re

def normalize(name):
    name = name.lower()
    name = re.sub(r"[^a-z0-9]+", "_", name)
    return name.strip("_")

# Map normalized aliases to canonical channel IDs
ALIASES = {
    "veh_speed_x": "vehicle_speed_longitudinal",
    "vehicle_speed_x": "vehicle_speed_longitudinal",
    "veh_spd_x": "vehicle_speed_longitudinal",
}

Always carry forward four metadata fields per channel: canonical name, unit, sample rate, and CFC class (or None if unfiltered). Store them in a small dataclass or dictionary so downstream pulse analysis never has to guess. The SVG below shows how the four formats feed into a single normalized channel record.

Practical Validation Checklist

After loading, run a short validation pass before analysis. The following checks catch the majority of real-world ingestion errors:

When a check fails, do not proceed with a default. Log the channel, its source file, and the specific mismatch. In a multi-file test, one bad channel can invalidate an entire occupant injury calculation, so the cost of a strict validation gate is far lower than the cost of a silent error in the final report.