4. Establishing a traceable data lineage

In crash test score recalculation, the final number on the report is only as defensible as the chain of evidence behind it. When an auditor asks why the chest resultant acceleration for a 56 km/h frontal impact changed from 42.1 g to 43.6 g between the original certification and a later recalculation, the answer cannot be "we re-ran the analysis." It must be a reconstructable path: which raw sensor file was used, which channel mapping was applied, which filter and offset correction were performed, which intermediate dataset was produced, and which version of each artifact was in play. That path is the data lineage. This chapter explains how to build it so that any score can be traced back to the physical measurement that produced it.

What Lineage Means in a Crash Test Context

Data lineage is the documented, verifiable relationship between every derived artifact and its inputs, transformations, and parameters. In a crash test laboratory, the lineage spans three distinct layers:

• Raw sensor files — the immutable outputs of the data acquisition system (DAS), typically per-channel time-series files in formats such as ISO-MME (ISO 6487-compliant), DIAdem DAT, or vendor-specific binary. These include the accelerometers, load cells, seat belt force transducers, and high-speed video timestamps.
• Preprocessing steps — the deterministic operations applied to raw data: channel identification and unit normalization, zero-offset correction, anti-alias and CFC filtering (e.g., CFC 60 for accelerometers per ISO 6487), time-zero alignment to the trigger event, and coordinate system transformation.
• Versioned intermediate datasets — the outputs of preprocessing that feed scoring: filtered acceleration traces, computed injury criteria inputs (HIC, chest deflection, femur force), and the derived score tables. Each must carry a version identifier and a pointer to its parent artifacts.

The critical principle is immutability of raw data. Raw files are never edited in place; every correction is a new derived artifact with its own identifier. This mirrors the "append-only" discipline used in regulated data systems and is what allows an auditor to replay the entire chain.

Designing the Lineage Record

A practical lineage record is a structured manifest — one per test or per recalculation run — that captures, for each artifact, its identity, its parents, the transformation applied, and the parameters. The following example shows a manifest fragment for a single accelerometer channel. The values are illustrative and represent a plausible teaching example, not measured results.

{
  "run_id": "RC-2024-0117-FRONTAL-56",
  "test_id": "CT-2024-0117",
  "artifacts": [
    {
      "artifact_id": "raw/CT-2024-0117/CH03_head_accel.dat",
      "type": "raw_sensor",
      "source": "DAS-2, channel 03, PCB 356A16, serial 88213",
      "immutable": true,
      "sha256": "9f2c...a71e"
    },
    {
      "artifact_id": "interim/CT-2024-0117/CH03_head_accel_cfc60_v1.csv",
      "type": "filtered_trace",
      "parents": ["raw/CT-2024-0117/CH03_head_accel.dat"],
      "transform": "zero_offset + CFC60_lowpass + time_zero_align",
      "params": {"cfc": 60, "offset_g": -0.42, "t_zero_ms": 12.5},
      "sha256": "3b81...c04d"
    },
    {
      "artifact_id": "scores/CT-2024-0117/HIC15_v2.json",
      "type": "derived_score",
      "parents": ["interim/CT-2024-0117/CH03_head_accel_cfc60_v1.csv"],
      "transform": "HIC15_integration",
      "params": {"window_ms": 15, "resultant": "a_x,a_y,a_z"},
      "sha256": "e5aa...19f2"
    }
  ]
}

Three fields do the heavy lifting. The parents array establishes the directed edge in the lineage graph. The transform and params fields make the operation reproducible — an auditor can re-apply CFC 60 with the same offset and time-zero and expect a bit-identical or tolerance-identical result. The sha256 hash provides tamper evidence: if the raw file's hash no longer matches, the lineage is broken and the recalculation must be flagged.

Preprocessing as a Reproducible, Parameterized Step

The most common audit failure is not a missing file but an undocumented parameter. A filter cutoff that was "tuned by feel," a time-zero that was "eyeballed," or a coordinate rotation applied in the wrong order will silently shift scores. The remedy is to treat preprocessing as a parameterized function whose inputs and outputs are both recorded. The diagram below shows the lineage graph for one channel, from raw DAS output through preprocessing to the derived score, with the parameters that must be captured at each edge.

Notice that the graph is not a simple linear pipeline: the audit check branches off the intermediate dataset and re-verifies both the hash of the raw parent and the reproducibility of the transformation. This is the operational meaning of traceability — the ability to independently re-derive the result, not merely to read a log.

Versioning Intermediate Datasets Without Ambiguity

Intermediate datasets are where versioning discipline most often collapses. A file named head_accel_filtered_final.csv is a liability; a file named CH03_head_accel_cfc60_v1.csv with a manifest entry is an asset. Adopt a naming and versioning convention with three properties:

1. Content-addressed or version-tagged names — include the channel, the transformation, and a monotonic version (v1, v2). Never overwrite; a re-filter with a different cutoff produces v2, and the manifest records why (e.g., "CFC 60 corrected from 100 per ISO 6487 review").
2. Parent pointers in the manifest — every intermediate dataset lists its raw parent and the exact parameter set. This makes the lineage graph traversable in both directions: from score back to sensor, and from sensor forward to every score that depends on it.
3. Change justification — a short free-text field explaining why a new version exists. Auditors care less about the number than about the reason; "recomputed after correcting time-zero from 12.0 to 12.5 ms" is auditable, "updated" is not.

Reconstructing a Recalculation: A Worked Walkthrough

Suppose an audit requests the basis for a revised HIC15 of 612 (example value). The reconstruction proceeds as follows. First, locate the score artifact HIC15_v2.json and read its parents field, which points to CH03_head_accel_cfc60_v1.csv. Second, verify the intermediate dataset's sha256 against the stored hash; a match confirms the file has not been altered since the manifest was written. Third, read the intermediate's parent pointer to the raw file and re-hash the raw file; a mismatch here is a hard stop — the lineage is broken and the score cannot be certified. Fourth, re-apply the recorded transformation (offset −0.42 g, CFC 60, time-zero 12.5 ms) to the raw file and compare the regenerated trace to the stored intermediate within a defined tolerance (for example, a maximum absolute deviation of 0.05 g over the impact window). Fifth, re-integrate HIC15 over the 15 ms window and compare to 612. If all checks pass, the score is traceable; if any fails, the manifest is flagged and the recalculation is held pending resolution.

Common Failure Modes and Their Symptoms

The through-line is that lineage is not a report produced after the fact; it is a property of how the data is handled during recalculation. If raw files are immutable, preprocessing is parameterized and recorded, and intermediate datasets are versioned with parent pointers and hashes, then the audit trail is a byproduct of correct practice rather than a separate documentation burden. The next time a score is questioned, the answer is a graph, not a guess.