2. Audit Restraint Response Dataset for Multi-Output Coverage

Why a Dataset Audit Comes Before Any Model Code

In a multi-output restraint-response model, the most expensive mistakes are not architectural — they are statistical. A shared trunk trained with a naive mean-squared-error loss over a partially labeled target tensor will silently learn from zeros that were never measured, and a random split will leak near-duplicate operating conditions across train and validation. Before writing a single line of model code, you need a coverage report: a quantified profile of which restraint response channels are actually populated, how skewed their class distributions are, and how strongly the channels co-vary. That report directly determines two downstream decisions — the per-output loss mask and the split strategy.

This case builds that audit as a small, self-contained Python script. The input is a tabular dataset in which each row is one test/operating record and each column is either an input feature or one of the restraint response targets. The output is a structured report: per-output availability, per-output class balance (for categorical channels) or distribution shape (for continuous channels), and a cross-channel correlation matrix that exposes redundancy and dependency.

Defining the Audit Contract

Assume the restraint response schema from the modeling contract has five outputs. Three are categorical severity/state channels and two are continuous physical quantities:

• belt_load_class — categorical, {0, 1, 2} (none / partial / full restraint load)
• pretension_state — categorical, {0, 1, 2} (slack / nominal / over-tensioned)
• anchor_condition — categorical, {0, 1} (compliant / degraded)
• peak_force_kN — continuous, kilonewtons
• displacement_mm — continuous, millimetres

Missing labels are encoded as NaN in the target columns only; input features are always complete. This convention matters: the audit must distinguish "label absent" from "label present with value 0", which is exactly the distinction the loss mask will later encode.

Step 1 — Load and Separate Feature and Target Columns

The script reads a CSV, splits columns by an explicit target list (never by position, which breaks when columns are reordered), and reports the raw shape. Explicit target lists keep the audit aligned with the modeling contract.

import numpy as np
import pandas as pd

TARGETS = ["belt_load_class", "pretension_state", "anchor_condition",
           "peak_force_kN", "displacement_mm"]
CATEGORICAL = {"belt_load_class", "pretension_state", "anchor_condition"}

df = pd.read_csv("restraint_responses.csv")
feature_cols = [c for c in df.columns if c not in TARGETS]

print(f"rows={len(df)}  features={len(feature_cols)}  targets={len(TARGETS)}")
targets = df[TARGETS].copy()

Step 2 — Per-Output Label Availability

Availability is the fraction of rows with a non-null value per target. It is the single most important number in the report, because it sets the ceiling on how much signal each head can learn and tells you whether a head is viable at all.

availability = targets.notna().mean().rename("availability")
counts = targets.notna().sum().rename("n_labeled")
avail = pd.concat([counts, availability], axis=1).sort_values("availability")

# Joint availability: rows where ALL targets are present
complete_rows = targets.notna().all(axis=1).mean()
print(avail)
print(f"fully-labeled rows: {complete_rows:.3f}")

Two derived figures are worth computing explicitly. Complete-case rate (all five targets present) tells you how large a fully supervised batch you can ever assemble. Pairwise co-availability — the fraction of rows where two specific targets are both present — bounds how much a shared trunk can learn joint structure between those two heads. A head pair with 0.05 co-availability cannot support a meaningful learned correlation, no matter how good the architecture is.

Step 3 — Class Imbalance for Categorical Channels

For each categorical target, compute the normalized class distribution over labeled rows only. Report the majority-class share and the imbalance ratio (max count / min count). A channel where one class exceeds ~80% will dominate a plain cross-entropy loss and needs class weighting or resampling — a decision the audit should surface, not the training loop.

for t in CATEGORICAL:
    vc = targets[t].dropna().value_counts(normalize=True).sort_index()
    raw = targets[t].dropna().value_counts()
    ratio = raw.max() / raw.min() if len(raw) > 1 else np.inf
    print(f"{t}: {dict(vc.round(3))}  imbalance_ratio={ratio:.2f}")

For the continuous channels, imbalance is not the right lens; instead report count, mean, standard deviation, min/max, and the fraction of values at the physical floor (e.g. peak_force_kN == 0), which often indicates a censored or default-filled label rather than a true measurement.

Step 4 — Cross-Channel Correlation Structure

Correlation across restraint channels reveals redundancy and dependency. Compute a pairwise correlation matrix on the labeled subset for each pair, using Pearson for continuous–continuous pairs and a rank-based measure (Spearman) when either side is ordinal. Crucially, compute each pair on its own co-available rows — do not drop rows globally, or you will silently shrink the sample to the complete-case subset and misrepresent the structure.

def pairwise_corr(a, b, method="spearman"):
    mask = a.notna() & b.notna()
    if mask.sum() < 30:
        return np.nan, int(mask.sum())
    return a[mask].corr(b[mask], method=method), int(mask.sum())

corr = pd.DataFrame(index=TARGETS, columns=TARGETS, dtype=float)
n_pair = pd.DataFrame(index=TARGETS, columns=TARGETS, dtype=int)
for i, a in enumerate(TARGETS):
    for b in TARGETS[i:]:
        r, n = pairwise_corr(targets[a], targets[b])
        corr.loc[a, b] = corr.loc[b, a] = r
        n_pair.loc[a, b] = n_pair.loc[b, a] = n
print(corr.round(2))

Reading the Coverage Report

The SVG below summarizes how the audit outputs map onto the two downstream decisions. Availability and co-availability feed the loss mask; imbalance and correlation feed the split and weighting design.

Step 5 — Emitting the Report and Setting Thresholds

The audit should end with explicit, machine-readable thresholds rather than prose. The following block writes the four CSVs and a short summary, and flags any target that fails a viability check. These flags are the handoff to the loss-masking pipeline.

MIN_AVAIL = 0.05      # below this, a head cannot learn
MIN_COAVAIL = 0.10    # below this, joint structure is unlearnable
MAX_IMBALANCE = 4.0   # above this, apply class weighting

avail.to_csv("availability.csv")
corr.round(3).to_csv("correlation.csv")
n_pair.to_csv("co_availability.csv")

flags = []
for t in TARGETS:
    if availability[t] < MIN_AVAIL:
        flags.append(f"DROP_HEAD {t}: avail={availability[t]:.3f}")
for i, a in enumerate(TARGETS):
    for b in TARGETS[i+1:]:
        if n_pair.loc[a, b] / len(df) < MIN_COAVAIL:
            flags.append(f"WEAK_PAIR {a}~{b}: coavail={n_pair.loc[a,b]/len(df):.3f}")

with open("summary.md", "w") as f:
    f.write("# Coverage Report\n\n")
    f.write(avail.round(3).to_string() + "\n\n")
    f.write("## Flags\n" + "\n".join(flags) + "\n")
print("\n".join(flags) if flags else "no flags raised")

Verification and Failure Symptoms

You can verify the audit without any model. Three checks are decisive:

• Availability sums to labeled counts. availability[t] * len(df) must equal targets[t].notna().sum() exactly. A mismatch means a target column was silently coerced (e.g. strings parsed as objects).
• Correlation matrix is symmetric with unit diagonal. If a diagonal entry is not 1.0, the pairwise routine is dropping rows inconsistently between the (a,b) and (b,a) calls.
• Co-availability never exceeds either marginal availability. If coavail(a,b) > min(avail(a), avail(b)), the mask logic is inverted.

Typical failure symptoms and their causes: an availability of exactly 1.0 for a channel you know is sparse usually means missing values were filled with a sentinel (0 or -1) upstream — check the raw CSV. A correlation of NaN for a pair with plenty of rows indicates zero variance in one channel (a constant column), which will also break standardization later. A complete-case rate far below the product of marginals signals that missingness is correlated across channels — labels tend to be absent together — which is a strong argument for grouped rather than random splitting.

Deliverable: Coverage Report Summary

The audit's concrete output is a compact table that a reviewer can act on directly. A filled example (illustrative values, not measured results) shows the expected shape:

With this report in hand, the loss mask is no longer a guess: each head's weight follows from its availability, class weights follow from the imbalance ratios, and the split must stratify on the categorical combination while preserving co-availability. The next stage — building the masked data pipeline — consumes exactly these numbers.