1. Evidence hierarchy fundamentals

Evidence hierarchy fundamentals 学习卡片

In vehicle development, a subsystem can pass every bench test in isolation and still fail in the assembled vehicle. The reverse also happens: a component that looks marginal on a rig performs flawlessly on the road. These outcomes are not anomalies to be explained away — they are the direct consequence of the fact that evidence gathered at different levels of the product hierarchy carries different weight, different scope, and different risk. The discipline of organizing that evidence deliberately, rather than accumulating it ad hoc, is what this course calls the evidence hierarchy.

What an Evidence Hierarchy Actually Is

An evidence hierarchy is a structured ordering of validation evidence by the level of the product at which it was produced, together with explicit rules for how evidence at one level supports, constrains, or supersedes evidence at another. It is not a document format and not a test plan. It is a reasoning framework that answers a deceptively simple question: given what we have measured, how confident are we that the vehicle will behave correctly in the customer's hands?

Three terms need to be separated cleanly, because they are routinely conflated in practice:

  • Evidence — an observation with provenance: a measurement, a test result, a simulation output, a field return, or a structured inspection record. Evidence must be traceable to a configuration, a stimulus, and a date. An opinion is not evidence; a screenshot without a configuration identifier is weak evidence.
  • Level — the position in the product decomposition at which the evidence was generated: component, subsystem, system, or complete vehicle. Level determines which interactions are present and which are absent by construction.
  • Hierarchy — the ordering and the transfer rules between levels. The hierarchy is what lets a reviewer say "this subsystem result is credible, but it does not cover the thermal coupling we see at vehicle level," and have that statement be a formal property of the framework rather than a personal opinion.

The essential insight is that level is not a proxy for quality. A meticulously instrumented component test is better evidence than a sloppy vehicle test — for the claims it can support. The hierarchy does not rank evidence as good or bad; it ranks evidence by scope of validity. Higher levels cover more interactions but usually with less control and less repeatability. Lower levels offer precision and diagnostic power but deliberately exclude the very interactions that cause integration defects.

Why the Hierarchy Exists

The purpose of the hierarchy is to make the limits of each piece of evidence explicit before a release decision depends on it. Four functions follow from that:

  1. Scope accounting. Every test environment suppresses some real-world influence — rig stiffness, harness length, ambient temperature, electromagnetic background, driver behavior. The hierarchy records which influences were present at each level so that gaps are visible rather than assumed away.
  2. Defect localization. When a failure appears at vehicle level, the hierarchy provides the diagnostic ladder: which lower-level result should have caught it, and why didn't it? That question is often more valuable than the fix itself.
  3. Evidence reuse and substitution. A validated subsystem result can legitimately substitute for a vehicle-level test only when the transfer assumptions are stated and checked. The hierarchy defines what those assumptions are.
  4. Decision traceability. Release decisions become auditable: each claim about vehicle behavior is linked to specific evidence at specific levels, with known residual risk.
A useful sanity check when reviewing any validation package: for each top-level requirement, ask "at which level was this actually demonstrated, and what interactions were absent at that level?" If the answer is vague, the hierarchy is not doing its job.

The Levels and What Each One Can and Cannot See

A typical four-level decomposition for an automotive subsystem — for example, an electric power steering (EPS) system — illustrates the trade-off concretely. At component level, the steering motor and its inverter are tested on a dynamometer with a fixed DC bus supply. Torque ripple, current-loop bandwidth, and thermal derating are measurable with high precision, but vehicle-level bus voltage sag, CAN timing under load, and steering-feel interactions are entirely absent. At subsystem level, the EPS rack, motor, ECU, and torque sensor run on a hardware-in-the-loop (HIL) bench with a steering robot and a simulated vehicle model. Now closed-loop behavior, assist curve shaping, and fault reactions are observable — but the bench model's tire and suspension representation is an assumption, not a measurement. At system level, the EPS integrates with the chassis domain controller and the brake system, exposing bus arbitration, shared-sensor conflicts, and cross-domain fault propagation. At vehicle level, the complete car on a proving ground or in a climate chamber reveals the residual effects: real tire dynamics, temperature soak, road-induced vibration, and driver perception.

The following SVG maps this hierarchy as a scope-and-transfer structure, showing what each level adds and which interactions remain invisible at each stage.

Evidence hierarchy from component to vehicle level Four stacked levels — component, subsystem, system, vehicle — each adding interactions and reducing control, with transfer assumptions between levels and residual risk at the top. Component Subsystem System Vehicle Motor + inverter on dynamometer, fixed DC bus Sees: torque ripple, current loop, thermal derating Absent: bus sag, CAN load, steering feel EPS rack + ECU + sensor on HIL bench, steering robot Sees: closed-loop assist, fault reactions, curve shaping Absent: real tire/suspension dynamics EPS + chassis controller + brake system integrated Sees: bus arbitration, shared sensors, cross-domain faults Absent: full thermal soak, road vibration Complete vehicle, proving ground / climate chamber Sees: real tires, temperature soak, driver perception Absent: nothing — but control and repeatability lowest Interaction coverage ↑ Control & repeatability ↓ assumption: bench model represents tire/suspension assumption: bus timing scales to full network load assumption: rig boundary conditions match vehicle Residual risk Interactions never exercised at any level remain as unquantified release risk. Level band Transfer with stated assumption Uncovered interaction risk

How the Hierarchy Drives Real Validation Work

Consider a concrete, reproducible example. A requirement states that EPS assist torque shall remain within ±5% of the nominal curve across a DC bus voltage range of 9–16 V. At component level, this is straightforward: sweep the bus supply on the dynamometer from 9 V to 16 V in 0.5 V steps at a fixed motor speed of 1500 rpm and a commanded torque of 3 N·m, log assist torque, and compute deviation from the nominal map. Suppose the result shows a maximum deviation of 3.1% at 9.5 V — the requirement appears satisfied. The hierarchy immediately raises the next question: at what level was the 9.5 V condition actually representative? On the vehicle, 9.5 V occurs during cold-crank with the EPS drawing peak current, and the bus impedance includes harness resistance and battery internal resistance that the bench supply does not model. The component evidence is valid but its scope excludes the very condition that matters.

The correct response is not to discard the component result but to promote the question upward: repeat the sweep at subsystem level with a battery emulator whose output impedance is set from measured vehicle harness data, then confirm at vehicle level during a cold-crank event with a current probe on the EPS feed. Each level narrows the residual uncertainty, and the hierarchy records exactly which uncertainty was closed at which level. If the vehicle-level measurement shows 6.2% deviation, the component result was not wrong — it was insufficiently scoped, and the hierarchy is what makes that distinction visible and actionable.

A common failure mode is treating a passed lower-level test as a closed requirement. In hierarchy terms, a component pass closes the requirement only for the interaction set present at component level. The remaining interaction set must be explicitly assigned to a higher level or accepted as residual risk — silently dropping it is how integration defects escape to the customer.

The Role in Subsystem-to-Vehicle Validation

Within the broader validation flow, the hierarchy plays three roles. First, it is a planning instrument: it forces the team to decide, before testing begins, which interactions will be exercised at which level, and therefore which transfer assumptions must be validated rather than assumed. Second, it is a review instrument: a reviewer can trace any release claim to its supporting evidence and immediately see the level at which it was demonstrated and the interactions that were absent. Third, it is a risk instrument: the interactions that no level exercises — the residual risk box in the diagram — become an explicit, owned item rather than an invisible gap.

The practical consequence is a shift in how test coverage is discussed. Instead of counting tests or tracking pass rates, the team tracks which interactions have been observed at which level, and which transfer assumptions have been confirmed by measurement. That framing is what allows a subsystem-to-vehicle validation program to make defensible release decisions under schedule pressure, because it distinguishes between "we have not tested this" and "we have tested this and the result does not transfer" — two very different problems that a flat test list cannot tell apart.

The remaining chapters build on this foundation: defining level-specific evidence types, specifying transfer assumptions and how to verify them, constructing the traceability links between requirements and evidence, and applying the hierarchy to release decisions and defect escape analysis.

← Previous 📖 Contents Next →