3. Coverage metrics taxonomy

Coverage in physical test planning is not a single number. It is a family of measures, each answering a different question about what a validation campaign has actually exercised. A brake-by-wire program can report 92% structural coverage of its controller software, 78% functional coverage of its actuation modes, 40% scenario coverage of its operational design domain, and 100% requirement coverage of its safety goals — and all four statements can be simultaneously true and simultaneously misleading. The planner's job is to know which measure is being quoted, what it can and cannot prove, and how the four families interact when a physical test campaign is used to substantiate virtual validation results.

The taxonomy below is organized by what is being counted. Structural measures count artifacts inside a model or implementation. Functional measures count behaviors the system can exhibit. Scenario-based measures count operating conditions the system may encounter. Requirement-based measures count obligations the system must satisfy. The four are complementary: structural coverage tells you the implementation has been touched, functional coverage tells you the behavior space has been sampled, scenario coverage tells you the environment space has been sampled, and requirement coverage tells you the contractual obligations have been discharged.

Structural coverage: what the implementation actually executed

Structural coverage is measured against a model or code artifact and answers a narrow but essential question: which parts of the implementation were reached by the test inputs? In a physical test context, the "implementation" is usually the controller software or the plant model that the test bench exercises. Common variants include statement coverage (each executable line executed at least once), branch or decision coverage (each outcome of each decision taken), modified condition/decision coverage (MC/DC, each condition shown to independently affect the decision outcome), state coverage (each state of a state machine entered), and transition coverage (each legal transition taken). For calibration-heavy systems, table-cell coverage — the fraction of cells in a lookup table that were addressed by the test inputs — is a practical structural measure that maps directly onto physical test design.

Structural coverage is necessary but never sufficient. A brake controller can achieve 100% statement coverage while never being exercised in the specific sequence of wheel-slip, yaw-rate, and driver-brake-input conditions that triggers a latent timing defect. The measure proves reachability, not correctness. In physical test planning, structural coverage is most useful as a gap detector: uncovered branches or table cells identify inputs the campaign has not yet applied, which then become candidate test cases for the next bench session.

Functional coverage: what behaviors were sampled

Functional coverage is defined by the test architect, not by the implementation. It enumerates the behaviors the system is expected to exhibit — operating modes, fault reactions, input equivalence classes, cross-feature combinations, and transition sequences — and records which of them were observed during a test run. A functional coverage model for an electric power-steering system might define bins for assist-motor current ranges, hand-wheel torque classes, vehicle-speed bands, and the cross-product of assist mode with fault state. Each bin is a named, countable item; coverage is the fraction of bins hit at least once.

The strength of functional coverage is that it can be specified before the implementation exists, which makes it the natural bridge between requirements and test design. The weakness is that it is only as good as the model: a coverage model that omits a critical cross-feature pair will report high coverage while missing the very interaction that matters. Cross-feature coverage — the pairwise or n-wise combination of independently varying parameters — is the standard technique for catching these omissions, and it is where most functional coverage models earn or lose their value.

Scenario-based coverage: what operating conditions were encountered

Scenario-based coverage measures the fraction of the operational design domain (ODD) that the test campaign has actually exercised. The ODD is decomposed into scenario dimensions — road class, surface friction, weather, lighting, traffic density, speed envelope, load state, and maneuver type — and each dimension is partitioned into representative classes. Coverage is then reported per dimension and, more usefully, as a coverage of the cross-product of dimensions that the system is expected to handle. For a physical test campaign, scenario coverage is the measure that most directly reflects real-world exposure, and it is the one most often under-reported because scenario enumeration is labor-intensive.

A practical technique is to define a scenario matrix with explicit cells and a target sample count per cell, then track which cells have been driven on the proving ground or executed on the bench. Cells that are physically dangerous or expensive to reproduce — low-friction split-mu braking, for example — are exactly the cells where virtual validation earns its keep, provided the virtual results are anchored to physical reference runs in adjacent cells.

Requirement-based coverage: what obligations were discharged

Requirement-based coverage counts the fraction of requirements, safety goals, performance targets, and regulatory clauses for which passing evidence exists. It is the measure closest to the contractual and certification view: a requirement is covered when a test has been executed, the result has been compared against the acceptance criterion, and the evidence has been recorded with traceability to the requirement identifier. Unlike the other three families, requirement coverage is binary per item — a requirement is either discharged or it is not — and the aggregate is a simple ratio of discharged to total.

The trap in requirement-based coverage is that it says nothing about the strength of the evidence behind each discharge. A requirement covered by a single nominal test at one operating point is formally covered but evidentially thin. Mature programs therefore pair requirement coverage with a confidence rating per item — for example, the number of independent test cases, the number of scenario cells exercised, and whether the result was reproduced on a second bench or vehicle. This is where the four families combine: requirement coverage provides the obligation list, functional and scenario coverage provide the sampling depth, and structural coverage provides the implementation reachability check.

Combining the families in a physical test plan

The four families are not alternatives to be chosen between; they are layers that must be reported together. A defensible coverage report for a physical test campaign states, for each requirement, which functional bins and scenario cells were exercised, what structural coverage was achieved on the relevant controller paths, and what the acceptance result was. The table below shows a compact reporting format that a test planner can populate directly from bench and proving-ground records.

In practice, the planner uses the taxonomy in two directions. Top-down, requirements and the ODD define the functional and scenario coverage models, which in turn define the test cases; structural coverage is then checked after execution to confirm the cases reached the intended implementation paths. Bottom-up, structural gaps found after a bench session are promoted into new functional bins or scenario cells, closing the loop. The taxonomy is therefore not a reporting artifact but a design tool: it tells the planner where the next test case should come from, and it tells the reviewer whether the evidence presented is deep enough to support the claim being made.