4. Data quality rule specification

A data quality monitoring system is only as good as the rules it enforces. Without a disciplined specification process, teams end up with hundreds of ad-hoc checks that overlap, contradict each other, or silently pass bad data. This chapter covers how to translate abstract quality dimensions into executable rules for three distinct industrial data domains: process tags, laboratory data, and asset records.

4.1 ISO 8000 and DAMA-DMBOK as Complementary Frameworks

ISO 8000 provides a principles-based framework for data quality. Its core assertion is that data quality is defined as fitness for purpose — there is no universal "good" dataset, only data suitable for a specific use. ISO 8000-61 defines a process reference model for data quality management, while ISO 8000-110 specifies requirements for exchanging master data. The standard does not prescribe specific thresholds; it requires that organizations define their own quality requirements and demonstrate conformance.

DAMA-DMBOK complements ISO 8000 by providing a practical taxonomy of quality dimensions. The most commonly applied dimensions in industrial settings are:

• Completeness — Are all required values present? (e.g., no nulls in a mandatory tag)
• Accuracy — Does the value correctly represent the real-world entity?
• Consistency — Do values agree across systems or over time?
• Timeliness — Is the data available when needed and current enough for its purpose?
• Validity — Does the value conform to its defined domain, format, or range?
• Uniqueness — Is each real-world entity represented exactly once?

The practical workflow is: use DAMA-DMBOK dimensions to classify what kind of quality issue you are addressing, then use ISO 8000's fitness-for-purpose principle to set the threshold for that dimension in your specific context. A temperature tag used for closed-loop control requires tighter timeliness than the same tag used for monthly reporting.

4.2 From Dimension to Executable Rule

A quality rule is a declarative statement that can be evaluated against a dataset to produce a pass/fail result. Every rule should specify five elements: the target (table, column, or tag), the dimension it addresses, the condition to evaluate, the threshold for acceptable violation rate, and the severity when breached.

Consider a simple example. A pressure transmitter tag PT-101.PV in a historian database should report values between 0 and 250 bar. The rule specification would be:

Rule ID:      RQ-PT-001
Target:       historian.tag_PT-101.PV
Dimension:    Validity
Condition:    value < 0 OR value > 250
Threshold:    < 0.1% of readings in 24h window
Severity:     High
Action:       Flag for engineering review; suppress from KPI calculation

This rule is unambiguous, testable, and directly traceable to a physical constraint (the transmitter's calibrated range). Contrast this with a vague rule like "pressure should be reasonable" — which cannot be automated and will be interpreted differently by different engineers.

4.3 Domain-Specific Constraints

The three data domains in industrial operations have fundamentally different quality characteristics. Applying the same rule template to all three is a common and costly mistake.

Process Tags

Process tags are high-frequency time-series values from sensors, controllers, and PLCs. Their quality rules must account for:

• Physical range constraints — Each tag has a calibrated range (e.g., 4–20 mA scaled to 0–100%). Values outside this range indicate sensor fault or wiring issues.
• Rate-of-change limits — A temperature that jumps 50°C in one second is physically implausible for most processes and likely indicates a communication error or spike.
• Frozen value detection — A tag reporting the exact same value for an extended period may indicate a stuck sensor or failed communication.
• Timestamp monotonicity — Historian timestamps must be strictly increasing; duplicates or out-of-order entries corrupt trend analysis.

A practical rate-of-change rule for a reactor temperature tag might specify: if |value(t) − value(t−1)| > 5°C within a 1-second interval, flag as suspect. The threshold of 5°C/s is derived from the thermal mass of the reactor and the maximum credible heating rate — not from a generic default.

Laboratory Data

Lab data arrives as discrete results with associated metadata: sample ID, method, analyst, instrument, and detection limits. Quality rules here focus on:

• Detection limit handling — Results below the limit of detection (LOD) must be flagged as "<LOD" rather than reported as zero or a numeric value.
• Method-specific ranges — A pH result of 14.2 is invalid because the method's valid range is 0–14.
• Holding time compliance — Samples analyzed beyond their maximum holding time (e.g., 48 hours for certain water samples) must be flagged as expired.
• Duplicate sample consistency — When duplicate samples are analyzed, the relative percent difference (RPD) should fall within method-defined limits.

For example, a rule for chloride analysis by ion chromatography might specify: RPD between duplicates must be ≤ 20% when the result is ≥ 10 mg/L; if RPD exceeds this, flag the sample batch for re-analysis.

Asset Records

Asset records are master data describing equipment: pumps, valves, vessels, instruments. Quality rules here address:

• Uniqueness of asset identifiers — Each functional location or equipment tag must appear exactly once in the master register.
• Referential integrity — Every asset must reference a valid parent location, cost center, and manufacturer.
• Mandatory attribute completeness — Critical fields (serial number, installation date, design pressure) must be populated.
• Cross-system consistency — The asset's location in the CMMS must match its location in the P&ID and the historian tag mapping.

A typical completeness rule: for all rotating equipment assets, the fields manufacturer, model, and serial_number must be non-null; violation rate must be 0%. Unlike process tags, asset records have zero tolerance for missing critical attributes because they drive maintenance planning and regulatory compliance.

4.4 Rule Specification Template

The following template consolidates the five required elements and adds traceability fields. It is designed to be stored in a metadata repository and version-controlled alongside the rules themselves.

Field	Description	Example
Rule ID	Unique identifier	RQ-LAB-007
Target	Table, column, or tag	lab_results.pH_value
Dimension	DAMA-DMBOK dimension	Validity
Condition	Boolean expression	value < 0 OR value > 14
Threshold	Acceptable violation rate	0% (zero tolerance)
Severity	Impact level	Critical
Action	Response on breach	Reject result; notify lab supervisor
Owner	Responsible role	Lab QA Manager
Review Date	Next scheduled review	2025-06-30

4.5 Common Pitfalls in Rule Specification

Over-specification. Writing rules for every conceivable edge case creates alert fatigue. Start with rules that address known failure modes and expand based on observed incidents.

Ignoring domain context. A rate-of-change limit appropriate for a temperature sensor will generate false positives on a flow meter during pump start/stop transitions. Rules must be parameterized per tag type or per asset class.

Static thresholds. A fixed range for a process tag may be valid during steady-state operation but invalid during startup, shutdown, or grade transitions. Consider state-aware rules that reference the current operating mode.

No ownership. A rule without an assigned owner will not be maintained. When the process changes, the rule becomes stale and either fails silently or generates noise.

The diagram above maps four DAMA-DMBOK dimensions to their corresponding industrial domains and shows how each becomes a concrete, executable rule with a condition, threshold, and severity. Notice that the same dimension (e.g., Validity) produces different rules depending on whether the target is a process tag, a lab result, or an asset record — the domain context determines the condition and threshold.