2. Time series data characteristics in equipment operations
Industrial time series data is not the clean, evenly spaced grid that textbooks often assume. On a real plant floor, a vibration channel may report at 25.6 kHz, a temperature transmitter at 1 Hz, and a PLC tag only when its value changes. Before any analytics can be trusted, you must understand five characteristics that shape every downstream decision: sampling rates, irregular timestamps, missing values, outliers, and overall data quality. This chapter explains what each one looks like in practice and how to assess it.
Sampling Rates and Multi-Rate Data
Sampling rate determines what a signal can physically reveal. The Nyquist–Shannon theorem states that to reconstruct a frequency component, you must sample at more than twice that frequency. A bearing defect at 3.2 kHz requires a sampling rate above 6.4 kHz; a 1 Hz temperature tag can never capture it. In equipment operations, three rate classes typically coexist:
- High-rate (kHz–MHz): vibration, acoustic emission, current signature analysis. Usually captured in bursts or triggered windows.
- Medium-rate (1–100 Hz): pressure, flow, speed, torque. Continuous streaming.
- Low-rate / event-driven: alarms, state changes, maintenance events, often logged only on transition.
When you join these streams, you must decide on a common time base. Resampling high-rate data down loses information; upsampling low-rate data fabricates it. A practical rule is to keep each channel at its native rate and align only at the analysis window (for example, a 10-minute feature window), rather than forcing everything into one uniform grid.
Irregular Timestamps
Even nominally fixed-rate sensors drift. Network latency, buffering, and clock skew produce timestamps that are close to but not exactly on the expected interval. Event-driven tags are irregular by design. Two problems follow:
- Clock alignment: different devices may use unsynchronized clocks. Without NTP or PTP synchronization, a 200 ms offset between a vibration sensor and a speed reference can corrupt order-tracking calculations.
- Duplicate and out-of-order records: retransmissions or gateway buffering can deliver the same timestamp twice or in the wrong sequence.
Handling this starts with a timestamp audit: compute the inter-arrival interval distribution per tag. A tag expected at 1 Hz that shows a median interval of 1.0 s but a 99th percentile of 4.7 s has a latency problem worth investigating before modeling.
Missing Values
Missing data in equipment operations is rarely random. It usually falls into recognizable patterns:
| Pattern | Typical Cause | Analytical Risk |
|---|---|---|
| Single-point gaps | Packet loss, brief comms dropout | Low; interpolation is safe |
| Block gaps | Sensor failure, maintenance, shutdown | High; interpolation fabricates data |
| Stuck-at values | Frozen sensor, saturated ADC | High; looks valid but is not |
| Systematic absence | Tag only logged when running | Biases statistics toward operating states |
The correct response depends on the pattern. Short gaps (a few samples) can be linearly interpolated or forward-filled with a documented flag. Long block gaps should be left as NaN and excluded from feature computation, or the window should be dropped entirely. Stuck-at values require a change-detection check: if a tag reports the identical value for longer than a physically plausible duration, mark it invalid.
Outliers
An outlier is a value that deviates from expected behavior. In equipment data, outliers are either fault indicators (a genuine spike from a developing defect) or measurement artifacts (a sensor glitch, EMI, or communication error). Distinguishing them is the core challenge.
Common detection methods and their trade-offs:
- Statistical (z-score, IQR): fast, but assumes a distribution and flags legitimate transient events.
- Rate-of-change limits: effective for physical signals where a temperature cannot jump 50 °C in one second. Requires domain-specific thresholds.
- Physical range checks: a pressure tag rated 0–10 bar reporting 47 bar is an artifact, not a fault.
- Contextual / model-based: compare against a predicted value from operating conditions. More accurate but needs a baseline model.
A practical layered approach: apply physical range checks first (cheap, catches gross errors), then rate-of-change limits, then statistical or model-based detection on the remainder. Flag rather than delete — a spike that repeats at a characteristic frequency may be the earliest sign of a fault.
Data Quality Assessment
Quality assessment turns these observations into measurable indicators. A useful per-tag quality report includes:
- Completeness: fraction of expected samples actually present over the period.
- Timeliness: distribution of inter-arrival intervals versus the nominal rate.
- Validity: fraction of samples within physical range and not stuck.
- Consistency: agreement between redundant or related tags (for example, two speed sensors on the same shaft).
These four dimensions can be combined into a single quality score per tag per day, which makes it easy to spot degrading sensors before they fail. The workflow below shows how raw multi-rate data flows through validation into a quality-scored, analysis-ready dataset.
Putting It Into Practice
Consider a pump monitoring setup with a vibration sensor at 25.6 kHz, a discharge pressure tag at 10 Hz, and a run-state tag logged on change. A daily quality check might reveal: pressure completeness 99.2% (acceptable), vibration completeness 87% (block gaps during scheduled stops — expected), and run-state showing 14 duplicate timestamps (gateway retransmission — needs deduplication). The vibration data is not "bad"; it simply has a known, explainable gap pattern. The run-state duplicates, however, would corrupt any state-transition count and must be fixed at the gateway.
The key takeaway is that data quality is not a preprocessing afterthought — it is an engineering discipline. Sampling rates define what is knowable, timestamps define when it happened, missing values and outliers define what can be trusted, and quality scoring makes all of this visible and actionable. Every subsequent chapter in this course assumes these foundations are in place.