2. Signal Preprocessing for Correlation: Filtering and Resampling Test Data

Why Preprocessing Decides Whether the Comparison Is Fair

In CAE–test correlation, the two signals rarely arrive on the same footing. A typical experimental acquisition runs at a high sampling rate (for example 51.2 kHz) and carries broadband noise, sensor resonance, and electrical interference. The CAE solver, by contrast, often writes output at a much coarser interval (for example 1 kHz or 2 kHz) because the time step is dictated by stability and cost, not by the measurement chain. If you overlay these two signals directly, you are comparing a noisy, oversampled measurement against a smooth, undersampled simulation. Any correlation metric computed on that pair mixes genuine model error with artifacts introduced by the mismatch itself.

Preprocessing has one job: put both signals on a common, defensible basis before any metric is computed. That means two operations applied in the correct order — low-pass filtering to remove content the CAE output cannot represent, and resampling to bring the test signal onto the CAE time base. Get the order wrong, or pick the cutoff carelessly, and the correlation score becomes meaningless even when the model is good.

The Governing Constraint: Nyquist and the CAE Time Base

Everything follows from the CAE output interval. If the CAE writes results every ΔtCAE seconds, its effective sampling frequency is fs,CAE = 1/ΔtCAE, and by the Nyquist criterion it can only represent content up to fNyq = fs,CAE/2. Content above that limit in the test signal has no counterpart in the simulation. If you resample the test signal without first removing that content, it aliases — high-frequency energy folds down into the low-frequency band and corrupts exactly the region you are trying to compare.

So the filter cutoff must sit at or below the CAE Nyquist frequency, and the filter must run before the decimation step. A practical rule is to set the cutoff slightly below Nyquist (for example 0.8–0.9 × fNyq) to leave room for the anti-alias filter's transition band, and to use a filter order high enough that the stopband attenuation is adequate (a 6th- to 8th-order Butterworth or an elliptic design is common).

Worked example (illustrative parameters). Suppose the test signal is sampled at fs,test = 51.2 kHz and the CAE output interval is ΔtCAE = 0.5 ms, giving fs,CAE = 2 kHz and fNyq = 1 kHz. Choose a low-pass cutoff of 800 Hz (0.8 × Nyquist). Apply an 8th-order Butterworth low-pass at 800 Hz to the test signal, then decimate from 51.2 kHz to 2 kHz — a decimation factor of 25.6, which in practice means resampling to a uniform 2 kHz grid rather than integer decimation. The result is a test signal band-limited to the same range the CAE output can represent, on the same time base.

Filter Choice and the Phase Question

Filter type matters as much as cutoff. A causal IIR filter (the default when you call a one-pass Butterworth) introduces frequency-dependent phase delay. That delay shifts the test signal in time relative to the CAE output and will show up in the correlation as a phase error that has nothing to do with the model. For correlation work, use a zero-phase filter — forward-backward filtering (for example filtfilt in SciPy or filtfilt in MATLAB) — which cancels the phase distortion at the cost of doubling the effective order and requiring care at the signal edges.

Edge handling deserves attention. Forward-backward filtering and resampling both need padding or reflection at the boundaries; without it, the first and last few percent of the record can show transients that inflate or deflate the correlation. Either pad the signal (reflect or constant extension) before filtering, or trim the affected edges from the comparison window and document the trimmed span.

Resampling: Interpolation, Not Decimation by Picking Samples

Once the test signal is band-limited, resample it onto the CAE time vector. Do not simply pick every Nth sample — that is decimation without interpolation and it introduces timing jitter whenever the two time bases are not integer-related. Use a proper resampler (polyphase FIR resampling, or band-limited sinc interpolation) that evaluates the test signal at each CAE timestamp. The CAE time vector is the reference grid; the test signal is the one that moves.

Two practical checks confirm the resampling is sound. First, verify the resampled test signal still passes through the original sample values at coincident times (interpolation should not shift the curve). Second, confirm the resampled length matches the CAE record length and that the time axes now align sample-for-sample.

Preprocessing Chain: Test Signal → Filter → Resample → Common Time Base Raw Test Signal fs,test = 51.2 kHz broadband noise + sensor resonance content up to 25.6 kHz Zero-Phase Low-Pass fc = 800 Hz 8th-order Butterworth forward-backward (filtfilt) no phase delay Resample polyphase FIR 51.2 kHz → 2 kHz evaluate at CAE timestamps band-limited, no aliasing Common Time Base fs = 2 kHz Δt = 0.5 ms ready for CORA CAE Output Δt = 0.5 ms → fs = 2 kHz Nyquist = 1 kHz reference grid Governing constraint: fc ≤ fs,CAE / 2 Filter BEFORE resampling — otherwise high-frequency energy aliases into the comparison band and corrupts the correlation score Failure modes to check before computing any metric Resample before filter Aliased high-frequency energy folds into low band → false magnitude/phase error Causal IIR filter Frequency-dependent phase delay shifts test in time → spurious phase sub-score Decimate by picking samples Timing jitter when time bases are not integer-related → shape distortion

Implementing the Chain

The following Python snippet implements the full chain with the illustrative parameters above. It is self-contained: it synthesizes a test signal and a CAE reference, filters, resamples, and reports the alignment check. Replace the synthetic signals with your loaded data; the processing calls stay the same.

import numpy as np
from scipy.signal import butter, filtfilt, resample_poly

# --- Illustrative parameters ---
fs_test = 51200.0          # Hz, experimental sampling rate
dt_cae  = 0.5e-3           # s, CAE output interval
fs_cae  = 1.0 / dt_cae     # 2000 Hz
fc      = 800.0            # Hz, low-pass cutoff (0.8 * Nyquist)
order   = 8

# --- Synthetic signals (replace with real data) ---
t_test = np.arange(0, 2.0, 1.0/fs_test)
rng = np.random.default_rng(0)
test = (np.sin(2*np.pi*50*t_test)
        + 0.3*np.sin(2*np.pi*300*t_test)
        + 0.05*rng.standard_normal(t_test.size))   # noise

t_cae = np.arange(0, 2.0, dt_cae)
cae = np.sin(2*np.pi*50*t_cae) + 0.3*np.sin(2*np.pi*300*t_cae)

# --- 1. Zero-phase low-pass (before resampling) ---
b, a = butter(order, fc/(fs_test/2), btype='low')
test_f = filtfilt(b, a, test)

# --- 2. Resample onto CAE time base ---
# 51200 -> 2000 Hz: ratio 128/5
test_rs = resample_poly(test_f, up=128, down=5)

# --- 3. Align lengths and verify ---
n = min(test_rs.size, cae.size)
test_rs, cae = test_rs[:n], cae[:n]
print("resampled length:", test_rs.size, "cae length:", cae.size)
print("max |test_rs - cae| (interior):",
      np.max(np.abs(test_rs[100:-100] - cae[100:-100])))

With the synthetic signals above, the interior maximum absolute difference between the resampled test signal and the CAE reference should be small (on the order of the injected noise amplitude, roughly 0.05), confirming that filtering and resampling preserved the shared 50 Hz and 300 Hz content. If you see a large residual, check the filter cutoff, the resampling ratio, and the edge regions in that order.

Verification and Failure Symptoms

Check Expected result Symptom if wrong
Filter cutoff vs CAE Nyquist fc ≤ fs,CAE / 2 (e.g. 800 Hz ≤ 1000 Hz) Aliased energy in low band; inflated magnitude error
Filter phase Zero-phase (filtfilt) Constant time shift; phase sub-score degrades
Resampling method Polyphase / sinc interpolation Timing jitter; shape distortion at non-integer ratios
Record length after resample Matches CAE length sample-for-sample Truncation or padding bias in the metric
Edge transients Padded or trimmed First/last few percent dominate the score
Order of operations is non-negotiable: filter first, resample second. The filter must run at the original high sampling rate so it can actually attenuate the content above the CAE Nyquist frequency. Resampling first destroys that content by folding it down, and no subsequent filter can recover it.
Do not over-filter. A cutoff far below the CAE Nyquist frequency (for example 200 Hz when Nyquist is 1 kHz) removes genuine dynamic content that the CAE output does represent, and the correlation will then under-report agreement in the 200–1000 Hz band. Set the cutoff to match the CAE band, not to make the curves look smooth.

Preprocessing Checklist

  • Record fs,test, ΔtCAE, and fNyq = 1/(2ΔtCAE) before touching the data.
  • Choose fc at 0.8–0.9 × fNyq; document the value and filter order.
  • Apply a zero-phase low-pass filter at fs,test; pad or trim edges.
  • Resample onto the CAE time vector with polyphase interpolation; never pick every Nth sample.
  • Confirm resampled length equals CAE length and time axes align sample-for-sample.
  • Verify the interior residual is consistent with expected noise, not with a systematic shift.
  • Store the preprocessing parameters alongside the correlation result so the comparison is reproducible.

Once both signals share a common, band-limited time base, the correlation metrics in the next stage measure model agreement rather than acquisition mismatch. The preprocessing record — cutoff, order, filter type, resampling ratio, and edge handling — becomes part of the correlation report, because a score is only as trustworthy as the basis on which it was computed.

← Previous 📖 Contents Next →