2. Exploratory data analysis for time series

Before a single model is trained, the shape of a time series must be understood. Exploratory data analysis (EDA) for time series is not the same as EDA for tabular data: observations are ordered, adjacent points are correlated, and the series can be decomposed into components that behave very differently. This chapter covers four core EDA activities — visualization, decomposition, stationarity testing, and autocorrelation analysis — and shows how each one informs later modeling decisions.

Visualization: The First Diagnostic

A raw line plot is the fastest way to detect trend, seasonality, level shifts, and outliers. Plot the series against a real datetime index, not a row counter, so that irregular sampling becomes visible. For a monthly series, a seasonal subseries plot (one line per month across years) reveals whether the seasonal pattern is stable. For high-frequency data, overlay a rolling mean and rolling standard deviation to see whether the level and variance drift over time.

Consider a synthetic monthly series y[t] generated as a linear trend plus a 12-month seasonal cycle plus noise. A quick plot immediately shows an upward slope and repeating peaks. If the amplitude of those peaks grows with the level, a multiplicative decomposition is more appropriate than an additive one. Visualization is also where data quality issues surface: gaps, duplicated timestamps, and impossible values (negative sales, zero variance stretches) should be fixed before any statistical test.

Decomposition: Separating Signal from Structure

Classical decomposition splits a series into trend, seasonal, and residual components. The additive model is y[t] = T[t] + S[t] + R[t]; the multiplicative model is y[t] = T[t] × S[t] × R[t]. Use a moving average to estimate the trend, then subtract (or divide) it out to isolate the seasonal component, and average the seasonal indices across full cycles to get a stable estimate.

STL (Seasonal-Trend decomposition using Loess) is more flexible than classical decomposition because it allows the seasonal component to change over time and lets you control smoothness with two parameters: seasonal (the seasonal smoother length, must be odd) and trend (the trend smoother length). A robust variant down-weights outliers, which is useful when the residual component contains spikes.

from statsmodels.tsa.seasonal import STL
stl = STL(series, period=12, seasonal=13, trend=25, robust=True)
res = stl.fit()
res.plot()  # trend, seasonal, resid panels

After fitting, inspect the residual panel. If residuals still show a clear pattern, the decomposition has not captured all structure — either the period is wrong or the series needs a transformation. The strength of trend and seasonality can be quantified with variance ratios: max(0, 1 - Var(R)/Var(T+R)) for trend and max(0, 1 - Var(R)/Var(S+R)) for seasonality. Values near 1 indicate a strong component; values near 0 indicate the component is negligible.

Stationarity: What It Means and How to Test It

A series is stationary when its statistical properties — mean, variance, and autocovariance — do not change over time. Most classical forecasting models (ARIMA, VAR) assume weak stationarity, so this is a gate that determines whether differencing or detrending is required.

The Augmented Dickey-Fuller (ADF) test has the null hypothesis that the series contains a unit root (non-stationary). A small p-value (typically < 0.05) rejects the null and suggests stationarity. The Kwiatkowski–Phillips–Schmidt–Shin (KPSS) test has the opposite null: the series is stationary. Running both is good practice because they have different alternative hypotheses and can disagree in the presence of trend-stationarity.

from statsmodels.tsa.stattools import adfuller, kpss
adf_p = adfuller(series, autolag="AIC")[1]
kpss_p = kpss(series, regression="c", nlags="auto")[1]
print(f"ADF p={adf_p:.4f}  KPSS p={kpss_p:.4f}")

If ADF fails to reject and KPSS rejects, the series is likely non-stationary. Apply first-order differencing y'[t] = y[t] - y[t-1] and retest. If the variance grows with the level, apply a log or Box-Cox transform before differencing. Seasonal differencing y[t] - y[t-m] handles repeating seasonal unit roots. Over-differencing is a real risk: it inflates variance and introduces negative autocorrelation at lag 1, so stop differencing as soon as the tests pass.

Autocorrelation and Partial Autocorrelation

The autocorrelation function (ACF) measures the linear correlation between y[t] and y[t-k] across all lags k. The partial autocorrelation function (PACF) measures the correlation at lag k after removing the effect of intermediate lags. Together they are the primary tool for identifying AR and MA orders.

Reading the plots: a slow, gradual decay in the ACF indicates non-stationarity and calls for differencing. A sharp cutoff in the ACF after lag q with a gradual decay in the PACF suggests an MA(q) process. A sharp cutoff in the PACF after lag p with gradual ACF decay suggests an AR(p) process. Spikes at multiples of the seasonal period (e.g., lags 12, 24, 36 for monthly data) indicate seasonal structure that requires seasonal differencing or a seasonal AR/MA term.

Always check the confidence bands: for a white noise series, roughly 95% of autocorrelations should fall within ±1.96/√N. The Ljung-Box test formalizes this by testing whether the first m autocorrelations are jointly zero. A significant result means the series still contains autocorrelation that a model should capture.

Putting It Together: A Practical EDA Checklist

The four activities above are not independent. Visualization suggests whether decomposition should be additive or multiplicative. Decomposition reveals whether the seasonal component is stable or evolving. Stationarity tests determine the differencing order d and D. ACF and PACF on the differenced series suggest the AR and MA orders. The following table summarizes the decision points and the typical failure symptoms that send you back to an earlier step.

Once these four steps are complete, you have a concrete set of modeling inputs: the transform type, the differencing orders d and D, the seasonal period m, and candidate AR/MA orders. These feed directly into model selection and hyperparameter search in the chapters that follow.