4. 时间窗口偏差校验:时间切片与分布对比
时间窗口偏差是风控样本偏差中最隐蔽的一类——模型在训练期表现良好,上线后却因时间分布漂移而失效。本节聚焦两个核心操作:时间切片(将连续时间划分为等长窗口)与分布对比(比较不同窗口的样本特征与标签分布),通过 SQL 实现可复用的校验框架。
4.1 时间切片 SQL:DATE_TRUNC 与 WINDOW 函数
时间切片的核心是将时间戳字段按固定粒度截断,生成可聚合的窗口标识。PostgreSQL / Redshift 使用 DATE_TRUNC,BigQuery 使用 DATE_TRUNC 或 TIMESTAMP_TRUNC,Snowflake 使用 DATE_TRUNC,Hive / Spark SQL 使用 TRUNC 或 TO_DATE 配合格式化。
关键概念: 切片粒度通常选择“天”“周”“月”,风控场景中“周”最常用——既能捕捉短期波动,又不会因日粒度噪声过大而误判。
基础时间切片示例(按周聚合):
-- 将每笔交易的时间戳截断到周起始日,并统计每周样本量
SELECT
DATE_TRUNC('week', trans_time) AS week_start,
COUNT(*) AS sample_cnt,
COUNT(DISTINCT user_id) AS user_cnt,
AVG(amount) AS avg_amount
FROM trans_log
WHERE trans_time >= '2024-01-01'
AND trans_time < '2024-07-01'
GROUP BY DATE_TRUNC('week', trans_time)
ORDER BY week_start;
使用 WINDOW 函数计算窗口间变化率:
-- 计算每周样本量的环比变化率,快速定位异常窗口
WITH weekly_stats AS (
SELECT
DATE_TRUNC('week', trans_time) AS week_start,
COUNT(*) AS sample_cnt
FROM trans_log
WHERE trans_time >= '2024-01-01'
GROUP BY 1
)
SELECT
week_start,
sample_cnt,
LAG(sample_cnt) OVER (ORDER BY week_start) AS prev_cnt,
ROUND(
(sample_cnt - LAG(sample_cnt) OVER (ORDER BY week_start))
/ NULLIF(LAG(sample_cnt) OVER (ORDER BY week_start), 0) * 100,
2
) AS pct_change
FROM weekly_stats
ORDER BY week_start;
实践建议: 将时间切片 SQL 封装为视图或临时表,后续所有分布对比查询都基于该视图,避免重复计算且保证窗口定义一致。
4.2 不同时间窗口的分布对比方法
分布对比的核心是回答三个问题:样本量是否稳定?特征分布是否漂移?标签(逾期/坏账)率是否突变?以下表格列出常用对比维度与 SQL 实现方式。
| 对比维度 | 校验指标 | SQL 实现方式 | 偏差阈值参考 |
|---|---|---|---|
| 样本量稳定性 | 窗口样本数、环比变化率 | COUNT + LAG WINDOW | 变化率 > ±30% 需预警 |
| 数值特征分布 | 均值、中位数、标准差 | AVG / PERCENTILE_CONT / STDDEV | 均值偏移 > 0.5σ 需关注 |
| 类别特征分布 | 各类别占比、PSI | COUNT + GROUP BY + 交叉表 | PSI > 0.1 需深入分析 |
| 标签分布 | 坏账率、逾期率 | SUM(flag) / COUNT(*) | 坏账率波动 > 20% 需排查 |
窗口间特征分布对比 SQL(以金额字段为例):
-- 对比两个时间窗口的金额分布统计量
WITH window_a AS (
SELECT
AVG(amount) AS avg_amt,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY amount) AS med_amt,
STDDEV(amount) AS std_amt,
COUNT(*) AS cnt
FROM trans_log
WHERE trans_time >= '2024-01-01' AND trans_time < '2024-04-01'
),
window_b AS (
SELECT
AVG(amount) AS avg_amt,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY amount) AS med_amt,
STDDEV(amount) AS std_amt,
COUNT(*) AS cnt
FROM trans_log
WHERE trans_time >= '2024-04-01' AND trans_time < '2024-07-01'
)
SELECT
'金额' AS feature_name,
a.avg_amt AS w1_avg, b.avg_amt AS w2_avg,
ROUND((b.avg_amt - a.avg_amt) / NULLIF(a.std_amt, 0), 4) AS std_shift,
a.med_amt AS w1_med, b.med_amt AS w2_med,
a.cnt AS w1_cnt, b.cnt AS w2_cnt
FROM window_a a, window_b b;
常见陷阱: 直接比较均值可能掩盖分布形态变化。例如金额均值不变,但高额交易占比从 5% 升至 30%,风控策略会完全失效。建议同时对比分位数或绘制分布直方图。
4.3 时间窗口偏差校验流程
下图展示从原始日志到偏差报告的完整校验链路,包含三个核心阶段:切片、对比、判定。
4.4 多窗口分布对比:基期 vs 当前期
实际风控校验中,通常选取一个稳定历史窗口作为基期,与当前窗口进行对比。基期选择原则:样本量充足、业务无重大变更、距离当前不超过 6 个月。
基期 vs 当前期对比 SQL(含 PSI 计算):
-- 计算金额字段的 PSI(群体稳定性指标)
WITH base AS (
SELECT
CASE
WHEN amount < 1000 THEN '0-1k'
WHEN amount < 5000 THEN '1k-5k'
WHEN amount < 20000 THEN '5k-20k'
ELSE '20k+'
END AS amt_bucket,
COUNT(*) * 1.0 / SUM(COUNT(*)) OVER() AS base_pct
FROM trans_log
WHERE trans_time >= '2024-01-01' AND trans_time < '2024-04-01'
GROUP BY 1
),
current AS (
SELECT
CASE
WHEN amount < 1000 THEN '0-1k'
WHEN amount < 5000 THEN '1k-5k'
WHEN amount < 20000 THEN '5k-20k'
ELSE '20k+'
END AS amt_bucket,
COUNT(*) * 1.0 / SUM(COUNT(*)) OVER() AS curr_pct
FROM trans_log
WHERE trans_time >= '2024-04-01' AND trans_time < '2024-07-01'
GROUP BY 1
)
SELECT
b.amt_bucket,
ROUND(b.base_pct, 4) AS base_pct,
ROUND(c.curr_pct, 4) AS curr_pct,
ROUND((c.curr_pct - b.base_pct) * LN(c.curr_pct / b.base_pct), 6) AS psi_contrib
FROM base b
JOIN current c ON b.amt_bucket = c.amt_bucket
UNION ALL
SELECT
'PSI = SUM(psi_contrib)' AS amt_bucket,
NULL, NULL,
ROUND(SUM((c.curr_pct - b.base_pct) * LN(c.curr_pct / b.base_pct)), 6)
FROM base b
JOIN current c ON b.amt_bucket = c.amt_bucket;
PSI 解读: PSI < 0.1 表示分布稳定;0.1–0.25 需关注;> 0.25 表示显著漂移,模型需要重新训练或特征工程调整。
4.5 时间窗口偏差的常见模式与应对
根据实际项目经验,时间窗口偏差通常表现为以下三种模式,每种模式对应不同的根因与处理策略。
| 偏差模式 | 典型表现 | 可能根因 | 应对策略 |
|---|---|---|---|
| 样本量骤降 | 某周样本量环比下降 > 50% | 数据管道故障、业务下线 | 排查 ETL 链路,补充缺失窗口 |
| 特征均值漂移 | 金额均值连续 3 周上升 | 用户群体变化、促销活动 | 增加分群校验,或引入时间衰减权重 |
| 坏账率突变 | 逾期率从 2% 跳升至 8% | 风控策略调整、外部环境变化 | 回滚策略或重新训练模型 |
自动化校验脚本框架:
-- 每周自动执行,输出偏差预警记录
WITH weekly_metrics AS (
SELECT
DATE_TRUNC('week', trans_time) AS week_start,
COUNT(*) AS sample_cnt,
AVG(amount) AS avg_amount,
SUM(CASE WHEN is_default = 1 THEN 1 ELSE 0 END) * 1.0 / COUNT(*) AS bad_rate
FROM trans_log
GROUP BY 1
)
SELECT
week_start,
sample_cnt,
avg_amount,
ROUND(bad_rate, 4) AS bad_rate,
CASE
WHEN sample_cnt < 0.7 * LAG(sample_cnt) OVER (ORDER BY week_start)
THEN '样本量骤降预警'
WHEN ABS(avg_amount - LAG(avg_amount) OVER (ORDER BY week_start))
/ NULLIF(LAG(avg_amount) OVER (ORDER BY week_start), 0) > 0.3
THEN '均值漂移预警'
WHEN ABS(bad_rate - LAG(bad_rate) OVER (ORDER BY week_start)) > 0.02
THEN '坏账率突变预警'
ELSE '正常'
END AS alert_flag
FROM weekly_metrics
ORDER BY week_start;
重要提醒: 时间窗口偏差校验应纳入风控模型的日常监控看板,而非仅在训练时做一次。建议设置每周自动运行的任务,并将预警结果推送到团队协作工具(如钉钉、Slack)。