4. 事件源接入:Prometheus告警接入、Zabbix告警接入、云监控告警接入、自定义Webhook
事件源接入是Runbook自动化故障自愈体系的“第一公里”。值班机器人必须能够统一接收来自不同监控系统的告警事件,并将其标准化为内部可处理的告警对象。本章详细阐述四种主流事件源的接入方式、数据格式转换及配置要点。
4.1 告警事件标准化模型
无论事件源来自何方,Runbook内部均采用统一的告警事件模型(Alert Event Model)进行抽象。该模型包含以下核心字段:
| 字段名 | 类型 | 说明 | 示例 |
|---|---|---|---|
alert_id |
String | 全局唯一告警标识 | prom-20250321-001 |
source |
String | 事件源类型 | prometheus |
alert_name |
String | 告警规则名称 | HighCPUUsage |
severity |
String | 告警级别 | critical |
status |
String | 告警状态 | firing / resolved |
instance |
String | 告警对象标识 | 10.0.1.5:9090 |
summary |
String | 告警摘要 | CPU usage > 90% |
description |
String | 详细描述 | Instance 10.0.1.5 CPU at 95% |
started_at |
Timestamp | 告警触发时间 | 2025-03-21T10:00:00Z |
labels |
Map | 自定义标签 | {“env”: “prod”, “team”: “sre”} |
4.2 Prometheus告警接入
Prometheus通过Alertmanager组件发送告警。Runbook通过接收Alertmanager的Webhook回调实现接入。
4.2.1 配置Alertmanager Webhook
在Alertmanager的配置文件 alertmanager.yml 中,添加一个指向Runbook Webhook接收端点的receiver:
receivers:
- name: 'runbook-webhook'
webhook_configs:
- url: 'http://runbook.example.com/api/v1/events/prometheus'
send_resolved: true
http_config:
basic_auth:
username: 'runbook_user'
password: 'runbook_pass'
route:
receiver: 'runbook-webhook'
group_wait: 10s
group_interval: 5m
repeat_interval: 1h
4.2.2 数据格式转换
Prometheus Alertmanager发送的JSON Payload结构如下,Runbook适配器需将其映射为标准化模型:
{
"receiver": "runbook-webhook",
"status": "firing",
"alerts": [
{
"status": "firing",
"labels": {
"alertname": "HighCPUUsage",
"instance": "10.0.1.5:9090",
"severity": "critical"
},
"annotations": {
"summary": "CPU usage > 90%",
"description": "Instance 10.0.1.5 CPU at 95%"
},
"startsAt": "2025-03-21T10:00:00Z",
"endsAt": "0001-01-01T00:00:00Z",
"generatorURL": "http://prometheus.example.com/graph?g0..."
}
]
}
alert_id=labels.alertname + “-” + labels.instance + “-” + startsAt(MD5哈希)source=“prometheus”alert_name=labels.alertnameseverity=labels.severitystatus=status(firing/resolved)instance=labels.instancesummary=annotations.summarydescription=annotations.descriptionstarted_at=startsAtlabels= 合并labels和annotations
4.3 Zabbix告警接入
Zabbix通过Action中的Media类型或Webhook脚本发送告警。推荐使用Zabbix内置的Webhook方法(Zabbix 5.0+)或自定义脚本调用Runbook API。
4.3.1 配置Zabbix Webhook Action
在Zabbix前端创建Action,配置Operation为“Send message”并选择Webhook Media类型。Webhook URL指向Runbook端点:
URL: http://runbook.example.com/api/v1/events/zabbix
Method: POST
Content-Type: application/json
Parameters:
- {“name”: “alert_subject”, “value”: “{ALERT.SUBJECT}”}
- {“name”: “alert_message”, “value”: “{ALERT.MESSAGE}”}
- {“name”: “alert_status”, “value”: “{EVENT.STATUS}”}
- {“name”: “trigger_id”, “value”: “{TRIGGER.ID}”}
- {“name”: “host_name”, “value”: “{HOST.NAME}”}
- {“name”: “event_id”, “value”: “{EVENT.ID}”}
- {“name”: “severity”, “value”: “{TRIGGER.SEVERITY}”}
- {“name”: “timestamp”, “value”: “{EVENT.DATE} {EVENT.TIME}”}
4.3.2 数据格式转换
Zabbix Webhook发送的JSON结构示例:
{
"alert_subject": "PROBLEM: High CPU on Server-01",
"alert_message": "CPU load is 95% on Server-01",
"alert_status": "PROBLEM",
"trigger_id": "12345",
"host_name": "Server-01",
"event_id": "67890",
"severity": "4",
"timestamp": "2025-03-21 10:00:00"
}
alert_id=event_idsource=“zabbix”alert_name= 从alert_subject提取(如“High CPU on Server-01”)severity= Zabbix数字级别映射为字符串(1=information, 2=warning, 3=average, 4=high, 5=disaster)status=alert_status(PROBLEM/OK 映射为 firing/resolved)instance=host_namesummary=alert_subjectdescription=alert_messagestarted_at=timestamp
4.4 云监控告警接入
阿里云、腾讯云、AWS等云厂商均提供告警回调功能,通过HTTP/HTTPS将告警消息推送到指定URL。Runbook需为每个云厂商实现独立的适配器。
4.4.1 阿里云云监控
在阿里云云监控控制台配置“报警回调”URL:http://runbook.example.com/api/v1/events/alicloud。阿里云发送的JSON结构如下:
{
"alertName": "CPU使用率过高",
"alertState": "ALERT",
"curValue": "95.0",
"dimensions": "{\"instanceId\":\"i-bp1xxxxx\"}",
"expression": "Average > 90",
"metricName": "CPUUtilization",
"namespace": "acs_ecs",
"regionId": "cn-hangzhou",
"timestamp": "2025-03-21 10:00:00"
}
4.4.2 腾讯云云监控
在腾讯云云监控策略中配置“接口回调”地址:http://runbook.example.com/api/v1/events/tencentcloud。腾讯云发送的JSON结构:
{
"sessionId": "xxxxx",
"alarmStatus": "1",
"alarmType": "metric",
"alarmObjInfo": {
"region": "ap-guangzhou",
"namespace": "qce/cvm",
"dimensions": {"InstanceId": "ins-xxxxx"}
},
"alarmPolicyName": "CPU告警策略",
"alarmPolicyType": "cvm",
"alarmLevel": "2",
"metricName": "cpu_usage",
"metricValue": "95.0",
"firstOccurTime": "2025-03-21 10:00:00"
}
4.4.3 AWS CloudWatch
AWS通过SNS Topic将CloudWatch告警推送到订阅的HTTP/HTTPS端点。Runbook接收端点:http://runbook.example.com/api/v1/events/aws。AWS SNS发送的JSON结构:
{
"Type": "Notification",
"MessageId": "xxxxx",
"TopicArn": "arn:aws:sns:us-east-1:xxxxx:MyTopic",
"Message": "{\"AlarmName\":\"HighCPU\",\"AlarmDescription\":\"CPU > 90%\",\"AWSAccountId\":\"xxxxx\",\"NewStateValue\":\"ALARM\",\"OldStateValue\":\"OK\",\"StateChangeTime\":\"2025-03-21T10:00:00.000+0000\",\"Region\":\"US East (N. Virginia)\",\"Trigger\":{\"MetricName\":\"CPUUtilization\",\"Namespace\":\"AWS/EC2\",\"Statistic\":\"Average\",\"Dimensions\":[{\"name\":\"InstanceId\",\"value\":\"i-xxxxx\"}]}}"
}
4.5 自定义Webhook接入
对于自研监控系统或第三方工具,Runbook提供通用Webhook接入点,允许用户自定义Payload格式并通过配置映射规则实现标准化。
4.5.1 通用Webhook端点
端点URL:http://runbook.example.com/api/v1/events/custom
请求方式:POST
Content-Type:application/json
4.5.2 配置映射规则
用户需在Runbook管理后台配置JSONPath映射规则,将自定义Payload字段映射到标准化模型。示例配置:
{
"source": "custom_monitor",
"mapping_rules": {
"alert_id": "$.event_id",
"alert_name": "$.rule_name",
"severity": "$.level",
"status": "$.state",
"instance": "$.host",
"summary": "$.title",
"description": "$.message",
"started_at": "$.time",
"labels": "$.tags"
}
}
4.5.3 自定义Payload示例
假设用户发送如下自定义Payload:
{
"event_id": "cust-001",
"rule_name": "DiskFull",
"level": "critical",
"state": "firing",
"host": "app-server-01",
"title": "磁盘使用率超过95%",
"message": "/data分区使用率98%",
"time": "2025-03-21T10:00:00Z",
"tags": {
"env": "prod",
"team": "sre"
}
}
根据上述映射规则,Runbook将生成标准化告警对象:
{
"alert_id": "cust-001",
"source": "custom_monitor",
"alert_name": "DiskFull",
"severity": "critical",
"status": "firing",
"instance": "app-server-01",
"summary": "磁盘使用率超过95%",
"description": "/data分区使用率98%",
"started_at": "2025-03-21T10:00:00Z",
"labels": {"env": "prod", "team": "sre"}
}
4.6 事件源接入架构总览
下图展示了Runbook事件源接入层的整体架构,包括各适配器、标准化管道和事件总线的关系:
4.7 接入配置清单与验证
下表总结了各事件源的接入配置要点和验证方法:
| 事件源 | 配置位置 | 关键参数 | 验证方法 |
|---|---|---|---|
| Prometheus | alertmanager.yml | webhook URL, send_resolved, basic_auth | 触发测试告警,查看Runbook日志是否收到 |
| Zabbix | Action → Operation → Webhook | URL, Method, Parameters宏变量 | 手动执行Action测试,检查Runbook事件列表 |
| 阿里云 | 云监控 → 报警联系人 → 报警回调 | 回调URL | 在云监控控制台发送测试报警 |
| 腾讯云 | 云监控 → 告警策略 → 接口回调 | 回调URL | 在告警策略中执行“测试”操作 |
| AWS | SNS Topic → HTTP订阅 | 订阅URL, 确认订阅 | 确认订阅后,发布测试消息到SNS Topic |
| 自定义 | Runbook管理后台 → 事件源配置 | JSONPath映射规则 | 使用curl发送测试Payload,检查标准化结果 |