1、音画同步基础:理解PTS、DTS和时间基,为什么视频会音画不同步?
做音视频开发这些年,我见过最多的新手问题就是:「明明视频能播,声音也对得上,为什么我处理完就不同步了?」
说实话,音画同步这个坑,几乎每个入行的人都会踩一次。我自己也不例外——刚用FFmpeg做转码时,输出文件声音比画面快了半秒,排查了一整天才找到原因。
这一章,我们就从最基础的概念讲起。搞懂了PTS、DTS和时间基,你就能明白音画不同步的根因在哪。
1.1 三个核心概念:PTS、DTS、时间基
先记住一句话:音画同步,本质上是时间对齐。视频里的每一帧画面、每一段音频,都需要一个「时间戳」告诉播放器:你该在什么时候显示我。
1.1.1 PTS(显示时间戳)
PTS全称是Presentation Time Stamp。说白了,就是「这一帧该什么时候显示给用户看」。
- 视频帧的PTS:告诉播放器,这一帧画面在哪个时间点显示
- 音频帧的PTS:告诉播放器,这一段声音在哪个时间点播放
播放器拿到PTS后,会不断对比当前系统时间。如果画面PTS到了,就显示画面;音频PTS到了,就播放声音。两者对齐,就是音画同步。
1.1.2 DTS(解码时间戳)
DTS全称是Decoding Time Stamp。它告诉解码器:你该在什么时候解码这一帧。
你可能会问:PTS和DTS不是一回事吗?
嗯,这里要注意——在视频编码中,帧的顺序不一定是显示的顺序。比如B帧(双向预测帧)需要参考前后的帧才能解码,所以解码顺序和显示顺序可能不同。
关键区别:
- PTS:播放器用,决定「什么时候显示」
- DTS:解码器用,决定「什么时候解码」
- 对于I帧和P帧,PTS = DTS;对于B帧,PTS ≠ DTS
1.1.3 时间基(Time Base)
时间基,就是时间戳的「刻度单位」。FFmpeg里用分数表示,比如 1/25 表示每个刻度是0.04秒,1/90000 表示每个刻度约11微秒。
我个人习惯把时间基理解成「尺子的最小刻度」。刻度越细,时间精度越高。
| 时间基 | 含义 | 适用场景 |
|---|---|---|
1/25 |
每刻度0.04秒 | 25fps视频 |
1/30 |
每刻度约0.033秒 | 30fps视频 |
1/90000 |
每刻度约11微秒 | MPEG-TS流 |
1/1000 |
每刻度1毫秒 | 常见音频 |
避坑指南:我曾经在转码时,视频流用 1/25 的时间基,音频流用 1/1000。结果输出文件音画不同步。原因就是两个流的时间基不一致,FFmpeg在混流时计算偏移量出了偏差。后来我统一用 1/1000 作为输出时间基,问题就解决了。
1.2 为什么视频会音画不同步?
搞清楚了三个概念,我们来看看实际中音画不同步的常见原因。我把它归纳为四类:
1.2.1 时间基不一致
这是最常见的原因。视频流和音频流使用不同的时间基,FFmpeg在计算PTS时出现精度损失或舍入误差,导致两者逐渐错位。
举个例子:视频时间基 1/25,音频时间基 1/44100。视频每帧0.04秒,音频每样本约22.7微秒。当FFmpeg把音频PTS从 1/44100 转换到 1/25 时,必然产生舍入误差。累积几百帧后,误差就肉眼可见了。
1.2.2 编码器参数设置错误
有些编码器允许你设置 -vsync 或 -async 参数。如果设置不当,FFmpeg会主动丢弃或复制帧,导致PTS序列被打乱。
我记得有一次,我用 -vsync vfr 处理一个可变帧率视频,结果输出文件的PTS跳跃不定,音画完全对不上。后来改用 -vsync cfr 强制恒定帧率,才恢复正常。
1.2.3 容器格式限制
不同的容器格式对时间戳的精度支持不同。比如:
- MP4:时间基通常为
1/90000或1/1000,精度较高 - AVI:时间基固定为
1/30,精度较低 - FLV:时间基为
1/1000,但只支持整数毫秒
如果你从MP4转成AVI,时间基从 1/90000 降到 1/30,精度损失可能导致音画不同步。
1.2.4 手动修改PTS
有些开发者为了「加速」或「慢放」,直接修改PTS值。但如果没有同时调整DTS和时间基,就会破坏解码顺序和显示顺序的关系。
警告:不要手动修改PTS而不调整DTS。我曾经见过一个项目,为了做视频抽帧,直接删掉了一些帧的PTS,结果解码器因为DTS顺序错乱而崩溃。正确的做法是用FFmpeg的 setpts 滤镜,它会自动处理DTS和时间基的转换。
1.3 知识体系总览
下面这张图,把本章的核心知识点串起来了。你可以把它当作一个「音画同步知识地图」:
1.4 实战:用FFmpeg查看PTS和时间基
光说不练假把式。我们直接上手,用FFmpeg看看一个视频文件的PTS和时间基长什么样。
先准备一个视频文件,然后运行:
ffprobe -v error -select_streams v:0 -show_entries stream=index,codec_type,time_base -show_frames -of csv=p=0 input.mp4
这条命令会输出视频流的时间基,以及每一帧的详细信息。输出大概长这样:
0,video,1/25
1,audio,1/44100
frame,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,