Timing Issues When Muxing Audio and Video with libav

Question

I have series of encoded packets, H.264 video and AAC audio. As they're coming on, I'm writing them to a video file, using av_write_frame.

Given the following situation in a row

10 seconds of video, then
10 seconds of video and audio, then
10 seconds of video.

Everything muxes fine and when played back via VLC or QuickTime, everything looks good. If I play it in Windows Media Player, the audio is played immediately.

It seems I'm doing something wrong, but checking the PTS of the audio stream packets, they are set to 10 seconds based on the time base of the audio stream.

It seems as though empty audio packets might be needed before the actual audio. Using similar code on the Mac side that uses AVFoundation, the resulting file has it's initial audio PTS at 0, so it's pre-filling with empty audio packets. — Stephen H. Gerstacker
What container were you using? Some muxers may handle this use-case differently. — Jack

Stephen H. Gerstacker Stephen H. Gerstacker · Accepted Answer · 2014-03-21T13:20:46

It seems that it's best to inject empty audio packets at the beginning of the stream. This is the only way that video playback in WMP would work. Every player handles the streams differently and this is the best way to ensure compatibility across players.

Timing Issues When Muxing Audio and Video with libav

1 Answers