Audio/Video synchronization issues using samplegrabber filter

Question

I am using DirectShow for a movie playback. I modify video/audio data and render everything myself in my app. Therefore, I am using SampleGrabber filters and NULL renderers.

The problem arises when I rewind the movie (using IMediaSeeking interface) - audio significantly lags behind the video (a few seconds lag).

I understand the a/v sync basics and timestamps. I do understand that render filters can drop frames depending on the presentation times.

But I guess SampleGrabber filters simply receive all data? Is it possible to get audio/video data already synchronized in my SampleCB callbacks?

Answer 1

On the sample grabber callback you receive data time stamped. If you keep the time stamps you get, and you make sure you have the data playback sync'ed using these time stamps in terms of playing data with equal time stamps on all streams at any time - then you will have your synchronization in good standing.

Sample grabbers receive all data, but it does not mean this data reaches grabbers in synchronized order. Matching time stamps is absolutely necessary. Because video might be temporally compressed and audio is typically not, your seeking might be getting you additional preroll data on the video leg, which you might be not processing correctly.