Seeking to video frame of h264 codec in mp4 container with FFmpeg. Packet pts is always 0

Question

I'm trying to seek to the nearest keyframe of a specific frame with FFmpeg but whenever I obtain the next frame with av_read_frame after seeking, the packet pts or dts are always 0. This only happens with h264/mp4 videos as it works correctly for some codecs in .avi container.

I have tried using avformat_seek_file and av_seek_frame but they give me the same result.

I also read that I shouldn’t be reading timestamps from the packet, so I tried decoding the packet first with avcodec_decode_video2 and reading AVFrame->pts information but this value is always invalid for h264/mp4 videos.

This is the relevant code of what I'm trying to do:

/*Relevant from header*/
AVCodecContext pCodecCtx;
AVFormatContext *pFormatCtx;
int videoStreamIndex;

int Class::getFrame(int desiredFrame, bool seek)
if(seek)
{
    /* We seek to the selected frame */
    if(avformat_seek_file(pFormatCtx, videoStreamIndex, 0, desiredFrame, desiredFrame, AVSEEK_FLAG_BACKWARD) < 0)
    //if(av_seek_frame(pFormatCtx, mVideoStream, desiredFrame, AVSEEK_FLAG_BACKWARD) < 0)
    {
    // error management
    }
    avcodec_flush_buffers(pCodecCtx);
}

AVPacket packet;
int frameFinished;
/* Loop until we find the next video frame */
while(av_read_frame(pFormatCtx, &packet) >= 0 )
{
    if(packet.stream_index == videoStreamIndex)
    {
        avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet);
        int pcktPts;

        /*** management of other codecs here using av_frame_get_best_effort_timestamp() ***/


        /* With this approach I have been getting correct pts info after many av_read_frame loops */
        if(pCodecCtx->codec->id == AV_CODEC_ID_H264 && videoPath.toLower().endsWith(".mp4"))
        {
            pcktPts = av_rescale_q(packet.pts, //pFrame->pts always invalid here
                                      pFormatCtx->streams[videoStreamIndex]->time_base,
                                      pFormatCtx->streams[videoStreamIndex]->codec->time_base);
            pcktPts = (pcktPts/pCodecCtx->ticks_per_frame);
        }

        if(pcktPts == desiredFrame) ....
        /* more irrelevant code for reading, converting frame, etc */

Perhaps I'm dealing with this kind of codec incorrectly, any idea will be highly appreciated.

As a note, I am only interested in video frames.

Maxito Maxito · Accepted Answer · 2015-04-28T18:52:12

Ok, I think I got something.

Seems like avformat_seek_file actually wants the pts of the packet you want to seek to and not the number of frame, so since I'm using av_rescale_q to convert the packet pts to actual frame number I supposed I had to do the opposite operation to transform desired frame number to packet pts.

Now before seeking, I transform the desired frame number like this:

int target = desiredFrame *
             (pFormatCtx->streams[videoStreamIndex]->time_base.den /
              pFormatCtx->streams[videoStreamIndex]->time_base.num) /
             (pFormatCtx->streams[videoStreamIndex]->codec->time_base.den /
              pFormatCtx->streams[videoStreamIndex]->codec->time_base.num )*
              pCodecCtx->ticks_per_frame;

And for now that seems to work as expected. Still I would take any suggestions as this is the first solution I thought of and may be a little naive and I'm not sure it will work for all cases.

Seeking to video frame of h264 codec in mp4 container with FFmpeg. Packet pts is always 0

1 Answers