FFMPEG API real-time encoding, skipped frames

Question

I am using FFMPEG to encode live video. If I can't keep up with the live encoding, I need to drop frames. Currently, I just skip the encoding of a raw frame, and the PTS for the next frame is the same. Meaning, even if there is a skipped frame, the pts is always 1,2,3,4,5,6....

This causes the playback of the file to skip ahead in time. Instead, I would like to add blank frames to the muxed file. I tried making PTS indicate a skipped frame by setting it to 1,2,4,5,8,...., but the FFMPEG h264 encoders depend on PTS being consecutive.

How do I pass empty data to my mp4/container, indicating a blank space in time, so that decoders won't appear to jump ahead in time, but instead just see a delay/freeze.

Gigone Lee Gigone Lee · Accepted Answer · 2015-10-24T06:10:14

You don't need to create empty frame, in this case, you should manage PTS yourself. for video PTS, it is always increased by 1 per frame. it is guaranteed so you can assure it.

I don't have any experience in live encoding yet, but when as long as live goes, sync with audio frame could be conflicted if you just drop the video frame.

To avoid that, you should save previous frame, and make the clone and fill it when you drop the frame, or you should drop audio frame too. but as i said, it is just gessing so i don't know what is best for you.

following is puedo code in this case :

// you should drop / or drop and make the clone of previous frame.

static int64_t last_pts = 0; // just make sure initialize once.
if(frame->pts != AV_NOPTS_VALUE && is_video == true)
{
  if(last_pts) frame->pts = last_pts + 1;
  last_pts = frame->pts;
}

FFMPEG API real-time encoding, skipped frames

1 Answers