Flush & Latency Issue with Fragmented MP4 Creation in FFMPEG

10
votes

I'm creating a fragmented mp4 for html5 streaming, using the following command:

-i rtsp://172.20.28.52:554/h264 -vcodec copy -an -f mp4 -reset_timestamps 1 -movflags empty_moov+default_base_moof+frag_keyframe -loglevel quiet -
  1. "-i rtsp://172.20.28.52:554/h264" because the source is h264 in rtp packets stream from an ip camera. For the sake of testing, the camera is set with GOP of 1 (i.e. all frames are key frames)
  2. "-vcodec copy" because I don't need transcoding, only remuxing to mp4.
  3. "-movflags empty_moov+default_base_moof+frag_keyframe" to create a fragmented mp4 according to the media source extensions spec.
  4. "-" at the end in order to output the mp4 to stdout. I'm grabbing the ouput and sending it to the webclient through web sockets.

Everything is working well, expect for a latency issue which I'm trying to solve. If I'm logging every time a data is coming in from stdout, with the timestamp of arrival, I get this output:

16/06/2015 15:40:45.239 got data size = 24

16/06/2015 15:40:45.240 got data size = 7197

16/06/2015 15:40:45.241 got data size = 32768

16/06/2015 15:40:45.241 got data size = 4941

16/06/2015 15:40:45.241 got data size = 12606

16/06/2015 15:40:45.241 got data size = 6345

16/06/2015 15:40:45.241 got data size = 6339

16/06/2015 15:40:45.242 got data size = 6336

16/06/2015 15:40:45.242 got data size = 6361

16/06/2015 15:40:45.242 got data size = 6337

16/06/2015 15:40:45.242 got data size = 6331

16/06/2015 15:40:45.242 got data size = 6359

16/06/2015 15:40:45.243 got data size = 6346

16/06/2015 15:40:45.243 got data size = 6336

16/06/2015 15:40:45.243 got data size = 6338

16/06/2015 15:40:45.243 got data size = 6357

16/06/2015 15:40:45.243 got data size = 6357

16/06/2015 15:40:45.243 got data size = 6322

16/06/2015 15:40:45.243 got data size = 6359

16/06/2015 15:40:45.244 got data size = 6349

16/06/2015 15:40:45.244 got data size = 6353

16/06/2015 15:40:45.244 got data size = 6382

16/06/2015 15:40:45.244 got data size = 6403

16/06/2015 15:40:45.304 got data size = 6393

16/06/2015 15:40:45.371 got data size = 6372

16/06/2015 15:40:45.437 got data size = 6345

16/06/2015 15:40:45.504 got data size = 6352

16/06/2015 15:40:45.571 got data size = 6340

16/06/2015 15:40:45.637 got data size = 6331

16/06/2015 15:40:45.704 got data size = 6326

16/06/2015 15:40:45.771 got data size = 6360

16/06/2015 15:40:45.838 got data size = 6294

16/06/2015 15:40:45.904 got data size = 6328

16/06/2015 15:40:45.971 got data size = 6326

16/06/2015 15:40:46.038 got data size = 6326

16/06/2015 15:40:46.105 got data size = 6340

16/06/2015 15:40:46.171 got data size = 6341

16/06/2015 15:40:46.238 got data size = 6332

As you can see, the first 23 lines (which contain data of about 1.5 secs of video) are arriving almost instantly, and then the delay between each 2 consecutive lines is ~70ms which makes sense because the video is 15 frames per sec. This behavior introduces a latency of about 1.5 sec.

It looks like a flushing issue because I don't see any reason why would ffmpeg need to hold the first 23 frames in memory, especially since each frame is a fragment of it's own inside the mp4. I couldn't however, find any method that would cause ffmpeg to flush this data faster.

Has anyone got a suggestion?

I'd like to note that this is a follow up question to this one: Live streaming dash content using mp4box

3
ffmpegvideo-streamingmp4mpeg-dashmedia-source
It occurred to me, that you have control over the blocksize used for buffering the output. Check ffmpeg.org/ffmpeg-all.html#toc-pipe and see if tweaking that value can help you there. – Pablo Montilla
@PabloMontilla I tried to play with some different values of blocksize and although it effected the output in some way, it didn't solve the initial delay. – galbarm
Hello @galbarm! I can't get video running on page with your ffmpeg params, always getting Skipping unrecognized top-level box: ftyp. (h264 ip cam). I also tried to change -vcodec to libx264, that case i get Skipping unrecognized top-level box: mdat. Can you please describe your code more or to gist it somewhere? Most interesting part is .addSourceBuffer param, i.e. codec string. Thanks in advance! – zarkone
Hi, @zarkone I'm also seeing the "skipping ftyp" error but it doesn't seem to have any functional effect. Here a gist of the client code, I'm sure it will help you: gist.github.com/galbarm/8cb1b684652de648ded3 – galbarm
Thanks for your snippet! – zarkone

3 Answers

5
votes

The key to removing the delay is to use the -probesize argument:

probesize integer (input)

Set probing size in bytes, i.e. the size of the data to analyze to get stream information. A higher value will enable detecting more information in case it is dispersed into the stream, but will increase latency. Must be an integer not lesser than 32. It is 5000000 by default.

By default the value is 5,000,000 bytes which was equivalent to ~1.5 sec of video. I was able to almost completely eliminate the delay by reducing the value to 200,000.

0
votes

I solved the latency issue by using the -g option to set the number of frames in the group. In my case I used -g 2. I suspect that if you don't make it explicit, the fragment either waits for the source to provide the keyframe or uses a really large default value to generate the keyframe before closing off the fragment and dumping it to stdout.

0
votes

Usually the buffering for stdout is disabled in case of console output. If you run ffmpeg from code, the buffering is enabled, so you will get your data only when the buffer is full or the command ends.

You have to eliminate the stdout buffering of your os. On windows its impossible imo, but on ubuntu for ex. There is http://manpages.ubuntu.com/manpages/maverick/man1/stdbuf.1.html