How to write RTP/H264 stream as a file

11
votes

I couldn't find a solution for my previous question so I decided to try it step by step.

The thing that I wanna do now is to store the RTP/H264 stream as a file.

What I found so far is the below:

(firstly my RTP/H264 is FU-A which is the form of)

| RTP HEADER 12bytes long | FU INDICATOR 1byte | FU HEADER 1byte | FU payload |

As I understood RFC 6184 document, I'm starting a NAL with a packet which has '1' at the first bit of FU Header and appending following packets that set '0' at the first bit until the last packet which has '1' at the second bit of FU Header.

I think that's how to get a complete NAL before the FU-A packetization and also what I found is I need to put the 'starting bits' (0x00000001) at the front of each complete NAL.

But no luck so far. The below is part of log

 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]7C/1111100 [5]85/10000101 [6]B8/10111000 [7]40/1000000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]7C/1111100 [5]85/10000101 [6]B8/10111000 [7]40/1000000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]7C/1111100 [5]85/10000101 [6]B8/10111000 [7]40/1000000 ...
 ========= adding the next NAL as 716
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]7C/1111100 [5]85/10000101 [6]B8/10111000 [7]40/1000000 ...
 ========= a NAL is summed up as 4866
 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= adding the next NAL as 139
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E2/11100010 [7]20/100000 ...
 ========= a NAL is summed up as 7061
 ========= the new NAL is as 1377
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]41/1000001 [5]E4/11100100 [6]40/1000000 [7]1A/11010 ...
 ========= a NAL is summed up as 1369
 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E6/11100110 [7]60/1100000 ...
 ========= adding the next NAL as 94
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]E6/11100110 [7]60/1100000 ...
 ========= a NAL is summed up as 1472
 ========= the new NAL is as 447
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]41/1000001 [5]E8/11101000 [6]80/10000000 [7]16/10110 ...
 ========= a NAL is summed up as 439
 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]EA/11101010 [7]A0/10100000  ...
 ========= adding the next NAL as 1174
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]EA/11101010 [7]A0/10100000 ...
 ========= a NAL is summed up as 2552
 ========= the new NAL is as 1400
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]EC/11101100 [7]C0/11000000 ...
 ========= adding the next NAL as 1364
 [0]0/0 [1]0/0 [2]0/0 [3]1/1 [4]5C/1011100 [5]81/10000001 [6]EC/11101100 [7]C0/11000000 ...
 ========= a NAL is summed up as 2742
 ========= the new NAL is as 1400
 ...

my question is,

  1. if I can get the complete NALs from the fragmented packets by FU-A, how can I make it as a file which is able to run by VLC or other player?

  2. I'm still confused if I have to keep the FU indicator and FU header or not. Somebody said I need to take them only for the very first packet (starting with '1' at the FU header)

Any advice will be really appreciated.

Thanks.

2
streamingffmpegh.264rtp
Hey, I am trying to do the exact same thing.. I read the RFC 6184 and tried to follow teh same steps and store it into a file.. But my file does not seem to play at all... Can you please post the solution you used? This question has over 7k views.. It will be helpful for all of us.. - mk..

2 Answers

6
votes

The FU, STAP, and MTAP NAL units are specific only to RTP packetization as they're designed to facilitate network transport. In other words, don't count on a decoder parsing them correctly. Ultimately, you need to reassemble NAL units as in the case of FU packets, or break them up in to multiple NAL units the case of STAP/MTAP.

Once you have a NAL unit (and this includes PPS, SPS, SEI, slice partitions, and all the other types in the 1-23 range), then you can write to disk along with the '0001' start codes per H.264 annex B.

Placing the H.264 annex B stream into a container such as MPEG-4 can be done with various command-line tools (I'm pretty sure ffmpeg can do it).

5
votes

  1. One option is to mux the data into a file format such as mp4 or avi to be able to play it with VLC. AFAIR avi was a bad fit for H.264 (can't remember the reason off hand). There are free libraries such as libmp4, or on if you're on windows using DirectShow, Geraint's mp4mux.

    Another option is to use ffmpeg to convert a .264 file into an mp4

    ffmpeg -i test.264 test.mp4

    This assumes that the .264 file contains NAL units separated by start codes.

  2. From RFC6184

    The FU payload consists of fragments of the payload of the fragmented NAL unit so that if the fragmentation unit payloads of consecutive FUs are sequentially concatenated, the payload of the fragmented NAL unit can be reconstructed. The NAL unit type octet of the fragmented NAL unit is not included as such in the fragmentation unit payload, but rather the information of the NAL unit type octet of the fragmented NAL unit is conveyed in the F and NRI fields of the FU indicator octet of the fragmentation unit and in the type field of the FU header. An FU payload MAY have any number of octets and MAY be empty.

    You would need to reconstruct the original NAL unit before writing it to the .264 file if you take the second approach outlined in 1.