How to know the packet order in RTP H264 Stream

0
votes

Let me remember somethings about H264 RTP packets.

If you want to observe some error about the sentences below feel free to do it.

1) IDRs, also named by I-Frames which are the whole image can be divided in packets.

2) When I receive a packet and it´s "fragment_type == 28" (not just that, but lets supose that I really get a fragment) it means I have an IDR fragment.

3) Each packet has a sequence number which is generated by the emissor. The sequence packet respect an order: if the packet A has sequence number = 20, the next packet sent by the emissor will be 21 and so on.

Now let´s go to my questions:

A) If I have an IDR to reconstruct, how can I know what packets belongs to this exactly IDR? Let me take an example: The IDR is fragmented in the packets A, B, C and D. Will Always the packet A have his sequence number 6606 (just an example) therefore B will have 6607 and C 6608 and D 6609 for example or the sequence number doesn´t metter? If sequence number doesn´t metter, I supose I have a field in the packet which say "hey! I´m a fragment of the frame "HelloWorld". And the other packet cames and say "Hey! I´m a fragment of the frame "HelloWorld" too. But other packet cames and say "Hey! I dont belong to the HelloWorld frame as my other friends, I belong to the frame "HelloHell". Well, what field is that which shows me what frame the IDR belong to?

B) How can I extract the SPS and the PPS from the packet? I read a lot of topics about that, but I can´t understand in which byte are this informations. Later of get it, how can I send this informations for my decoder?

C) User Cipi, I invoke you to help me with some code examples about the question B. Dude, you are so helpful. Almost everything I did until know was because of you help. Can you help me?

Thank you people! I´m working hard in my software to get this stream from an IP câmera (SONY SNC-EP580).

1
h.264rtp

1 Answers

2
votes

1) IDRs, also named by I-Frames which are the whole image can be divided in packets.

Any frame can be divided into multiple packets. This is typically related to the MTU of the network.

2) When I receive a packet and it´s "fragment_type == 28" (not just that, but lets supose that I really get a fragment) it means I have an IDR fragment.

Incorrect as stated above.

3) Each packet has a sequence number which is generated by the emissor. The sequence packet respect an order: if the packet A has sequence number = 20, the next packet sent by the emissor will be 21 and so on.

A) If I have an IDR to reconstruct, how can I know what packets belongs to this exactly IDR?

The first packet of a fragmentation unit will have the start bit set in the FU header (second byte of payload). The last packet of the fragmentation unit will have the end bit set. Read section 5.8 in RFC6184 for more info on fragmentation units.

How can I extract the SPS and the PPS from the packet?

It depends on the packetization mode used. In single NAL unit mode, each NAL unit (e.g. SPS or PPS) will be in its own RTP packet. In non-interleaved mode, you typically have to split the STAP packet into the original NAL units.

Later of get it, how can I send this informations for my decoder?

This depends on the decoder, so you have to initialise, others you can just pass the NAL unit e.g. SPS or PPS to as you would any other NAL unit.

I invoke you to help me with some code examples about the question B.

Again, this is decoder specific, the open source VLC and ffmpeg code contains examples of how to interface with libavcodec.