I receive h.264 stream (baseline profile) through RTP/UDP, and when sending them to decoder, if (P)frame loss occurred, I wanna skip frames until next I frame received. How can I detect the frame loss? H.264 spec is too confused to me..
3 Answers
2
votes
From the slice header, use the frame_num (decoding order) parameters as well as the TopFieldPictureCount and BottomFieldPictureCount. The latter two are collectively referred to as Picture Order Count (POC). POC determines display order.
Let's say your GOP's display order is as follows: IBBPBBPBBP... This will be the sequence of your POC parameter. The corresponding decoding order will be IPBBPBBPBB... Note that you need to decode the first I and P in order to decode the first 2 B-frames. Similarly, you need to decode the second P-frame in order to (in conjunction with the first P-frame) decode the second 2 B-frames.
With that, I would use the POC parameter to ensure that my frames have been decoded properly. If you know your GOP beforehand, you can make sure that you are receiving frames 3, 6, 9, etc. using the GOP above as an example, where the I-frame is frame 0. If the GOP is unknown, you decode a few frames and determine the GOP from the POC pattern. For instance, the above GOP would have a real-time POC of 0, 3, 1, 2, 6, 4, 5, 9, 7, 8, ... Alternatively, the slice headers can tell you exactly what frame you are dealing with.
1
votes
My first instinct is that your decoder should be able to recover from a missing P frame, but in case your decoder library can not...
My first guess would be the frame_num field of the slice_header().
The syntax of the slice_header is documented in section 7.3.3 of ISO 14496-10.
The definition for frame_num starts on page 88 of my copy and goes on for about a page (so it is not trivial to interpret if you want to handle all video).
While scanning for references to frame_num in the rest of the document I also noticed there's a gaps_in_frame_num_value_allowed_flag (in the SPS, section 7.3.2.1) that enables the frame_num to get complicated in the general case.
