h264 to FLV conversion, ambiguous FrameType definition: What differentiates a Key Frame vs an Inter Frame?

0
votes

I'm trying to write out an Annex B formatted h264 bitstream in an FLV container over RTMP in an Android app. After reading this SO answer and several more, I understand my Annex B bitstream first has to be converted to AVCC before being wrapped in an FLV container.

That said, the FLV container spec is a bit unclear in the wrapping of its video tags. It requires a FrameType which can be:

  1. keyframe (For AVC a seekable frame)
  2. interframe (For AVC a non seekable frame)
  3. Disposable interframe (h263 only)
  4. Generated keyframe (serverside only)
  5. Video info / command frame (The message string contains a byte, 0 or 1 meaning either start of client seeking video frame sequence, or end of client seeking frame sequence)

When writing IDR or non-IDR AVC frames, i.e VCL NAL units, it's clear that options 1 and 2 should be used respectively. However, my h264 bitstream also contains non-VCL NAL units, specifically SEI payload type 47 (Display Orientation). It's not clear whether this fits when going from AVC to FLV per the spec. Do I simply consider the SEI a non seekable frame? What is a "frame" from FLV's point of view?

2
androidvideo-streamingh.264rtmpflv

2 Answers

1
votes

It’s not ambiguous. What flv calls a frame, h264 calls an access unit. Like the link you posted said, an access unit can contain more than one NAL. An SEI by itself is not an access unit, hence it is not a frame, hence it should never be by itself in an flv tag, hence the question does not make sense.

An SEI is sent along as part of the same frame (AU) payload and the flv frame type is that if the VCL nalus in that au.

-1
votes

According to "High Efficiency Video Coding: Coding Tools and Specification" by Mathias Wien, all SEI (Supplemental Enhancement Information) messages are defined as information not needed for decoding and reconstruction of the video. In general, not all video formats can represent such information. For this one specifically, a display orientation message does not represent any specific video frame, because it is a flag that persists until changed.