How to decide using MTAP over STAP in h264 rtp payload

Question

I've been digging into encoding and streaming h264 over the past week. To night I'm implementing the rtp h264 payload.

According to RFC 3984 ("RTP Payload Format for H.264 Video - February 2005") multiple new NALU were introduced. Among them MTAP (multi time aggregation packet) and the STAP (single time aggre...).

As the names indicating, in STAP mode, all units is assumed to have the same timestamps. Does not that mean we can't use STAP for VCL NAL units?

For example one may use STAP for transmitting NAL types 7 or 8 (SPS, PPS) but can't use STAP for types 1,2,3?

Answer 1

You can use STAP packets for aggregating both VCL and Non-VCL NALUs that have the same presentation time, which should be the case if they are part of the same frame.

You're encoder should be providing a series of NALUs for the frame and they should have the same presentation time.

I've worked with encoders that produced a NALU byte stream containing all NALUs for the frame. This byte stream was assigned a single presentation time. I've also seen encoders that produced individual NALUs, which multiple would have the same presentation time if they were part of the same frame.