Including SPS and PPS in a raw h264 track

2
votes

Should a raw h.264 video track contain at the start SPS and PPS data in order for a player (at least VLC Player) to be able to play it properly?

In this Android question, I see that in this specific context - MediaCodec decoder on Android API - it is said that the two buffers need to be provided. Is this generalized. Do I provide it as 0,0,0,1,[sps],0,0,0,1,[pps]?

[EDIT]

I've added SPS and PPS as is, and now VLC Player wants to play the video. It seems to decode the proper amount of frames at the good framerate, though the frames are pink and green noise, with movements that seem to follow original movie movements. I feel like there are missing informations regarding the format of my video track data. When I demux an MP4 with FFmpeg, the provided sps at the beginning of the raw h.264 stream is richer than mine.

FFMPEG h.264 raw video track stream:

SPS 00 00 00 01 67 42 c0 0d 9a 74 03 c0 11 3f 2e 02 20 00 00 03 00 20 00 00 06 51 e2 85 54

PPS 00 00 00 01 68 ce 3c 80

The rest... 00 00 00 01 65 etc....

1
h.264
What are you using to stream the raw H264 stream? A NAL unit may not actually start with the 4 bytes 00 00 00 01, it could just be three bytes 00 00 01 especially if you are using FFMPEG to create the raw H264 stream. FFMPEG will automatically spit out the SPS and PPS information into NAL units for you. Also if you are streaming with something like Live555 you need to strip the leading and trailing NAL units (00 00 00 01 or 00 00 01), and I think you may even need to terminate with a zero too? can't remember exactly. – ALM865
Also H264 streams NAL units are usually capped at some arbitrary maximum size (like 200000 bytes). You need to tell your encoder to create NAL units smaller than whatever the limit in the program that you are using the stream is (VLC is 200000 bytes max, Live555 is something like 300000?). Anyways if you don't do that you'll start dropping frames at the other end and start seeing weird stuff happen – ALM865
I'm using Android MediaCodec. When feeding the encoder with images, at the beginning of the process, I'm receiving an array with 0,0,0,1,67,etc,0,0,0,1,68,etc, then it starts to outputs data starting with 0,0,0,1,65. – Léon Pelletier
You need to keep sending the sps /pps data, I'm not sure how FFMPEG does it but I guess sending the sps / pps data every 'I frame' wouldn't hurt if you can. The thing is with streaming is that unless you're using multicast, you don't actually send any data until someone connects, so if you don't keep sending the sps / pps information the streamer (whether it is live555 or something else) it won't be able to send any details of the stream with the stream. hope that makes sense – ALM865
Also I'm sure you've already checked this but most H264 codecs only support frames in YUV format, so if you are using RGB without converting them to YUV first then you could get funny things like greenish/purple movies. It's a slim chance that that's the problem but it worth looking into I guess... – ALM865

1 Answers

1
votes

which library are you using to get stream ? for example if you use live555 to get stream you need to put sps and pps information before frames because ffmpeg wants these information for decoding.

00 00 00 01 for h264 format

00 00 00 01 sps 00 00 00 01 pps 00 00 00 01 data frame