h.264 MPEG-TS to MP4 file

Question

I've been kind of going through an exercise trying to understand streaming a bit more. So please bear with my ignorance :)

I have mpeg-ts files with an h.264 stream within. I've been able to write a small program (in Java) following the ISO 13818-1 spec to extract PES packets. From there I can easily extract the PES packet data.

So my understanding is h.264 is mp4 but I suppose it isn't as easy as just chaining all the PES packet data into a file called *.mp4 to create a working mp4 file.

I'm not clear what actually the PES packet data is at this point.

But basically, I'm trying to get a handle on what that algorithm would basically look like to remux the data I have to an mp4 file.

Answer 1

H.264 is a video compression standard while MP4 is a container file format.

The packetized elementary stream (PES) encapsulates the elementary stream (ES) which in your case is a H.264 bitstream. There are multiple ways to format a H.264 bitstream.

Since your input is an MPEG-2 Transport Stream it uses the Annex B format. This means the bitstream is made of a sequence of network abstraction layer units (NALUs) prefixed with start codes.

The MP4 format uses a different format where each NALU is prefixed with a field containing its length.

You will need to convert the bitstream to the proper format and mux it into the new container. Take a look at this answer and find the MP4 standard for more information.