How can i mux H264 stream into MP4 file via libavformat

Question

I want to realize an application that firstly decode a multi-media file(such as test.mp4 file, video codec id is H264), get a video stream and an audio stream, then make some different in the audio stream, at last encode the video stream(use libx264) and audio stream into a result file(result.mp4). To promote the efficiency, i omitted the decode and encode of video stream, i get the video packet via function "av_read_frame", then output it directly into the result file via function "av_write_frame". But there is no picture in the output file, and the size of output file is fairly small.

I tracked the ffmpeg code and found that in the function "av_write_frame->mov_write_packet->ff_mov_write_packet", it will call function "ff_avc_parse_nal_units" to obtain the size of nal unit, but the return value is very small(such as 208 bytes).

I find that the H264 stream in the MP4 file is not stored in Annex-B format, so it can't find start code(0x000001）， now my problem is how can I change the H264 stream to Annex-B format, and make it work? I added start code at the beginning of every frame manually, but it still not work.

Anyone can give me any hint?Thanks very much. Following is the codes similar with my:

    // write the stream header, if any
    av_write_header(pFormatCtxEnc);

    .........
    /**
    * Init of Encoder and Decoder
    */

    bool KeyFlag = false;
    bool KeyFlagEx = false;
    // Read frames and save frames to disk
    int iPts = 1;
    av_init_packet(&packet);
    while(av_read_frame(pFormatCtxDec, &packet)>=0)
    {
            if (packet.flags == 1)
                    KeyFlag = true;

            if (!KeyFlag)
                    continue;

            if (m_bStop)
            {
                    break;
            }

            // Is this a packet from the video stream?
            if(packet.stream_index == videoStream)
            {
                    currentframeNum ++;
                    if (progressCB != NULL && currentframeNum%20 == 0)
                    {
                            float fpercent = (float)currentframeNum/frameNum;
                            progressCB(fpercent,m_pUser);
                    }

                    if (currentframeNum >= beginFrame && currentframeNum <= endFrane)
                    {
                            if (packet.flags == 1)
                                    KeyFlagEx = true;

                            if (!KeyFlagEx)
                                    continue;
                            packet.dts = iPts ++;
                            av_write_frame(pFormatCtxEnc, &packet);

                    }
            }
            // Free the packet that was allocated by av_read_frame
    }

    // write the trailer, if any
    av_write_trailer(pFormatCtxEnc);

    /**
    * Release of encoder and decoder
    */

    return true;

FFmpeg's MP4 multiplexer can be set to accept streams both with and without start codes. Initial NALs are typically parameter sets and are OK to be small. av_write_frame is the right thing to do. What is your code? Maybe someone will notice a problem. — Roman R.
Thanks, how to set the ffmpeg's MP4 multiplexer to accept streams without start code? Does it need to set some special parameters? The above is the codes similar with my. The return value of function "ff_avc_parse_nal_units" is almost 0 for every frame, and most frames output with nothing. — Brian

Alex I Alex I · Accepted Answer · 2013-01-06T04:05:56

You might try this: libavcodec/h264_mp4toannexb_bsf.c. It converts bitstream without start codes to bitstream with start codes.

Using your source file, does ffmpeg -i src.mp4 -vcodec copy -an dst.mp4 work? Does it work if you add -bsf h264_mp4toannexb? (all using the same version/build of ffmpeg as you are trying to use programmatically of course)

How can i mux H264 stream into MP4 file via libavformat

1 Answers