FFMPEG library- transcode raw image to h264 stream, and the output file does not contains pts and dts info

Question

I am trying using ffmpeg c++ library to convert several raw yuyv image to h264 stream, the image come from memory and passed as string about 24fps, i do the convention as the following steps:

init AVFormatContext,AVCodec,AVCodecContext and create new AVStream. this step i mainly refer to ffmpeg-libav-tutorial, and AVFormatContext use customize write_buffer() function(refer to simplest_ffmpeg_mem_handler)
receive raw frame data, set width and height(1920x1080), and set pts and dts. here i manually set the output fps to 24, and use a global counter to count num of frames, and the pts is calculated by this counter, code snippet(video_avs is AVStream,output_fps is 24 and time_base is 1/24):

    input_frame->width = w;  // 1920
    input_frame->height = h;  // 1080
    input_frame->pkt_dts = input_frame->pts = global_pts;
    global_pts += video_avs->time_base.den/video_avs->time_base.num / output_fps.num * output_fps.den;

convert it from yuyv to yuv422(because h264 does not support yuyv) and resize it from 1920x1080 to 640x480(because i need this resolution output), use sws_scale()
use avcodec_send_frame() and avcodec_receive_packet() to get the output packet. set output_packet duration and stream_index, then use av_write_frame() to write frame data.

    AVPacket *output_packet = av_packet_alloc();
    int response = avcodec_send_frame(encoder->video_avcc, frame);
    while (response >= 0) {
        response = avcodec_receive_packet(encoder->video_avcc, output_packet); // !! here output_packet.size is calculated
        if (response == AVERROR(EAGAIN) || response == AVERROR_EOF) {
            break;
        } 
        else if (response < 0) {
            printf("Error while sending packet to decoder");  // ??av_err2str(response)会报错
            return response;
        }

        // duration = next_pts - this_pts = timescale / fps = 1 / timebase / fps
        output_packet->duration = (encoder->video_avs->time_base.den / encoder->video_avs->time_base.num) / (output_fps.num / output_fps.den);
        output_packet->stream_index = 0;
        int response = av_write_frame(encoder->avfc, output_packet);  // packet order are not ensure
        if (response != 0) { printf("Error %d while receiving packet from decoder", response); return -1;}   
    }
    av_packet_unref(output_packet);
    av_packet_free(&output_packet);

in write_buffer() function, video stream output is stored to string variable, and then i write this string to file with ostream, and suffix mp4.

after all the above steps, the output.mp4 cannot be played, the ffprobe output.mp4 -show_frames output is (image):

Input #0, h264, from '/Users/ming/code/dev/haomo/output.mp4':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High 4:2:2), yuv422p(progressive), 640x480, 24.92 fps, 24 tbr, 1200k tbn, 48 tbc
[FRAME]
media_type=video
stream_index=0
key_frame=1
pkt_pts=N/A
pkt_pts_time=N/A
pkt_dts=N/A
pkt_dts_time=N/A
best_effort_timestamp=N/A
best_effort_timestamp_time=N/A

Note that before and after calling av_write_frame() in step 4, the passed argument output_packet contains correct pts and dts info, i cannot figure out why the output stream lost these info.

zhumm zhumm · Accepted Answer · 2020-11-05T02:31:19

I figure it out, the output stream is a raw h264 stream, and I directly store this stream into a file with a suffix of ".mp4", so it is actually not a correct mp4 file.

Then it store the stream into output.h264 file, and use ffmpeg to convert it to mp4 file: ffmpeg -framerate 24 -i output.h264 -c copy output.mp4, finally this output.mp4 contains right pts data and can be played.

FFMPEG library- transcode raw image to h264 stream, and the output file does not contains pts and dts info

1 Answers