FFMpeg, libav, Problem with sws_scale and yuv to rgb conversion - colors are not correct

Question

I've been trying to create my own video player in Delphi, and ffmpeg works great, until i hit a snag with color spaces, or color conversion.

Been trying everything for a couple of days, but can't figure it out. I'm using AV_PIX_FMT_RGB48LE as output...

Old context creation:

      videoConvContext := sws_getContext(AVStreamInit.codec.width, AVStreamInit.codec.height, AVStreamInit.codec.pix_fmt, AVStreamInit.codec.width, AVStreamInit.codec.height, AV_PIX_FMT_RGB48LE, SWS_POINT, nil, nil, nil);

New context creation:

    dstRange := 1;
    srcRange := 1;
    if AVStreamInit.codec.color_range = AVCOL_RANGE_MPEG then
      srcRange := 0;

    videoConvContext := sws_alloc_context();

    av_opt_set_int(videoConvContext, 'sws_flags', SWS_POINT or SWS_PRINT_INFO, 0);

    av_opt_set_int(videoConvContext, 'srcw', AVStreamInit.codec.width, 0);
    av_opt_set_int(videoConvContext, 'srch', AVStreamInit.codec.height, 0);
    av_opt_set_int(videoConvContext, 'src_format', Integer(AVStreamInit.codec.pix_fmt), 0);

    av_opt_set_int(videoConvContext, 'dstw', AVStreamInit.codec.width, 0);
    av_opt_set_int(videoConvContext, 'dsth', AVStreamInit.codec.height, 0);
    av_opt_set_int(videoConvContext, 'dst_format', Integer(AV_PIX_FMT_RGB48LE), 0);

    i1 := sws_getCoefficients2(Integer(AVStreamInit.codec.colorspace));
    i2 := sws_getCoefficients2(SWS_CS_ITU709);

    ret := sws_setColorspaceDetails2(videoConvContext, i1, srcRange, i2, dstRange, 0, 1 shl 16, 1 shl 16);

    sws_init_context(videoConvContext, nil, nil);

Results are exactly the same....

Output from sws_scale does not have correct colors, here are 2 screenshot

ok - from mpc-hc

not ok - from my player via sws_scale (direct raw extract so no color processing done) not ok

This video has pix_fmt AV_PIX_FMT_YUV420P10LE, but it happens on other YUV inputs, but not so obvious. Also on other RGB outputs(8bit etc)... Here the Netflix logo is pure red, but sws_scale output gives orange-ish...

Sws_scale code just for reference:

sws_scale(videoConvContext, @AVPacketBuffer.AVFrameCopy.Data, @AVPacketBuffer.AVFrameCopy.linesize, 0, AVStream.codec.height, @BitmapXBuffer.data, @linesize);

BitmapXBuffer.data is a signle buffer of correct size...

AVFrameCopy - is a copy of original frame done with:

  if Assigned(AVPacketBuffer.AVFrameCopy) then
      av_frame_free(@AVPacketBuffer.AVFrameCopy);

    AVPacketBuffer.AVFrameCopy := av_frame_alloc();
    AVPacketBuffer.AVFrameCopy.format := AVPacketBuffer.AVFrame.format;
    AVPacketBuffer.AVFrameCopy.width := AVPacketBuffer.AVFrame.width;
    AVPacketBuffer.AVFrameCopy.height := AVPacketBuffer.AVFrame.height;
    AVPacketBuffer.AVFrameCopy.channels := AVPacketBuffer.AVFrame.channels;
    AVPacketBuffer.AVFrameCopy.channel_layout := AVPacketBuffer.AVFrame.channel_layout;
    AVPacketBuffer.AVFrameCopy.nb_samples := AVPacketBuffer.AVFrame.nb_samples;
    av_frame_get_buffer(AVPacketBuffer.AVFrameCopy, 32);
    av_frame_copy(AVPacketBuffer.AVFrameCopy, AVPacketBuffer.AVFrame);
    av_frame_copy_props(AVPacketBuffer.AVFrameCopy, AVPacketBuffer.AVFrame);

Thanks!

Shot out of the dark (I'm working on something similar but still don't understand half of your code): Maybe two color channels get switched. Did you try AV_PIX_FMT_BGR48LE instead? But since you tried everything this is probably not the right answer... — Renate Schaaf
Sorry no, rgb values for on are like fe0000, 0000 on bg always, and on bad one are like ce5000, no ligic i could find :) — Milika

Milika Milika · Accepted Answer · 2020-07-23T13:55:26

To answer my own question - above code is correct, what confused me was dst_table, but it is relevant only if destination color space is YUV.

Also Netflix logo is orangeish, but not that correct orange. Converting from sRGB to Linear RGB solved it for me, but thats only in my case.

So, treat above code as an example :D

FFMpeg, libav, Problem with sws_scale and yuv to rgb conversion - colors are not correct

1 Answers