ffmpeg poor YUV to RGB in sws_scale() (was: ffmpeg blocky chroma decoding)

2
votes

I'm using ffmpeg-3.2.4-win32, static and dev versions from zeranoe page, to decode some h264 mp4 videos I have created with x264 encoder. The videos have lots of static zones, 2d videogame type.

When I decode their frames using either ffmpeg.exe or avcodec_decode_video2() API function, I get poor chroma resolution, blocky and quite visible in the static zones of the video (edit: avcodec_decode_video2() returns correct YUV, the problem is the YUV->RGB conversion, see my own answer below).

With ffmpeg.exe I get poor chroma only if I output the frames to png, but if I output them to jpg, chroma seems much better (edit: this is due to poor YUV to RGB conversion of ffmpeg's sws_scale()).

Here is an example of the same frame obtained from a mp4 video created with a static image, the output frame is zoomed in to clearly see the effect.

https://drive.google.com/file/d/0B5KI1D-N1kHpV3lGaERJRjNkcms/view

1: Original frame before compressing.

2: YUV 4:2:0 conversion before compressing

3: Video output from MPC HC + LAV decoder. This is YUV decoded output.

4: Video frames decoded with ffmpeg.exe to jpeg, quality is mostly ok, just some jpeg macroblocking is visible.

5: Video frames decoded with ffmpeg.exe to png, very blocky chroma. This is RGB output, using avcodec_decode_video2() and sws_scale() conversion to RGB produces the same blocky chroma.

The command lines for decoding the frames to jpg and png are these:

ffmpeg -i testcase.mp4 -vframes 5 -qscale:v 2 output%03d.jpg

ffmpeg -i testcase.mp4 -vframes 5 output%03d.png

You can download mp4 and bat files that showcase the effect here:

https://drive.google.com/drive/folders/0B5KI1D-N1kHpcUdGd2IyM2pXMTg

I have tried with ffmpeg 3.2, but same thing happens.

1
videoffmpegrgbyuv

1 Answers

5
votes

After doing more tests, I have narrowed the issue to the YUV -> RGB conversion of sws_scale().

I was wrong in my first explanation, since the YUV output of ffmpeg avcodec_decode_video2() is OK, that's why external players, ffplay and the jpeg output is OK too. The RGB output is what is wrong.

For YUV to RGB with sws_scale() I have tried destination formats AV_PIX_FMT_RGB24 and AV_PIX_FMT_BGRA and tried the flags SWS_BILINEAR, SWS_FAST_BILINEAR, SWS_POINT with same bad results.

Fix:

Ok, I found the fix in an old ffmpeg bugtrack at https://trac.ffmpeg.org/ticket/1582

In order to do good quality YUV to RGB conversion, you have to add in ffmpeg.exe:

-sws_flags full_chroma_int+accurate_rnd

Or in ffmpeg API sws_getContext() the flags:

SWS_FULL_CHR_H_INT | SWS_ACCURATE_RND

With these, the RGB output is OK.