FFmpeg decoding H264

2
votes

I am decoding a H264 stream using FFmpeg on the iPhone. I know the H264 stream is valid and the SPS/PPS are correct as VLC, Quicktime, Flash all decode the stream properly. The issue I am having on the iPhone is best shown by this picture.

enter image description here

It is as if the motion vectors are being drawn. This picture was snapped while there was a lot of motion in the image. If the scene is static then there are dots in the corners. This always occurs with predictive frames. The blocky colors are also an issue.

I have tried various build settings for FFmpeg such as turning off optimizations, asm, neon, and many other combinations. Nothing seems to alter the behavior of the decoder. I have also tried the Works with HTML, Love and Peace releases, and also the latest GIT sources. Is there maybe a setting I am missing, or maybe I have inadvertently enabled some debug setting in the decoder.

Edit

I am using sws_scale to convert the image to RGBA. I have tried various different pixel formats with the same results.

sws_scale(convertCtx, (const uint8_t**)srcFrame->data, srcFrame->linesize, 0, codecCtx->height, dstFrame->data, dstFrame->linesize);

I am using PIX_FMT_YUV420P as the source format when setting up my codec context.

1
iphoneffmpegh.264
do you start the decoding from keyframes? - alex-i
That is a good question. I do believe so, but I will double check. I know if I configure the encoder to only generate key frames then all I get is a blocky image. No arrows as I would expect. - Steve McFarlin
I confirmed the stream is starting with a key frame. I thought maybe my FU-A RTP packet assembler might have been bad, but I just check it again and it is correct. - Steve McFarlin

1 Answers

3
votes

What you're looking at is ffmpeg's motion vector visualization. Make sure that none of the following debug flags are set:

avctx->debug & FF_DEBUG_VIS_QP
avctx->debug & FF_DEBUG_VIS_MB_TYPE
avctx->debug_mv

Also, keep in mind that decoding H264 video using the CPU will be MUCH slower and less power-efficient on iOS than using the hardware decoder.