Matlab VideoReader reading frames as green images or just not at all

Question

I'm trying to read a video file and store the frames as a series of images. I'm using VideoReader but for some reason I'm having trouble. I want to store the frames of two videos encoded differently and measure the structural similarity and PSNR between the two on a frame-by-frame basis.

Essentially I've three video files, an original (which reads fine), one compressed with VP9 using ffmpeg, and one compressed with H.624 using ffmpeg. The original video was originally just a set of frames merged into a .avi video using VirtualDub. The compressed videos are also .avi container.

The VP9 video appeared to work fine but when I open the images using imshow() they appear to be just a solid green color. The video opens fine on VLC so I'm not sure what the problem is.

The H.264 video doesn't read at all. When it attempts to enter the "while hasFrame()" loop, it skips over it which leads to believe Matlab thinks the video frames aren't there? Again, this video opens fine in VLC and the three videos look almost identical.

Anyone got any ideas why this is happening? Is it to do with the way Matlab decodes the video or some parameters set by ffmpeg?

Original vs VP9 in Matlab

Console output for ffmpeg - VP9

Console output for ffmpeg - H264

Main file:

test_vid = 'vp9.avi';
images = readVideos(test_vid);

for i=1:length(images)

  % Convert from cells to matrices
  image1 = cell2mat(images(1,i));
  image2 = cell2mat(images(2,i));

  % Do stuff with the images here

  subplot(1,2,1);
  imshow(image1);
  subplot(1,2,2);
  imshow(image2);
end

ReadVideos():

function images = readVideos(test_video)

    % Video directories
    test_video_dir = strcat('src/', test_video);
    v_original = VideoReader('src/input.avi');
    v_test = VideoReader(test_video_dir);

    % Read original video
    i = 1;
    v_original.CurrentTime = 5;
    while hasFrame(v_original)
        frame = readFrame(v_original);
        originalImages{i} = frame;
        i = i + 1;
    end

    % Read test video
    i = 1;
    v_test.CurrentTime = 5;
    while hasFrame(v_test)
         frame = readFrame(v_test);
         testImages{i} = frame;
         i = i + 1;
    end

    images = cat(1, originalImages, testImages);
end

On a side note, is Matlab the best choice for the task or is there specialised software out there for doing this?

Gyan Gyan · Accepted Answer · 2019-05-02T13:32:26

Matlab's Videoreader relies, for the most part, on the decoders provided by the OS. In general, these decoders have limited support for streams with pixel formats other than the most common ones. That appears to be the case here.

Your VP9 output is gbrp i.e. planar (8-bit) RGB. Your H.264 output is YUV but with 4:4:4 chroma sampling. Both of these are non-standard.

To read these files with Videoreader, convert to a standard pixel format:

ffmpeg -i input.avi -pix_fmt yuv420p -c:v libvpx-vp9 vp9.avi

and

ffmpeg -i input.avi -pix_fmt yuv420p -c:v libx264 h264.avi

However, you may encounter issues when carrying out quality metric comparisons since they require both operands to have the same pixel format. Matlab may or may not convert them automatically.

You could just perform the quality check in ffmpeg.

ffmpeg -i input.avi -pix_fmt gbrp -c:v libvpx-vp9 vp9.mkv

and

ffmpeg -i input.avi -pix_fmt gbr24 -c:v libx264rgb h264.mkv

And then

ffmpeg -i input.avi -i vp9.mkv -i h264.mkv -filter_complex "[1][0]psnr=f=vp9-psnr.txt:stats_version=2;[1][0]ssim=f=vp9-ssim.txt;[2][0]psnr=f=h264-psnr.txt:stats_version=2;[2][0]ssim=f=h264-ssim.txt" -f null -

See http://www.ffmpeg.org/ffmpeg-filters.html#psnr and http://www.ffmpeg.org/ffmpeg-filters.html#ssim

Matlab VideoReader reading frames as green images or just not at all

1 Answers