Solved: Python YU12 to RGB with CV2 failed

Question

I tried to get frames from a webcam and process them in python. The webcam tells me that it uses the YU12 codec. The unprocessed frame(1280x720) looks like: You should see in the picture a cup of coffee, my arm and my monitor in the background. For some reason the picture looks odd. Look at the pot handle.

If I try to convert it to RGB I get the following error:

cv2.error: OpenCV(4.1.2) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<1>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::::SizePolicy)1u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]' Invalid number of channels in input image: 'VScn::contains(scn)' where 'scn' is 3

import os
import sys
import cv2

videoSource = 0


def getFrame():
    """"""

    cv_cam_0 = cv2.VideoCapture(0)
    if not cv_cam_0.isOpened():
        raise Exception('video source: %s could not be opened' %(str(videoSource)))

    codec_char_code = int(cv_cam_0.get(cv2.CAP_PROP_FOURCC))
    a = chr(0x000000FF&  codec_char_code)
    b = chr((0x0000FF00& codec_char_code)  >> 8)
    c = chr((0x00FF0000& codec_char_code)  >> 16)
    d = chr((0xFF000000& codec_char_code)  >> 24)

    print('codec 4 char code: ' + a+b+c+d)

    ret, raw_frame = cv_cam_0.read()
    cv2.imwrite('/tmp/test0.jpg', raw_frame)
    rgbFrame = cv2.cvtColor(raw_frame, cv2.COLOR_YUV2RGB_I420)
    cv2.imwrite('/tmp/testConvert.jpg', rgbFrame)



def main(args):
    getFrame()
    sys.exit()


if __name__ == "__main__":
    main(sys.argv)

if I use mplayer the picture from the webcam looks fine. For debugging purpose output from mplayer:

Could not find matching colorspace - retrying with -vf scale... Opening video filter: [scale] Movie-Aspect is undefined - no prescaling applied. [swscaler @ 0x5638ca496560] bicubic scaler, from yuyv422 to yuv420p using MMXEXT [swscaler @ 0x5638ca496560] using unscaled yuyv422 -> yuv420p special converter VO: [xv] 1920x1080 => 1920x1080 Planar YV12 Selected video codec: [rawyuy2] vfm: raw (RAW YUY2)

Answer 1

Right know I have no idea what is wrong. What I did is to use v4l2-ctl maybe someone can give me a hint.

v4l2-ctl -d /dev/video2 --all

Format Video Capture:

Width/Height      : 1280/720
Pixel Format      : 'YUYV' (YUYV 4:2:2)
Field             : None
Bytes per Line    : 2560
Size Image        : 1843200
Colorspace        : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization      : Default (maps to Limited Range)
Flags             : 

Answer 2

I did some further investigation. For a test I changed the code to use pygame to grab images from the webcam. In short: the camera is working and shows beautiful images. For some reasons open video has some problems to decode the webcam frames. Maybe I miss some parameters but currently I do not know which parameters are missing.

import pygame
import pygame.camera

def getFrame():
    """"""
    pygame.init()
    pygame.camera.init()
    cam = pygame.camera.Camera('/dev/video2',(1280, 720))
    cam.start()
    screen = pygame.display.set_mode((1280, 720),0)

    while(True):
        image = cam.get_image()
        screen.blit(image,(0,0))
        pygame.display.flip()


 def main(args):
    getFrame()
    sys.exit()


if __name__ == "__main__":
    main(sys.argv)

Answer 3

Good news, problem is solved. Open video selects a wrong decoder. In my case "YU12" but the webcam uses: YUYV. I have to set it manually - (Function: set(cv2.CAP_PROP_FOURCC, fourcc)). Working code is below:

import os
import sys
import cv2

videoSource = 0

def getFrame():
    """"""


    cv_cam_0 = cv2.VideoCapture(videoSource)
    if not cv_cam_0.isOpened():
        raise Exception('video source: %s could not be opened' %(str(videoSource)))

    cv_cam_0.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
    cv_cam_0.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)

    fourcc = cv2.VideoWriter_fourcc(*'YUYV')
    ret = cv_cam_0.set(cv2.CAP_PROP_FOURCC, fourcc)

    codec_char_code = int(cv_cam_0.get(cv2.CAP_PROP_FOURCC))
    a = chr(0x000000FF&  codec_char_code)
    b = chr((0x0000FF00& codec_char_code)  >> 8)
    c = chr((0x00FF0000& codec_char_code)  >> 16)
    d = chr((0xFF000000& codec_char_code)  >> 24)

    print('codec 4 char code: ' + a+b+c+d)


    #ret, raw_frame = cv_cam_0.read()
    ret = cv_cam_0.grab()
    ret, raw_frame = cv_cam_0.retrieve()

    cv2.imwrite('/tmp/testRaw.png', raw_frame)

def main(args):
    getFrame()
    sys.exit()


if __name__ == "__main__":
    main(sys.argv)