Creating cv2.VideoCapture() object directly from numpy array image data

Question

The purpose is to take data from a virtual camera (from a camera in Gazebo simulation, updating every second) and use Detectron2 (requires data come from cv2.VideoCapture) to recognize other objects in the simulation. The virtual camera of course does not appear in lspci so I can't simply use cv2.VideoCapture(0).

So my code is

bridge = CvBridge()
cv_image = bridge.imgmsg_to_cv2(data, desired_encoding='bgr8') #cv_image is numpy.ndarray, size (100,100,3)
cap = cv2.VideoCapture()
ret, frame = cap.read(image=cv_image)
print(ret, frame)

but it just prints False None, I assume because there's nothing being captured in cap. I

f I replace line 2 with cap = cv2.VideoCapture(cv_image) I get the error,

TypeError: only size-1 arrays can be converted to Python scalars

since I believe it requires either and integer (representing webcam number) or string (representing video file).

And for reference,

cv_image = bridge.imgmsg_to_cv2(data, desired_encoding='bgr8') # cv_image is numpy.ndarray
cv2.imshow('image', cv_image)
cv2.waitKey(1)

displays the image perfectly fine. Could there be a way to use imshow() or something similar as input for VideoCapture()?

However, cap = cv2.VideoCapture(cv2.imshow('image', cv_image))opens a blank window and gives me,

[ERROR:0] global /io/opencv/modules/videoio/src/cap.cpp (116) open VIDEOIO(CV_IMAGES): raised OpenCV exception:

OpenCV(4.2.0) /io/opencv/modules/videoio/src/cap_images.cpp:293: error: (-215:Assertion failed) !_filename.empty() in function 'open'

How can I create a cv2.VideoCapture() object that can use the image data that I have? Or what's something that might point me in the right direction?

Ubuntu 18.04 and Python 3.6 with opencv-python 4.2.0.34

Victoria Kepler Victoria Kepler · Accepted Answer · 2020-05-26T13:35:42

From what I found on Gazebo tutorials page:

In Rviz, add a ''Camera'' display and under ''Image Topic'' set it to /rrbot/camera1/image_raw.

In your case it probably won't be /rrbot/camera1/ name, but the one you are setting in .gazebo file

<cameraName>rrbot/camera1</cameraName>
<imageTopicName>image_raw</imageTopicName>
<cameraInfoTopicName>camera_info</cameraInfoTopicName>

So you can create subscriber and use cv2.VideoCapture() for every single image from that topic.

Creating cv2.VideoCapture() object directly from numpy array image data

2 Answers