OpenCV python 3.7 handle cv2.VideoCapture(0) asynchronously

Question

I am new to python and openCV. I hope this question makes sense. I want to use a asyncio/multithreading in python in order process cv2.VideoCapture(0) in openCV asynchronously.

The reason: I can only create one cv2.VideoCapture(0) object and I cannot copy it - as far as I know. Here the errors I got.

TypeError: cannot pickle 'cv2.VideoCapture' object

TypeError: 'cv2.VideoCapture' object is not subscriptable

First I want to show the video capturing by the PCs cam in a window:

cap = cv2.VideoCapture(0)

def one():
        while cv2.waitKey(1) < 0:
        hasFrame, frame = cap.read()
        cv2.imshow('Capturing1', frame)

While I process the same frames in a different function:


def two():
    while cv2.waitKey(1) < 0:
        hasFrame, frame = cap.read()
        frameWidth = frame.shape[1]
        frameHeight = frame.shape[0]

        inpBlob = cv2.dnn.blobFromImage(frame, 1.0 / 255, (inWidth, inHeight),
                                        (0, 0, 0), swapRB=False, crop=False)

        net.setInput(inpBlob)
        output = net.forward()

        H = output.shape[2]
        W = output.shape[3]
        # Empty list to store the detected keypoints
        points = []
        for i in range(nPoints):
            threshold = 0.1
            # confidence map of corresponding body's part.
            probMap = output[0, i, :, :]

            # Find global maxima of the probMap.
            minVal, prob, minLoc, point = cv2.minMaxLoc(probMap)

            # Scale the point to fit on the original image
            x = (frameWidth * point[0]) / W
            y = (frameHeight * point[1]) / H

            if prob > threshold:
                # cv2.circle(frameCopy, (int(x), int(y)), 8, (0, 255, 255), thickness=-1, lineType=cv2.FILLED)
                # cv2.putText(frameCopy, "{}".format(i), (int(x), int(y)), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2, lineType=cv2.LINE_AA)

                # Add the point to the list if the probability is greater than the threshold
                points.append((int(x), int(y)))
                print(points)
            else:
                points.append(None)

I want to do this, because the

 cv2.imshow('Capturing1', frame)

in def one() is slown down by the code in def two() if I would combine those two functions in one.

Many thanks for help. I hope it makes sense to you.

why do you want to copy VideoCapture ? Why not copy frame ? Better run VideoCapture in one process and send frame to other process using queue. — furas
always put full error message (starting at word "Traceback") in question (not comment) as text (not screenshot). There are other useful information. — furas
Thanks @furas for your comments. How can I run VideoCapture in one process and send frame to other process using queue? Can you provide an example pls. Not sure how to implement this. Like this: stackoverflow.com/questions/65646894/… — Anton Hörl

furas furas · Accepted Answer · 2021-01-10T18:23:51

You don't have to copy VideoCapture and probably you couldn't use two copies to read camera because only one VideoCapture can access camera.

You can use threads to run two or more processes at the same time - and because threads share memory so you don't have to use queue or pickle to send frame from one thread to another.

You can create one VideoCapture in main thread and use one thread to read frame and other threads to run few processes which create few frames, but GUI (displayng frame) you have to run in main thread.

import cv2
import threading
import time

# --- functions ---

def read_frame():
    global has_frame
    global frame

    while running:
        has_frame, frame = cap.read()
        #time.sleep(.1)  # 0.1s to use less CPU

def one():
    global frame1

    while running:
        if has_frame:
            # processing frame
            frame1 = frame.copy()
            frame1 = cv2.putText(frame1, "One: fast process", (10, 30), cv2.FONT_HERSHEY_PLAIN, 2, (255, 255, 255))
        time.sleep(.1)  # 0.1s to use less CPU

def two():
    global frame2
    
    while running:
        if has_frame:
            # processing frame
            frame2 = frame.copy()
            frame2 = cv2.putText(frame2, "Two: slow process", (10, 30), cv2.FONT_HERSHEY_PLAIN, 2, (255, 255, 255))
            # simulate long running code
            time.sleep(2)        
        time.sleep(.1)  # 0.1s to use less CPU

#--- main ---

# - init ---

# create variables at start with default values
has_frame = False 
frame = None   # original frame from camera

frame1 = None  # frame after processing
frame2 = None  # frame after processing

cap = cv2.VideoCapture(0)  # create only one access to camera

# - create threads -

t0 = threading.Thread(target=read_frame)
t1 = threading.Thread(target=one)
t2 = threading.Thread(target=two)

# - start threads -

running = True  # use in threads to stop loops

t0.start()
t1.start()
t2.start()

# - GUI has to be in main thread -

while cv2.waitKey(100) != 27:  # ESC  # 100ms to use less CPU

    if frame1 is not None:
        cv2.imshow('Capturing1', frame1)

    if frame2 is not None:
        cv2.imshow('Capturing2', frame2)

    #time.sleep(.1)  # 0.1s to use less CPU
    
# - stop threads -

running = False

t0.join()
t1.join()
t2.join()

# - quit -

cv2.destroyAllWindows()
cap.release()

EDIT: Author of cv2 created also class to run VideoCapture in separated Thread.

from imutils.video import WebcamVideoStream

See details in Increasing webcam FPS with Python and OpenCV no PyImageSearch.com

OpenCV python 3.7 handle cv2.VideoCapture(0) asynchronously

1 Answers