Screen Capture with OpenCV and Python-2.7

Question

I'm using Python 2.7 and OpenCV 2.4.9.

I need to capture the current frame that is being shown to the user and load it as an cv::Mat object in Python.

Do you guys know a fast way to do it recursively?

I need something like what's done in the example below, that captures Mat frames from a webcam recursively:

import cv2

cap = cv2.VideoCapture(0)
while(cap.isOpened()):
    ret, frame = cap.read()
    cv2.imshow('WindowName', frame)
    if cv2.waitKey(25) & 0xFF == ord('q'):
        cap.release()
        cv2.destroyAllWindows()
        break

In the example it's used the VideoCapture class to work with the captured image from the webcam.

With VideoCapture.read() a new frame is always being readed and stored into a Mat object.

Could I load a "printscreens stream" into a VideoCapture object? Could I create a streaming of my computer's screen with OpenCV in Python, without having to save and delete lots of .bmp files per second?

I need this frames to be Mat objects or NumPy arrays, so I can perform some Computer Vision routines with this frames in real time.

It's hard to tell what your question actually is. Could you edit your question to clarify exactly what it is? For example, what do you mean 'recursively'? frame already contains the image. Why don't you just use it directly? — Aurelius
@Aurelius From What I understand, he's just looking for a way to inject printscreens as frames instead of images from the webcam. Since the monitor is not included in the device list taken by cv2.VideoCapture, You just need to grab the printscreen from elsewhere, such as PIL Image.Imagegrab.grab(), convert it to a numpy array, and inject it in the code shown above as a frame... — Raoul

Phani Rithvij Phani Rithvij · Accepted Answer · 2020-05-29T08:07:41

This is the updated answer for the answer by @Neabfi

import time

import cv2
import numpy as np
from mss import mss

mon = {'top': 160, 'left': 160, 'width': 200, 'height': 200}
with mss() as sct:
    # mon = sct.monitors[0]
    while True:
        last_time = time.time()
        img = sct.grab(mon)
        print('fps: {0}'.format(1 / (time.time()-last_time)))
        cv2.imw('test', np.array(img))
        if cv2.waitKey(25) & 0xFF == ord('q'):
            cv2.destroyAllWindows()
            break

And to save to a mp4 video

import time

import cv2
import numpy as np
from mss import mss


def record(name):
    with mss() as sct:
        # mon = {'top': 160, 'left': 160, 'width': 200, 'height': 200}
        mon = sct.monitors[0]
        name = name + '.mp4'
        fourcc = cv2.VideoWriter_fourcc(*'mp4v')
        desired_fps = 30.0
        out = cv2.VideoWriter(name, fourcc, desired_fps,
                              (mon['width'], mon['height']))
        last_time = 0
        while True:
            img = sct.grab(mon)
            # cv2.imshow('test', np.array(img))
            if time.time() - last_time > 1./desired_fps:
                last_time = time.time()
                destRGB = cv2.cvtColor(np.array(img), cv2.COLOR_BGRA2BGR)
                out.write(destRGB)
            if cv2.waitKey(25) & 0xFF == ord('q'):
                cv2.destroyAllWindows()
                break


record("Video")

Screen Capture with OpenCV and Python-2.7

3 Answers