Speed up reading video frames from camera with Opencv

Question

Currently, i have my application implemented in Python to read camera frames via USB. I use Opencv, but for camera with 60fps and 1920x1080 resolution, i could not capture the this speed. I try something: cv2.VideoCapture(0, cv2.CAP_FFMPEG), but i could not the frames (capture read returns False). Could you please suggest any solutions or ways to improve the overall speed or even processing them in GPU.

    # Opencv 3.4.9 & Python 3.6.9
    import cv2    
    capture = cv2.VideoCapture(0, cv2.CAP_FFMPEG)

    while True:
        ret, frame = capture.read()
        if ret:
            cv2.imshow('Demo', frame)
            cv2.waitKey(1)

Opencv is built from source:

#!/usr/bin/env bash

set -e

VERSION='3.4.9'
DOWNLOAD_URL="https://github.com/opencv/opencv/archive/${VERSION}.zip"
CONTRIB_DOWNLOAD_URL="https://github.com/opencv/opencv_contrib/archive/${VERSION}.zip"
CURRENT_DIR="$(pwd)"
config_opencv_with_cuda() {
  cmake -D CMAKE_BUILD_TYPE=RELEASE \
        -D CMAKE_INSTALL_PREFIX=/usr/local \
        -D INSTALL_PYTHON_EXAMPLES=OFF \
        -D WITH_CUDA=ON \
        -D ENABLE_FAST_MATH=1 \
        -D CUDA_FAST_MATH=1 \
        -D WITH_CUBLAS=1 \
        -D WITH_V4L=ON \
        -D WITH_FFMPEG=ON \
        -D WITH_GSTREAMER=ON \
        -D BUILD_opencv_cnn_3dobj=OFF \
        -D BUILD_opencv_dnn_modern=OFF \
        -D OPENCV_EXTRA_MODULES_PATH=$CURRENT_DIR/opencv_contrib-$VERSION/modules \
        -D BUILD_DOCS=OFF \
        -D BUILD_PERF_TESTS=OFF \
        -D BUILD_TESTS=OFF \
        -D BUILD_EXAMPLES=OFF ..
}
echo 'Uninstalling pip installation'
pip3 uninstall opencv-python opencv-contrib-python

echo 'Installing dependencies'
apt-get update && apt-get install -y \
  build-essential cmake pkg-config \
  libjpeg-dev libtiff5-dev libpng12-dev \
  libavcodec-dev libavformat-dev libswscale-dev libv4l-dev \
  libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
  libxvidcore-dev libx264-dev \
  libgtk2.0-dev libgtk-3-dev \
  gfortran

wget -O opencv.zip "${DOWNLOAD_URL}"
wget -O opencv_contrib.zip "${CONTRIB_DOWNLOAD_URL}"
unzip opencv.zip && unzip opencv_contrib.zip
cd opencv-$VERSION
mkdir build
cd build

if [[ -d /usr/local/cuda ]]; then
  config_opencv_with_cuda
else
  config_opencv_without_cuda
fi

make -j $(nproc)
make install
ldconfig

can you try to remove imshow and waitkey and just count frames and seconds? With 60 fps it might be necessary to use a faster and asynchronous image displaying. What kind of device do you run the code on? You only have 17 ms for all the processing of one frame! — Micka
@Micka I know about imshow effect on this example, but i want to try to change to ffmpeg backend for opencv, but what i got is VideoCapture always return False and i could not get the frame with cv2.VideoCapture(0, cv2.CAP_FFMPEG) — Tuyen Vo Quang
sorry, dont know whether ffnpeg videocapture should work in all cases. Currently your question title is misleading. — Micka

Ahx Ahx · Accepted Answer · 2020-09-19T18:36:21

The reason you cannot capture this speed is that VideoCapture blocks the application, while the next frame is being read, decode, and returned.

If I'm not mistaken you are reading a movie file from your USB.

You can use FileVideoStream object for speeding up your frames. FileVideoStream uses queue structure for reading, decoding, and returning. As a result application is not blocked during the processing.

You need to install imutils

pip: pip install imutils
anaconda: conda install -c conda-forge imutils

Example usage:

import cv2
from imutils.video import FileVideoStream as Fvs

fvs = Fvs(path="usb.mp4").start()

four_cc = cv2.VideoWriter_fourcc(*"mp4v")
writer = cv2.VideoWriter("out.mp4", four_cc, 60,
                         (1920, 1080), isColor=True)

while True:
    frame = fvs.read()

    # You write your processing code below here
    

    # If you are sure each usb frame size is (1920, 1080) comment the below state.
    frame = cv2.resize(frame, (1920, 1080))

    writer.write(frame)
    cv2.imshow("out", frame)

    if cv2.waitKey(0) & 0xFF == ord("q"):
        break

cv2.destroyAllWindows()
fvs.stop()
writer.release()

Speed up reading video frames from camera with Opencv

1 Answers