OpenCV (cv2 in Python) VideoCapture not releasing camera after deletion

Question

I am relatively new to Python, just having learnt it over the past month or so and have hacked this together based off examples and others' code I found online.

I have gotten a Tkinter GUI to display the feed from a webcam as a loop of continuously updated images on a canvas. Quitting the GUI and re-running the script every other time results in this error:

Exception in Tkinter callback
Traceback (most recent call last):
    File "C:\Python27\lib\lib-tk\Tkinter.py", line 1410, in __call__
        return self.func(*args)
    File "C:\Python27\lib\lib-tk\Tkinter.py", line 495, in callit
        func(*args)
   File "C:\...\cv2_cam_v8.py", line 20, in update_video
        (self.readsuccessful,self.f) = self.cam.read()
SystemError: NULL object passed to Py_BuildValue

When the error happens no images get read and the videofeed recieves no images to update the canvas. The script runs normally with no errors the first time and every second time. From previous tests with the VideoCapture function in the cv2 module, I found that I had to delete the camera object to release it so that subsequent runs are able to capture the camera stream with no issue. Checks on the namespace by typing who in the console do not show cam so I know it is being deleted properly after the GUI is closed. I do not understand why cv2's read function is giving an error. I think it is only happening every second time because when the error occurs, some garbage collection or error handling deletes or frees up something to do with the camera but I do not know what this is...

Here is my code:

import cv2
import Tkinter as tk
from PIL import Image, ImageTk


class vid():      
    def __init__(self,cam,root,canvas):
        self.cam = cam
        self.root = root
        self.canvas = canvas

    def update_video(self):
        (self.readsuccessful,self.f) = self.cam.read()
        self.gray_im = cv2.cvtColor(self.f, cv2.COLOR_RGB2GRAY)
        self.a = Image.fromarray(self.gray_im)
        self.b = ImageTk.PhotoImage(image=self.a)
        self.canvas.create_image(0,0,image=self.b,anchor=tk.NW)
        self.root.update()
        self.root.after(33,self.update_video)


if __name__ == '__main__':
    root = tk.Tk()
    videoframe = tk.LabelFrame(root,text='Captured video')
    videoframe.grid(column=0,row=0,columnspan=1,rowspan=1,padx=5, pady=5, ipadx=5, ipady=5)
    canvas = tk.Canvas(videoframe, width=640,height=480)
    canvas.grid(column=0,row=0)
    cam = cv2.VideoCapture(2)
    x = vid(cam,root,canvas)
    root.after(0,x.update_video)
    button = tk.Button(text='Quit',master=videoframe,command=root.destroy)
    button.grid(column=0,row=1)
    root.mainloop()
    del cam

Refactoring the code like this:

def update_video(cam,root,canvas):
    (readsuccessful,f) = cam.read()
    gray_im = cv2.cvtColor(f, cv2.COLOR_RGB2GRAY)
    a = Image.fromarray(gray_im)
    b = ImageTk.PhotoImage(image=a)
    canvas.create_image(0,0,image=b,anchor=tk.NW)
    root.update()
    root.after(33,update_video(cam,root,canvas))

if __name__ == '__main__':
    root = tk.Tk()
    videoframe = tk.LabelFrame(root,text='Captured video')
    videoframe.grid(column=0,row=0,columnspan=1,rowspan=1,padx=5, pady=5, ipadx=5, ipady=5)
    canvas = tk.Canvas(videoframe, width=640,height=480)
    canvas.grid(column=0,row=0)
    cam = cv2.VideoCapture(2)
    root.after(0,update_video(cam,root,canvas))
    button = tk.Button(text='Quit',master=videoframe,command=root.destroy)
    button.grid(column=0,row=1)
    root.mainloop()
    del cam

does not display the button in the GUI and gives this error after closing the window:

RuntimeError: Too early to create image

I have 3 questions

1 - How can I prevent either exception? UPDATE: changing "root.after(0,update_video(cam,root,canvas))" to "root.after(0,lambda: update_video(cam,root,canvas))" and "update_video(cam,root,canvas)" to "update_video(cam,root,canvas,event=None)" OR passing the arguments to the callback using this format: "root.after(time_to_wait, callback, arguments, master)" fixes the second error (and others I did not post). Also as kobejohn pointed out, adding a try: except block also fixes the second error. Please see his answer for more details.

2 - Is there a faster, more efficient function than .read() in cv2? Edit: Is there a way to refactor my code to get higher framerates? The read function is the only one listed in the docs and I just read somewhere that if it is not in the docs, then it is not available. This method only gives me about 5fps, where 10-20fps would be much more acceptable. UPDATE: From the discrepancies between kobejohn's tests and mine with different cameras, the low framerate is a result of poor quality webcams. Better quality webcams yield higher framerates.

3 - I have been reading that update() should be avoided as much as possible but how do I get the canvas to redraw the image otherwise (or implement update_idletasks() with this code)?. Do I have to implement some sort of threading or can I avoid that? UPDATE: I have gotten the code to work without using the update() method but have to look at implementing threading anyway because when I start recording the videofeed from a button the main GUI, it freezes/ becomes unresponsive.

The finished program will be used in Ubuntu and windows (possibly on macs as well). I am running Windows 7, IDE is Spyder 2.1.11 (Python 2.7.3).

Thank you in advance, any advice and/or solutions will be much appreciated!

Regards,

S. Chia

Answer 1

Solved! OpenCV 2.4.2/ cv2 in python

For some strange reason, I could not find the 'release' method before and other forums, pages specifically mentioned that the python bindings to opencv did not include the release method. Perhaps this only applied when using 'import cv'. I did my initial prototyping using the latter and for some reason missed the 'release' method in cv2 when I was looking for a ReleaseCapture method.

Just found it in the docs: http://docs.opencv.org/modules/highgui/doc/reading_and_writing_images_and_video.html

import cv2

cam=cv2.VideoCapture(0)
cam.release

Answer 2

Can you try this code and see what FPS you get? I included an FPS calculation so we can compare notes. (edit: also what errors. I didn't get the errors you got in the original code and I get zero errors with the code below)

I started from scratch just to see if I came up with something different. There are a few differences:

1. There was a (minor?) bug: the opencv default color channels are BGR rather than RGB. So change your grascale conversion from cv2.COLOR_RGB2GRAY --> cv2.COLOR_BGR2GRAY. You can see in the VideoCapture example they do something similar.
2. I used a simple label to display the image instead of a canvas. I haven't used the canvas before, so I'm not sure what you need to do with it. With a simple label, you have to keep a reference to the image you are displaying so it doesn't get garbage collected. You can see that in update_image().
3. For callbacks, I used lambdas with arguments (as you mentioned in your comment). Otherwise, when you make a function call with arguments, you are running the callback immediately instead of registering it. Ends up looking like it is working, but it's not doing quite what you would think. Alternately, you can use functools.partial if you prefer to package up your arguments and send that as an uncalled function.
4. Also for the callback, I added a try: except block for the case that the callback starts running after root has been destroyed. I don't know if this is the 'right' way to do it, but it works as far as I know.

With this code, I get 15 FPS and no errors on windows 7:

from collections import deque
import cv2
import Image, ImageTk
import time
import Tkinter as tk

def quit_(root):
    root.destroy()

def update_image(image_label, cam):
    (readsuccessful, f) = cam.read()
    gray_im = cv2.cvtColor(f, cv2.COLOR_BGR2GRAY)
    a = Image.fromarray(gray_im)
    b = ImageTk.PhotoImage(image=a)
    image_label.configure(image=b)
    image_label._image_cache = b  # avoid garbage collection
    root.update()


def update_fps(fps_label):
    frame_times = fps_label._frame_times
    frame_times.rotate()
    frame_times[0] = time.time()
    sum_of_deltas = frame_times[0] - frame_times[-1]
    count_of_deltas = len(frame_times) - 1
    try:
        fps = int(float(count_of_deltas) / sum_of_deltas)
    except ZeroDivisionError:
        fps = 0
    fps_label.configure(text='FPS: {}'.format(fps))


def update_all(root, image_label, cam, fps_label):
    update_image(image_label, cam)
    update_fps(fps_label)
    root.after(20, func=lambda: update_all(root, image_label, cam, fps_label))


if __name__ == '__main__':
    root = tk.Tk()
    # label for the video frame
    image_label = tk.Label(master=root)
    image_label.pack()
    # camera
    cam = cv2.VideoCapture(0)
    # label for fps
    fps_label = tk.Label(master=root)
    fps_label._frame_times = deque([0]*5)  # arbitrary 5 frame average FPS
    fps_label.pack()
    # quit button
    quit_button = tk.Button(master=root, text='Quit',
                            command=lambda: quit_(root))
    quit_button.pack()
    # setup the update callback
    root.after(0, func=lambda: update_all(root, image_label, cam, fps_label))
    root.mainloop()

Answer 3

Set the environment variable before you initialize the camera object in opencv.

os.environ['OPENCV_VIDEOIO_PRIORITY_MSMF'] = '0'

This released the camera even after closing the camera object in my code.