How to mark features which are present in both arrays in OpenCV?

2
votes

I'm trying to write a Python 3.7 script to detect faces and features using the OpenCV Haar classifier files which stores images as n-dimensional Numpy arrays. Right now I'm only working with two features: - The entire face - The eyes Both of these are obtained using two different classifiers. The Code detects the presence of both these features in the image and then marks them with a rectangle using the cv2.rectangle() function inside a for loop for each feature i.e, one for detected faces and one for detected eyes.

I want the script to mark the eye-rectangles ONLY if those points were found in the face array as well as the eye array. numpy.intersect1d() only finds intersection within one-dimensional arrays.

I've even tried

for x,y,w,h in eyes and x,y,w,h in faces:

and all it does is return this error message:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Any help would be greatly appreciated.

This is being attempted on Windows 10 64-bit, the code is written in Pycharm 2019, OpenCV is imported as CV2.

# Creating the cascade classifier objects.
face_cascade = 
cv2.CascadeClassifier("haarcascade_frontalface_default.xml")
eye_cascade = cv2.CascadeClassifier("haarcascade_eye.xml")

# Entering and displaying the original image.
img = cv2.imread("filename.jpg", 1)
cv2.imshow("Original Images",img)
cv2.waitKey(0)
cv2.destroyAllWindows()

# Convert the image to Black and White and store in a variable.
gry_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

faces = face_cascade.detectMultiScale(gry_img, scaleFactor = 1.05, minNeighbors = 8)
eyes = eye_cascade.detectMultiScale(gry_img, scaleFactor = 1.30, minNeighbors = 12)

# Now, we mark the detected features with a rectangle on the original image.
for x,y,w,h in faces:
    img = cv2.rectangle(img, (x,y), (x+w, y+h), (255, 21, 21), 3) # Blue.


for x,y,w,h in eyes:
    img = cv2.rectangle(img, (x,y), (x+w, y+h), (255, 255, 24), 3) # Cyan.

cv2.imshow("Detected Faces and Eyes",img)
cv2.waitKey(0)
cv2.destroyAllWindows()

I need the eye-features to be marked only if they were ALSO found in the face-features array.

2
pythonpython-3.xnumpyopencvhaar-classifier

2 Answers

0
votes

A better approach to solve this problem is already suggested in the OpenCV docs. It suggests that instead of passing the whole image for detecting the eyes, you should first detect the face, then crop the image, and use this cropped image for eyes detection. For cropping the face image you can use numpy slicing as:

for x,y,w,h in faces:
    face_img = gry_img[y:y+h, x:x+w]
    # Now detect the eyes in this `face_img` only.
    eyes = eye_cascade.detectMultiScale(face_img, scaleFactor = 1.30, minNeighbors = 12)
0
votes

Oh boy, do I feel stupid now. Well here goes:

As ZdaR said, the OpenCV documentation does indeed give a perfect example on how to achieve the result I required, the gist of which is:

for x,y,w,h in faces:
    face_img = gry_img[y:y+h, x:x+w]
    # Now detect the eyes in this `face_img` only.
    eyes = eye_cascade.detectMultiScale(face_img, scaleFactor = 1.30, minNeighbors = 12)

What I needed to do other than this, apparently was to crop the original image to the face coordinates as well for the eye-features to be drawn upon. I'm still not sure if this should work, but for now it seems that adding that extra line does work.

Code that finally worked:

faces = face_cascade.detectMultiScale(gry_img, scaleFactor = 1.05, minNeighbors = 10)
for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y),(x + w, y + h), (255, 0, 0), 3)
    gry_eye = gry_img[y:y+h, x:x+w]
    eye_img = img[y:y + h, x:x + w]
    eyes = eye_cascade.detectMultiScale(gry_eye, scaleFactor = 1.05, minNeighbors = 5)
    for (x, y, w, h) in eyes:
        cv2.rectangle(eye_img, (x, y), (x + w, y + h), (255, 255, 24), 3)

Well, this has been a learning experience. Thanks to ZdaR for the help!