Preprocessing methods for face recognition in Python

Question

I am working on a face recognition project where I am recognizing the faces of the person in movement which means the person keeps moving and I have to detect and recognize face.

To do this I am using caffemodel for face detection and modified knn approach for face recognition. It works fine but most of the case it gives false recognition. Because the persons are moving, thus its really hard to get a good face image. Below are few examples of the face captured:

Camera is placed a bit far from where the person is due to which its not very clear and frontal face. Face image size is 100x120. I have to use these images for face recognition. I was wondering if anyone can guide on some of the pre processing methods I can use before recognition so that accuracy can be improved. Please help. Thanks

Answer 1

If you have different degradations (blur, extreme pose, illumination, etc.) knn approach is bad. If you have data, what I suggest you use a small neural network which is trained with degradations and poses (Megaface). It would have much better recognition accuracies. Of course, you should use smaller networks for real-time applications. Other than my suggestion, there are couple of preprocessing methods for face recognition. The first one is face alignment which warps the faces to get the same alignment for better accuracy. To make alignment there are landmark prediction models that find 68 landmark points (nose, eyes, mouths e.g.) on the face. However, in extreme poses (top right) alignment is not enough.

Answer 2

Since you are dong face recognition so need face details only not surroundings. Surrounding objects reduce accuracy because of false positive. You can follow the following steps to enhance image quality:

Step 1: Detect face only using haarcascade

face_cascade = cv2.CascadeClassifier(‘haarcascade_frontalface_default.xml’)
faces_detected = face_cascade.detectMultiScale(img, scaleFactor=1.1, minNeighbors=5)
(x, y, w, h) = faces_detected[0]
cv2.rectangle(img, (x, y), (x+w, y+h), (0, 255, 0), 1);
cv2.imshow(img)

Step 2: Crop image

p = 20 #padding 
img_cropped = img[y-p+1:y+h+p, x-p+1:x+w+p]

Step 3: reshape to the original size

im_reshape = cv2.resize(img_cropped , (img.shape[0], img.shape[1]), interpolation=cv2.INTER_LINEAR)

Step 3: Normalize image to fix brightness

norm_img = np.zeros((img.shape[0], img.shape[1]))
norm_img = cv2.normalize(img, norm_img, 0, 255, cv2.NORM_MINMAX)

Step 3: norm_img to caffemodel

This method can increase the accuracy by a significant amount.

Answer 3

The best preprocessing method you could do is removing the lightning effect.

Assume you have the following image:

How can you find the face? or where is the face?

In the current situation, it is impossible to find the face in the image even with the human eye.

If we apply GaussianBlur:

At least we can see where the face is in the image.

import cv2

img = cv2.imread('input.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
smooth = cv2.GaussianBlur(gray, (125,125), 0)
division = cv2.divide(gray, smooth, scale=255)
cv2.imwrite('output.jpg',division)

Now, how do we locate the face?

Above is a challenging image in which you can test your method. Since the above image can not be detected by Haar, and face-recognition. If your method finds it, you can be sure about your method.

For instance:

The following image can be found by Haar-like features. Therefore your method must detect the below image. Otherwise, you should change your method.

• Apply Gaussian Blur

• Detect the face

• Using Haar

img = cv2.imread('face_shaded_division.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
face_cascade = cv2.CascadeClassifier('/opencv/data/haarcascades/haarcascade_frontalface_default.xml')

faces = face_cascade.detectMultiScale(img, 1.3, 5)
for (x,y,w,h) in faces:
    img = cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
    roi_gray = gray[y:y+h, x:x+w]
    roi_color = img[y:y+h, x:x+w]
cv2.imwrite('output.png', img)
cv2.imshow('img', img)
cv2.waitKey(0)
cv2.destroyAllWindows()

Possible Question Why don't you mention about face-alignment?

The most challenging fact is the lightning effect in face-recognition. If your method solves that, all you can do is training your network with more samples, or you could use pre-trained networks.

Answer 4

Face alignment increases model success almost 1% based on the google research.

I recommend you to use deepface. It applies some pre-processing steps in the background such as detection and alignment. Besides, its face recognition module wraps state-of-the-art face recognition techniques based on deep learning. KNN based approaches are not modern anymore.

#!pip install deepface
from deepface import DeepFace

#real-time web cam implementation
DeepFace.stream("C:/database")

#this will look for img in the my_db and img could be exact image path (c:/img1.jpg) or numpy array.
obj = DeepFace.find(img, db_path = "C:/workspace/my_db", model_name = "VGG-Face")