Pose independent face detection

Question

I'm working on a project where I need to detect faces in very messy videos (recorded from an egocentric point of view, so you can imagine..). Faces can have angles of yaw that variate between -90 and +90, pitch with almost the same variation (well, a bit lower due to the human body constraints..) and possibly some roll variations too.

I've spent a lot of time searching for some pose independent face detector. In my project I'm using OpenCV but OpenCV face detector is not even close to the detection rate I need. It has very good results on frontal faces but almost zero results on profile faces. Using haarcascade .xml files trained on profile images doesn't really help. Combining frontal and profile cascades yield slightly better results but still, not even close to what I need.

Training my own haarcascade will be my very last resource since the huge computational (or time) requirements.

By now, what I'm asking is any help or any advice regarding this matter. The requirements for a face detector I could use are:

very good detection rate. I don't mind a very high false positive rate since using some temporal consistency in my video I'll probably be able to get rid of the majority of them
written in c++, or that could work in a c++ application

Real time is not an issue by now, detection rate is everything I care right now.

I've seen many papers achieving these results but i couldn't find any code that I could use.

I sincerely thank for any help that you'll be able to provide.

To get reliable results for face recognition, you need not a most powerful computer with geniuosly devised algorithm or exhaustingly trained system, but additional dimension in the image. 3D pictures (when available) will solve the task. — SChepurin
I'm not actually interested in face recognition. I just need the detection right now. I will use the detected faces for some face pose estimation using two different SVM (one for yaw and one for pitch). I trained these SVMs on a head pose estimation dataset and it yielded quite interesting results. I just couldn't get enough detections on a "real" video due to the complexity of human head poses in real enviroments. — powder

Zaw Lin Zaw Lin · Accepted Answer · 2013-08-05T05:51:20

perhaps not an answer but too long to put into comment.

you can use opencv_traincascade.exe to train a new detector that can detect a wider variety of poses. this post may be of help. http://note.sonots.com/SciSoftware/haartraining.html. i have managed to trained a detector that is sensitive within -50:+50 yaw by using feret data set. for my case, we did not want to detect purely side faces so training data is prepared accordingly. since feret already provides convenient pose variations it might be possible to train a detector somewhat close to your specification. time is not an issue if you are using lbp features, training completes in 4-5 hours at most and it goes even faster(15-30min) by setting appropriate parameters and using fewer training data(useful for ascertaining whether the detector is going to produce the output you expected).

Pose independent face detection

1 Answers