Face mask in opencv

Question

Input: face image

Problem: thresholded image before applying Canny to find contours but does not return face mask

Desired output if different face is input,it should generate a proper face mask(face area white and background white)

Tried with apple picture..works fine

            #include <opencv2/highgui/highgui.hpp>
            #include <opencv2/core/core.hpp>
            #include <opencv2/imgproc/imgproc.hpp>

            using namespace cv;
            using namespace std;

            int main(){
              Mat right=imread("front.jpg");
              Mat img1;
              cvtColor(right, img1, CV_RGB2GRAY);
              threshold(img1,img1,160,255,cv::THRESH_BINARY);
              Canny(img1, img1, 128, 350);
              vector< vector<Point> > contours;
              findContours(img1, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
              Mat mask = Mat::zeros(img1.rows, img1.cols, CV_8UC1);
              drawContours(mask, contours, -1, Scalar(255), CV_FILLED);
              normalize(mask.clone(), mask, 0.0, 255.0, CV_MINMAX, CV_8UC1);

              imshow("original", right);
              imshow("thresh",img1);
              imshow("mask", mask);

              waitKey(0);
              return 0;
    }

Here is the image that I used

enter image description here

Please ignore the first 3 comments below

So you can define the eye-corner: the point in the middle of the rect (vertically) which is nearest to the ear position. Or if you know the face direction, you can define the eye-corner, the outermost position in the middle of the eye rect. — masoud
I think a more robust approach would be to threshold the inner ROI and do a simple Harris-corner detection there. It would most likely pick up the two eye corners. Of course, you will need to do some additional checks to see if that corner is towards the ear or the nose (dataset dependent). — scap3y
@Steph It seems like threshold binary inverted works fine for the above image, some thing like threshold(img1,img1,160,255,cv::THRESH_BINARY_INV); see the result here i.stack.imgur.com/VtB3Q.jpg , also you don't need to do canny before find contour. — Haris
Hi there @Micka. Remember the blending function you wrote, well I want to have a separate function which will generate the face mask of aligned images and then use it in the blending function. But it's not working :/ Here is the image of the findContours for the above image i.imgur.com/5BaqwqJ.jpg — Steph
So why don't you set the color range of the background (that greyish color, but it has to be a range of RGB, not a single RGB, even better is HSV color space, check out HSV color segmentation) to black and all the other pixels to white. — Rui Marques

Haris Haris · Accepted Answer · 2014-03-27T06:53:48

Using this below code the mask creation working perfectly for the sample image you provided in the above commants.

Here assumes, your background is of any colour with no other object.

The below code will do

Find edge
Strengthen the edge by morphology operation.
Find biggest contour in edge( always the boundary of your foreground ) and draw it by filling.

Sometimes your contour may not be closed at the bottom(no edges at bottom) so the filling contour wont work, so for make it closed the code first find the bounding rect for biggest contour(foreground) and then draw bottom of rect to the edge image, then find largest contour again will give you the proper mask image.

Rect R;

Mat findLargestContour(Mat thr){

vector< vector <Point> > contours; // Vector for storing contour
vector< Vec4i > hierarchy;
int largest_contour_index=0;
int largest_area=0;
Mat dst(thr.rows,thr.cols,CV_8UC1,Scalar::all(0)); //create destination image

findContours( thr, contours, hierarchy,CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE );
for( int i = 0; i< contours.size(); i++ ) // iterate through each contour.
      {
       double a=contourArea( contours[i],false);  //  Find the area of contour
       if(a>largest_area){
       largest_area=a;
       largest_contour_index=i;                //Store the index of largest contour
       }
      }
drawContours( dst,contours, largest_contour_index, Scalar(255,255,255),CV_FILLED, 8, hierarchy );
R= boundingRect( contours[largest_contour_index]);
return dst;

}

int main( )
{

              Mat right=imread("1.jpg");
           // blur(right,right,Size(3,3));
              Mat gray;
              cvtColor(right, gray, CV_RGB2GRAY);

              int borderW=10;
              //Mat ROI=gray(Rect(borderW,borderW,img1.cols-2*borderW,img1.rows-2*borderW));
              Canny(gray, gray, 30, 255);

              Size kernalSize (5,5);
              Mat element = getStructuringElement (MORPH_RECT, kernalSize, Point(1,1)  );
              morphologyEx(gray, gray, MORPH_CLOSE, element );
              imshow("canny", gray);

              Mat largestCon=findLargestContour(gray.clone());
              line(largestCon, Point(R.x,R.y+R.height), Point(R.x+R.width,R.y+R.height), Scalar(255),2,8,0);

              Mat mask=findLargestContour(largestCon.clone());

              Mat A;
              right.copyTo(A,mask);
              imshow("original", right);
              imshow("dst", A);
              imshow("mask", mask);

              waitKey(0);


  return 0;
  }

See some sample mask

enter image description here

Face mask in opencv

1 Answers