Kinect V2 High Definition Face Tracking - Multiple Faces

Question

I've been working for quite a while with Kinect V2 in WPF. Currently, I've written a module to extract the contours of the face from the colour stream via HighDefinitionFaceFrameSource and HighDefinitionFaceFrameReader. Essentially, I am using the faceModel and faceAllignment properties from the HightDefinitionFaceFrameResult to calculate the path around the face. I then map it to the colour stream, which gives me an image of the person's face. I then draw the colour stream to an WriteableBitmap, which I display in my XAML window. The path data is discarded for now.

This method works great for a single person. However, when I tried to expand it to six people, it slowed down the rendering of my visualization to a halt! It seems like only the rendering is affected, as the face frames seem to be arriving as usual, the CPU and memory usage is unaffected as well. This seems to happen once I process more than one High Definition face frame.

I also noticed that Microsoft SDK examples only tracks one face with HighDefinitionFaceFrameSource. Perhaps, there is a limitation in the SDK?

Has anyone managed to have six people tracked with HignDefinitionFaceFrameSource & Reader?

did you ever find a answer to this? I'm trying to control HD face tracking of more than just one person at a time but haven't found any examples yet. — Sergio S

Levko Ivanchuk Levko Ivanchuk · Accepted Answer · 2016-02-05T18:37:01

After further testing, it became apparent to me that the Kinect SDK will not allow you to have multiple HighDefinitionFaceFrameSources have a set tracking ID. So, it seems to me that only one face is allowed to be tracked at the time. Note that I might be wrong here, and perhaps someone from Microsoft's Kinect team can give a more definitive answer, but that is what I found out.

Here is how I managed to work around this limitation. I am iterating over the list of tracked bodies and setting the tracking ID for each face individually. This allows me to track a face for some time, grab all the data I need, stop tracking it and move on to the next body face and repeat. This has many drawbacks, of course - your face tracking data stream is not as 'fluid' as it should have been, the visualization has data 'gaps' - small, but noticeable. However, this is just a workaround and I am still in search of a better solution.

In responce to your comment, Sergio, you can set a face as active by assigning it a current, valid tracking ID.

        // update the face frame source to track this body

        if (FaceFrameSources[bodyFrameIndex].TrackingId != trackingId)
        {
            FaceFrameSources[bodyFrameIndex].TrackingId = trackingId;
        }


        //set all the tracking face data to false
        // update the high definition face frame source to track this body / face

        if (_highDefinitionFaceFrameSources[bodyFrameIndex].TrackingId != trackingId)
        {
            _highDefinitionFaceFrameSources[bodyFrameIndex].TrackingId = trackingId;
        }

Here, the trackingId is the current tracking ID from your list of Body objects. Your skeleton tracker should keep this value updated on every frame.

Setting the TrackingId value in the HighDefinitionFaceFrameSource to 0 would disable tracking for that object.

Kinect V2 High Definition Face Tracking - Multiple Faces

1 Answers