Kinect v2 Relative Cursor Location

Question

As a side note, this question has nothing to do with SharpDX, it is purely a Kinect 2.0 SDK question.

I am migrating a completed Kinect 1.8 project to the Kinect 2.0 SDK. In this program I have a WPF front end, but 99% of the code is written in SharpDX for C#. The program hides the KinectRegions cursor and uses the cursor location and grip data as the input to the SharpDX code. With this new version of the Kinect SDK however, I can't find any way to get the relative cursor data (hand position relative to the user). I tried using skeleton data to extrapolate the cursor location, just a simple primary shoulder location - primary hand location. This has the issue of when the hand occludes the shoulder, the cursor would shoot around. If I switch shoulders by reflecting across the spine when the occlusion occurs, I get a momentary jump. I can think of a way to get this to work but it will take quite a bit of code. I want to make sure there is no other way before I dive into that. Thanks in advance for any help!

redhotspike redhotspike · Accepted Answer · 2015-04-21T18:39:25

You should check out Mike Taulty's blog. He utilizes the KinectCoreWindow to grab the pointer movement. One note of caution though: this event gets raised for both hands even if they're not "active." I mitigated this by utilizing the body frame to set which hand was higher (y) to denote that that hand was "active."

...
var window = KinectCoreWindow.GetForCurrentThread();
window.PointerMoved += window_PointerMoved;
...

void window_PointerMoved(object sender, KinectPointerEventArgs e)
{
    if ((!rightHand && e.CurrentPoint.Properties.HandType == HandType.LEFT) ||
        (rightHand && e.CurrentPoint.Properties.HandType == HandType.RIGHT))
    {
         //do something with this hand pointer's location
    }
}

void _bodyReader_FrameArrived(object sender, BodyFrameArrivedEventArgs e)
{
    //get and check frame
    //...

    using (frame)
    {
        frame.GetAndRefreshBodyData(_bodies);
        foreach(Body body in _bodies)
        {
            if(body.IsTracked)
            {
                CameraSpacePoint left = body.Joints[JointType.HandLeft].Position;
                CameraSpacePoint right = body.Joints[JointType.HandRight].Position;

                if (left.Y > right.Y)
                    rightHand = false;
                else
                    rightHand = true;

                break; //for this example I'm just looking at the first
                       //tracked body - other handling is required if you
                       //want to keep track of more than one body's hands
            }
        }
    }
}

The other part is that your previous application could hide the KinectRegion cursor; I have not as of yet found out how to do that with the Kinect v2 (which is what brought me to this question actually lol).

Kinect v2 Relative Cursor Location

1 Answers