Gradual cursor move algorithm - Kinect SDK

Question

I am building a Kinect SDK WPF Applicaiton and using the Kinect to move a "cursor"/hand object.

The problem i am having is that at 30 frames a second the cursor is actually jumping around a bit erratically because of the precision of the Kinect (i.e. while holding your hand still the object moves within a 5px space).

I am planning on writing an algorithm that doesn't simply move the X/Y of my "cursor" sprint to the right position on the screen, but behaves more like a "move the hand towards this X/Y co-ordinate" so that it is a more smooth movement.

Can someone point me to a good one that someone else has written so i can avoid reinventing the wheel.

I understand that this is probably pretty common, but as i am more of a business developer i am not sure of the name for such a feature so apologies in advance if its a n00b question.

Answer 1

When I worked with the Kinect, I just used some simple math (which I think is called linear regression) to move to a point some distance between the cursor's current location and its target location. Get the location of the cursor, get the location the user's hand is at (translated to screen coordinates), then move the cursor to some point between those.

float currentX = ..., currentY = ..., targetX = ..., targetY = ...; 
float diffX = targetX - currentX;
float diffY = targetY - currentY;
float delta = 0.5f; // 0 = no movement, 1 = move directly to target point. 

currentX = currentX + delta * diffX;
currentY = currentY + delta * diffY;

You'll still get jittering, depending on the delta, but it will be much smoother and generally in a smaller area.

On a related note, have you taken a look at the Kinect's skeleton smoothing parameters? You can actually let the SDK handle some of the filtering.

Answer 2

Consider your input values (those jumping positions) as a signal with both low and high frequency parts. The low frequencies represent the rough position/movement while the high frequency parts contain the fast jumping within smaller distances.

So what you need or look for is a low pass filter. That filters out the high frequency parts and leaves the rough (but as accurate as the Kinect can get) position over, if you manage to set it up with the right parameter. This parameter is the crossover frequency for the filter. You have to play around a bit and you will see.

An implementation example for time-discrete values would be from here (originally from wikipedia):

static final float ALPHA = 0.15f;

protected float[] lowPass( float[] input, float[] output ) {
    if ( output == null ) return input;

    for ( int i=0; i<input.length; i++ ) {
        output[i] = output[i] + ALPHA * (input[i] - output[i]);
    }
    return output;
}

You can put the last values of both the X and Y components of your position vectors into this function to smooth them out (input[0] for X and input[1] for Y, output[0] and output[1] are results of the previous function call).

Like I already said, you have to find a good balance for the smoothing factor ALPHA (0 ≤ ALPHA ≤ 1):

• Too big and the signal will not get smoothed enough, the effect wont be sufficient
• Too small and the signal will be smoothed 'too much', the cursor will lag behind the users movement, too much inertia

(If you look at the formula newout = out + alpha * (in - out), you see that with a alpha value of 0, you just take the old out value again, therefore the value will never change; while with a value of 1 you have newout = out + in - out that means you dont smooth anything but always take the newest value)

Answer 3

One very simple idea for solving this problem would be to display the cursor at a location that's the average of some past number of positions. For example, suppose that you track the last five locations of the hand and then display the cursor at that position. Then if the user's hand is relatively still, the jerkiness from frame to frame should be reasonably low, because the last five frames will have had the hand in roughly the same position and the noise should cancel out. If the user then moves the cursor across the screen, the cursor will animate as it moves from its old position to the new position, since as you factor in the last five positions of the hand the average position will slowly interpolate between its old and new positions.

This approach is very easily tweaked. You could transform the data points so that old points are weighted more or less than new points, and could adjust the length of the history you keep.

Hope this helps!