Step-by-step cursor movement algorithm - Kinect SDK

Question

I am creating a WPF Applicaiton Kinect SDK application and using Kinect to move the cursor / hand object.

The problem I am facing is that at 30 frames per second the cursor actually skips a bit due to the accuracy of Kinect (i.e. holding your hand, still the object moves in a space of 5 pixels).

I plan to write an algorithm that not only moves the X / Y of my "cursor" sprint to the desired position on the screen, but behaves more like "moving your hand to this X / Y software," the ordinate ", so this is a smoother movement.

Can someone point me to what someone wrote, so I can avoid reusing the wheel.

I understand that this is probably quite common, but since I am more of a business developer, I’m not sure of the name for such a function, so I apologize in advance if this is a question n00b.

Answer 1

When I was working with Kinect, I just used simple math (which I think is called linear regression) to jump to a point at some distance between the current location of the cursor and its target location. Get the location of the cursor, find the location where the user’s hand is located (translated to screen coordinates), then move the cursor to some point between them.

float currentX = ..., currentY = ..., targetX = ..., targetY = ...; float diffX = targetX - currentX; float diffY = targetY - currentY; float delta = 0.5f; // 0 = no movement, 1 = move directly to target point. currentX = currentX + delta * diffX; currentY = currentY + delta * diffY; 

You will still tremble, depending on the delta, but it will be much smoother and generally in a smaller area.

In connection with the note, did you take a look at the smoothing parameters of the Kinect skeleton? In fact, you can let the SDK handle some filtering.

Answer 2

Consider your input values ​​(these jump positions) as a signal with low-frequency and low-frequency parts. The low frequencies represent a rough position / movement, while the high-frequency parts contain rapid spasmodic movement at shorter distances.

So you need or need to look for a lowpass filter. This filters out the high-frequency parts and leaves a rough (but accurate, like Kinect) position if you manage to adjust it using the correct parameter. This parameter is the crossover frequency for the filter. You need to play a little and you will see.

An example implementation for discrete time values ​​will be here (originally from wikipedia ):

 static final float ALPHA = 0.15f; protected float[] lowPass( float[] input, float[] output ) { if ( output == null ) return input; for ( int i=0; i<input.length; i++ ) { output[i] = output[i] + ALPHA * (input[i] - output[i]); } return output; } 

You can put the last values ​​of the X and Y components of your position vectors into this function to smooth them ( input[0] for X and input[1] for Y, output[0] and output[1] are the results of the previous function call).

As I said, you need to find a good balance for the ALPHA smoothing factor (0 ≤ ALPHA ≤ 1):

• Too large and the signal will not be smooth enough, the effect will not be sufficient
• Too small, and the signal will be smoothed "too much", the cursor will lag behind the movement of users, too much inertia.

(If you look at the formula newout = out + alpha * (in - out) , you will see that with an alpha value of 0 you again take the old out value, so the value will never change, and with a value of 1 you newout = out + in - out , this means that you do not smooth out anything, but always take the latest value)

Answer 3

One very simple idea to solve this problem would be to display the cursor in a place where the average value has a certain number of last positions. For example, suppose you track the last five locations of a hand, and then display the cursor at that position. Then, if the user’s hand is relatively motionless, the jerk from frame to frame should be low enough, because the last five frames will have the arm in about the same position, and the noise should be canceled. If the user then moves the cursor around the screen, the cursor will be animated as it moves from its old position to the new position, because since you are considering the last five positions of the hand, the middle position will slowly interpolate between its old and new positions.

This approach is very easy to adjust. You can convert data points so that old points are weighted more or less than new points, and can adjust the length of your story.

Hope this helps!