How to detect start and end of a gesture in kinect?

1
votes

I am working on one-shot learning of gestures. Most of the gestures involve moving the left and the right hand and the hand joints are easily detectable using skeletal tracing library of Kinect SDK. I am facing the problem as to how to guess the start of the gesture and when it ends so that I can feed the coordinates of the hand joint trajectory to my algorithm that finally classifies the gesture?

1
computer-visionkinectgesturegesture-recognitionopenni
Are you looking to detect the start/end of a gesture the system is learning, or the start/end of a gesture that the system already knows? I read both in your question, so I'm unsure. – Nicholas Pappas
I need it while the system is learning the gesture. I need to start recording the trajectory when the user starts the gesture. How does one detect this starting point? I guess I will use a similar technique while comparing a query gesture with the ones the system has already learnt. – web_ninja

1 Answers

4
votes

There is no way you can detect the beginning of a unknown gesture within a learning engine. There must be some discrete action that tells the system that a gesture is about to be started for it to learn. Without this discrete action the system can not know what motion is the beginning of the gesture, v.s. a motion between, v.s. a motion moving towards the beginning, v.s. an arbitrary motion the engine should care nothing about.

There are a few discrete actions that might work, depending on your situation:

  1. a keyboard or mouse action
  2. a known gesture to signify a new gesture is to begin/end
  3. use voice recognition to notify the engine that you are starting/ending
  4. some action with a short countdown timer for the user to get to "position 1" of the gesture and begin when prompted.
  5. have a single origin for all gestures - holding your hand there for a short period to signify beginning of learning action.

Without some form of a discrete action, the system just can not know what you want. It will always guess, and you will always run into a situation where the system guesses wrong.

For executing on a known gesture, your method depends on how you store the data and the complexity of the gesture. Here are two gesture libraries that you can review to see how they work:

They may also help give ideas of how you want to start/end gestures, based on how the gesture data is stored for each situation.