Building HMM from the data

Question

I have a problem with Hidden Markov Models. I'm writing an app in C# on Kinect for gesture recognition (simple gestures as swipes, waves, circles etc.). I wrote a system for gesture recording, after that I created a feature extractor that counts velocities between points and I also count orientations (the angle between pairs of points) - I think it's a simple vector quantization method, because I created a codebook with codewords 1-18. Codewords are angles' intervals every 20 degrees (e.g. angle 12dg is a codeword-1, 45dg = codeword-3 etc.).

Now I guess I need to create a HMM and use some Forward-Backward algorithm for learning. Then some Bayes classification method and finish.

My questions:

1. Am I wrong in my steps?
2. How to create a HMM? Could anyone describe how it should look like programatically?
3. How to perform LIVE gesture matching for tests?

Answer 1

Disclaimer: I am the author of some of the tutorials/software I am going to list.

1. Your steps seems fine. They are very similar, for example, to the method used by Lee and Kim (minus the threshold models part).

2. Perhaps you can follow this guide on HMMs in C#. It was written by me, and have some examples on how to perform the gesture recognition. However, the guide creates continuous-density HMMs rather than discrete HMMs. Your vector quantization makes your features discrete, so I guess you would be more interested in using discrete-density HMMs. The only changes you have to do to make it work with discrete models is to remove the generic arguments in the model creation/learning.

3. To perform live gestures, you can either use some kind of specific marker to signal the starting and ending of a gesture, or you can use a technique similar to Lee and Kim's threshold models. The framework described in the aforementioned guide has support for those.

I have worked on a similar project using the same technology and I can say HMMs work just fine for the task.