Hand shape analysis with Hidden Markov Model

0
votes

I am currently working on a dynamic gesture recognition system. I opted to go for the curvature feature of the hand shape. At the moment I am able to obtain a list of angles of the hand contour curvature.

The problem is that I am stuck with what should I do next. Little information can be found in the numerous literature I am finding. This set of angles per gesture/posture should be "PROCESSED" before passing the processed output to the Hidden Markov Model for classification.

But what should this process consist of? I encountered: Fourier Transform, B Spline, and a lot of functions yet I do not know how to pass the set of angles (e.g. -23, 90, 45, 156, ...) to such functions.

Thank you for your patience.

The following is a picture showing what stage I have achieved so far:

enter image description here

1
opencvmachine-learningclassificationgesture-recognitionhidden-markov-models
what are you using for HMM classification ? ( just adding a comment to get notified of improvements :] ) - berak
Thank you. I am new to computer vision and machine learning. Can you kindly explain what you mean by your question please? If you mean as a tool, I have encountered Jahmm - a java based approach for HMM. It's quite interesting but there is not a lot of support about it considering that I'm very new to this area! - test
yes, i meant that, thanks for clarifying - berak
The problem is that I still do not know how I will use this tool since I do not know what type of input it will take. - test

1 Answers

1
votes

Can you explain how you have obtained these angles? You could group consecutive angles together and may be relate them in some manner: say by subtracting them or processing them in some way to find which fingers have been opened/closed or to find the direction in which the hand is headed. For example, say the centroid of your palm was at (x,y) in one frame and in (a,b) in the other frame. You could then subtract these to find the direction in which your hand is headed.

Now since you have found the angles of the hand contour curvature, you have the number of different points and the change in angles. By pairing consecutive angles and finding the number of points on the contour you could try and detect which fingers are open and which are closed. Look into the Convex-Hull algorithm if your trying to do finger-tip detection. But since, you say your gestures are dynamic, that must mean that your hand is continually moving in space while performing a certain gesture. Please clarify this. Once processed, the result shall form the Observation Sequence, which shall serve as input to your HMM.