HMM: Automatically determine hidden states?

Question

I'm fairly new to HMMs so please excuse me if any of my questions sound pretty naive. I am trying to train three HMM models for classification. These models will be emotions such as happiness, anger, and fear. I understand that I will need to have one HMM for each of the emotions but I don't understand how to determine what the hidden states will be. Each of my samples will include video and audio data with some actor acting out the emotion so there does not seem to be any clear hidden states. Is there a way that I can use all my samples(observation data) for happiness and automatically determine the number of hidden states? I don't need to know the label of the hidden state, I just need to be capable of classifying new samples into one of these emotions.

Thanks for you help.

Answer 1

If you want to model each emotion as an HMM, then the changes in the audio and visual features (I'm not sure what features you will be using) may be captured by the changes in the underlying Markov chain. For example, the speed of utterances will surely be different between sadness and fear. If you know what your state-conditional densities are (assuming your features again) then you can use a method such as AIC or BIC to determine the number of states (see Walter and Zucchini).

Answer 2

In an HMM, the hidden states are what you want to automatically identify. So, if you are trying to identify what emotion the actor is showing (I'm not entirely sure from your question, but that's what it sounds like), the hidden states would be the actor's emotions and the visible outputs of those states would be the video and audio data.