Can't get the Speech Recognizer and a KinectRegion to work together

Question

The Kinect SDK 1.8 has a menu demo with a tile list that uses the KinectSensorChooser and a KinectRegion and it works great, the SDK also have a speech recognition sample but it uses a KinectSensor object directly, no sensor chooser.

I wanted to combine both KinectRegion GUI and Audio but can't get it to work, there are no exceptions at all but it won't respond.

When I bind a Microsoft.Speech recognizer to a KinectSensorChooser.Kinect property the speech recognition event never triggers, as if it doesn't receive any audio.

When I bind a KinectRegion to a raw KinectSensor object the region doesn't work, meaning it won't display the hand cursor nor respond to hand movements.

The samples that I'm talking about are: Controls Basic WPF (C#) and Speech Basics-WPF (C#).

Any help is appreciated. Thanks

Binding the Speech Recognition Engine to a KinectSensorChooser - Doesn't work

speechEngine.SetInputToAudioStream(this.sensorChooser.Kinect.AudioSource.Start(), new    SpeechAudioFormatInfo(EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, null));

Binding the Speech Recognition Engine to a raw KinectSensor - Works

speechEngine.SetInputToAudioStream(this.sensor.AudioSource.Start(), new    SpeechAudioFormatInfo(EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, null));

Binding a KinectRegion to a KinectSensor - Doesn't work

var regionSensorBinding = new Binding("Kinect") { Source = this.sensor };
BindingOperations.SetBinding(this.kinectRegion, KinectRegion.KinectSensorProperty, regionSensorBinding);

Binding a KinectRegion to a KinectSensorChooser - Works

var regionSensorBinding = new Binding("Kinect") { Source = this.sensorChooser };
BindingOperations.SetBinding(this.kinectRegion, KinectRegion.KinectSensorProperty, regionSensorBinding);

In what order are you initializing audio and skeleton tracking? See the Kinect for Windows Known Issues: msdn.microsoft.com/en-us/library/dn435682.aspx - "Audio is not processed if skeleton stream is enabled after starting audio capture" — Nicholas Pappas

paul-2011 paul-2011 · Accepted Answer · 2013-11-25T14:51:35

I kept the KinectSensorChooser initialization at the main method of the Window (creator) and moved the speech audio initialization code to the Window_Openeded event and it worked. I suppose the timing was not good enough to initialize the sensor and audio during the window creation. Thank you

Can't get the Speech Recognizer and a KinectRegion to work together

1 Answers