Speech recognition and signal visualization simultaneously

Question

I'm using Kinect + Microsoft.Speech for speech recognition.

To get the input audio stream one has to do the following

Stream kinectAudioStream = myKinect.AudioSource.Start();

and to start speech recognition

speechRecognitionEngine.SetInputToAudioStream(kinectAudioStream, new   SpeechAudioFormatInfo(...));
speechRecognitionEngine.RecognizeAsync(RecognizeMode.Multiple);

My target is to visualize the input signal in real-time (like a oscilloscope) and do speech recognition simultaneously .

I tried some things like add speech recognition to the Microsoft "AudioBasics-WPF C# Sample" in multiple threads. It works but the latency is very very very bad. To much interruptions in speech recognition and visualization.

Is there a way to increase the performance? Has anyone an idea?

So what is the system load during the process. Are interruptions due to high system load or due to some other factor. — Nikolay Shmyrev
Also, when you say that the latency is very bad, just how bad is it? 1 second? 10 seconds? What is your host platform? — Eric Brown
I think the latency is about 1 second more in comparison to “normal“ speech recognition or visualization. Also I got trouble with quality of recognition. The software runs only on desktop systems. Is there a way to optimize multiple accesses on the same stream? — hagem

Eric Brown Eric Brown · Accepted Answer · 2013-07-19T20:51:15

Well, your best option is to drop down to the Kinect DMO and connect a SmartTee filter, but that requires that you write in C++, as the KinectAudioSource doesn't expose the filter itself.

Speech recognition and signal visualization simultaneously

1 Answers