Android record audio while doing speech recognition

Question

I am doing speech recognition using a third party cloud service on Android, and it works well with Android API SpeechRecognizer. Code below:

    Intent recognizerIntent =
            new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
    recognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,
            RecognizerIntent.LANGUAGE_MODEL_WEB_SEARCH);

    // accept partial results if they come
    recognizerIntent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true);

    //need to have a calling package for it to work
    if (!recognizerIntent.hasExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE)) {
        recognizerIntent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, "com.example.speechrecognition");
    }
    recognizer = SpeechRecognizer.createSpeechRecognizer(context);
    recognizer.setRecognitionListener(this);
    recognizer.startListening(recognizerIntent);

At the same time, I want to record the audio with different audio setting, such as frequency, channels, audio-format, etc. Then I will continuously analyze this audio buffer. I use the AudioRecord for the purpose. This works well only I turn off the speech recognition.

If I record audio and speech recognition at the same time, then error happens.

E/AudioRecord: start() status -38

How to implement this kind of feature, I also tried native audio - SLRecordItf, also doesn't work.

because of android policy, in the same time just one application has access to microphone api. — faraz khonsari
maybe you should find some speech recognizer that work with byte of sound not microphone. — faraz khonsari
It might be due to the simultaneous use of the audio input hardware of the phone. The speech recognizer and audio recorder both use the same hardware for listening. If one is currently using the hardware, the other might not be able to use it. — Shreyansh Patni

brandall brandall · Accepted Answer · 2016-09-29T10:49:08

As the comments state, only one mic access is permitted/possible at a time.

For the SpeechRecognizer the attached RecognitionListener has a callback of onBufferReceived(byte[] buffer) but unfortunately, Google's native recognition service does not supply any audio data to this, it's very frustrating.

Your only alternative is to use an external service, which won't be free. Google's new Cloud Speech API has an Android example.

Android record audio while doing speech recognition

1 Answers