How to get speech also in audio file in botframework using directline?

Question

I am using botframework channel directline in C# for my chatbot recently I added Bing speech for text to speech and vice versa but is there any chance that when user speaks to have beside plain text also the audio file as attachment in message activity.

Thanks in advance.

Regards

You can access audio and video with a skype bot docs.microsoft.com/en-us/bot-framework/dotnet/… — Eric Dahlvang

Nicolas R Nicolas R · Accepted Answer · 2018-01-23T15:01:24

In the case of the Webchat channel, you can have a look to the sources to understand how it is using speech recognition.

In particular, you can see that all speech part is made by the webchat before sending the message to the bot (sources):

const startListeningEpic: Epic<ChatActions, ChatState> = (action$, store) =>
    action$.ofType('Listening_Starting')
    .do((action : ShellAction) => {
        var locale = store.getState().format.locale;
        var onIntermediateResult = (srText : string) => { store.dispatch({ type: 'Update_Input', input: srText, source:"speech" })};
        var onFinalResult = (srText : string) => {
                srText = srText.replace(/^[.\s]+|[.\s]+$/g, "");
                onIntermediateResult(srText);
                store.dispatch({ type: 'Listening_Stop' });
                store.dispatch(sendMessage(srText, store.getState().connection.user, locale));
            };
        var onAudioStreamStart = () => { store.dispatch({ type: 'Listening_Start' }) };
        var onRecognitionFailed = () => { store.dispatch({ type: 'Listening_Stop' })};
        Speech.SpeechRecognizer.startRecognizing(locale, onIntermediateResult, onFinalResult, onAudioStreamStart, onRecognitionFailed);
    })
    .map(_ => nullAction)

Here the bot code on the web app is called with sendMessage(srText..., without audio.

How to get speech also in audio file in botframework using directline?

1 Answers