Getting timestamps for each transcribed word through Google Cloud Speech API?

Question

I'm hoping to transcribe an audio file via the Google Cloud Speech API. This simple script takes a wav as input and transcribes it with pretty high accuracy.

import os
import sys
import speech_recognition as sr

with open("~/Documents/speech-to-text/speech2textgoogleapi.json") as f:
  GOOGLE_CLOUD_SPEECH_CREDENTIALS = f.read()
name = sys.argv[1] # wav file
r = sr.Recognizer()
all_text = []
with sr.AudioFile(name) as source:
  audio = r.record(source)
  # Transcribe audio file
  text = r.recognize_google_cloud(audio, credentials_json=GOOGLE_CLOUD_SPEECH_CREDENTIALS)
all_text.append(text)
with open("~/Documents/speech-to-text/transcript.txt", "w") as f:
  f.write(str(all_text))

How can I use the API to extract other meaningful information from the speech audio? Specifically, I'm looking to get a timestamp for each word, but other info (eg. pitch, amplitude, speaker recognition, etc.) would be extremely welcome. Thanks in advance!

Victor M Herasme Perez Victor M Herasme Perez · Accepted Answer · 2018-03-29T17:34:13

There is actually an example on how to do this in the Speech API in

Using Time offsets(TimeStamps):

Time offset (timestamp) values can be included in the response text for your recognize request. Time offset values show the beginning and end of each spoken word that is recognized in the supplied audio. A time offset value represents the amount of time that has elapsed from the beginning of the audio, in increments of 100ms.

Time offsets are especially useful for analyzing longer audio files, where you may need to search for a particular word in the recognized text and locate it (seek) in the original audio. Time offsets are supported for all our recognition methods: recognize, streamingrecognize, and longrunningrecognize. See below for an example of longrunningrecognize.....

This is the code sample for Python:

def transcribe_gcs_with_word_time_offsets(gcs_uri):
    """Transcribe the given audio file asynchronously and output the word time
    offsets."""
    from google.cloud import speech
    from google.cloud.speech import enums
    from google.cloud.speech import types
    client = speech.SpeechClient()

    audio = types.RecognitionAudio(uri=gcs_uri)
    config = types.RecognitionConfig(
        encoding=enums.RecognitionConfig.AudioEncoding.FLAC,
        sample_rate_hertz=16000,
        language_code='en-US',
        enable_word_time_offsets=True)

    operation = client.long_running_recognize(config, audio)

    print('Waiting for operation to complete...')
    result = operation.result(timeout=90)

    for result in result.results:
        alternative = result.alternatives[0]
        print('Transcript: {}'.format(alternative.transcript))
        print('Confidence: {}'.format(alternative.confidence))

        for word_info in alternative.words:
            word = word_info.word
            start_time = word_info.start_time
            end_time = word_info.end_time
            print('Word: {}, start_time: {}, end_time: {}'.format(
                word,
                start_time.seconds + start_time.nanos * 1e-9,
                end_time.seconds + end_time.nanos * 1e-9))

Hope this helps.

Getting timestamps for each transcribed word through Google Cloud Speech API?

1 Answers