IBM watson speech-to-text job hung in “processing” state

Question

I'm using https://stream.watsonplatform.net/speech-to-text/api/v1 with mime type audio/mp3, communicating with async interface (http callback) on these events:

recognitions.failed
recognitions.started
recognitions.completed

Most of the jobs run well, but I've had one (e9be207c-348d-11e8-bd95-874f1d4b7552) that did this:

• 2018-03-31/2:47:55: client started job
• 2018-03-31/2:48:03: called back with "recognitions.started"
• 2018-03-31/3:00:52: called back with "recognitions.completed"
• 2018-03-31/3:00:52: client fetched job results, found state was still "processing"

It's been in that state ever since. Audio attribs are ...

> mediainfo ~/Downloads/11394.aud_watson.mp3
General
Complete name                            : /home/dougs/Downloads/11394.aud_watson.mp3
Format                                   : MPEG Audio
File size                                : 6.96 MiB
Overall bit rate mode                    : Variable
Writing library                          : Lavf56.40.101

Audio
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 3
Bit rate mode                            : Variable
Channel(s)                               : 1 channel
Sampling rate                            : 44.1 KHz
Compression mode                         : Lossy
Stream size                              : 6.96 MiB (100%)

... with duration 12 min 15 secs.

Any suggestions regarding how to diagnose what's happening on the watson server?

Answer 1

in recent days this API of the service has had some instability issues due to very heavy load and traffic coming in spikes. This has caused a small percentage of the jobs to get stuck into "processing" state.

Edited: this issue is resolved now. We pushed to production an upgraded version of this API, with higher capacity, and we have not seen any job loss since then. Please verify on your end.