How can I play an ArrayBuffer stream on a browser with web audio API?

Question

I have an independent device that uses ffmpeg and streams audio to my node server. I encode the stream with the ffmpeg command

ffmpeg -f alsa -i hw:0 -acodec mp2 -f mp3 -r 30 http://localhost:8086/abc

output:

ffmpeg version 0.8.8-4:0.8.8-0ubuntu0.12.04.1, Copyright (c) 2000-2013 the Libav developers
built on Oct 22 2013 12:31:55 with gcc 4.6.3
[alsa @ 0x20ef9c0] Estimating duration from bitrate, this may be inaccurate
 Input #0, alsa, from 'hw:0':
  Duration: N/A, start: 10088.609013, bitrate: N/A
  Stream #0.0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
Output #0, mp3, to 'http://localhost:8086/abc':
  Metadata:
    TSSE            : Lavf53.21.1
    Stream #0.0: Audio: mp2, 48000 Hz, 2 channels, s16, 128 kb/s
Stream mapping:
  Stream #0.0 -> #0.0
Press ctrl-c to stop encoding
size=     462kB time=29.54 bitrate= 128.0kbits/s

I'm using web sockets to then write the stream to all connected clients. The stream looks something like this

request.on('data', function(data){
    console.log(data);
    audioSocket.broadcast(data, {binary:true});
});

//stream output
<Buffer 78 00 dc ff 3b 00 1e 00 27 00 0d 00 72 00 f7 ff 4f 00 0e 00 7d 00 f5 ff 43 00 e2 ff 15 00 e5 ff 3e 00 db ff e9 ff d6 ff 24 00 ad ff ad ff a3 ff a3 ff 53 ...>

I'm trying to play this stream using the web audio api but I haven't been able to get past the decodeAudioData step. It always calls the error callback and err is always null.

var audio = new WebSocket('ws://localhost:8088/');
audio.binaryType = "arraybuffer";
var context = new webkitAudioContext();

audio.onmessage = function(data){
    var buf  = data.data;
    d = data;
    context.decodeAudioData(
        buf,
        function(buffer){    console.log(buffer);    },     //success handler
        function(err){   console.log('error: ', err)    }   //error handler
    );
}

How can I get this stream played?

Sorry, not super familiar with ffmpeg... that's mp2 or mp3? Only the latter can be decoded (as far as I know) by most browsers. — Kevin Ennis
Do you really want WebAudio for this? To play an audio stream HTML5 Audio is enough. <audio src="http://localhost:8086/abc" />, and no need for a socket. — katspaugh

Steve -Cutter- Blades Steve -Cutter- Blades · Accepted Answer · 2018-10-12T12:19:18

It sounds like you're trying to create a media stream, rather than decoding an actual file. I could be wrong, but I would imagine that your socket is streaming data in, rather than a pure 'load', and in that instance decodeAudioData wouldn't serve your needs. Look at the AudiContext.createMediaStreamSource api.

How can I play an ArrayBuffer stream on a browser with web audio API?

1 Answers