Get MediaStreamTrack(audio) from Video

Question

I want to record audio from video element alongside recording from canvas. I have

var stream = canvas.captureStream(29);

Now I am adding audioTrack of video to the stream.

var vStream = video.captureStream();
stream.addTrack(vStream.getAudioTracks()[0]);

But this slows down the performance with every video added. As captureStream() is very heavy on video and it also requires a flag to be switched on in Chrome. Is there a way of creating only audio MediaStream from video element without using captureStream().

Kaiido Kaiido · Accepted Answer · 2017-06-01T08:11:20

Yes, you can use the Web Audio API's method createMediaElementSource which will grab the audio from your mediaElement, and then the createMediaStreamDestination method, which will create an MediaStreamDestination node, which contains an MediaStream.

You then just have to connect it all, and you've got your MediaStream with your MediaElement's audio.

// wait for the video starts playing
vid.play().then(_=> {
  var ctx = new AudioContext();
  // create an source node from the <video>
  var source = ctx.createMediaElementSource(vid);
  // now a MediaStream destination node
  var stream_dest = ctx.createMediaStreamDestination();
  // connect the source to the MediaStream
  source.connect(stream_dest);
  // grab the real MediaStream
  out.srcObject = stream_dest.stream;
  out.play();
  });

The video's audio will be streamed to this audio elements : <br>
<audio id="out" controls></audio><br>
The original video element : <br>
<video id="vid" crossOrigin="anonymous" src="https://dl.dropboxusercontent.com/s/bch2j17v6ny4ako/movie720p.mp4?dl=0" autoplay></video>

Note that you could also connect more sources to this stream, and also that you can combine it with an other video stream with the new MediaStream([stream1, stream2]) Constructor (It's currently the only way to combine different streams on FF, until this bug is fixed, should be soon though).

Get MediaStreamTrack(audio) from Video

1 Answers