What is the need of MediaStream interface in WebRTC?

3
votes

In WebRTC we have MediaStream and MediaStreamTrack interfaces.

MediaStreamTrack represents a audio or video stream of a media source. So a consumer like video or audio tag can simply take an MediaStreamTrack object and and get the stream from it. So what is the need for MediaStream interface?

According to official documentation, MediaStream synchronises one or more tracks. Does that mean it combines multiple streams from tracks and produces a single stream so that we have video data with audio?

For example: Does a video tag read the stream from MediaStream object or reads streams from the individual tracks?

This concept is not explained clearly anywhere.

Thanks in advance.

1
javascripthtmlwebrtcpeerjs

1 Answers

2
votes

MediaStream has devolved into a simple container for tracks, representing video and audio together (a quite common occurrence).

It doesn't "combine" anything, it's merely a convenience keeping pieces together that need to be played back in time synchronization with each other. No-one likes lips getting out of sync with spoken words.

It's not even really technically needed, but it's a useful semantic in APIs for:

  • Getting output from hardware with camera and microphone (usually video and audio), and
  • Connecting it (the output) to a sinc, like the html video tag (which accepts video and audio).
  • Reconstituting audio and video at the far end of an RTCPeerConnection that belong together, in the sense that they should generally be played in sync with each other (browsers have more information about expectations on the far end this way, e.g. if packets from one track are lost but not the other).

Whether this is a useful abstraction may depend on the level of detail you're interested in. For instance the RTCPeerConnection API, which is still in Working Draft stage, has over the last year moved away from streams as inputs and outputs to dealing directly with tracks, because the working group believes that details matter more when it comes to transmission (things like tracking bandwidth use etc.)

In any case, going from one to the other will be trivial:

var tracks = stream.getTracks();
console.log(tracks.map(track => track.kind)); // audio,video
video.srcObject = new MediaStream(tracks);

once browsers implement the MediaStream constructor (slated for Firefox 44).