How to get multiple streams from WebRTC PeerConnection

Question

According to RTCPeerConnection.ontrack documentation, "ontrack" event suppose to fire for each incoming streams. I have a PeerConnection with two video streams, after connection, "ontrack" fires two times (up to here everything is OK). But both times it sends same stream out, so I end up with two identical video, I am sure sender is sending two different streams, dimension and frame rate of them are different and I can clearly see in chrome://webrtc-internals/ that 2 video streams have different frame size/rate.

Here is PeerConnection ontrack code:

        this.peerConnection.ontrack = function(evt) {
            console.log("PeerConnection OnTrack event: ", evt.streams);
            that.emit('onRemoteStreamAdded', evt.streams);
        };

I don't assume evt.streams has 1 object, so I did not write evt.streams[0].

Here is Chrome console log:

As it is obvious from log getRemoteStreams() returns only one object. How is it possible ontrack fires two time when it has only one stream, and why second RTCRtpTransceiver does not make a new stream?

Keivan Keivan · Accepted Answer · 2021-02-17T17:09:25

I solved it after few hours of struggling with different browsers and reading documentations several times!

Problem starts at MediaStream.id, it suppose to be unique but <video> element in HTML5 only listens to first track inside each stream. PeerConnection adds new transceivers (as MediaStreamTrack) to same MediaStream, so no matter how many times ontrack handler fires, you get exact same MediaStream objects, but each time you have new unique MediaStreamTrack inside the RTCTrackEvent.

Solution is to create new MediaStream object for each new MediaStreamTrack inside ontrack handler.

this.peerConnection.ontrack = function(event) {
    that.emit('onRemoteStreamAdded', new MediaStream([event.track]));
};

Or, more like standard examples:

pc.ontrack = function(event) {
  document.getElementById("received_video").srcObject = new MediaStream([event.track]);
};

How to get multiple streams from WebRTC PeerConnection

2 Answers