WebRTC - can't display remote mediaStream

0
votes

This is driving me up the walls. I've been working on a webRTC communication app. Basic use case is failing me.

So I can create the RTCPeerConnection just fine, add a freshly grabbed mediaStream to it, peers exchange offer/answer/iceCandidates, connection successful and I can see the mediaStream object arrive correctly at the other end of the connection when the .onaddstream callback is called.

All fine up to here but now I am unable to paint the media stream in a video element. I've tried replacing the src of an existing video element and also creating a new video element from scratch. Nothing seems to work.

Just in case I also checked the tracks on the mediaStream received. All looks fine. One for audio, one for video and both enabled = true.

Anyone bumped into this?

UPDATED

Offer:

v=0
↵o=- 6956171358659097378 2 IN IP4 127.0.0.1
↵s=-
↵t=0 0
↵a=group:BUNDLE audio video
↵a=msid-semantic: WMS IqUsuHBZTllg61x1bh2v6hK34e2odmLWZvlc
↵m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
↵c=IN IP4 0.0.0.0
↵a=rtcp:9 IN IP4 0.0.0.0
↵a=ice-ufrag:vXOp
↵a=ice-pwd:J74PQYw8MI2YKF//VXvmpsA/
↵a=fingerprint:sha-256 CB:B5:85:14:ED:2E:1E:8D:0C:2E:69:86:58:F4:B1:52:D0:C4:8B:1B:C8:72:08:5C:16:B6:E3:F8:FF:EE:E2:FE
↵a=setup:actpass
↵a=mid:audio
↵a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
↵a=sendrecv
↵a=rtcp-mux
↵a=rtpmap:111 opus/48000/2
↵a=rtcp-fb:111 transport-cc
↵a=fmtp:111 minptime=10;useinbandfec=1
↵a=rtpmap:103 ISAC/16000
↵a=rtpmap:104 ISAC/32000
↵a=rtpmap:9 G722/8000
↵a=rtpmap:0 PCMU/8000
↵a=rtpmap:8 PCMA/8000
↵a=rtpmap:106 CN/32000
↵a=rtpmap:105 CN/16000
↵a=rtpmap:13 CN/8000
↵a=rtpmap:110 telephone-event/48000
↵a=rtpmap:112 telephone-event/32000
↵a=rtpmap:113 telephone-event/16000
↵a=rtpmap:126 telephone-event/8000
↵a=ssrc:1609504167 cname:b7o7HCuRjcMovZC0
↵a=ssrc:1609504167 msid:IqUsuHBZTllg61x1bh2v6hK34e2odmLWZvlc 96ee4822-2115-4704-879c-9ebd87bd7819
↵a=ssrc:1609504167 mslabel:IqUsuHBZTllg61x1bh2v6hK34e2odmLWZvlc
↵a=ssrc:1609504167 label:96ee4822-2115-4704-879c-9ebd87bd7819
↵m=video 9 UDP/TLS/RTP/SAVPF 96 98 100 102 127 97 99 101 125
↵c=IN IP4 0.0.0.0
↵a=rtcp:9 IN IP4 0.0.0.0
↵a=ice-ufrag:vXOp
↵a=ice-pwd:J74PQYw8MI2YKF//VXvmpsA/
↵a=fingerprint:sha-256 CB:B5:85:14:ED:2E:1E:8D:0C:2E:69:86:58:F4:B1:52:D0:C4:8B:1B:C8:72:08:5C:16:B6:E3:F8:FF:EE:E2:FE
↵a=setup:actpass
↵a=mid:video
↵a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
↵a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
↵a=extmap:4 urn:3gpp:video-orientation
↵a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
↵a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
↵a=sendrecv
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:96 VP8/90000
↵a=rtcp-fb:96 ccm fir
↵a=rtcp-fb:96 nack
↵a=rtcp-fb:96 nack pli
↵a=rtcp-fb:96 goog-remb
↵a=rtcp-fb:96 transport-cc
↵a=rtpmap:98 VP9/90000
↵a=rtcp-fb:98 ccm fir
↵a=rtcp-fb:98 nack
↵a=rtcp-fb:98 nack pli
↵a=rtcp-fb:98 goog-remb
↵a=rtcp-fb:98 transport-cc
↵a=rtpmap:100 H264/90000
↵a=rtcp-fb:100 ccm fir
↵a=rtcp-fb:100 nack
↵a=rtcp-fb:100 nack pli
↵a=rtcp-fb:100 goog-remb
↵a=rtcp-fb:100 transport-cc
↵a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
↵a=rtpmap:102 red/90000
↵a=rtpmap:127 ulpfec/90000
↵a=rtpmap:97 rtx/90000
↵a=fmtp:97 apt=96
↵a=rtpmap:99 rtx/90000
↵a=fmtp:99 apt=98
↵a=rtpmap:101 rtx/90000
↵a=fmtp:101 apt=100
↵a=rtpmap:125 rtx/90000
↵a=fmtp:125 apt=102
↵a=ssrc-group:FID 3183126486 3711718742
↵a=ssrc:3183126486 cname:b7o7HCuRjcMovZC0
↵a=ssrc:3183126486 msid:IqUsuHBZTllg61x1bh2v6hK34e2odmLWZvlc b5ae9b5a-0555-4a68-814d-cfd124dd4b27
↵a=ssrc:3183126486 mslabel:IqUsuHBZTllg61x1bh2v6hK34e2odmLWZvlc
↵a=ssrc:3183126486 label:b5ae9b5a-0555-4a68-814d-cfd124dd4b27
↵a=ssrc:3711718742 cname:b7o7HCuRjcMovZC0
↵a=ssrc:3711718742 msid:IqUsuHBZTllg61x1bh2v6hK34e2odmLWZvlc b5ae9b5a-0555-4a68-814d-cfd124dd4b27
↵a=ssrc:3711718742 mslabel:IqUsuHBZTllg61x1bh2v6hK34e2odmLWZvlc
↵a=ssrc:3711718742 label:b5ae9b5a-0555-4a68-814d-cfd124dd4b27

Answer:

v=0
↵o=- 191937705071419457 2 IN IP4 127.0.0.1
↵s=-
↵t=0 0
↵a=group:BUNDLE audio video
↵a=msid-semantic: WMS
↵m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
↵c=IN IP4 0.0.0.0
↵a=rtcp:9 IN IP4 0.0.0.0
↵a=ice-ufrag:DxU/
↵a=ice-pwd:apFEGyelJKYHQg9eehCSHJ4A
↵a=fingerprint:sha-256 0E:7A:BF:BB:54:04:C1:71:55:2B:6F:36:80:58:71:C0:F9:65:34:92:70:F4:EB:71:1A:97:00:30:7D:CE:E7:CB
↵a=setup:active
↵a=mid:audio
↵a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
↵a=recvonly
↵a=rtcp-mux
↵a=rtpmap:111 opus/48000/2
↵a=rtcp-fb:111 transport-cc
↵a=fmtp:111 minptime=10;useinbandfec=1
↵a=rtpmap:103 ISAC/16000
↵a=rtpmap:104 ISAC/32000
↵a=rtpmap:9 G722/8000
↵a=rtpmap:0 PCMU/8000
↵a=rtpmap:8 PCMA/8000
↵a=rtpmap:106 CN/32000
↵a=rtpmap:105 CN/16000
↵a=rtpmap:13 CN/8000
↵a=rtpmap:110 telephone-event/48000
↵a=rtpmap:112 telephone-event/32000
↵a=rtpmap:113 telephone-event/16000
↵a=rtpmap:126 telephone-event/8000
↵m=video 9 UDP/TLS/RTP/SAVPF 96 98 100 102 127 97 99 101 125
↵c=IN IP4 0.0.0.0
↵a=rtcp:9 IN IP4 0.0.0.0
↵a=ice-ufrag:DxU/
↵a=ice-pwd:apFEGyelJKYHQg9eehCSHJ4A
↵a=fingerprint:sha-256 0E:7A:BF:BB:54:04:C1:71:55:2B:6F:36:80:58:71:C0:F9:65:34:92:70:F4:EB:71:1A:97:00:30:7D:CE:E7:CB
↵a=setup:active
↵a=mid:video
↵a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
↵a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
↵a=extmap:4 urn:3gpp:video-orientation
↵a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
↵a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
↵a=recvonly
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:96 VP8/90000
↵a=rtcp-fb:96 ccm fir
↵a=rtcp-fb:96 nack
↵a=rtcp-fb:96 nack pli
↵a=rtcp-fb:96 goog-remb
↵a=rtcp-fb:96 transport-cc
↵a=rtpmap:98 VP9/90000
↵a=rtcp-fb:98 ccm fir
↵a=rtcp-fb:98 nack
↵a=rtcp-fb:98 nack pli
↵a=rtcp-fb:98 goog-remb
↵a=rtcp-fb:98 transport-cc
↵a=rtpmap:100 H264/90000
↵a=rtcp-fb:100 ccm fir
↵a=rtcp-fb:100 nack
↵a=rtcp-fb:100 nack pli
↵a=rtcp-fb:100 goog-remb
↵a=rtcp-fb:100 transport-cc
↵a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
↵a=rtpmap:102 red/90000
↵a=rtpmap:127 ulpfec/90000
↵a=rtpmap:97 rtx/90000
↵a=fmtp:97 apt=96
↵a=rtpmap:99 rtx/90000
↵a=fmtp:99 apt=98
↵a=rtpmap:101 rtx/90000
↵a=fmtp:101 apt=100
↵a=rtpmap:125 rtx/90000
↵a=fmtp:125 apt=102

addIceCandidates is called correctly on the RTCPeerConnenction.

2
videowebrtcmediastream

2 Answers

0
votes

Check the ice connection state, it should be connected/completed. Then only media will flow from source to destination.

function gotRemoteStream(e) {
  if (remoteVideo.srcObject !== e.streams[0]) {
    remoteVideo.srcObject = e.streams[0];
    console.log('pc received remote stream');
  }
}
//pc.onaddstream = gotRemoteStream; // this got deprecated
pc.ontrack = gotRemoteStream;

Try the Demo & Check the source.

0
votes

Found the issue. The problem was due to a bad communication between developers regarding and API name change. All communications between peers were working correctly but the side that created the offer was not adding the remote description to it's RTC Connection. Therefore the iceConnectionStatus remaind as new and the signallingStatus was 'has-local-descripion'. Once the remote description was set correctly all communications completed and the stream came through.

A bit of nasty one to stop because: - Offer was created & added correctly. Then sent. At this point iceCandidates were being obtained and sent correctly as well. - Offer was received correctly, set as remote and answer created correctly. At this point iceCandidates were being obtained and sent correctly. - The onaddstream event was being triggered and the side that expected to receive the stream.

In my opinion, this event should not be triggered unless the iceConnection is completed. I think it was what had me most confused.

@Ajay, Thanks for your time!