WebRTC to Gstreamer Bridge

Question

I'm trying to stream audio from a browser to a gstreamer pipeline on a server.

I'm currently using Kurento, and modifying the Hello World example to try to connect an RTP Endpoint to the pipeline -- but am having trouble.

I know the media is getting there because when I swap in a Recording Endpoint, I get a valid recording.

The Kurento Node JS is:

pipeline.create("RtpEndpoint", {}, function(error, rtpEndpoint) {
        if (error) {
            console.log("Recorder problem");
            return sendError(res, 500, error);
        }

        console.log("Creating WebRtcEndpoint");
        pipeline.create('WebRtcEndpoint', function(error, webRtcEndpoint) {
            if (error) {
                return sendError(res, 500, error);
            }

            console.log("Processing sdpOffer at server and generating sdpAnswer");
            webRtcEndpoint.processOffer(sdpOffer, function(error, sdpAnswer) {
                if (error) {
                    webRtcEndpoint.release();
                    return sendError(res, 500, error);
                }

                console.log("Connecting loopback");
                webRtcEndpoint.connect(webRtcEndpoint, function(error) {
                    if(error){
                        webRtcEndpoint.release();
                        return sendError(res, 500, error);
                    }
                    console.log("Sending sdpAnswer to client");
                    console.log(sdpAnswer);

                    webRtcEndpoint.connect(rtpEndpoint, function(error) {
                        if(error) {
                            webRtcEndpoint.release();
                            return sendError(res, 500, error);
                        }
                        rtpEndpoint.generateOffer(function(error, offer) {
                            fs.writeFile('/tmp/test.sdp',offer);
                            console.log("RTP OFFER GENERATED.");
                        });
                    });

                    res.type('application/sdp');
                    res.send(sdpAnswer);
                });     
            });
        });
    });

and my GStreamer pipeline is:

gst-launch-1.0 -vvvv filesrc location=/tmp/test.sdp ! sdpdemux ! decodebin ! autovideosink

which returns

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'autovideosink0-actual-sink-glimage': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstUDPSrc:udpsrc0: timeout = 10000000000
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstUDPSrc:udpsrc2: timeout = 10000000000
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0/GstRtpSession:rtpsession0.GstPad:send_rtcp_src: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0.GstGhostPad:send_rtcp_src_0: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0.GstGhostPad:send_rtcp_src_0.GstProxyPad:proxypad4: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0/GstRtpSession:rtpsession1.GstPad:send_rtcp_src: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0.GstGhostPad:send_rtcp_src_1: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstUDPSink:udpsink1.GstPad:sink: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0.GstGhostPad:send_rtcp_src_1.GstProxyPad:proxypad7: caps = application/x-rtcp
ERROR: from element /GstPipeline:pipeline0/GstSDPDemux:sdpdemux0: Could not read from resource.
Additional debug info:
gstsdpdemux.c(1213): gst_sdp_demux_handle_message (): /GstPipeline:pipeline0/GstSDPDemux:sdpdemux0:
Could not receive any UDP packets for 10.0000 seconds, maybe your firewall is blocking it.
Execution ended after 0:00:10.062018001
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

It doesnt work on FFPMEG, VLC, etc -- results are similar to "Attempt 5.3" here: https://altanaitelecom.wordpress.com/2015/02/26/continue-streaming-broadcasting-live-video-call-to-non-webrtc-supported-browsers-and-media-players/

I don't think theres a firewall issue as the pipeline and kurento instance are on the same virtual machine (which has no firewall) -- and the recording endpoint works. Is it being linked badly? Is there an easier way?

lulop lulop · Accepted Answer · 2015-07-23T13:48:12

Using RtpEndpoint is tricky because you need to complete de SDP negotiation. This means that somewhere after the

rtpEndpoint.generateOffer(...

you should be invoking

rtpEndpoint.processAnswer(sdpAnswer, ...)

The tricky part is that you need to obtain the sdpAnswer from your gstreamer pipeline and this is not trivial if you want to it just using gst-launch. Probably your best option is to write a small programm creating the pipeline and generating the sdpAnswer so that you can give it back to the rtpEndpoint through your signaling mechanism.

WebRTC to Gstreamer Bridge

1 Answers