Play stream from gstreamer in browser

Question

I want to play stream from gstreamer in a web browser.

I played around a with RTP, WebRTC and SDP files but, while VLC was able to connect to stream by simple SDP, browsers were not. I later understood that WebRTC requires secure connection which only complicates things and is not needed for my purposes. I stumbled upon Media Source Extension (MSE) of html5, which seems that it could help, but I'm not able to find some comprehensive tutorial or appropriate specs on how to get gstreamer to stream correct data and later how to play them using MSE. I'm also not sure about latency with using MSE.

So is there a way to play stream from gstreamer in a browser? Thanks.

Márius Rak Márius Rak · Accepted Answer · 2019-09-04T11:54:41

Using node webrtc project, I was able to combine output from gstreamer with webrtc call. For gstreamer, there is a project which enables it's use with node gstreamer superficial. So basically, you need to run gstremaer process from node process, which can then control output from gstremaer. On every gstreamer frame there is a callback called which takes the frame and can send it to webrtc calls.

Then an webrtc calls needs to be implemented. There is required some signaling protocol for calls. One side of the call will be the server and another will be the client's browser, instead of two browsers. Then a video track will be created where frames from gstreamer superficial will be pushed.

const { RTCVideoSource } = require("wrtc").nonstandard;
const gstreamer = require("gstreamer-superficial");

const source = new RTCVideoSource();
// This is WebRTC video track which should be used with addTransceiver see below
const track = source.createTrack();

const frame = {
        width: 1920,
        height: 1080,
        data: null
};

const pipeline = new gstreamer.Pipeline("v4l2src ! videorate ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=25/1 ! videoconvert ! video/x-raw,format=I420 ! appsink name=sink");
const appsink = pipeline.findChild("sink");
const pull = function() {
        appsink.pull(function(buf, caps) {
                if (buf) {
                        frame.data = new Uint8Array(buf);
                        try {
                                source.onFrame(frame);
                        } catch (e) {}
                        pull();
                } else if (!caps) {
                        console.log("PULL DROPPED");
                        setTimeout(pull, 500);
                }
        });
};

pipeline.play();
pull();

// Example:
const useTrack = SomeRTCPeerConnection => SomeRTCPeerConnection.addTransceiver(track, { direction: "sendonly" });

Play stream from gstreamer in browser

1 Answers