gstreamer video into a window in python

Question

I have the following pipelines to stream video:

sender:

gst-launch-1.0 rpicamsrc preview=0 ! 'video/x-h264, width=1280, height=720, framerate=15/1,profile=high' ! queue ! rtph264pay ! udpsink host=192.168.0.8 port=50000

receiver:

gst-launch-1.0 udpsrc port=50000 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" ! rtph264depay ! decodebin ! autovideosink

This works fine, but I'd like to do the receiver in python, and direct the video stream into a window, somehow like this:

import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst, Gtk, GdkX11,GstVideo

GObject.threads_init()
Gst.init(None)

class VideoReceiver:
  def __init__(self):

    self.window = Gtk.Window()
    self.window.connect('destroy', self.stop)
    self.window.set_default_size(320, 200)

    self.drawingarea = Gtk.DrawingArea()
    self.window.add(self.drawingarea)

    self.window.show_all()
    self.xid = self.drawingarea.get_property('window').get_xid()

    self.pipeline = Gst.parse_launch ('udpsrc name=udpsrc port=50000'
                                      ' caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264" ! '
                                      'rtph264depay ! decodebin ! autovideosink')

    self.bus = self.pipeline.get_bus()
    self.bus.add_signal_watch()
    self.bus.connect('message::error', self.on_error)

    self.bus.enable_sync_message_emission()
    self.bus.connect('sync-message::element', self.on_sync_message)

  def start(self):
    self.pipeline.set_state(Gst.State.PLAYING)
    Gtk.main()

  def stop(self, window):
    self.pipeline.set_state(Gst.State.NULL)
    Gtk.main_quit()

  def on_sync_message(self, bus, msg):
    if msg.get_structure().get_name() == 'prepare-window-handle':
        print('prepare-window-handle')
        msg.src.set_property('force-aspect-ratio', True)
        msg.src.set_window_handle(self.xid)

  def on_error(self, bus, msg):
    print('on_error():', msg.parse_error())

vr1=VideoReceiver()
vr1.start()

But the window just simply closes when the streaming starts, and the program ends without errors. Any ideas what can be wrong, and how could I direct the video output into a window?

Output if running without root permissions:

$ GST_DEBUG=3 python3.2 test.py

** (test.py:3275): WARNING **: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files
prepare-window-handle
0:00:04.134038733  3275  0x1c72260 ERROR            egladaption gstegladaptation_egl.c:311:gst_egl_adaptation_create_surface:<autovideosink0-actual-sink-eglgles> Can't create surface
0:00:04.135032949  3275  0x1c72260 ERROR            egladaption gstegladaptation.c:461:gst_egl_adaptation_init_surface:<autovideosink0-actual-sink-eglgles> Can't create surface
0:00:04.135378104  3275  0x1c72260 ERROR            egladaption gstegladaptation.c:657:gst_egl_adaptation_init_surface:<autovideosink0-actual-sink-eglgles> Couldn't setup EGL surface
0:00:04.135678780  3275  0x1c72260 ERROR            eglglessink gsteglglessink.c:2132:gst_eglglessink_configure_caps:<autovideosink0-actual-sink-eglgles> Couldn't init EGL surface from window
0:00:04.135971436  3275  0x1c72260 ERROR            eglglessink gsteglglessink.c:2144:gst_eglglessink_configure_caps:<autovideosink0-actual-sink-eglgles> Configuring caps failed
0:00:04.137130443  3275  0x1c78a60 ERROR            eglglessink gsteglglessink.c:2167:gst_eglglessink_setcaps:<autovideosink0-actual-sink-eglgles> Failed to configure caps
0:00:04.137830336  3275  0x1c78a60 ERROR            eglglessink gsteglglessink.c:2167:gst_eglglessink_setcaps:<autovideosink0-actual-sink-eglgles> Failed to configure caps
0:00:04.138175544  3275  0x1c78a60 WARN                GST_PADS gstpad.c:3620:gst_pad_peer_query:<sink:proxypad1> could not send sticky events
0:00:04.157868139  3275  0x1c78a60 ERROR            eglglessink gsteglglessink.c:2167:gst_eglglessink_setcaps:<autovideosink0-actual-sink-eglgles> Failed to configure caps
0:00:04.158217826  3275  0x1c78a60 ERROR            eglglessink gsteglglessink.c:2167:gst_eglglessink_setcaps:<autovideosink0-actual-sink-eglgles> Failed to configure caps
0:00:04.158321940  3275  0x1c78a60 WARN                GST_PADS gstpad.c:3620:gst_pad_peer_query:<sink:proxypad1> could not send sticky events
0:00:04.184023215  3275  0x1c78a60 ERROR            eglglessink gsteglglessink.c:2167:gst_eglglessink_setcaps:<autovideosink0-actual-sink-eglgles> Failed to configure caps
0:00:04.184216600  3275  0x1c78a60 WARN                GST_PADS gstpad.c:3620:gst_pad_peer_query:<sink:proxypad1> could not send sticky events
0:00:04.185187274  3275  0x1c78a60 ERROR            eglglessink gsteglglessink.c:2167:gst_eglglessink_setcaps:<autovideosink0-actual-sink-eglgles> Failed to configure caps
0:00:04.185499825  3275  0x1c78a60 ERROR            eglglessink gsteglglessink.c:2167:gst_eglglessink_setcaps:<autovideosink0-actual-sink-eglgles> Failed to configure caps
0:00:04.186118000  3275  0x1c78a60 WARN             omxvideodec gstomxvideodec.c:2817:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error: Internal data stream error.
0:00:04.186551488  3275  0x1c78a60 WARN             omxvideodec gstomxvideodec.c:2817:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error: stream stopped, reason not-negotiated
0:00:04.187462163  3275  0x1c78a60 ERROR            eglglessink gsteglglessink.c:2167:gst_eglglessink_setcaps:<autovideosink0-actual-sink-eglgles> Failed to configure caps
0:00:04.187758151  3275  0x1c78a60 ERROR            eglglessink gsteglglessink.c:2167:gst_eglglessink_setcaps:<autovideosink0-actual-sink-eglgles> Failed to configure caps
on_error(): (GError('Internal data stream error.',), 'gstomxvideodec.c(2817): gst_omx_video_dec_loop (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:\nstream stopped, reason not-negotiated'

Output as root:

GST_DEBUG=3 sudo python3.2 test.py

** (test.py:3205): WARNING **: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files
prepare-window-handle

So no error from Gstreamer, the window simply closes when the stream starts.

run the python receiver with GST_DEBUG=3 and check the warnings/errors.. but are you not missing a + sign in parse_launch after second line - caps...? — nayana
No, it works both with + and without, but I corrected the code to make it consistent. Unfortunately there's no warning or error messages from Gstreamer, the window simply closes and the program ends when the stream starts. — Zorgmorduk
If I run the program without sudo, then I do have error message, but when I run as root, then the window only closes. I'll post the exact output in both cases. — Zorgmorduk
I am not sure.. the sudo thing may be caused by not propagated GST_DEBUG(or some permissions?).. what is the OS? maybe escape the double quotes? try to prepare the pipeline string in variable and print it to see how it looks like.. also use ximagesink instead of auto.. here is some nice tutorial - chapter 3.2 is what you want.. — nayana

Rolf of Saxony Rolf of Saxony · Accepted Answer · 2015-11-17T08:37:58

I have not used gtk for this purpose only wxpython so just some thoughts.
Try changing:

self.bus.connect('message::error', self.on_error)

to

self.bus.connect('message', self.on_error)

Then pick the bones out of the message there i.e.

t = message.type
if t == Gst.MessageType.ERROR:
    error_0 = Gst.Message.parse_error (message)[0]
    error_1 = Gst.Message.parse_error (message)[1]
    report errors here.......
elif t == Gst.MessageType.EOS:
    print ("End of Audio")
return True

From on_error and on_sync_message always return True
and finally,
try firing the set_window_handle at a window not a drawing area.

gstreamer video into a window in python

1 Answers