Gstreamer tee and mux

Question

The following Gstreamer pipeline works OK on i.MX6 platform:

gst-launch-1.0 imxv4l2videosrc ! imxvideoconvert_ipu deinterlace=3 ! clockoverlay ! tee name = tp tp. ! queue2 ! imxg2dvideosink tp. ! queue2 ! vpuenc_h264 bitrate=5000 ! tee name=tp2 tp2. ! queue2 max-size-buffers=0 max-size-time=0 max-size-bytes=0 ! qtmux name=mux ! filesink location=./test.mp4 tp2. ! queue2 max-size-buffers=0 max-size-time=0 max-size-bytes=0 ! rtph264pay ! udpsink host=192.168.30.151 port=5000

However, when I try to link tp2 to qtmux queue using Gstreamer API in the following way:

    /* Build the gst_RecPipeline in streaming mode */
    gst_bin_add_many(GST_BIN(recPipe.pipeline), recPipe.tee2, recPipe.rtpQueue, recPipe.rtpPay, recPipe.udpSink, NULL);
    if (gst_element_link_many(recPipe.videoQueue, recPipe.vidEnc, recPipe.tee2, NULL) != TRUE ||
            gst_element_link_many(recPipe.videoMuxQueue, recPipe.mux, recPipe.fileSink, NULL) != TRUE ||
            gst_element_link_many(recPipe.rtpQueue, recPipe.rtpPay, recPipe.udpSink, NULL) != TRUE) {
        g_printerr("Elements could not be linked.\n");
        gst_object_unref(recPipe.pipeline);
        return -1;
    }

    /* Manually link the Tee, which has "Request" pads */
    tee_src_pad_template = gst_element_class_get_pad_template(GST_ELEMENT_GET_CLASS (recPipe.tee2), "src_%u");
    tee_1_pad = gst_element_request_pad(recPipe.tee2, tee_src_pad_template, NULL, NULL);
    gst_pad_set_caps(tee_1_pad, caps);
    q1_pad = gst_element_get_static_pad (recPipe.videoMuxQueue, "sink");
    gst_pad_set_caps(q1_pad, caps);
    tee_2_pad = gst_element_request_pad(recPipe.tee2, tee_src_pad_template, NULL, NULL);
    q2_pad = gst_element_get_static_pad (recPipe.rtpQueue, "sink");

    if ((err = gst_pad_link(tee_1_pad, q1_pad)) != GST_PAD_LINK_OK) {
        g_printerr("Tee2 for q1 could not be linked, err=%d.\n", err);
        gst_object_unref(recPipe.pipeline);
        return -1;
    }
    if (gst_pad_link(tee_2_pad, q2_pad) != GST_PAD_LINK_OK) {
        g_printerr("Tee2 for q2 could not be linked.\n");
        gst_object_unref(recPipe.pipeline);
        return -1;
    }

the call gst_pad_link(tee_1_pad, q1_pad) fails with an error -4 (GST_PAD_LINK_NOFORMAT)

If I remove recPipe.mux from the pipeline, it works OK.

Any ideas why I cannot link tee to mux and how to fix it?

Answer 1

The issue actually was an attempt to link muxer directly without using request pads. After I properly implemented linking with muxer request pads, the code works fine.

Answer 2

qtmux and rtph264pay require different formats for their h264 stream. The payloader wants 'byte-stream' while qtmux wants 'avc'.

I don't know what format does vpuenc_h264 provide but you could use h264parse on both branches (after the tee) to convert between h264 stream formats.

Also, the muxer has request pads as well. So you should request the sink pads similarly to what you do with tee.