Playback custom stream with QMediaPlayer

Question

I searched thoroughly to find an answer to my problem but no other post has been helpful so far. I am developing an application in Qt where I need to playback a video stream which is received through a custom protocol. I found myself trying in every possible way to feed these packets in QMediaPlayer with no success. My idea was to write incoming packets in a QBuffer and then read them from QMediaPlayer. Follows my trial:

/// VideoPlayer.h
class VideoPlayer : public QWidget
{
public slots:
    void play();
    void handlePacket(QByteArray);
    [...]

private:
    QMediaPlayer mediaPlayer;
    QBuffer      buffer;
};

/// VideoPlayer.cpp
VideoPlayer::VideoPlayer(QWidget *parent)
: QWidget(parent)
, mediaPlayer(0, (QMediaPlayer::StreamPlayback))
{
    buffer.open(QBuffer::ReadWrite);
}

void VideoPlayer::handlePacket(QByteArray packet)
{
    buffer.buffer().append(packet);
}

void VideoPlayer::play()
{
    mediaPlayer.setMedia(QMediaContent(), &buffer);
    mediaPlayer.play();
}

With the above QMediaPlayer plays back data in the buffer at the moment of calling mediaPlayer.setMedia(QMediaContent(), &buffer) but seems to ignore that new packets were appended to the buffer. May it be because I am accessing the internal QByteArray directly (I checked that QIODevice::readyRead signal is emitted and it is)? I found no way to make QMediaPlayer play new incoming data other than calling setMedia again. Is there a way to notify QMediaPlayer that media length has changed?

Is there an easier way to make this? I thought about writing my own QIODevice or somehow integrate my packet receiver in the Qt framework to provide my custom stream as a QMediaContent?

Are there any other libraries or methods which would allow me to accomplish this task?

I am using Qt 5.4. Thanks in advance for your help.

possible duplicate of How do i play a stream with QMediaPlayer — p.i.g.
Can you provide more info on what type of media is being played (container/raw RTP or UDP/...). My guess is that since various formats have packet boundaries maybe you are skipping some data that tells the media player new data is available? Anyhow subclassing QIODevice sounds like a good approach. — Rudolfs Bundulis

serg.v.gusev serg.v.gusev · Accepted Answer · 2015-06-16T11:31:48

My answer is not real answer, just sharing my experience and research results. This focused on audio part of stream.

I'm currently working with kind of same issue. I have custom protocol which contains one encoded image and one encoded audio portion within one incoming [from network] data frame. I need to handle image and audio separately:

decode image and send raw RGB data to another process which will display it (lol nevermind)
play audio portion whatever way

Main: i can use only Qt library. This because libVLC [which i use right now for these things] buffering audio/video and i didn't found any way to disable it and play stream in real-time. I should note that incoming stream is from video camera so i need to play it as real-time as possible.

During my quick research, i've found a way to plays something:

// init method
{
    // ...
    mPlayer = new QMediaPlayer(this, QMediaPlayer::StreamPlayback);
    mBuffer = new QBuffer(this);
    mBuffer->open(QIODevice::ReadWrite);
    // ...
}

// trying to play audio portion
void MyCoolPlayer::handleFrameAudio(const QByteArray &audioBlob)
{
    mBuffer->seek(0);
    mBuffer->write(audioBlob);
    mBuffer->seek(0);
    mPlayer->setMedia(QMediaContent(), mBuffer);
    mPlayer->play();
}

This plays some distorted sound but at least it plays something.

Also, if you have some audio/video decoder, you can use:

QPainter::drawImage to draw decoded rgb image data on empty widget
QAudioOutput to plays decoded pcm audio data

Playback custom stream with QMediaPlayer

3 Answers