QT, C++: fast way to draw live image stream from camera on QGraphicsview

Question

I'm writing a QT GUI application in wich a live stream of a connected camera is shown on a QGraphicsview. Therefore an openCV image is first converted to a QImage and than to a QPixmap. This is added to the QGraphicsScene of the QGraphicsView.

The bandwidth is not a problem, the cameras are connected via ethernet or USB.

I am testing the performance with the Analyze Toole build in Visual Studio 2012 and it shows that the conversion to the QPixmap is very slow and takes 60% of the computation time (of displaying the image), so that I end up with 1 FPS or so. The images are 2560 by 1920 or even bigger. Scaling the cv::Ptr stream_image befor converting it to a QImage improves the performance significantly but I need all the image detail in the image.

EDIT Here is some code how I do the conversion:

cv::Ptr<IplImage> color_image;
// stream_image is a cv::Ptr<IplImage> and holds the current image from the camera
if (stream_image->nChannels != 3) {
    color_image = cvCreateImage(cvGetSize(stream_image), IPL_DEPTH_8U, 3);
    cv::Mat gr(stream_image);
    cv::Mat col(color_image);
    cv::cvtColor(gr, col, CV_GRAY2BGR);
}
else {
    color_image = stream_image;
}

QImage *tmp = new QImage(color_image->width, color_image->height, QImage::Format_RGB888);
memcpy(tmp->bits(), color_image->imageData, color_image->width * color_image->height * 3);


// update Scene
m_pixmap = QPixmap::fromImage(*tmp); // this line takes the most time!!!
m_scene->clear();
QGraphicsPixmapItem *item = m_scene->addPixmap(m_pixmap);
m_scene->setSceneRect(0,0, m_pixmap.width(), m_pixmap.height());


delete tmp;
m_ui->graphicsView->fitInView(m_scene.sceneRect(),Qt::KeepAspectRatio);


m_ui->graphicsView->update();

EDIT 2 I tested the method from from Thomas answer, but it is as slow as my method.

QPixmap m_pixmap = QPixmap::fromImage(QImage(reinterpret_cast<uchar const*>(color_image->imageData), 
color_image->width, 
color_image->height, 
QImage::Format_RGB888));

EDIT 3 I tried to incorporate Thomas second suggestion:

color_image = cvCreateImage(cvGetSize(resized_image), IPL_DEPTH_32F, 3);
//[...]   
QPixmap m_pixmap = QPixmap::fromImage(QImage( 
reinterpret_cast<uchar const*>( color_image->imageData), 
color_image->width, 
color_image->height, 
QImage::Format_RGB32));

But that crashes when the drawEvent of the Widget is called.

Q: Is there a way to display the image stream in a QGraphicsView without converting it to a QPixmap first or any other fast/performant way? The QGraphicsView is importent since I want to add overlays to the image.

Answer 1

I have figured out a solution that works for me but also tested a little with different methods and how they perform:

Method one is performant even in debug mode and takes only 23.7 % of the execution time of the drawing procedure (using ANALYZE in VS2012):

color_image = cvCreateImage(cvGetSize(stream_image), IPL_DEPTH_8U, 4);
cv::Mat gr(stream_image);
cv::Mat col(color_image);
cv::cvtColor(gr, col, CV_GRAY2RGBA,4);

QPixmap m_pixmap = QPixmap::fromImage(QImage(reinterpret_cast<uchar const*>( color_image->imageData), 
                                      color_image->width, 
                                      color_image->height, 
                                      QImage::Format_ARGB32));

Method two is still performant in debug mode taking 42,1% of the execution time. when the following enum is used in QPixmap::fromeImage instead

QImage::Format_RGBA8888

Method three is the one I showed in my question and it is very slow in debug builds being responsible for 68,3% of the drawing workload.

However, when I compile in release all three methods are seamingly equally performant.

Answer 2

This is what I usually do. Use one of the constructors for QImage that uses an existing buffer and then use QPixmap::fromImage for the rest. The format of the buffer should be compatible with the display, such as QImage::Format_RGB32. In this example a vector serves as the storage for the image.

std::vector<QRgb> image( 2560 * 1920 );

QPixmap pixmap = QPixmap::fromImage( QImage( 
    reinterpret_cast<uchar const*>( image.data() ), 
    2560, 
    1920, 
    QImage::Format_RGB32 ) );

Note the alignment constraint. If the alignemnt is not 32-bit aligned, you can use one of the constructors that takes a bytesPerLine argument.

Edit:

If your image is 32bit, then you can write.

QPixmap pixmap = QPixmap::fromImage( QImage( 
    reinterpret_cast<uchar const*>( color_image->imageData ), 
    color_image->width, 
    color_image->height, 
    QImage::Format_RGB32 ) );