ISampleGrabber::BufferCB to IplImage; display in OpenCV shows garbled image - C++

Question

I'm using DirectShow to access a video stream, and then using the SampleGrabber filter and interface to get samples from each frame for further image processing. I'm using a callback, so it gets called after each new frame. I've basically just worked from the PlayCap sample application and added a sample filter to the graph.

The problem I'm having is that I'm trying to display the grabbed samples on a different OpenCV window. However, when I try to cast the information in the buffer to an IplImage, I get a garbled mess of pixels. The code for the BufferCB call is below, sans any proper error handling:

STDMETHODIMP BufferCB(double Time, BYTE *pBuffer, long BufferLen)
{
    AM_MEDIA_TYPE type;
    g_pGrabber->GetConnectedMediaType(&type);
    VIDEOINFOHEADER *pVih = (VIDEOINFOHEADER *)type.pbFormat;

    BITMAPINFO* bmi = (BITMAPINFO *)&pVih->bmiHeader;
    BITMAPINFOHEADER* bmih = &(bmi->bmiHeader);
    int channels = bmih->biBitCount / 8;

    mih->biPlanes = 1;
    bmih->biBitCount = 24;
    bmih->biCompression = BI_RGB;

    IplImage *Image = cvCreateImage(cvSize(bmih->biWidth, bmih->biHeight), IPL_DEPTH_8U, channels);     

    Image->imageSize = BufferLen;
    CopyMemory(Image->imageData, pBuffer, BufferLen);
    cvFlip(Image);  

    //openCV Mat creation
    Mat cvMat = Mat(Image, true);
    imshow("Display window", cvMat);                   // Show our image inside it.
    waitKey(2);


    return S_OK;
}

My question is, am I doing something wrong here that will make the image displayed look like this:

Am I missing header information or something?

Roman R. Roman R. · Accepted Answer · 2014-02-20T08:51:45

The quoted code is a part of the solution. You create here an image object of certain width/height with 8-bit pixel data and unknown channel/component count. Then you copy data from another buffer of unknown format.

The only chance for it to work well is that all unknowns amazingly match without your effort. So you basically need to start with checking what media type is exactly on Sample Grabber's input pin. Then, if it is not what you wanted, you have to update your code respectively. It might also be important what is the downstream connection of the SG, and whether it is connected to video renderer in particular.

ISampleGrabber::BufferCB to IplImage; display in OpenCV shows garbled image - C++

1 Answers