Render off screen (with FBO and RenderBuffer) and pixel transfer of color, depth, stencil

Question

I'have to render off screen in OpenGL and then pass the image to a QImage. Plus, just for exercise I'd like to transfer to the CPU also the depth and the stencil buffer.

For drawing offscreen I've used Frame Buffer Object with Render Buffer (and not with texture because I don't need the texture).

Pixel transfer operation with color buffers (the actual raw image) works.. I see what I'm expecting. But depth and stencil are not working.. strange image for depth and nothing with for the stencil.

First, the easy part, what I'm actually drawing:

    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
    glLoadIdentity();
    glTranslatef(-1.5f,0.0f,-6.0f);
    glBegin(GL_TRIANGLES);
    glColor3f(1.0f,0.0f,0.0f);
    glVertex3f( 0.0f, 1.0f, 0.0f);
    glColor3f(0.0f,1.0f,0.0f);
    glVertex3f(-1.0f,-1.0f, -1.0f);
    glColor3f(0.0f,0.0f,1.0f);
    glVertex3f( 1.0f,-1.0f, 0.0f);
    glEnd();

Here the initalization of the FBO and of the 3 render buffer:

// frame buffer object
glGenFramebuffers(1, &fboId);
glBindFramebuffer(GL_FRAMEBUFFER, fboId);

// render buffer as color buffer
glGenRenderbuffers(1, &colorBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, colorBuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA, width, height);
glBindRenderbuffer(GL_RENDERBUFFER, 0);
// attach render buffer to the fbo as color buffer
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, colorBuffer);

// render buffer as depth buffer
glGenRenderbuffers(1, &depthBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, depthBuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT, width, height);
glBindRenderbuffer(GL_RENDERBUFFER, 0);
// attach render buffer to the fbo as depth buffer
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthBuffer);

glGenRenderbuffers(1, &stencilBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, stencilBuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_STENCIL_INDEX, width, height);
glBindRenderbuffer(GL_RENDERBUFFER, 0);
// attach render buffer to the fbo as stencil buffer
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_STENCIL_BUFFER, GL_RENDERBUFFER, stencilBuffer);

And finally here 3 methods for the pixel transfer:

QImage FBO::getImage()
{
    // this is called Pixel Transfer operation: http://www.opengl.org/wiki/Pixel_Transfer
    uchar *pixels;
    pixels = new uchar[width * height * 4];
    for(int i=0; i < (width * height * 4) ; i++ ) {
        pixels[i] = 0;
    }

    glBindFramebuffer(GL_FRAMEBUFFER, fboId);
    glReadPixels( 0,0,  width, height, GL_RGBA, GL_UNSIGNED_BYTE, pixels);

    glBindFramebuffer(GL_FRAMEBUFFER, 0);

    QImage qi = QImage(pixels, width, height, QImage::Format_ARGB32);
    qi = qi.rgbSwapped();

    return qi;
}

QImage FBO::getDepth()
{
    // this is called Pixel Transfer operation: http://www.opengl.org/wiki/Pixel_Transfer
    uchar *pixels;
    pixels = new uchar[width * height * 4];
    for(int i=0; i < (width * height * 4) ; i++ ) {
        pixels[i] = 0;
    }

    glBindFramebuffer(GL_FRAMEBUFFER, fboId);
    glBindRenderbuffer(GL_RENDERBUFFER, depthBuffer);
    glReadPixels( 0,0,  width, height, GL_DEPTH_COMPONENT, GL_FLOAT, pixels);

    glBindFramebuffer(GL_FRAMEBUFFER, 0);

    QImage qi = QImage(pixels, width, height, QImage::Format_ARGB32);
    qi = qi.rgbSwapped();

    return qi;
}

QImage FBO::getStencil()
{
    // this is called Pixel Transfer operation: http://www.opengl.org/wiki/Pixel_Transfer
    uchar *pixels;
    pixels = new uchar[width * height * 4];
    for(int i=0; i < (width * height * 4) ; i++ ) {
        pixels[i] = 0;
    }

    glBindFramebuffer(GL_FRAMEBUFFER, fboId);
    glBindRenderbuffer(GL_RENDERBUFFER, stencilBuffer);
    glReadPixels( 0,0,  width, height, GL_STENCIL_INDEX, GL_FLOAT, pixels);

    glBindFramebuffer(GL_FRAMEBUFFER, 0);

    QImage qi = QImage(pixels, width, height, QImage::Format_ARGB32);
    qi = qi.rgbSwapped();

    return qi;

}

Here the 2 screenshot (color and depth, with stencil I get an empty QImage):

The color one is exactly what I'm drawing (flipped but it is normal I think). The depth.. I'm expecting a white image with a gradient gray triangle.. For sure I'm making some mistake in the format of the image (GL_FLOAT?) but I've tried some combinations and this is the best result.. a violet background with a glitchy colors inside it.. and with the stencil buffer.. i'm expecting a white triangle outline in a black background but.. no idea why I don't see anything..

EDIT:

Ok, never use stencil buffer alone, so I've refactored a little bit.

when declaring the FBO:

// render buffer for both depth and stencil buffer
glGenRenderbuffers(1, &depthStencilBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, depthStencilBuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH24_STENCIL8, width, height);
glBindRenderbuffer(GL_RENDERBUFFER, 0);
// attach render buffer to the fbo as depth buffer
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_RENDERBUFFER, depthStencilBuffer);

pixel transfer of the depth:

QImage FBO::getDepth()
{
    std::vector<uchar> pixels;
    pixels.reserve(width * height*4);
    for(int i=0; i < (width * height*4) ; i++ ) {
        pixels.push_back(0);
    }

    glBindFramebuffer(GL_FRAMEBUFFER, fboId);
    glBindRenderbuffer(GL_RENDERBUFFER, depthStencilBuffer);
    glReadPixels( 0,0,  width, height,  GL_DEPTH24_STENCIL8, GL_UNSIGNED_BYTE, pixels.data());

    glBindFramebuffer(GL_FRAMEBUFFER, 0);

    std::vector<uchar> _pixels;
    _pixels.reserve(width * height*4);
    for(int i=0; i < (width * height*4) ; i++ ) {
        uchar p_red = pixels[i];
        uchar p_green = pixels[i+1];
        uchar p_blue = pixels[i+2];

        uchar p_stencil = pixels[i+3];

        _pixels.push_back(p_red);
        _pixels.push_back(p_green);
        _pixels.push_back(p_blue);

        _pixels.push_back(255); // alpha
    }

    QImage qi = QImage(_pixels.data(), width, height, QImage::Format_ARGB32);
    //qi = qi.rgbSwapped();

    return qi;
}

the stencil is similar but using p_stencil with rgb component

.. the result image is a black image for both depth and stencil

EDIT

thanks to Nicolas answer I managed to use a renderbuffer for both depth and stencil buffer and extract the depth component to fit a QImage::Format_ARGB32 with this code:

QImage FBO1::getDepth()
{
    // sizeof( GLuint ) = 4 byte
    // sizeof( uchar ) = 1 byte

    std::vector<GLuint> pixels(width * height);
    glBindFramebuffer(GL_FRAMEBUFFER, fboId);
    glBindRenderbuffer(GL_RENDERBUFFER, depthStencilBuffer);
    glReadPixels( 0,0,  width, height,  GL_DEPTH_STENCIL, GL_UNSIGNED_INT_24_8, pixels.data());
    glBindFramebuffer(GL_FRAMEBUFFER, 0);

    std::vector<uchar> _pixels;
    for(int i=0; i < (width * height) ; i++ ) {
        GLuint color = pixels[i];

        float temp = (color & 0xFFFFFF00) >> 8; // 24 bit of depth component

        // remap temp that goes from 0 to 0xFFFFFF (24 bits) to
        // _temp that goes from 0 to 0xFF (8 bits)
        float _temp = (temp / 0xFFFFFF) * 0xFF;

        _pixels.push_back((uchar)_temp);
        _pixels.push_back((uchar)_temp);
        _pixels.push_back((uchar)_temp);
        _pixels.push_back(255);
    }

    QImage qi = QImage(_pixels.data(), width, height, QImage::Format_ARGB32);
    qi = qi.rgbSwapped();

    return qi;
}

.. Still some problems with stencil component (the following code does not work: it produces glitches image):

QImage FBO1::getStencil()
{
    // sizeof( GLuint ) = 4 byte
    // sizeof( uchar ) = 1 byte

    std::vector<GLuint> pixels(width * height);
    glBindFramebuffer(GL_FRAMEBUFFER, fboId);
    glBindRenderbuffer(GL_RENDERBUFFER, depthStencilBuffer);
    glReadPixels( 0,0,  width, height,  GL_DEPTH_STENCIL, GL_UNSIGNED_INT_24_8, pixels.data());
    glBindFramebuffer(GL_FRAMEBUFFER, 0);

    std::vector<uchar> _pixels;
    for(int i=0; i < (width * height); i++ ) {
        GLuint color = pixels[i];

        uchar temp = (color & 0x000000FF); // last 8 bit of depth component

        _pixels.push_back(temp);
        _pixels.push_back(temp);
        _pixels.push_back(temp);
        _pixels.push_back(255);
    }

    QImage qi = QImage(_pixels.data(), width, height, QImage::Format_ARGB32);
    qi = qi.rgbSwapped();

    return qi;
}

Answer 1

The depth.. I'm expecting a white image with a gradient gray triangle..

Your code doesn't suggest that.

uchar *pixels;
pixels = new uchar[width * height * 4];

This creates an array of integers. It's also a memory leak; use std::vector<uchar> instead.

glReadPixels( 0,0,  width, height, GL_DEPTH_COMPONENT, GL_FLOAT, pixels);

This tells OpenGL to write floats to your array of integers. So OpenGL will treat pixels as an array of GLfloats when writing.

QImage qi = QImage(pixels, width, height, QImage::Format_ARGB32);

I'll assume that this is going to interpret this as some kind of 8-bit-per-component integer image format. So you're interpreting floats as 8-bit integers. There's no reason to expect this to behave rationally.

You should either be reading the depth buffer as floats and converting that to your pixel colors in a more reasonable way, or you should be reading the depth buffer as an integer value, letting OpenGL do the greyscale conversion for you.

Also, you should always be using a combined depth/stencil image, not two separate renderbuffers.

---

glReadPixels( 0,0,  width, height,  GL_DEPTH24_STENCIL8, GL_UNSIGNED_BYTE, pixels.data());

You don't seem to understand how pixel transfers work.

First, the pixel transfer format specifies which components that you're reading. It does not specify their sizes. GL_DEPTH24_STENCIL8 is an image format, not a pixel transfer format. If you want to read the depth and stencil from an image, you use GL_DEPTH_STENCIL. Pixel transfer formats don't have sizes.

So this function is just giving you an OpenGL error.

The size comes from the second parameter, the pixel transfer type. In this case GL_UNSIGNED_BYTE means that it will read the depth and stencil, convert each into an unsigned, 8-bit value, and store two of those per-pixel into pixels.data().

Depth buffers only store 1 value per pixel. Depth/stencil only store 2. You cannot copy them into a 4-component-per-pixel format with OpenGL. Therefore, however you build your QImage later, it must me some method that takes 1 or 2 values per pixel.

Generally speaking, if you want to read the depth buffer, and you want the depth buffer's data to actually be meaningful, you do this:

std::vector<GLuint> pixels(width * height);  //std::vector value-initializes its elements.

glBindFramebuffer(GL_FRAMEBUFFER, fboId);
glReadPixels( 0,0,  width, height,  GL_DEPTH_STENCIL, GL_UNSIGNED_INT_24_8, pixels.data());

How you put those in a QImage for visualization is up to you. But this gets you an array of unsigned ints, where the high 24 bits are the depth component, and the low 8 bits are the stencil.

Answer 2

Well, the depth picture looks like one can excpect from this code:

 glReadPixels( 0,0,  width, height, GL_DEPTH_COMPONENT, GL_FLOAT, pixels);

 glBindFramebuffer(GL_FRAMEBUFFER, 0);

 QImage qi = QImage(pixels, width, height, QImage::Format_ARGB32);

QImage doesn't know how to deal with floats, so it interprets each every group of 32 bits (within a float) as if they were ARGB components, 8 bits each. A float has a 25 bits mantissa in the lower bits, which nicely maps into those components. Those bands you see is simply the increasing mantissa, cropped to some number of bits.

QImage is really a very limited thing (it can't even deal with HDR configurations, like 16 bits per channel, which is kind of frustrating). Anyway, your best bet is to convert this floats into the range 0…255 and pass that as a grayscale image to QImage.

Also separate depth stencil is ditremental for performance. Always use a combined format, where possible.

… well, there comes Nicol Bolas' answer, which is right the same as I wrote. Plus he pointed out the memory leak.