Getting OpenGL stencil buffer clipping working on glkit

Question

I'm trying to reproduce in a GLKViewController the effect in this stencil buffer tutorial here in which a stencil is used for clipping.

In my drawinrect method I have:

- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect {


    [self.effect prepareToDraw];

    glEnableVertexAttribArray(GLKVertexAttribPosition);
    glVertexAttribPointer(GLKVertexAttribPosition, 3, GL_FLOAT, GL_FALSE, sizeof(Vertex), (const GLvoid *) offsetof(Vertex, Position));
    glEnableVertexAttribArray(GLKVertexAttribTexCoord0);
    glVertexAttribPointer(GLKVertexAttribTexCoord0, 2, GL_FLOAT, GL_FALSE, sizeof(Vertex), (const GLvoid *) offsetof(Vertex, TexCoord));

    glBindBuffer(GL_ARRAY_BUFFER, _vertexBuffer);
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, _indexBuffer);

    glClearColor(0.5, 0.5, 0.5, 1.0);
    glClear( GL_DEPTH_BUFFER_BIT | GL_COLOR_BUFFER_BIT);



    glClear(GL_DEPTH_BUFFER_BIT);
    glEnable(GL_STENCIL_TEST);
    glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
    glDepthMask(GL_FALSE);
    glStencilFunc(GL_NEVER, 1, 0xFF);
    glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP);  // draw 1s on test fail (always)

    // draw stencil pattern
    glStencilMask(0xFF);
    glClear(GL_STENCIL_BUFFER_BIT);  // needs mask=0xFF




    glDrawElements(GL_TRIANGLES, 1*6, GL_UNSIGNED_SHORT, 0);

     glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
    glDepthMask(GL_TRUE);
    glStencilMask(0x00);


    // draw only where stencil's value is 1
    glStencilFunc(GL_EQUAL, 1, 0xFF);


    glDrawElements(GL_TRIANGLES, nLiveSquares*6, GL_UNSIGNED_SHORT, 0);


    glDisable(GL_STENCIL_TEST);

}

The vertexbuffer contains info for squares, which hold textures.

The effect I'm trying to get is to have the first square be used to define the stencil shape and then when I draw the scene, anything which lies outside of the first square is clipped. Hence, I have used drawElements twice - firstly to draw the clipping square and then to draw all the squares.

I also have another method used for the OpenGL set-up, called once at the start of the program. As suggested by an answer to this stackoverflow question, this includes:

GLuint depthStencilRenderbuffer;

glGenRenderbuffers(1, &depthStencilRenderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, depthStencilRenderbuffer);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthStencilRenderbuffer);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_STENCIL_ATTACHMENT, GL_RENDERBUFFER, depthStencilRenderbuffer);

Running this results in nothing except a screen containing the grey clear color. It appears that everything has been stenciled out (or is not drawn to the screen).

Have you set your GLKView's drawableStencilFormat to something appropriate? — rickster
I have to ask: Why are you clearing your depth buffer twice in a row? glClear ( GL_DEPTH_BUFFER_BIT | GL_COLOR_BUFFER_BIT); followed by glClear (GL_DEPTH_BUFFER_BIT). — Andon M. Coleman

Christian J. B. Christian J. B. · Accepted Answer · 2014-05-04T21:17:20

Solution:

i) I changed the initialization to:

GLuint depthStencilRenderbuffer;
glBindRenderbufferOES(GL_RENDERBUFFER_OES, depthStencilRenderbuffer);
glRenderbufferStorageOES(GL_RENDERBUFFER_OES,
                         GL_DEPTH24_STENCIL8_OES,
                         self.view.bounds.size.width,
                         self.view.bounds.size.height);

I'm not sure what the difference is, but this seems to work.

ii) In the set up I use:

GLKView *view = (GLKView *)self.view;
view.context = self.context;
view.drawableDepthFormat = GLKViewDrawableDepthFormat24;
view.drawableStencilFormat = GLKViewDrawableStencilFormat8;

Thanks to @rickster for his suggestion.

Getting OpenGL stencil buffer clipping working on glkit

1 Answers