Using stencil test to clip items being drawn in OpenGL-ES

3
votes

I'm currently working on an OpenGL project in Android. Right now, I'm trying to create a "clip" for items to be confined to when drawing, using the stencil buffer. My problem is that I would like to be able to create a clip shape, and then "remove" portions of that clipped shape later on (much like in an Android Canvas calling clipPath() with Region.Op.UNION and then another clipPath() call with Region.Op.DIFFERENCE)

Basically, my intention is to only draw to portions of the screen with a stencil buffer bit of value 1. Therefore, as I understand it, I should be able to draw to the stencil buffer with different glStencilOp functions that will allow me to draw with 1's in some areas, and then perhaps replace them later on with the default 0 in portions of those areas.

As an example test, I tried to clip a "doughnut" on the screen, using something like this (Note: I am using the ShapeRenderer from the LibGDX open source project to draw shapes): 

// Beginning LibGDX shapeRenderer.
shapeRenderer.begin(ShapeType.FILLED);

gl20.glClear(GL20.GL_STENCIL_BUFFER_BIT); // Clear stencil buffer.
gl20.glEnable(GL20.GL_STENCIL_TEST); // Enable stencil test.
gl20.glColorMask(false, false, false, false); // Disable color drawing.
gl20.glStencilMask(0xFF); // Set 0xFF as stencil mask.

// Have stencil test always fail, replace drawn stencil bits with 1.
gl20.glStencilFunc(GL20.GL_NEVER, 1, 0xFF); 
gl20.glStencilOp(GL20.GL_REPLACE, GL20.GL_KEEP,GL20.GL_KEEP);

/* Draw outer circle, libGDX call. 600px radius. */
shapeRenderer.circle(screenRect.width() / 2, screenRect.height() / 2, 600);

// Now, replace newly drawn stencil bits with 0.
gl20.glStencilOp(GL20.GL_ZERO, GL20.GL_KEEP, GL20.GL_KEEP);

/* Draw inner circle, libGDX call. 200px radius. */
shapeRenderer.circle(screenRect.width() / 2, screenRect.height() / 2, 200);

// Re-enable color drawing. 
gl20.glColorMask(true, true, true, true);
// Now, items drawn will be confined to areas with '1' stencil bit.
gl20.glStencilFunc(GL20.GL_EQUAL, 1, 0xFF);
gl20.glStencilOp(GL20.GL_KEEP, GL20.GL_KEEP, GL20.GL_KEEP);
gl20.glStencilMask(0x00);

/* Draw a filled, colored rectangle to the entire screen. */
shapeRenderer.setColor(Color.RED);
shapeRenderer.drawRect(screenRect);

shapeRenderer.end();

Now, as I understand it, this should result in a colored 'doughnut' drawn to the screen; However, as the code is written above, nothing is drawn to the screen.

However, if I only clip the outer circle (and not try to get rid of the inner circle with the "gl20.glStencilOp(GL20.GL_ZERO, GL20.GL_KEEP, GL20.GL_KEEP)" call and subsequent inner circle drawing, a circle the size of the intended outer circle draws on the screen.

I've looked through the documentation that I could find, and everything that I've read so far indicates that this should work. Is there something that I'm missing here? Any help is greatly appreciated.

3
androidcanvasopengl-es

3 Answers

1
votes

I've found the problem; I was drawing the two circles in between one begin() and end() call in the ShapeRenderer. Therefore, both circles were being drawn at the time of the end() call. This means that the shapes weren't even drawing to the stencil buffer, seeing as I was calling these functions before the end() call:

// Now, items drawn will be confined to areas with '1' stencil bit.
gl20.glStencilFunc(GL20.GL_EQUAL, 1, 0xFF);
gl20.glStencilOp(GL20.GL_KEEP, GL20.GL_KEEP, GL20.GL_KEEP);
gl20.glStencilMask(0x00);

If I make sure that I use a begin() call before and end() call after each shape being drawn, it will draw at the expected time, therefore making sure that it is drawing the expected values to the stencil buffer.

0
votes

The logic looks OK - what geometry are you drawing?

At a guess, based on the fact that you see the outer circle rendered on screen when you don't render the inner one, it looks like you are using the outer circle geometry in both cases (so you set all the stencil values, then clear all of them, so nothing has a stencil value of 1 when you come to blit the rectangle).

0
votes

I implemented your solution in WebGL and it works fine (try it out):

http://ezekiel.vancouver.wsu.edu/~cs442/stackoverflow/stencil.html

Here is how I am mimicking your code:

    gl.enable(gl.STENCIL_TEST);

    gl.colorMask(false, false, false, false);
    gl.stencilMask(0xFF);
    gl.stencilFunc(gl.NEVER, 1, 0xFF);
    gl.stencilOp(gl.REPLACE, gl.KEEP, gl.KEEP);
    fillCircle(1); // radius of 1 maps to canvas width

    gl.stencilOp(gl.ZERO, gl.KEEP, gl.KEEP);
    fillCircle(0.333);

    gl.colorMask(true, true, true, true);
    gl.stencilFunc(gl.EQUAL, 1, 0xFF);
    gl.stencilOp(gl.KEEP, gl.KEEP, gl.KEEP);
    gl.stencilMask(0x00);

    gl.uniform3fv(program.color, [1, 0, 0]);
    fillCircle(3);  // fill whole canvas width red

You'll see you get exactly what you wanted.