Unexpected texture blending result with mask

Question

I'm trying to implement a blending effect in GLSurfaceView with GLES20.

The view has a color (red) background, on top of which a mask texture and an image texture with alpha channel are blended.

Here are an example mask and image:

,

The expected result:

Here is a similar question, which guided me so far: OpenGL - mask with multiple textures

Unfortunately I couldn't achieve a similar result. Here are relevant parts of my onDraw() method:

// Bind default FBO and use shader program
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glUseProgram(program);

// Set the vertex attributes
glVertexAttribPointer(texCoordHandle, 2, GL_FLOAT, false, 0, mTexVertices);
glEnableVertexAttribArray(texCoordHandle);
glVertexAttribPointer(posCoordHandle, 2, GL_FLOAT, false, 0, mPosVertices);
glEnableVertexAttribArray(posCoordHandle);

// Clear the buffer color - draw red color background
glClear(GL_COLOR_BUFFER_BIT);

// Apply transformation - irrelevant
glUniformMatrix4fv(mvpMatrixHandle, 1, false, matrix, 0);

// Bind the mask texture for drawing
glUniform1i(contentTextureHandle, 0);

glBindTexture(GL_TEXTURE_2D, textures[MASK_TEX]);

// Write alpha value 1.0 to white regions and 0.0 to black - don't change the RGB values
glBlendFuncSeparate(
   GL_ZERO,      // remove the source rgb colors
   GL_ONE,       // keep the destination rgb colors
   GL_SRC_COLOR, // !! put high alpha (1.0) where image is white
   GL_ZERO);     // remove destination alpha

glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

// Bind the content image
glBindTexture(GL_TEXTURE_2D, textures[CONTENT_TEX]);

// Change the blend function to write the regions with low alpha
glBlendFuncSeparate(
   GL_ONE_MINUS_DST_ALPHA, // black regions (alpha 0): keep src rgb, white regions: remove src rgb
   GL_DST_ALPHA,           // black regions: remove dst rgb, white regions: keep dst rgb
   GL_ONE_MINUS_DST_ALPHA, // black regions: keep src alpha, white regions: remove src alpha
   GL_DST_ALPHA);          // black regions: remove dst rgb, white regions: keep dst alpha

glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

However, the result of combining these blend function produces only the red background.

I've tried all sorts of functions for two days now, but I think the above one should be right. Also, while debugging I tried glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA); instead and both images show up on top of each other (i.e. there is no problem with the textures or the GL setup):

My choice of the above blending function is based on the presumption that the formula behind funcSeparate is :

 Orgb = srgb  *Srgb + drgb * Drgb 
 Oa = sa * Sa + da * Da

O: output, s - source, d - destination, rest are params from glBlendFuncSeparate​(Srgb, Drgb, Sa, Da)

Any ideas why the textures don't blend as expected or how to achieve the desired result?

---

Resolved

I followed Reto Koradi's advice and incorporated the blending inside my fragment shader:

  "precision mediump float;\n" +
  "uniform sampler2D tex_sampler;\n" +
  "uniform sampler2D mask_sampler;\n" +
  "varying vec2 v_texcoord;\n" +
  "void main() {\n" +
  "  vec4 mask = texture2D(mask_sampler, v_texcoord);\n" +
  "  vec4 text = texture2D(tex_sampler, v_texcoord);\n" +
  "  gl_FragColor = vec4(text.r * (1.0 - mask.r), text.g * (1.0 - mask.r), text.b * (1.0 - mask.r), text.a * (1.0 - mask.r));\n" +
  "}\n";

Note that I'm using premultiplied bitmaps with blend function .glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA).

I'm planning to cut down the (1.0 - mask.r) parts by switching to masks with reversed colors (i.e. showing the texture in the white zones instead of the blacks).

It was a bit tricky getting the two textures' bonding correctly (glActiveTexture and glUniform1i), but there's plenty of resources online how to do that.

Answer 1

Your solution seems to be based on a misunderstanding of how GL_SRC_COLOR as part of the blend function affects the alpha component. GL_SRC_COLOR uses the (R, G, B) of the source color as multiplier for the RGB part of the source, and the A of the source color as multiplier for the A of the source. So for the source part, it essentially just results in the square of each component.

So when you use this blend function:

glBlendFuncSeparate(GL_ZERO, GL_ONE, GL_SRC_COLOR, GL_ZERO);

the GL_SRC_COLOR applied to the alpha component uses the alpha value of the source color as the multiplier. This blend equation is equivalent to:

glBlendFuncSeparate(GL_ZERO, GL_ONE, GL_SRC_ALPHA, GL_ZERO);

It looks like the alpha of your mask texture is a constant 1.0, which means that the alpha values are all set to 1.0 when you render your mask texture with this blend function. Then, when you render your android texture, none of it shows up because the blend equation is set to keep the existing content for all pixels that have a destination alpha of 1.0.

What you need to fix this is make sure that the alpha output of your mask texture rendering represents the mask value you want to apply. You could do this by changing the texture itself to contain the mask value in the alpha component. But since ES 2.0 is all shader based, you can also handle this very easily in your fragment shader. Say you currently have something like this in your fragment shader:

vec4 texVal = texture2D(...);
gl_FragColor = texVal;

Since your mask texture is black and white, you can use any color value as the output alpha. You're not using the color output of this rendering stage, so those components can be anything. If we just leave them at 0.0, it could look like this:

vec4 texVal = texture2D(...);
gl_FragColor = vec4(0.0, 0.0, 0.0, texVal.r);

What you end up doing turns out to be similar to the answer I gave to a related question recently: LibGDX texture blending with OpenGL blending function. The main difference is that I suggested to use the color mask for suppressing the color output while rendering the mask, while you're using a separate blend function.