Background color issue when blending with transparency in OpenGL

Question

Let's take the simplest case of rendering two overlapping transparent rectangles, one red and one green, both with alpha=0.5. Assume that the drawing order is from back to front, meaning that the rectangle farther from the camera is drawn first.

In realistic scenarios, irrespective of which rectangle happens to be in front, the overlapping color should be the same, i.e. RGBA = [0.5, 0.5, 0.0, 0.5].

In practice, however, assuming that we are blending with weights SRC_ALPHA and ONE_MINUS_SRC_ALPHA, the overlapping color is dominated by the color of the front rectangle, as in this image:

I believe this happens because the first rectangle is blended with the background color, and the second rectangle is then blended with the resultant color. With this logic, assuming white background, the overlapping color in the two cases works out to be:

Red on top: 0.5*(0.5*[1,1,1,0] + 0.5*[0,1,0,0.5]) + 0.5*[1,0,0,0.5] = [0.75, 0.50, 0.25, 0.375]
Green on top: 0.5*(0.5*[1,1,1,0] + 0.5*[1,0,0,0.5]) + 0.5*[0,1,0,0.5] = [0.50, 0.75, 0.25, 0.375]

which explains the dominance of the color on top. In principle, this could be easily corrected if all the objects were blended first, and the resultant color is blended with the background color.

Is there a way to achieve this in OpenGL?

Answer 1

Drawing transparent surfaces depends a lot on order. Most issues happen because you're using depth tests and writing to the depth buffer (in which case the result depends not only on which triangle is in front, but also on which triangle is drawn first). But if you ignore depth and just want to draw triangles one after another, your results still depend on the order in which you draw them, unless you use certain commutative blend functions.

Since you've been talking about stained glass, here's one option that works roughly like stained glass:

glBlendFunc(GL_ZERO, GL_SRC_COLOR)

This essentially multiplies each color channel of the destination by the corresponding color channel of the source. So if you draw a triangle with color (0.5, 1.0, 1.0), then it will basically divide the red channel of whatever it's been drawn onto by two. Drawing on a black destination will keep the pixel black, just like stained glass does.

To reduce the "opacity" of your stained glass, you'll have to mix your colors with (1.0, 1.0, 1.0). The alpha value is ignored.

As a bonus, this blend function is independent of the order you draw your shapes (assuming you've locked the depth buffer or disabled depth testing).

Answer 2

Ideally, irrespective of which rectangle happens to be in front, the overlapping color should be the same

No, because when you use "SourceAlpha, InvSourceAlpha" blending, the the formula for calculating the final color is:

destRGB = destRGB * (1-sourceAlpha) + sourceRGB * sourceAlpha

This causes that the color of the rectangle which is drawn first, is multiplied by the alpha channel and add to the framebuffer. When the second rectangle is drawn, then the content of the framebuffer (which includes the color of the first rectangle) is multiplied again, but now by the inverse alpha channel of the second rectangle.
The color of the second rectangle is multiplied by alpha channel of the 2nd rectangle only:

destRGB = (destRGB * (1-Alpha_1) + RGB_1 * Alpha_1) * (1-Alpha_2) +  RGB_2 * Alpha_2

or

destRGB = destRGB * (1-Alpha_1)*(1-Alpha_2) + RGB_1 * Alpha_1*(1-Alpha_2) + RGB_2 * Alpha_2

While RGB_2 is multiplied by Alpha_2, RGB_1 is multiplied by Alpha_1 * (1-Alpha_2).
So the result depends on the drawing order, if the color in the framebuffer is modified by the alpha channel of the new (source) color.

---

If you want to achieve an order independent effect, then the the color of the framebuffer must not be modified by the alpha channel of the source fragment. e.g.:

destRGB = destRGB * 1 + sourceRGB * sourceAlpha

Which can be achieved by the parameter GL_ONE for the destination factor of glBlendFunc:

glBlendFunc(GL_SRC_ALPHA, GL_ONE);