OpenGL pre-multiplying alpha in GLSL gives unexpected result for DDS textures

Question

I have the following situation. I work on OpenGL renderer (OpenGL 4.2 so no fixed stuff).I DON"T use hardware blending for the reasons related to all sort of image effects I do in shaders.I have as my input textures of 2 types:

1) PNG 24 with alpha channel. 2)DDS (DXT5) with alpha channel.

In the fragment shader I pre-muliply alpha before output:

Fragment shader:

     outputColor=texture(colorMap,interpolateAtSample(uvsOut , gl_SampleID));
      /////// PREMULTIPLY ALPHA //////////
     outputColor.a *= alpha;
     outputColor.rgb *= outputColor.a;

PNG based textures work fine: enter image description here

But DDS looks like this: enter image description here

You can see the halo + black edge. First thing I thought was the problem with alpha multiplication.But then I changed Texture filtering for DDS from GL_LINEAR to GL_NEAREST and it removed both the halo and black edge. While this "hack" essentially fixed the problem I still would like to understand why GL_LINEAR causes this sort of issues to DDS compressed textures.

Almost forgot: If I remove pre - multiplication it solves the issue for DDS but creates one for PNG based textures.

UPDATE: There is another pass which comes after the MSAA resolve and that is blend pass.As I mentioned , I don't do any blend in this case but the output code in the blend shader looks like this:

         ///Prevent background bleeding:
         blendTex.rgb/=blendTex.a;
         //base texture is the texture under the current one.
         //This particular technique does "NORMAL" blend-just returns blendTex.
         vec4 blendedRes=blendTechnique( baseTex , blendTex) ;
         outputColor= vec4( blendedRes.rgb * blendTex.a, blendTex.a);

"I DON"T use hardware blending for the reasons related to all sort of image effects I do in shaders." That's going to make it rather difficult to diagnose your problem. What kind of blending are you using? Do you have some code? — Nicol Bolas
I do no blending for this particular example but alpha pre-multiply only.The chunk of the code at the top of the article does just that.Another thing I haven't mentioned -I do MSAA pass before this one.Can it be related? — Michael IV
"I do no blending for this particular example but alpha pre-multiply only." That's highly unlikely, since I can see partially through the fuzzy side area of the quad. So you are doing blending of some form. What blending are you using? Maybe you forgot to turn it off? — Nicol Bolas
I updated the question.I do have a blending pass as second pass. — Michael IV

Michael IV Michael IV · Accepted Answer · 2012-11-09T15:34:36

I found the source of the issue.My code was in fact correct 100%.The problem was in the specific DDS texture I used for tests.It was corrupted.When I tried different DDS (DXT5) it worked fine both with GL_NEAREST and GL_LINEAR filtering.The important detail to remember is that DDS CAN NOT be pre-multiplied by default,so if one needs it to be ,do it on GPU.

OpenGL pre-multiplying alpha in GLSL gives unexpected result for DDS textures

1 Answers