NULL checks in GLSL

1
votes

I'm trying to check inside the shader (GLSL) if my vec4 is NULL. I need this for several reasons, mostly to get specific graphics cards compatible, since some of them pass a previous color in gl_FragColor, and some don't (providing a null vec4 that needs to be overwritten). Well, on a fairly new Mac, someone got this error:

java.lang.RuntimeException: Error creating shader: ERROR: 0:107: '==' does not operate on 'vec4' and 'int'
ERROR: 0:208: '!=' does not operate on 'mat3' and 'int'

This is my code in the fragment shader:

void main()
{
    if(gl_FragColor == 0) gl_FragColor = vec4(0.0, 0.0, 0.0, 0.0); //Line 107
    vec4 newColor = vec4(0.0, 0.0, 0.0, 0.0);

    [...]

    if(inverseViewMatrix != 0) //Line 208
    {
        [do stuff with it; though I can replace this NULL check with a boolean]
    }

    [...]

    gl_FragColor.rgb = mix(gl_FragColor.rgb, newColor.rgb, newColor.a);
    gl_FragColor.a += newColor.a; 
}

As you can see, I do a 0/NULL check for gl_FragColor at the start, because some graphics cards pass valuable information there, but some don't. Now, on that special mac, it didn't work. I did some research, but couldn't find any information on how to do a proper NULL check in GLSL. Is there even one, or do I really need to make separate shaders here?

3
macosopenglnullglslfragment-shader
I don't think vector data types in GLSL can be NULL. They are always something, unlike pointers in C/C++ that can be NULL, and references in Java. – Victor Zamanian
Well, unless I also changed something else, that NULL check is exactly what solved a bug (alpha-related) on some other graphics card. It also works on most, so I can't really imagine why it wouldn't on this? I can only guess it would always return false on an vec4 - int comparison... =/ – Ivorius

3 Answers

9
votes

All variables meant for reading, i.e. input variables always deliver sensible values. Being an output variable, gl_FragColor is not one of these variables!

In this code 

void main()
{
    if(gl_FragColor == 0) gl_FragColor = vec4(0.0, 0.0, 0.0, 0.0); //Line 107
    vec4 newColor = vec4(0.0, 0.0, 0.0, 0.0);

The very first thing you do is reading from gl_FragColor. The GLSL specification clearly states, that the value of an output varialbe as gl_FragColoris, is undefined when the fragment shader stage is entered (point 1):

The value of an output variable will be undefined in any of the three following cases:

  1. At the beginning of execution.
  2. At each synchronization point, unless
    • the value was well-defined after the previous synchronization point and was not written by any invocation since, or
    • the value was written by exactly one shader invocation since the previous synchronization point, or
    • the value was written by multiple shader invocations since the previous synchronization point, and the last write performed by all such invocations wrote the same value.
  3. When read by a shader invocation, if
    • the value was undefined at the previous synchronization point and has not been writen by the same shader invocation since, or
    • the output variable is written to by any other shader invocation between the previous and next synchronization points, even if that assignment occurs in code following the read.

Only after an element of an output variable has been written to for the first time its value is defined. So the whole thing you do there makes no sense. That it "didn't work" is completely permissible and an error on your end.

You're invoking undefined behaviour and technically it would be permissible for your computer to become sentinent, chase you down the street and erase all of your data as an alternative reaction to this.

In GLSL a vec4 is a regular datatype just like int. It's not some sort of pointer to an array which could be a null pointer. At best it has some default value that's not being overwritten by a call to glUniform.

1
votes

Variables in GLSL shaders are always defined (otherwise, you'll get a linker error). If you don't supply those values with data (by not loading the appropriate uniform, or binding attributes to in or attribute variables), the values in those variables will be undefined (i.e., garbage), but present.

0
votes

Even if you can't have null values, you can test undefined variables. This is a trick that I use to debug my shaders:

  ...
  /* First we test for lower range */
  if(suspect_variable.x < 0.5) {
      outColour = vec4(0,1,0,0); /* Green if in lower range*/
  } else if(suspect_variable.x >= 0.5) { /*Then for the higher range */
      outColour = vec4(1,0,0,0); /* Red if in higher range */
  } else {
      /* Now we have tested for all real values.
         If we end up here we know that the value must be undefined */
      outColour = vec4(0,0,1,0); /* Blue if it's undefined */
  }

You might ask, what could make a variable undefined? Out of range access of an array would cause it to be undefined;

const int numberOfLights = 2;
uniform vec3 lightColour[numberOfLights];
...
for(int i = 0; i < 100; i++) {
   /* When i bigger than 1 the suspect_variable would be undefined */
   suspect_variable = suspect_variable * lightColour[i];
}

It is a simple and easy trick to use when you do not have access to real debugging tools.