OpenGL Projection/Matrix confusion

Question

I'm trying to project the normal of a point and colour it based on if it is (from the perspective of the camera/viewing plane) leaning left of vertical, or right of vertical.

To do this I'm taking my point normal and multiplying it by the gl_ModelViewProjectionMatrix, and checking if it has a positive or negative x value. However all my points go red (indicating left of vertical, which definitely isnt the case). If I change ModelViewProjectionMatrix to gl_ProjectionMatrix I get red and green points, however they obviously aren't coloured from the camera's perspective without the ModelViewMatrix.

Am I misunderstanding something here?

Shader example:

void main(){
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;

    vec4 temp = vec4(vNormal.x, vNormal.y, vNormal.z, 1);
    vMajorTransformed = gl_ModelViewProjectionMatrix * temp;

    if (vMajorTransformed.x < 0) {
        gl_FrontColor = vec4(1, 0, 0, 1);
    } else {
        gl_FrontColor = vec4(0, 1, 0, 1);
    }

    gl_PointSize = 3;
}

Can I ask why you appear to be transforming your normal into clip-space (by including the projection matrix)? Usually the farthest you would want to take the normal is into view-space if you were going to use it in the fragment shader for lighting calculations. You also may want to avoid using the ModelView matrix to transform your normals if you have non-uniform scaling in any direction, there is a special 3x3 sub-matrix computed by taking the inverse transpose that will allow you to transform normals correctly even under these circumstances. — Andon M. Coleman
That was my misunderstanding of what I was trying to do- I thought I required the clip-space resulting coords. inverse(transpose(gl_ModelViewMatrix))? Seems to be working. Had no idea about that - got to put more time into my euclidean geometry- thanks. — jProg2015
By the way, since you are using an older version of GLSL here that still supports the old fixed-function matrix built-ins, GL already computes the normal matrix for you. It is called gl_NormalMatrix, and it is different from gl_ModelViewMatrix in that it is 3x3 - you would not need to create a 4D vector or worry about W to transform your normal if you multiply by this matrix. (e.g. vMajorTransformed = gl_NormalMatrix * vNormal, assuming vMajorTransformed is vec3). — Andon M. Coleman

Angew is no longer proud of SO Angew is no longer proud of SO · Accepted Answer · 2014-02-03T22:59:16

A normal is a direction, not a point. This means its w component should be 0. Otherwise, you can end up applying a translation to it, which will definitely mess up the x, y, z coordinates (I guess that's happening in your case).

Also note that if your modelview matrix contains anything other than a rotation and a translation, normals will get messed up again. They have to be transformed by the inverse transpose instead. Inverse and transpose cancel out on a rotation-only matrix, but for other transformations, you have to use them.

And you probably don't want to apply projection to them - the test you're after (x coordinate) can be evaluated in camera coordinates (which is generally easier to reason about than projected space).

OpenGL Projection/Matrix confusion

1 Answers