Directional per-face lighting using Geometry Shader?

Question

Because of limitations I have to calculate vertex normals inside my geometry shader. Because of this limitation the normals are per-face instead of interpolated between faces on the vertex, meaning per-pixel directional lighting is just a waste. Because the normals are only known from the Geometry Shader onward, I can't do any lighting math in the Vertex Shader.

The normals that are calculated in the Geometry Shader are in screenspace, but they are correct from some tests that I have done.

I can provide the direction of the light in world-space or multiplied by the view/projection/transposed/inverted matrix.

The following code some-what works, but changes on camera/lightdirection angle:

void FlipFaceGS(triangle PS_IN input[3], inout TriangleStream<PS_IN> OutputStream)
{
    PS_IN v1 = input[0];
    PS_IN v2 = input[1];
    PS_IN v3 = input[2];

    float3 faceEdgeA = v2.pos - v1.pos;
    float3 faceEdgeB = v3.pos - v1.pos;
    float3 faceNormal = normalize(cross(faceEdgeA, faceEdgeB));

    v1.norm = faceNormal;
    v2.norm = faceNormal;
    v3.norm = faceNormal;

    float4 newColor = (v1.color * Light1Color * max(0, dot(v1.norm, Light1Direction)));
    newColor.a = v1.color.a;//we don't want to affect the alpha levels

    v1.color = newColor;
    v2.color = newColor;
    v3.color = newColor;

    OutputStream.Append(v1);
    OutputStream.Append(v2);
    OutputStream.Append(v3);
    OutputStream.RestartStrip();
}

On the above example, the lightdirection is multiplied by the camera's ViewProjection matrix and normalized before being sent to the GPU.

What is the math needed to correctly calculate the light in the Geometry Shader?

This is not doing "per-vertex" lighting. This is doing "per-face" lighting with vertex replication. This is probably going to be slow since you are doing vertex replication in the GS, although you are at least producing the same number of outputs as you are taking inputs which means you won't get hit by the worst-case GS performance. — Chuck Walbourn
@ChuckWalbourn I fixed the title, if you know of any vertex-order independant lighting options I'd like to hear them :) — ManIkWeet

Nico Schertler Nico Schertler · Accepted Answer · 2015-03-13T21:18:12

I assume that your input vertices' positions are already transformed to clip space. Then, at least you're calculating lighting in the same space. However, clip space is not suited for lighting calculations because perspective transforms are not angle-preserving. Either do the calculations in world or in view space. In either case, you would also have to pass the vertices' position in the according space to the geometry shader. And don't pass the normal to the pixel shader if you don't need to.

Directional per-face lighting using Geometry Shader?

1 Answers