How to generate normals in GLSL

Question

I have created randomly generated terrain using simplex noise. My code looks like this:

      float [ ] vertices = new float [ SIZE * SIZE * 3 ];

      short [ ] indices = new short [ ( SIZE - 1 ) * ( SIZE - 1 ) * 2 * 3 ];

      for ( int x = 0 , index = 0 ; x < SIZE ; x ++ )
      {
         for ( int z = 0 ; z < SIZE ; z ++ )
         {
            vertices [ index ++ ] = x;
            vertices [ index ++ ] = ( float ) SimplexNoise.noise ( x , z );
            vertices [ index ++ ] = z;
         }
      }

      for ( int i = 0 , index = 0 ; i < ( SIZE - 1 ) ; i ++ )
      {
         int offset = i * SIZE;

         for ( int j = 0 ; j < ( SIZE - 1 ) ; j ++ )
         {
            indices [ index ++ ] = ( short ) ( j + offset );
            indices [ index ++ ] = ( short ) ( j + offset + 1 );
            indices [ index ++ ] = ( short ) ( j + offset + 1 + SIZE );
            indices [ index ++ ] = ( short ) ( j + offset + 1 + SIZE );
            indices [ index ++ ] = ( short ) ( j + offset + SIZE );
            indices [ index ++ ] = ( short ) ( j + offset );
         }
      }

That gives me the indices and vertices but unless I use GL_LINES it looks like a mob of color. I can't continue progress on the terrain until I actually know what I am looking at but the only way I know how to use normals is to load them in with a model and send them off to the shader. I have no clue how to actually generate them. I have searched around and read a lot about getting the normals of the surrounding faces and normalizing them and some other stuff but I didn't understand most of it and to the best of my knowledge you only have access to a single vertex in the vertex shader at a time so I am not sure how you would process the entire triangle. As you can see much help would be appreciated.

EDIT:

I have did some research and have now calculated the normals of all the triangles:

      float [ ] triNormals = new float [ indices.length ];

      for ( int i = 0 , index = 0 ; i < indices.length ;  )
      {
         float x , y , z;

         x = vertices [ ( 3 * indices [ i ] ) ];
         y = vertices [ ( 3 * indices [ i ] ) + 1 ];
         z = vertices [ ( 3 * indices [ i ++ ] ) + 2 ];

         Vector3f p1 = new Vector3f ( x , y , z );

         x = vertices [ ( 3 * indices [ i ] ) ];
         y = vertices [ ( 3 * indices [ i ] ) + 1 ];
         z = vertices [ ( 3 * indices [ i ++ ] ) + 2 ];

         Vector3f p2 = new Vector3f ( x , y , z );

         x = vertices [ ( 3 * indices [ i ] ) ];
         y = vertices [ ( 3 * indices [ i ] ) + 1 ];
         z = vertices [ ( 3 * indices [ i ++ ] ) + 2 ];

         Vector3f p3 = new Vector3f ( x , y , z );

         Vector3f u = Vector3f.subtract ( p2 , p1 );
         Vector3f v = Vector3f.subtract ( p3 , p1 );

         Vector3f normal = Vector3f.crossProduct ( u , v );

         triNormals [ index ++ ] = normal.x;
         triNormals [ index ++ ] = normal.y;
         triNormals [ index ++ ] = normal.z;
      }

Now I just need to know how to calculate the normals of the vertices using the normals of the surrounding triangles.

jozxyqk jozxyqk · Accepted Answer · 2015-10-31T09:51:46

Generating per-triangle or face normals, which look flat and have sharp edges is done by normalizing the cross product of two edges. As @neuo says, this could be done in a geometry shader, however it sounds like you're after "smooth" normals.

Smooth normals are not necessarily implicit from the vertex geometry, but a decent guess is based off the adjacent triangles (the average or weighted average of). Unless you have adjacency information (as in tessellation shaders (?)), this isn't possible in GLSL as you render. It would need to be done in a separate pass (maybe using transform feedback) and probably just as easy to process on the CPU.

In GL, there is one normal for every vertex*. So start with a new zeroed array equal to your vertices (or with space between vertices if you interleave). For every triangle formed by your indices array (a, b, c), find the triangle normal using the cross product (b - a) × (c - a) and normalize it. Then add the triangle normal to the per vertex normals of the vertices of each triangle (i.e. a, b and c). Finally run through every normal and normalize. Done.

Depending on the mesh, this simple method can work pretty well. In cases where triangles are really small or skinny, the equal weighting can cause strange results:

One approach to fixing this is to scale the normal contribution from each triangle by its area. To do this, simply don't normalize the cross product result before you add it to each normal.

for (int t = 0; t < numIndices / 3; ++t)
{
    unsigned int t1 = dataIndices[t*3+0];
    unsigned int t2 = dataIndices[t*3+1];
    unsigned int t3 = dataIndices[t*3+2];
    const vec3f& a = verts[t1];
    const vec3f& b = verts[t2];
    const vec3f& c = verts[t3];
    vec3f u = b - a;
    vec3f v = c - a;
    vec3f n = u.cross(v);
    //if (n.size() > 0.0) n.normalize(); //removes weighting based on triangle area
    norms[t1] += n;
    norms[t2] += n;
    norms[t3] += n;
}
for (int v = 0; v < numVertices; ++v)
    norms[v].normalize();

^src

* To get face normals you'd have to duplicate vertices, which is why the image is drawn with two red normals for each vertex position. You could also use the keyword flat on your in/out variables but then everything would be flat.

How to generate normals in GLSL

2 Answers