OpenGL 3D terrain lighting artefacts

Question

I'm doing per-pixel lighting(phong shading) on my terrain. I'm using a heightmap to generate the terrain height and then calculating the normal for each vertex. The normals are interpolated in the fragment shader and also normalized.

I am getting some weird dark lines near the edges of triangles where there shouldn't be. http://imgur.com/L2kj4ca

I checked if the normals were correct using a geometry shader to draw the normals on the terrain and they seem to be correct. http://imgur.com/FrJpdXI

There is no point using a normal map for the terrain it will just give pretty much the same normals. The problem lies with the way the normals are interpolated across a triangle.

I am out of idea's how to solve this. I couldn't find any working solution online.

Terrain Vertex Shader:

#version 330 core

layout (location = 0) in vec3 position;
layout (location = 1) in vec3 normal;
layout (location = 2) in vec2 textureCoords;

out vec2 pass_textureCoords;
out vec3 surfaceNormal;
out vec3 toLightVector;
out float visibility;

uniform mat4 transformationMatrix;
uniform mat4 viewMatrix;
uniform mat4 projectionMatrix;

uniform vec3 lightPosition;

const float density = 0.0035;
const float gradient = 5.0;

void main()
{
    vec4 worldPosition = transformationMatrix * vec4(position, 1.0f);
    vec4 positionRelativeToCam = viewMatrix * worldPosition;

    gl_Position = projectionMatrix * positionRelativeToCam;
    pass_textureCoords = textureCoords;

    surfaceNormal = (transformationMatrix * vec4(normal, 0.0f)).xyz;
    toLightVector = lightPosition - worldPosition.xyz;

    float distance = length(positionRelativeToCam.xyz);
    visibility = exp(-pow((distance * density), gradient));
    visibility = clamp(visibility, 0.0, 1.0);
}

Terrain Fragment Shader:

#version 330 core

in vec2 pass_textureCoords;
in vec3 surfaceNormal;
in vec3 toLightVector;
in float visibility;

out vec4 colour;

uniform vec3 lightColour;
uniform vec3 fogColour;

uniform sampler2DArray blendMap;
uniform sampler2DArray diffuseMap;

void main()
{
    vec4 blendMapColour = texture(blendMap, vec3(pass_textureCoords, 0));

    float backTextureAmount = 1 - (blendMapColour.r + blendMapColour.g + blendMapColour.b);
    vec2 tiledCoords = pass_textureCoords * 255.0;
    vec4 backgroundTextureColour = texture(diffuseMap, vec3(tiledCoords, 0)) * backTextureAmount;
    vec4 rTextureColour = texture(diffuseMap, vec3(tiledCoords, 1)) * blendMapColour.r;
    vec4 gTextureColour = texture(diffuseMap, vec3(tiledCoords, 2)) * blendMapColour.g;
    vec4 bTextureColour = texture(diffuseMap, vec3(tiledCoords, 3)) * blendMapColour.b;

    vec4 diffuseColour = backgroundTextureColour + rTextureColour + gTextureColour + bTextureColour;

    vec3 unitSurfaceNormal = normalize(surfaceNormal);
    vec3 unitToLightVector = normalize(toLightVector);

    float brightness = dot(unitSurfaceNormal, unitToLightVector);
    float ambient = 0.2;
    brightness = max(brightness, ambient);
    vec3 diffuse = brightness * lightColour;

    colour = vec4(diffuse, 1.0) * diffuseColour;
    colour = mix(vec4(fogColour, 1.0), colour, visibility);
}

The way my terrain tiles are triangulated is the following: imgur.com/t8DOkTJ I tried another triangulation method but i just got different dark lines near the edges of the triangles. The effect I'm looking for on my terrain is like this: imgur.com/9Jx2g24 This is from vanilla wow. — Simon Mathay
Please supply two more images: geometry wireframe + normals encoded into color from the pixel shader. This should bring us closer to the solution. — mrVoid
I'm sry mrVoid what do you mean? The geometry shader and fragment shader code for drawing the normal vectors? — Simon Mathay
He means to turn the normals of the vertices into the color component by using surfaceNormal as the color value. — Colin Basnett
I used this code to visualize the normal vectors: learnopengl.com/#!Advanced-OpenGL/Geometry-Shader Scroll down to "Visualizing normal vectors" — Simon Mathay

jubertal jubertal · Accepted Answer · 2016-02-24T18:18:45

This can be either two issues :

1. Incorrect normals : There is different types of shading : Flat shading, Gouraud shading and Phong shading (different of Phong specular) example :

You usually want to do a Phong shading. To do that, OpenGL make your life easier and interpolate for you the normals between each vertex of each triangle, so at each pixel you have the correct normal for this point: but you still need to feed it proper normal values, that are the average of the normals of every triangles attached to this vertex. So in your function that create the vertex, the normals and the UVs, you need to compute the normal at each vertex by averaging every triangle normal attached to this vertex. illustration

2. Subdivision problem : The other possible issue is that your terrain is not subdivided enough, or your heightmap resolution is too low, resulting to this kind of glitch because of the difference of height between two vertex in one triangle (so between two pixels in your heightmap).

Maybe if you can provide some of your code and shaders, maybe even the heightmap so we can pin exactly what is happening in your case.

OpenGL 3D terrain lighting artefacts

2 Answers