GLSL calculating normal on a sphere mesh in vertex shader using noise function by sampling is producing strange graphical errors

Question

I'm trying to calculate the normals on surface of a sphere in the vertex shader because I defer my noise calculation to the vertex shader. The normal results are good when my theta (sampling angle) is closer to 1 but for more detailed terrain & a smaller theta my normals become very incorrect. Here's what I mean:

Accurate normals

Accurate normals

More detailed noise with a higher theta

More detailed noise with a higher theta

Zoomed in onto surface of last image. Red indicates ridges, blue indicates incorrect shadows

More detailed noise with a higher theta zoomed in

The code I use to calculate normals is:

vec3 calcNormal(vec3 pos)
{
    float theta = .1; //The closer this is to zero the less accurate it gets
    vec3 vecTangent = normalize(cross(pos, vec3(1.0, 0.0, 0.0))
        + cross(pos, vec3(0.0, 1.0, 0.0)));
    vec3 vecBitangent = normalize(cross(vecTangent, pos));
    vec3 ptTangentSample = getPos(pos + theta * normalize(vecTangent));
    vec3 ptBitangentSample = getPos(pos + theta * normalize(vecBitangent));

    return normalize(cross(ptTangentSample - pos, ptBitangentSample - pos));
}

I call calcNormal with

calcNormal(getPos(position))

where getPos is the 3D noise function that takes and returns a vec3 and position is the original position on the sphere.

Should you maybe normalize the arguments of getPos() to project it back on the sphere? Is it correct to assume that the sphere has unit radius and is centered at the origin? — Nico Schertler
@NicoSchertler OH! That's right. Thanks! I changed the ptTangentSample and ptBitangentSample lines to getPos( normalize (...)) — samsun96

samsun96 samsun96 · Accepted Answer · 2016-09-02T16:23:36

Thanks to @NicoSchertler the correct version of calcNormal is

vec3 calcNormal(vec3 pos)
{
    float theta = .00001; 
    vec3 vecTangent = normalize(cross(pos, vec3(1.0, 0.0, 0.0))
     + cross(pos, vec3(0.0, 1.0, 0.0)));
    vec3 vecBitangent = normalize(cross(vecTangent, pos));
    vec3 ptTangentSample = getPos(normalize(pos + theta * normalize(vecTangent)));
    vec3 ptBitangentSample = getPos(normalize(pos + theta * normalize(vecBitangent)));

     return normalize(cross(ptTangentSample - pos, ptBitangentSample - pos));
  }

GLSL calculating normal on a sphere mesh in vertex shader using noise function by sampling is producing strange graphical errors

1 Answers