How do I update normals after positioning vertices in vertex shader?

Question

Short version: I'm manipulating the position of vertices in a vertex shader, but when I calculate the normals based on the vertex position, the normals are calculated based on the original vertex position. Shouldn't the vertex shader know where the new vertices are?

Long version: I'm writing a custom shader in three.js R.58 based on the three.js normalmap shader. I'm passing a flat texture (0.5, 0.5, 1.0 lavender) to the shader as a tNormal, and then setting the position of a planeGeometry's vertices in the vertex shader to bend it into a sphere.

Here's the bit of the vertex shader where normals are calculated:

vec3 newPosition = mix( position, goalPosition, mixAmount );

vec4 mvPosition = modelViewMatrix * vec4( newPosition, 1.0 );
vViewPosition = -mvPosition.xyz;

vNormal = normalize( normalMatrix * normal );

//tangent and binormal vectors
vTangent = normalize( normalMatrix * tangent.xyz );

vBinormal = cross( vNormal, vTangent ) * tangent.w;
vBinormal = normalize( vBinormal );

This code works for default sphereGeometry and planeGeometry. However, when I deform a plane into a sphere, the shading doesn't work as expected – the fragment shader seems to think the sphere is still a plane – or something.

Here's the sphere with a pointLight, showing warping specular highlight:

enter image description here

Demo: http://meetar.github.io/planebending/plane_bending.html

And that only shows up if the original plane is facing the pointlight - at other rotations, the sphere is black no matter where the camera is, suggesting that the plane normals are still somehow being calculated as though the vertices hadn't been repositioned.

I've set geometry.dynamic = true as well as geometry.normalsNeedUpdate and geometry.tangentsNeedUpdate, and I'm calling geometry.computeTangents(), but nothing seems to work.

I was under the impression that in GLSL, the components of the scene used to calculate normals (such as the normalMatrix and the tangent) take into account any vertex manipulation by the vertex shader. What am I missing? Is this something specific to three.js?

Edit:

Checking in the console, I see that every face in the deformed plane still has a worldspace normal of (0, 0, 1), as it did in its undeformed state, as opposed to a sphereGeometry instance, whose normals vary depending on their orientation.

Here's another demonstration: these two objects have the same material. The left is SphereGeometry, the right is a deformed PlaneGeometry. When the plane deforms, the normals don't seem to update to reflect the faces' new orientation, and the specular doesn't show up properly.

enter image description here

Demo: http://meetar.github.io/planebending/plane_bending_06.html

Nothing is done automatically in the vertex shader; that is actually the point. Some things happen automatically afterwards (e.g. perspective division), but normalization of output varyings is not one of those things. In the fixed-function pipeline, there was a feature to automatically re-normalize, but that does not apply in WebGL. — Andon M. Coleman
That said, generally in the vertex the way you tackle this is by transforming the lighting vectors into tangent space using the Tangent, Bitangent and Normal vectors. Then all lighting calculation is done in the fragment shader in the same coordinate space as the normal is sampled in (tangent space). There are alternatives, such as transforming the normals into world/view space in the fragment shader but that approach is more expensive and primarily used for deferred shading. Unfortunately, without including the shaders relevant to this question, I cannot write an actual answer. — Andon M. Coleman
Hmm, okay. I'll modify my question to try to make more sense. — meetar
How does this differ from your prior question stackoverflow.com/questions/17528878/…? Also, I would advise you to update to the current version of three.js. — WestLangley
No. The data is one-way in this case -- from the CPU to the GPU. — WestLangley

meetar meetar · Accepted Answer · 2014-01-15T21:15:10

Short version: You can't!

Long version: Even if the vertex shader is moving the vertices, it only knows the updated position of the current vertex – to calculate new face normals, it would have to know the updated position of all the neighbor vertices, and so far, WebGL doesn't allow that.

Although the vertex shader does calculate normals, they're all based on the original normals, which are passed to the vertex shader along with the original vertex positions. Those calculations are mostly just the standard normal-related things required for the fragment shader, such as accounting for light position, transformations, and so on, and they all assume that the vertices are going to hold still relative to each other in object space.

It's relatively easy to update normals with the CPU and pass those to the vertex shader, but if you must do it in the vertex shader, there are some sneaky ways to fake it, such as using bump maps; and if the vertices are being moved based on any kind of parametric computation, you can generate some neighbors on the fly and check the normal between them, which is definitely cheating.

Sources:

How do I update normals after positioning vertices in vertex shader?

1 Answers