47
votes

Talking about bump mapping, specular highlight and these kind of things in OpenGL Shading Language (GLSL)

I have:

  • An array of vertices (e.g. {0.2,0.5,0.1, 0.2,0.4,0.5, ...})
  • An array of normals (e.g. {0.0,0.0,1.0, 0.0,1.0,0.0, ...})
  • The position of a point light in world space (e.g. {0.0,1.0,-5.0})
  • The position of the viewer in world space (e.g. {0.0,0.0,0.0}) (assume the viewer is in the center of the world)

Now, how can I calculate the Binormal and Tangent for each vertex? I mean, what is the formula to calculate the Binormals, what I have to use based on those informations? And about the tangent?

I'll construct the TBN Matrix anyway, so if you know a formula to construct the matrix directly based on those informations will be nice!

Oh, yeh, I have the texture coordinates too, if needed. And as I'm talking about GLSL, would be nice a per-vertex solution, I mean, one which doesn't need to access more than one vertex information at a time.

---- Update -----

I found this solution:

vec3 tangent;
vec3 binormal;

vec3 c1 = cross(a_normal, vec3(0.0, 0.0, 1.0));
vec3 c2 = cross(a_normal, vec3(0.0, 1.0, 0.0));

if (length(c1)>length(c2))
{
    tangent = c1;
}
else
{
    tangent = c2;
}

tangent = normalize(tangent);

binormal = cross(v_nglNormal, tangent);
binormal = normalize(binormal);

But I don't know if it is 100% correct.

4
mayebe a question for gamedev.stackexchange.com ?nkint
@user464230: No, it's okay here, too. 3D graphics isn't limited to games.datenwolf
That "solution" assumes you'll apply your normal map texture on a planar face with the normal = x. This will not work if applied to an arbitrary model. What you really need to do is solve the system of equations I gave you down there for each face and use the mean values for the vertices. If you want to do serious 3D programming you'll have to learn, how to translate linear algebra - like I showed below - into source code.datenwolf
"If you want to do serious 3D programming you'll have to learn, how to translate linear algebra - like I showed below - into source code."... Man, really, You don't meet me!!! Don't tell me "you'll to learn"... You don't have any idea of what I've done... Take care with your words!user464230
@user464230: No, I've no knowledge of what you've actually done. But you asked about how to calculate tangent and binormal. So what you can expect is a mathematical description of the process. If you just want some source code you can copy and paste, well, there's plenty of it out there. But to understand how to use it effectively you must understand what it does on a mathematical level. Which requires you to learn to read and understand linear algebra. The way you ask your question(s) and commented on my post clearly shows me, that you've not learned how to properly read math, yet.datenwolf

4 Answers

38
votes

The relevant input data to your problem are the texture coordinates. Tangent and Binormal are vectors locally parallel to the object's surface. And in the case of normal mapping they're describing the local orientation of the normal texture.

So you have to calculate the direction (in the model's space) in which the texturing vectors point. Say you have a triangle ABC, with texture coordinates HKL. This gives us vectors:

D = B-A
E = C-A

F = K-H
G = L-H

Now we want to express D and E in terms of tangent space T, U, i.e.

D = F.s * T + F.t * U
E = G.s * T + G.t * U

This is a system of linear equations with 6 unknowns and 6 equations, it can be written as

| D.x D.y D.z |   | F.s F.t | | T.x T.y T.z |
|             | = |         | |             |
| E.x E.y E.z |   | G.s G.t | | U.x U.y U.z |

Inverting the FG matrix yields

| T.x T.y T.z |           1         |  G.t  -F.t | | D.x D.y D.z |
|             | = ----------------- |            | |             |
| U.x U.y U.z |   F.s G.t - F.t G.s | -G.s   F.s | | E.x E.y E.z |

Together with the vertex normal T and U form a local space basis, called the tangent space, described by the matrix

| T.x U.x N.x |
| T.y U.y N.y |
| T.z U.z N.z |

Transforming from tangent space into object space. To do lighting calculations one needs the inverse of this. With a little bit of exercise one finds:

T' = T - (N·T) N
U' = U - (N·U) N - (T'·U) T'

Normalizing the vectors T' and U', calling them tangent and binormal we obtain the matrix transforming from object into tangent space, where we do the lighting:

| T'.x T'.y T'.z |
| U'.x U'.y U'.z |
| N.x  N.y  N.z  |

We store T' and U' them together with the vertex normal as a part of the model's geometry (as vertex attributes), so that we can use them in the shader for lighting calculations. I repeat: You don't determine tangent and binormal in the shader, you precompute them and store them as part of the model's geometry (just like normals).

(The notation between the vertical bars above are all matrices, never determinants, which normally use vertical bars instead of brackets in their notation.)

18
votes

Generally, you have 2 ways of generating the TBN matrix: off-line and on-line.

  • On-line = right in the fragment shader using derivative instructions. Those derivations give you a flat TBN basis for each point of a polygon. In order to get a smooth one we have to re-orthogonalize it based on a given (smooth) vertex normal. This procedure is even more heavy on GPU than initial TBN extraction.

    // compute derivations of the world position
    vec3 p_dx = dFdx(pw_i);
    vec3 p_dy = dFdy(pw_i);
    // compute derivations of the texture coordinate
    vec2 tc_dx = dFdx(tc_i);
    vec2 tc_dy = dFdy(tc_i);
    // compute initial tangent and bi-tangent
    vec3 t = normalize( tc_dy.y * p_dx - tc_dx.y * p_dy );
    vec3 b = normalize( tc_dy.x * p_dx - tc_dx.x * p_dy ); // sign inversion
    // get new tangent from a given mesh normal
    vec3 n = normalize(n_obj_i);
    vec3 x = cross(n, t);
    t = cross(x, n);
    t = normalize(t);
    // get updated bi-tangent
    x = cross(b, n);
    b = cross(n, x);
    b = normalize(b);
    mat3 tbn = mat3(t, b, n);
    
  • Off-line = prepare tangent as a vertex attribute. This is more difficult to get because it will not just add another vertex attrib but also will require to re-compose all other attributes. Moreover, it will not 100% give you a better performance as you'll get an additional cost of storing/passing/animating(!) vector3 vertex attribute.

The math is described in many places (google it), including the @datenwolf post.

The problem here is that 2 vertices may have the same normal and texture coordinate but different tangents. That means you can not just add a vertex attribute to a vertex, you'll need to split the vertex into 2 and specify different tangents for the clones.

The best way to get unique tangent (and other attribs) per vertex is to do it as early as possible = in the exporter. There on the stage of sorting pure vertices by attributes you'll just need to add the tangent vector to the sorting key.

As a radical solution to the problem consider using quaternions. A single quaternion (vec4) can successfully represent tangential space of a pre-defined handiness. It's easy to keep orthonormal (including passing to the fragment shader), store and extract normal if needed. More info on the KRI wiki.

2
votes

Based on the answer from kvark, I would like to add more thoughts.

If you are in need of an orthonormalized tangent space matrix you have to do some work any way. Even if you add tangent and binormal attributes, they will be interpolated during the shader stages and at the end they are neither normalized nor they are normal to each another.

Let's assume that we have a normalized normalvector n, and we have the tangent t and the binormalb or we can calculate them from the derivations as follows:

// derivations of the fragment position
vec3 pos_dx = dFdx( fragPos );
vec3 pos_dy = dFdy( fragPos );
// derivations of the texture coordinate
vec2 texC_dx = dFdx( texCoord );
vec2 texC_dy = dFdy( texCoord );
// tangent vector and binormal vector
vec3 t = texC_dy.y * pos_dx - texC_dx.y * pos_dy;
vec3 b = texC_dx.x * pos_dy - texC_dy.x * pos_dx;

Of course an orthonormalized tangent space matrix can be calcualted by using the cross product, but this would only work for right-hand systems. If a matrix was mirrored (left-hand system) it will turn to a right hand system:

t = cross( cross( n, t ), t ); // orthonormalization of the tangent vector
b = cross( n, t );             // orthonormalization of the binormal vector 
                               //   may invert the binormal vector
mat3 tbn = mat3( normalize(t), normalize(b), n );

In the code snippet above the binormal vector is reversed if the tangent space is a left-handed system. To avoid this, the hard way must be gone:

t = cross( cross( n, t ), t ); // orthonormalization of the tangent vector
b = cross( b, cross( b, n ) ); // orthonormalization of the binormal vectors to the normal vector 
b = cross( cross( t, b ), t ); // orthonormalization of the binormal vectors to the tangent vector
mat3 tbn = mat3( normalize(t), normalize(b), n );

A common way to orthogonalize any matrix is the Gram–Schmidt process:

t = t - n * dot( t, n ); // orthonormalization ot the tangent vectors
b = b - n * dot( b, n ); // orthonormalization of the binormal vectors to the normal vector 
b = b - t * dot( b, t ); // orthonormalization of the binormal vectors to the tangent vector
mat3 tbn = mat3( normalize(t), normalize(b), n );

Another possibility is to use the determinant of the 2*2 matrix, which results from the derivations of the texture coordinates texC_dx, texC_dy, to take the direction of the binormal vector into account. The idea is that the determinant of a orthogonal matrix is 1 and the determined one of a orthogonal mirror matrix -1.

The determinant can eihter be calcualted by the GLSL function determinant( mat2( texC_dx, texC_dy ) or it can be calcualated by it formula texC_dx.x * texC_dy.y - texC_dy.x * texC_dx.y.

For the calculation of the orthonormalized tangent space matrix, the binormal vector is no longer required and the calculation of the unit vector (normalize) of the binormal vector can be evaded.

float texDet = texC_dx.x * texC_dy.y - texC_dy.x * texC_dx.y;
vec3 t = texC_dy.y * pos_dx - texC_dx.y * pos_dy;
t      = normalize( t - n * dot( t, n ) );
vec3 b = cross( n, t );                      // b is normlized because n and t are orthonormalized unit vectors
mat3 tbn = mat3( t, sign( texDet ) * b, n ); // take in account the direction of the binormal vector
0
votes

There is a variety of ways to calculate tangents, and if the normal map baker doesn't do it the same way as the renderer you'll get subtle artifacts. Many bakers use the MikkTSpace algorithm, which isn't the same as the fragment derivatives trick.

Fortunately, if you have an indexed mesh from a program that uses MikkTSpace (and no texture coordinate triangles with opposite orientations share an index) the hard part of the algorithm is mostly done for you, and you can reconstruct the tangents like this:

#include <cmath>
#include "glm/geometric.hpp"
#include "glm/vec2.hpp"
#include "glm/vec3.hpp"
#include "glm/vec4.hpp"

using glm::vec2;
using glm::vec3;
using glm::vec4;

void makeTangents(uint32_t nIndices, uint16_t* indices,
                  const vec3 *positions, const vec3 *normals,
                  const vec2 *texCoords, vec4 *tangents) {
  uint32_t inconsistentUvs = 0;
  for (uint32_t l = 0; l < nIndices; ++l) tangents[indices[l]] = vec4(0);
  for (uint32_t l = 0; l < nIndices; ++l) {
    uint32_t i = indices[l];
    uint32_t j = indices[(l + 1) % 3 + l / 3 * 3];
    uint32_t k = indices[(l + 2) % 3 + l / 3 * 3];
    vec3 n = normals[i];
    vec3 v1 = positions[j] - positions[i], v2 = positions[k] - positions[i];
    vec2 t1 = texCoords[j] - texCoords[i], t2 = texCoords[k] - texCoords[i];

    // Is the texture flipped?
    float uv2xArea = t1.x * t2.y - t1.y * t2.x;
    if (std::abs(uv2xArea) < 0x1p-20)
      continue;  // Smaller than 1/2 pixel at 1024x1024
    float flip = uv2xArea > 0 ? 1 : -1;
    // 'flip' or '-flip'; depends on the handedness of the space.
    if (tangents[i].w != 0 && tangents[i].w != -flip) ++inconsistentUvs;
    tangents[i].w = -flip;

    // Project triangle onto tangent plane
    v1 -= n * dot(v1, n);
    v2 -= n * dot(v2, n);
    // Tangent is object space direction of texture coordinates
    vec3 s = normalize((t2.y * v1 - t1.y * v2)*flip);
    
    // Use angle between projected v1 and v2 as weight
    float angle = std::acos(dot(v1, v2) / (length(v1) * length(v2)));
    tangents[i] += vec4(s * angle, 0);
  }
  for (uint32_t l = 0; l < nIndices; ++l) {
    vec4& t = tangents[indices[l]];
    t = vec4(normalize(vec3(t.x, t.y, t.z)), t.w);
  }
  // std::cerr << inconsistentUvs << " inconsistent UVs\n";
}

In the vertex shader, they are rotated into world space:

  fragNormal = (model.model * vec4(inNormal, 0)).xyz;
  fragTangent = vec4((model.model * vec4(inTangent.xyz, 0)).xyz, inTangent.w);

Then the binormal and world space normal are calculated like this (see http://mikktspace.com/):

  vec3 binormal = fragTangent.w * cross(fragNormal, fragTangent.xyz);
  vec3 worldNormal = normalize(normal.x * fragTangent.xyz +
                               normal.y * binormal +
                               normal.z * fragNormal);

(The binormal is usually calculated per pixel, but some bakers give you the option to calculate it per vertex and interpolate it. This page has information about specific programs.)