Which coordinate system does gluUnProject gives me?

Question

enter image description here

As shown in the picture my object has its own "local coordinate system".(Actually I have a pointcloud instead of a mesh)

Then I apply transformations/scaling..etc to ModelView matrix and transformed coordinate system is "modelview coordinate system".

Now I am tring to pick of "vertex" using gluUnproject as follows.

GLint viewport[4];
GLdouble modelview[16];
GLdouble projection[16];
GLfloat winX, winY, winZ;


glGetDoublev(GL_MODELVIEW_MATRIX, modelview);
glGetDoublev(GL_PROJECTION_MATRIX, projection);
glGetIntegerv(GL_VIEWPORT, viewport);

winX = (GLfloat) mousePosition.x();
winY = (GLfloat) viewport[3] - (float) mousePosition.y();
glReadPixels((int)winX , int(winY) , 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT,&winZ);

GLdouble posX, posY, posZ;
gluUnProject( winX, winY, winZ, modelview, projection, viewport, &posX, &posY, &posZ);

Does this "gluUnProject" should give me the location relative to the "local coordinate system" or is it relative to the "modelview coordinate system" ?

datenwolf datenwolf · Accepted Answer · 2011-11-15T09:37:03

Modelview is not a (particular) coordinate system, but the transformation from object local space (model-) to eye space (-view).

In the picture above we see object space (the tripod "in" the teapot), and world space (the large tripod). But then there's eye space, for which no tripod is given, though we can imagine it easy enough: One vertical axis up and one horizontal axis right in the middle and a axis sticking out of the picture.

What we don't "see" is the projection, i.e. the transformation that projects the picture to the screen (technically clip space, then from there to NDC by a additional step).

So we have the following transformations:

Modelview, MV, from object space to eye space (this completely bypasses the world space)
Projection P, from world space to screen space.

A vertex goes through those

 v_eye = MV * v
 v_clip = P * v_eye = P * (MV * v) = P * MV * v

thereafter the clip to NDC-transformation is applied.

 [x,y,depth] = clip_to_NDC( v_clip ) = clip_to_NDC(P * MV * v)

What gluUnProject does is the reversal of just the above expression:

 gluUnProject(MV, P, x, y, depth):

 v_clip = clip_to_NDC^-1([x,y,depth])
 v = MV^-1 * P^-1 * v_clip = MV^-1 * P^-1 * clip_to_NDC^-1([x,y,depth])

Or in other words given the modelview and projection matrix, gluUnProject will project back from screen space into the space transformed there by the transformation chain.

It also helps to remember that OpenGL is not a scene graph, i.e. it doesn't keep track of all the transformations, objects you've drawn, etc. It "lives for the moment" if you want to say so.

Which coordinate system does gluUnProject gives me?

1 Answers