Finding backup winZ for glm::unProject

Question

I have an OpenGL program that basically just renders a bunch of points. I need to find the world-space coordinates from an arbitrary cursor position, which I do like this:

glm::mat4 modelview = graphics.view*graphics.model;
glm::vec4 viewport = { 0.0, 0.0, windowWidth, windowHeight };

float winX = cursorX;
float winY = viewport[3] - cursorY;
float winZ;

glReadPixels(winX, winY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ);
glm::vec3 screenCoords = { winX, winY, winZ };

glm::vec3 cursorPosition = glm::unProject(screenCoords, modelview, graphics.projection, viewport);

This works fine if the cursor is on an object, but if it's on an empty part of the screen (i.e. most of it), it's assumed to be on the far clipping plane (winZ = 1), and glm::unProject returns inf values. Ideally, I'd like to pass it a different winZ corresponding to the xy plane at z=0 in world space coordinates, but I can't figure out how to get that value.

z_win=1 being un-projected to infinity would imply that you're using an infinite far plane. Then, you should never feed z_win=1 into the unproject function. "I'd like to pass it a different winZ corresponding to the xy plane at z=0 " in which space? I'd assume that xy-plane is still supposed to be parallel to the near plane? — derhass
That's a good point, I should have mentioned that I'm using an infinitePerspective projection. "xy plane at z=0" is in world space coordinates, and it is parallel to the near plane — Elmo

Rabbid76 Rabbid76 · Accepted Answer · 2018-07-29T07:48:01

As there is a glm::unProject function, the is a glm::project function too.

If you want to know the depth of a point, which is located on parallel plane to the view space, with a z-coordinate of 0, then you have to glm::project a point on this plane. It can be any point on the plane, because you are only interested in its z-coordinate. You can use the origin of the world for this:

glm::vec3 world_origin{ 0.0f, 0.0f, 0.0f };
glm::vec3 origin_ndc = glm::project(screenCoords, view, graphics.projection, viewport);
float depth0         = world_origin[2];

where view is the view matrix, which transforms from world space to camera space (got from glm::lookAt).

So the implementation may look like this:

glm::mat4 modelview = graphics.view * graphics.model;
glm::vec4 viewport  = { 0.0, 0.0, windowWidth, windowHeight };

float winX = cursorX;
float winY = viewport[3] - cursorY;
float depth;
glReadPixels(winX, winY- cursorY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &depth);

const float epsi = 0.00001f;
if ( depth > 1.0f-epsi )
{
    glm::vec3 world_origin{ 0.0f, 0.0f, 0.0f };
    glm::vec3 origin_ndc = glm::project(world_origin, graphics.view, graphics.projection, viewport);
    depth                = origin_ndc[2];  
}

glm::vec3 screenCoords{ winX, winY, depth };
glm::vec3 cursorPosition = glm::unProject(screenCoords, modelview, graphics.projection, viewport);

Finding backup winZ for glm::unProject

1 Answers