Create ray from mouse coordinates for 3D picking

Question

My Question

Can someone please link a good article/tutorial/anything or maybe even explain how to correctly cast a ray from the mouse coordinates to pick objects in 3D?
I already have the Ray and intersection works, now I only need to create the ray from the mouse click.

I would just like have something which I know actually should work, thats why I ask the professionals here, not something where I am unsure if it is even correct in the first place.

State right now

I have a ray class, which actually works and detects intersection if I set the origin and direction to be the same as the camera, so when I move the camera it actually selects the right thing.

Now I would like to actually have 3D picking with the mouse, not camera movement.
I have read so many other questions about this, 2 tutorials, and especially so much different math stuff, since I am really not good at it.
But that didn't help me much, because the people there often use some "unproject" functions, which seem to actually be deprecated and which I have no idea how to use and also don't have access to.

Right now I set the ray origin to the camera position and then try to get the direction of the ray from the calculations in this tutorial.
And it works a little bit, meaning the selection works when the camera is pointed at the object and also sometimes along the whole y-axis, I have no idea what is happening.

If someone wants to take a look at my code right now:

public Ray2(Camera cam, float mouseX, float mouseY) {
    origin = cam.getEye();
    float height = 600;
    float width = 600;
    float aspect = (float) width / (float) height;


    float x = (2.0f * mouseX) / width - 1.0f;
    float y = 1.0f - (2.0f * mouseX) / height;
    float z = 1.0f;
    Vector ray_nds = vecmath.vector(x, y, z);

    Vector4f clip = new Vector4f(ray_nds.x(), ray_nds.y(), -1.0f, 1.0f);
    Matrix proj = vecmath.perspectiveMatrix(60f, aspect, 0.1f, 100f);
    proj = proj.invertRigid();
    float tempX = proj.get(0, 0) * clip.x + proj.get(1, 0) * clip.y
            + proj.get(2, 0) * clip.z + proj.get(3, 0) * clip.w;
    float tempY = proj.get(0, 1) * clip.x + proj.get(1, 1) * clip.y
            + proj.get(2, 1) * clip.z + proj.get(3, 1) * clip.w;
    float tempZ = proj.get(0, 2) * clip.x + proj.get(1, 2) * clip.y
            + proj.get(2, 2) * clip.z + proj.get(3, 2) * clip.w;
    float tempW = proj.get(0, 3) * clip.x + proj.get(1, 3) * clip.y
            + proj.get(2, 3) * clip.z + proj.get(3, 3) * clip.w;


    Vector4f ray_eye = new Vector4f(tempX, tempY, tempZ, tempW);
    ray_eye = new Vector4f(ray_eye.x, ray_eye.y, -1.0f, 0.0f);

    Matrix view = cam.getTransformation();
    view = view.invertRigid();

    tempX = view.get(0, 0) * ray_eye.x + view.get(1, 0) * ray_eye.y
            + view.get(2, 0) * ray_eye.z + view.get(3, 0) * ray_eye.w;
    tempY = view.get(0, 1) * ray_eye.x + view.get(1, 1) * ray_eye.y
            + view.get(2, 1) * ray_eye.z + view.get(3, 1) * ray_eye.w;
    tempZ = view.get(0, 2) * ray_eye.x + view.get(1, 2) * ray_eye.y
            + view.get(2, 2) * ray_eye.z + view.get(3, 2) * ray_eye.w;
    tempW = view.get(0, 3) * ray_eye.x + view.get(1, 3) * ray_eye.y
            + view.get(2, 3) * ray_eye.z + view.get(3, 3) * ray_eye.w;

    Vector ray_wor = vecmath.vector(tempX, tempY, tempZ);
    // don't forget to normalise the vector at some point
    ray_wor = ray_wor.normalize();
    direction = ray_wor;
}

Michael IV Michael IV · Accepted Answer · 2014-09-18T19:44:23

First,unproject() method is the way to go.It is not deprecated at all.You can find it implemented in GLM math library for example.Here is my implementation of Ray based 3D picking:

    // let's check if this renderable's AABB is clicked:
    const glm::ivec2& mCoords = _inputManager->GetMouseCoords();

    int mouseY = _viewportHeight - mCoords.y;

    //unproject twice to build a ray from near to far plane"
    glm::vec3 v0 = glm::unProject(glm::vec3(float(mCoords.x), float(mouseY), 0.0f),_camera->Transform().GetView(),_camera->Transform().GetProjection(), _viewport);
    glm::vec3 v1 = glm::unProject(glm::vec3(float(mCoords.x), float(mouseY), 1.0f),_camera->Transform().GetView(),_camera->Transform().GetProjection(), _viewport);

    glm::vec3 dir  = (v1 - v0); 

    Ray r(_camera->Transform().GetPosition(),dir);

    float ishit ;

     //construct AABB:
    glm::mat4 aabbMatr = glm::translate(glm::mat4(1.0),renderable->Transform().GetPosition());

    aabbMatr = glm::scale(aabbMatr,renderable->Transform().GetScale());

    //transforms AABB vertices(need it if the origianl bbox is not axis aligned as in this case)
    renderable->GetBoundBox()->RecalcVertices(aabbMatr);

     //this method makes typical Ray-AABB intersection test:
    if(r.CheckIntersectAABB(*renderable->GetBoundBox().get(),&ishit)){

        printf("HIT!\n");

    }

But I would suggest you also to take a look at color based 3d picking which is pixel perfect and even easier to implement.

Create ray from mouse coordinates for 3D picking

My Question

State right now

1 Answers