How to keep mouse coordinates within window height/width limits

Question

Suppose I have a 600 by 600 window created through glfwCreateWindow().

I have a few models rendered in the scene and to move around them in the 3D space I use a camera class which moves around with my cursor as well as AWSD/spacebar keys.

To make the movement "seamless" I use glfwSetInputMode(this->window, GLFW_CURSOR, GLFW_CURSOR_DISABLED).

Now then the problem is that I am also trying to implent a mouse picker of sorts (casting ray from cursor position to the scene).

So I implemented the mouse picker from a video I found on yt, but it doesnt work properly. The reason being is that my mouse X and Y coordinates get bigger than width and height if for example I rotate on spot (and I can increase it this way indefinitely).

I understand that this is happening because I have no cursor to be bound within resolution limits due to glfwSetInputMode(this->window, GLFW_CURSOR, GLFW_CURSOR_DISABLED). I am asking how I should correct this so that I can keep the seamlessness as well as be able to limit the cursor coordinates within width and height (otherwise the mouse picker function wont work because normalized mouse coordinates go over [1,1] which completely breaks it and so on).

I will be grateful for any answers.

EDIT:

@httpdigest's answer put into very elementary code:

        //store offsetX and offsetY values (and dont forget to give them 0 as initial value)
        overshootX = mouseX - offsetX;
        overshootY = mouseY - offsetY;

        if (overshootX > width) {
            offsetX = offsetX + overshootX - width;
        }
        if (overshootX < 0) {
            offsetX = offsetX + overshootX;
        }
        if (overshootY > height) {
            offsetY = offsetY + overshootY - height;
        }
        if (overshootY < 0) {
            offsetY = offsetY + overshootY;
        }

        float withinWindowCursorPosX = mouseX - offsetX;
        float withinWindowCursorPosY = mouseY - offsetY;

Kai Burjack Kai Burjack · Accepted Answer · 2021-01-02T18:19:45

One way of doing it is to store an offset (initially (0, 0)) by how much the cursor moved beyond the bounds of the window/viewport (separate for X and Y).

Essentially, everytime when you either read the current cursor position or are being told of an update by an event, you calculate overshoot = cursorPos - offset (separate for X and Y). If overshoot exceeds (width, height) you increment offset by the excess it exceeded (width, height). Likewise, if overshoot is less than (0, 0) you decrement offset by the negative overshoot.

That way, you have a "sliding" window/rectangle which gets slided around as you keep on moving the mouse cursor beyond either edge of the screen.

Now, whenever you want to obtain the corrected cursor position (within your window), you simply use withinWindowCursorPos = cursorPos - offset.

You can then use withinWindowCursorPos for your picking calculations that require a cursor within the window/viewport.

How to keep mouse coordinates within window height/width limits

1 Answers