3D perspective 'grab' panning with DirectX

Question

I am implementing a pan tool in our software's 3D view which is supposed to work much like the grab tool of, say, Photoshop or Acrobat Reader. That is, the point the user grabs onto with the mouse (clicks and holds, then moves the mouse) stays under the mouse cursor as the mouse moves.

This is a common paradigm and one that's been asked about on SO before, the best answer being to this question about the technique in OpenGL. There is another that also has some hints, and I have been reading this very informative CodeProject article. (It doesn't explain many of its code examples' variables etc, but from reading the text I think I understand the technique.) But, I have some implementation issues because my 3D environment's navigation is set up quite differently to those articles, and I am seeking some guidance.

My technique - and this might be fundamentally flawed, so please say so - is:

The scene 'camera' is stored as two D3DXVECTOR3 points: the eye position and a look point. The view matrix is constructed using D3DXMatrixLookAtLH like so:
```
const D3DXVECTOR3 oUpVector(0.0f, 1.0f, 0.0f); // Keep up "up", always.
D3DXMatrixLookAtLH(&m_oViewMatrix, &m_oEyePos, &m_oLook, &oUpVector);
```

When the mouse button is pressed, shoot a ray through that pixel and find: the coordinate (in unprojected scene / world space) of the pixel that was clicked on; the intersection of that ray with the near plane; and the distance between the near-plane point and object, which is the length between those two points. Store this and the mouse position, and the original navigation (eye and look).

// Get the clicked-on point in unprojected (normal) world space
D3DXVECTOR3 o3DPos;
if (Get3DPositionAtMouse(roMousePos, o3DPos)) { // fails if nothing under the mouse

// Mouse location when panning started
m_oPanMouseStartPos = roMousePos;

// Intersection at near plane (z = 0) of the ray from camera to clicked spot
D3DXVECTOR3 oRayVector;
CalculateRayFromPixel(m_oPanMouseStartPos, m_oPanPlaneZ0StartPos, oRayVector);

// Store original eye and look points
m_oPanOriginalEyePos = m_oEyePos;
m_oPanOriginalLook = m_oLook;

// Store the distance between near plane and the object, and the object position
m_dPanPlaneZ0ObjectDist = fabs(D3DXVec3Length(&(o3DPos - m_oPanPlaneZ0StartPos)));
m_oPanOriginalObjectPos = o3DPos;

Get3DPositionAtMouse is a known-ok method which picks a 3D coordinate under the mouse. CalculateRayFromPixel is a known-ok method which takes in a screen-space mouse coordinate and casts a ray, and fills the other two parameters with the ray intersection at the near plane (Z = 0) and the normalised ray vector.

When the mouse moves, cast another ray at the new position, but using the old (original) view matrix. (Thanks to Nico below for pointing this out.) Calculate where the object should be by extending the ray from the near plane the distance between the object and near plane (this way, the original object and new object points should be in parallel plane to the near plane.) Move the eye and look coordinates by this much. Eye and Look are set from their original (when panning started) values, with the difference being from the original mouse and new mouse positions. This is to reduce any precision loss from incrementing or decrementing by granular (integer) pixel movements as the mouse moves, ie it calculates the whole difference in navigation every time.

// Set navigation back to original (as it was when started panning) and cast a ray for the mouse
m_oEyePos = m_oPanOriginalEyePos;
m_oLook = m_oPanOriginalLook;
UpdateView();
D3DXVECTOR3 oRayVector;
D3DXVECTOR3 oNewPlaneZPos;
CalculateRayFromPixel(roMousePos, oNewPlaneZPos, oRayVector);

// Now intersect that ray (ray through the mouse pixel, using the original navigation)
// to hit the plane the object is in.  Function uses a "line", so start at near plane
// and the line is of the length of the far plane away
D3DXVECTOR3 oNew3DPos;
D3DXPlaneIntersectLine(&oNew3DPos, &m_oPanObjectPlane, &oNewPlaneZPos, &(oRayVector * GetScene().GetFarPlane()));

// The eye/look difference /should/ be as simple as:
//  const D3DXVECTOR3 oDiff = (m_oPanOriginalObjectPos - oNew3DPos);
// But that lags and is slow, ie the objects trail behind.  I don't know why.  What does
// work is to scale the from-to difference by the distance from the camera relative to
// the whole scene distance
const double dDist = D3DXVec3Length(&(oNew3DPos - m_oPanOriginalEyePos));
const double dTotalDist = GetScene().GetFarPlane() - GetScene().GetNearPlane();
const D3DXVECTOR3 oDiff = (m_oPanOriginalObjectPos - oNew3DPos) * (1.0 + (dDist / dTotalDist));

// Adjust the eye and look points by the same amount, so orthogonally changed
m_oEyePos = m_oPanOriginalEyePos + oDiff;
m_oLook = m_oPanOriginalLook + oDiff;

Diagram

This diagram is my working sketch for implementing this:

3D panning sketch

and hopefully explains the above much more simply than the text. You can see a moving point, and where the camera has to move to keep that point at the same relative position. The clicked-on point (the ray from the camera to the object) is just to the right of the straight-ahead ray representing the center pixel.

The problem

But, as you've probably guessed, this doesn't work as I hope. What I wanted to see was the clicked-on object moving with the mouse cursor. What I actually see is that the object moves in the direction of the mouse, but not enough, ie it does not keep the clicked-on point under the cursor. Secondly, the movement flickers and jumps around, jittering by up to twenty or thirty pixels sometimes, then flickers back. If I replace oDiff with something constant this doesn't occur.

Any ideas, or code samples showing how to implement this with DirectX (D3DX, DX matrix order, etc) will be gratefully read.

Edit

Commenter Nico below pointed out that when calculating the new position using the mouse cursor's moved position, I needed to use the original view matrix. Doing so helps a lot, and the objects stay near the mouse position. However, it's still not exact. What I've noticed is that at the center of the screen, it is exact; as the mouse moves further from the center, it gets out by more and more. This seemed to change based on how far away the object was, too. By pure 'I have no idea what I'm doing' guesswork, I scaled this by a factor of the near/far plane and how far away the object was, and this brings it very close to the mouse cursor, but still a few pixels away (1 to, say, 30 at the extreme edge of the screen, which is enough to make it feel wrong.)

default default · Accepted Answer · 2013-02-28T19:15:07

Here's how i solve this problem.

float fieldOfView = 45.0f;

float halfFOV = (fieldOfView / 2.0f) * (DEGREES_TO_RADIANS);
float distanceToObject = // compute the world space distance from the camera to the object you want to pan
float projectionToWorldScale = distanceToObject * tan( halfFov );
Vector mouseDeltaInScreenSpace = // the delta mouse in pixels that we want to pan

Vector mouseDeltaInProjectionSpace = Vector( mouseDeltaInScreenSpace.x * 2 / windowPixelSizeX, mouseDeltaInScreenSpace.y * 2 / windowPixelSizeY ); // ( the "*2" is because the projection space is from -1 to 1)

// go from normalized device coordinate space to world space (at origin)
Vector cameraDelta = -mouseDeltaInProjectionSpace * projectionToWorldScale;

// now translate your camera by "cameraDelta".

Note this works for an field of view apsect ratio of 1, i think you would have to break up the "scale" into separate x and y components if they vertical field of view was different than the horizontal field of view

Also, you mentioned a "look at" vector. I'm not sure how my math would need to change for that since my camera is always looking straight down the z-axis.

3D perspective 'grab' panning with DirectX

Diagram

The problem

Edit

2 Answers