How do I determine the camera rotation in world space (not orientation of a Charuco board) in OpenCV?

0
votes

In OpenCV, I am using a Charuco board, have calibrated the camera, and use SolvePnP to get rvec and tvec. (similar to the sample code). I am using a stationary board, with a camera on a circular rig which orbits the board. I'm a noob with this, so please bear with me if this is something simple I am missing.

I understand that I can use Rodrigues() to get the 3x3 rotation matrix for the board orientation from rvec, and that I can transform the tvec value into world coordinates using -R.t() * tvec (in c++).

However, as I understand it, this 3x3 rotation R gives the orientation of the board with respect to the camera, so it's not quite what need. I want the rotation of the camera itself, which is offset from R by (I think) the angle between tvec and the z axis in camera space. (because the camera is not always pointing at the board origin, but it is always pointing down the z axis in camera space). Is this correct?

How do I find the additional rotation offset and convert it to a 3x3 rotation matrix which I can combine with R to get the actual camera orientation?

Thanks!

1
opencvaruco

1 Answers

1
votes

Let's say you capture N frames of the Charuco board from the camera. Then, you have N transformations taking a point in the camera frame to the same point in the Charuco board frame. This is obtained from the Charuco board poses for every frame.

Say we denote a linear transformation from one coordinate frame to another as

T4x4 = [R3x3, t3x1; 01x3, 1]

board12

So, we could write:

Pboard = T1Pc1
Pboard = T2Pc2
.
.
.
Pboard = TNPcN

From what I understand, you need camera's rotation say from the starting point (assume that camera is at zero rotation in the frame 1). So, you could express every subsequent frame in terms of Pc1 instead of Pboard.

So, we could say
T2Pc2 = T1Pc1
or, Pc2 = T2-1T1Pc1
Similarly, PcN = TN-1T1Pc1

You could recover RN for taking point from camera position N to camera position 1 by looking at the rotation part of TN-1T1.