using aruco to estimate the world position of camera

Question

In OpenCV, I am using a Charuco board, have calibrated the camera, and use estimate to get rvec and tvec. (similar to the sample code). I am using a stationary board, with a camera moves around. I'm a noob with this, so please bear with me if this is something simple I am missing.

I understand that I can use Rodrigues() to get the 3x3 rotation matrix for the board orientation from rvec, and that I can transform the tvec value into world coordinates using worldPos = -R.t() * tvec (in python) to get position of camera

It seems that I got everything i need, however, if I rotate the camera physically, WorldPos also changes hugely! I am sure it’s not because the camera has been moved since I tried many times.

I don’t know where the problem is, thanks for your suggestion.

And the code is below(in python).

self.retval, self.rvec, self.tvec =   aruco.estimatePoseBoard(self.corners, self.ids, board, self.cameraMatrix, self.distanceCoefficients)
self.dst, jacobian = cv2.Rodrigues(self.rvec)
self.rvec_trs = self.dst.transpose()
self.worldPos = - self.rvec_trs * self.tvec
self.worldPos = [self.worldPos[0][0],self.worldPos[1][1], self.worldPos[2][2]]

what do you mean with "if I rotate the camera"? physically? how much is hugely? you should explain better with maybe an example and numbers and the code that you have used, for people to understand the problem better. — api55
are you sure that rvec and tvec are the camera extrinsics and not the object/board pose relative to the camera? — Micka
@Micka I think it's the board pose relative to the camera. I found that I need to get the camera extrinsic to estimate the worldPos of camera. So can you give me some suggestions? thank you! — linfeng chen

linfeng chen linfeng chen · Accepted Answer · 2018-10-17T06:42:17

Finally I solved it with the help of @Micka. The code is below(in python), hope it can help others who have the same problem.

self.retval, self.rvec, self.tvec = aruco.estimatePoseBoard(self.corners, self.ids, board, self.cameraMatrix, self.distanceCoefficients
self.dst, jacobian = cv2.Rodrigues(self.rvec)
self.extristics = np.matrix([[self.dst[0][0],self.dst[0][1],self.dst[0][2],self.tvec[0][0]],
                             [self.dst[1][0],self.dst[1][1],self.dst[1][2],self.tvec[1][0]],
                             [self.dst[2][0],self.dst[2][1],self.dst[2][2],self.tvec[2][0]],
                             [0.0, 0.0, 0.0, 1.0]
                ])
self.extristics_I = self.extristics.I  # inverse matrix
self.worldPos = [self.extristics_I[0,3],self.extristics_I[1,3],self.extristics_I[2,3]]

But I still don't know why worldPos = -R.t() * tvec cannot get the right answer. Hope someone can tell me. Thanks!

using aruco to estimate the world position of camera

1 Answers