Multi-camera calibration with opencv: Two cameras facing each other

Question

I want to compute the extrinsic calibration of two cameras w.r.t each other and am using cv::stereoCalibrate() function to do this. However, the result does not correspond to the reality. What could be wrong ?

Setup: Two cameras mounted 7 meters high, facing each other while looking downwards. They have lot of field of view intersection and I captured checkerboard images that I used in calibration.

I am not flipping any of the images.

Do I need to flip the images ? or do I need to do something else to tell that the cameras are actually facing each other ?

Note: The same function perfectly calibrates cameras that are next to each other facing in the same direction (like any typical stereo camera).

Thanks

How many images did you use for calibration? Do they have enough rotation variations? — hiroki
I am using 50 images with decent rotation and skew variation. — user3755683
What I would do: calibrate each camera independtly using cv::calibrateCamera(). Estimate the transformation between the two camera frames once they have been definitely fixed: chessboard pattern visible for each camera + cv::solvePnP(). The transformation c1Mc2 can be estimated as c1Mo x (c2Mo)^-1. — Catree
Thanks. Could you clarify what c1, c2 and Mo are. I am assuming c1 and c2 are the camera centers and Mo is the transformation ? solvePnP() produces the rotation and translation w.r.t. each camera for each image of the pattern. For example, for a given pattern, lets say we obtain R1, T1 (for the first camera) and R2, T2 (for the second camera) through the solvePnP() function. How can we compute Mo from these rotation-translation pairs ? — user3755683
The notation c1Mo corresponds to the homogeneous transformation between the object frame and the camera1 frame : a 4x4 matrix with (R | T). You will have to create c1Mo as R1 is a Rodrigues rotation vector, but basically c1Mo is the same as R1, T1, just a different notation. — Catree

alexisrozhkov alexisrozhkov · Accepted Answer · 2017-02-01T13:25:08

In order to "tell that the cameras are actually facing each other" you have to specify imagePoints1 and imagePoints2 correctly, such that points with matching indices correspond to a same physical point.

If in your case function works perfectly when the cameras are oriented in the same direction and doesn't work with your configuration - discrepancy between point indexing might be a probable reason (most likely points are flipped both vertically and horizontally).

One way to debug this is to either draw indices near the points on each of the frames, or color-code them and make sure they match between the images.

One question though - why do you use cv::stereoCalibrate()? The setting you described doesn't seem to be a good use-case for it. If you want to estimate extrinsic parameters of cameras you can use cv::calibrateCamera(). The only downside is that it assumes that intrinsic parameters are same for all provided views (all images were taken with same or very similar cameras). If it is not the case - indeed cv::stereoCalibrate() would be a better fit (but the manual suggests that you still estimate each camera intrinsic parameters individually using cv::calibrateCamera())

Multi-camera calibration with opencv: Two cameras facing each other

1 Answers