OpenCV Camera Calibration: very large k3 value in distCoeffs

0
votes

I run cv2.calibrateCamera on 67 images with 19x3 chessboard corner detections in each. I find reprojection error around 0.3. When I undistort images, they look great. They had little distortion before, and appear to be essentially perfect afterward. This is with high quality scientific cameras, however they view is underwater which complicates matters.

Later, I use these camera calibrations (and those from the other camera in a stereo setup) as input to stereoCalibrate, which returns error about 0.3. When I try to stereorectify and remap, I get nonsense: black images, or warped images, depending on the alpha parameter in cv2.stereoRectify.

This led me back to the original camera matrices. My distortion coefficients from the camera calibration are

array([[-1.44561083e-01, -3.21620254e+01,  0.00000000e+00,
 0.00000000e+00,  1.72034253e+03]])

K3 seems very large! Any idea what is going on here? Could this be ruining my stereo rectification later on?

1
opencvcamera-calibrationstereo-3d
Stereo calibration can be quiet tricky. When you calibrated each camera individually did you click pictures simultaneously for both cameras? - Rakshith G B
Yes- the calibration images are carefully taken and I can't think of any points for improvement. They are with a specially designed and professionally printed pvc calibration board, high resolution scientific cameras with little distortion - kevinkayaks
As kevinkayaks said it may be result of overfitting. Try to simplify model for camera distortions by passing CALIB_ZERO_TANGENT_DIST or CALIB_FIX_K%i as flags argument to cv2.calibrateCamera. - tstanisl

1 Answers

1
votes

Since few people seem to follow opencv questions until they need the answers, I'll share what I learned.

With calibration datasets involving highly ordered orientations of the calibration board, stereo calibration algorithms easily overfit. For example, 40 of my calibration images were related by 1cm incremental variations, as well at 17 lateral orientations. Cameras were very high quality with basically no distortion. This resulted in massive overfitting: the opencv algorithm was learning the pattern in the calibration image orientations I think.

The solution I found in this case was to use the Bouquet toolbox in Matlab. It is superior to opencv in that it permits one to constrain many parameters in the optimization. This helps overcome the overfitting issue I was getting which was leading to large image distortions.