How to get a good cv2.stereoCalibrate after successful cv2.calibrateCamera

Question

Hi everyone I've been digging a bit into computer vision using Python and OpenCV and was trying to calibrate two cameras I've bought in order to do some 3D stereo reconstruction but I'm having some problems with it.

I've followed mostly this tutorial in order to calibrate the cameras separately (I apply it to both of them) and then I intend to use the cv2.stereoCalibrate to get the relative calibration.

With the single camera calibration everything seems to be working correctly, I get a very low re-proyect error and as far as my knowledge goes the matrices seems to look OK. Here I leave the results of the single camera calibration.

cameraMatrix1 and distCoeffs1:

[[ 951.3607329     0.          298.74117671]
 [   0.          954.23088299  219.20548594]
 [   0.            0.            1.        ]]

[[ -1.07320015e-01  -5.56147908e-01  -1.13339913e-03   1.85969704e-03
    2.24131322e+00]]

cameraMatrix2 and distCoeffs2:

[[ 963.41078117    0.          362.85971342]
 [   0.          965.66793023  175.63216871]
 [   0.            0.            1.        ]]

[[ -3.31491728e-01   2.26020466e+00   3.86190151e-03  -2.32988011e-03
   -9.82275646e+00]]

So after having those I do the following (I fix the intrinsics as I already know them from the previous calibrations):

stereocalibration_criteria = (cv2.TERM_CRITERIA_MAX_ITER + cv2.TERM_CRITERIA_EPS, 100, 1e-5)
stereocalibration_flags = cv2.CALIB_FIX_INTRINSIC
stereocalibration_retval, cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, R, T, E, F = cv2.stereoCalibrate(objpoints,imgpoints_left,imgpoints_right,cameraMatrix1,distCoeffs1,cameraMatrix2,distCoeffs2,gray_left.shape[::-1],criteria = stereocalibration_criteria, flags = stereocalibration_flags)

I've tried several times to change the flags of the stereoCalibrate and switch the matrices to see if I was mistaken in the order and that mattered but I'm still blocked with this and get a retval of around 30 (and after that I try to rectify the images and of course the result is a disaster).

I've also tried using some calibration images from the internet and I do get the same result so I assume that the problem is not with the images I've taken. If anyone can point me in the right direction or knows what could be it will be very very welcome.

Mikel Díez Buil Mikel Díez Buil · Accepted Answer · 2018-06-01T18:27:10

Turns out that the order of the images I was using was not the same for right and left camera... I was using

images_left = glob.glob('Calibration/images/set1/left*' + images_format)
images_right = glob.glob('Calibration/images/set1/right*' + images_format)

When I should have been using something more like:

images_left = sorted(glob.glob('Calibration/images/set1/left*' + images_format))
images_right = sorted(glob.glob('Calibration/images/set1/right*' + images_format))

This is because glob gets the images in an apparently random order so I was trying to match the wrong images. Now I finally get a 0.4 retval, which is not that bad.

How to get a good cv2.stereoCalibrate after successful cv2.calibrateCamera

1 Answers