Undistortion crops the image and distort edges

Question

I’m trying to calibrate and undistort image from fish-eye camera.
My code is:

import cv2
import os
import numpy as np

CHECKERBOARD = (5,7)
subpix_criteria = (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 0.1)
calibration_flags = cv2.fisheye.CALIB_RECOMPUTE_EXTRINSIC+cv2.fisheye.CALIB_CHECK_COND+cv2.fisheye.CALIB_FIX_SKEW
R = np.zeros((1, 1, 3), dtype=np.float64)
T = np.zeros((1, 1, 3), dtype=np.float64)
objp = np.zeros( (CHECKERBOARD[0]*CHECKERBOARD[1], 1, 3) , np.float64)
objp[:,0, :2] = np.mgrid[0:CHECKERBOARD[0], 0:CHECKERBOARD[1]].T.reshape(-1, 2)
_img_shape = None
objpoints = [] # 3d point in real world space
imgpoints = []

N_OK = len(objpoints)

images = os.listdir('./images/')

for fname in images:
    img = cv2.imread(fname)
    img = cv2.imread('./images/'+fname)
    #print(fname + str(os.path.exists('./images/'+fname)))
    ext = os.path.splitext(fname)[-1].lower()
    if ext == ".jpg":
        print(img.shape[:2])
        if _img_shape == None:
            _img_shape = img.shape[:2]
        else:
            assert _img_shape == img.shape[:2], "All images must share the same size."

        gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
        ret, corners = cv2.findChessboardCorners(gray, CHECKERBOARD,cv2.CALIB_CB_ADAPTIVE_THRESH+cv2.CALIB_CB_FAST_CHECK+cv2.CALIB_CB_NORMALIZE_IMAGE)
        if ret == True:
            objpoints.append(objp)
            cv2.cornerSubPix(gray,corners,(3,3),(-1,-1),subpix_criteria)
            imgpoints.append(corners)

N_OK = len(objpoints)
K = np.zeros((3, 3))
D = np.zeros((4, 1))
rvecs = [np.zeros((1, 1, 3), dtype=np.float64) for i in range(N_OK)]
tvecs = [np.zeros((1, 1, 3), dtype=np.float64) for i in range(N_OK)]
rms, K, D, rvecs, tvecs = \
    cv2.fisheye.calibrate(
        objpoints,
        imgpoints,
        gray.shape[::-1],
        K,
        D,
        rvecs,
        tvecs,
        calibration_flags,
        (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 1e-6)
    )

DIM=_img_shape[::-1]
K=np.array(K.tolist())
D=np.array(D.tolist())

And function to undistort:

def undistort(img_path):    
    img = cv2.imread(img_path)
    h,w = img.shape[:2]    
    nk = K.copy()
    map1, map2 = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3), K, DIM, cv2.CV_16SC2)
    undistorted_img = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)    
    cv2.imwrite('calibresult1.png',undistorted_img)

It gives following image: undistorted image

While original image is: original image

The center seems to be undistorted, but corners are distorted and image itself is cropped. I'm not sure that the calibration process is correct. If anyone has experience with it, I would be happy if you look at the code and probably find errors.

Cropping is unavoidable, because after undistorting the edges of the image will not be complete. You can't restore parts of the image that were never captured. — Mark Ransom

Alejandro Silvestri Alejandro Silvestri · Accepted Answer · 2019-10-25T15:38:14

Fast answer: bad calibration. Your undistorted image was obtained with (very) wrong intrinsics and distortion coefficients.

Sorry I'm not debugging your code, but code can be fine and still not getting the right calibration. Not everything is code: you must choose useful chessboard poses and many images to improve calibration.

Recommendation: with fisheye calibration, let start trying to get only intrinsics (camera center and focal point), and avoid computing distortion coefficients.

Undistortion crops the image and distort edges

1 Answers