OpenCV - How to get real world distance from a 2D image using a chessboard as reference

Question

After checking several pieces of codes, I took several shots, found the chessboard corners and use them to get the camera matrix, distortion coefficients, rotation, and translation vectors. Now, can someone tell me which python opencv function do I need to calculate the distance in the real world from the 2D image? project points? For example, using a chessboard as a reference (see picture), if the tile size is 5cm, the distance for 4 tiles should be 20 cm. I saw some functions like projectPoints,findHomography, solvePnP but I am not sure which one do I need to solve my problem and get the transformation matrix between the camera world and the chessboard world. 1 single camera, same position of the camera for all cases but not exactly over the chessboard, and chessboard is placed over a planar object (table)

# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
    objp = np.zeros((nx * ny, 3), np.float32)
    objp[:, :2] = np.mgrid[0:nx, 0:ny].T.reshape(-1, 2)

    # Arrays to store object points and image points from all the images.
    objpoints = []  # 3d points in real world space
    imgpoints = []  # 2d points in image plane.

    # Make a list of calibration images
    images = glob.glob(path.join(calib_images_dir, 'calibration*.jpg'))
    print(images)
    # Step through the list and search for chessboard corners
    for filename in images:

        img = cv2.imread(filename)

        imgScale = 0.5
        newX,newY = img.shape[1]*imgScale, img.shape[0]*imgScale
        res = cv2.resize(img,(int(newX),int(newY)))

        gray = cv2.cvtColor(res, cv2.COLOR_BGR2GRAY)

        # Find the chessboard corners
        pattern_found, corners = cv2.findChessboardCorners(gray, (nx,ny), None)

        # If found, add object points, image points (after refining them)
        if pattern_found is True:
            objpoints.append(objp)

            # Increase accuracy using subpixel corner refinement
            cv2.cornerSubPix(gray,corners,(5,5),(-1,-1),(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.1 ))
            imgpoints.append(corners)

            if verbose:
                # Draw and display the corners
                draw = cv2.drawChessboardCorners(res, (nx, ny), corners, pattern_found)
                cv2.imshow('img',draw)
                cv2.waitKey(500)

    if verbose:
        cv2.destroyAllWindows()

    #Now we have our object points and image points, we are ready to go for calibration
    # Get the camera matrix, distortion coefficients, rotation and translation vectors
    ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None)
    print(mtx)
    print(dist)
    print('rvecs:', type(rvecs),' ',len(rvecs),' ',rvecs)
    print('tvecs:', type(tvecs),' ',len(tvecs),' ',tvecs)

    mean_error = 0
    for i in range(len(objpoints)):
        imgpoints2, _ = cv2.projectPoints(objpoints[i], rvecs[i], tvecs[i], mtx, dist)
        error = cv2.norm(imgpoints[i],imgpoints2, cv2.NORM_L2)/len(imgpoints2)
        mean_error += error

    print("total error: ", mean_error/len(objpoints))


    imagePoints,jacobian = cv2.projectPoints(objpoints[0], rvecs[0], tvecs[0], mtx, dist)
    print('Image points: ',imagePoints)

So, just to make sure I understood, you calibrated with several images of a chessboard and now you want to take another image of the same chessboard and compute the distance between the camera and the board? — Milo
@Milo I calibrated the camera to obtain the camera parameters. With cam matrix and distortion coefficients,My new idea is to use the getPerspective method to obtain a better view of the chessboard, so how can I get the size in real world if the chessboard can be seen as if the camera is exactly over the chessboard. Is there any opencv method for that? I found info about use a ratio px/mm but I wanna know if there is a better method for that. Now, is there any good way to convert from pixels to a unit in meters? Any method inside opencv? — Pablo Gonzalez
@PabloGonzalez To help you, I really need to understand your final goal. You have a calibrated camera over a table, in which will be a chessboard (but not directly below the camera). You would like to generate a "flat" image of the chessboard, as the one you posted, but all of this with which goal? Just generate the "flat" image? Compute the size of a new (unknown) board? Compute the position of the board with respect to the camera? Maybe you can find some information in this answer: stackoverflow.com/questions/55220229/… — Milo
@Milo I want to measure the size of the tile so I can know the mm/pixel ratio, and finally calculate lengths and sizes in real world using the camera.Similar to stackoverflow.com/questions/49530738/… — Pablo Gonzalez

Douglas Brion Douglas Brion · Accepted Answer · 2019-03-21T13:17:31

You are indeed right, and I think you should use solvePnP for this problem. (Read more on perspective-n-point problems here: https://en.wikipedia.org/wiki/Perspective-n-Point.)

The Python OpenCV solvePnP function takes the following parameters and returns an ouput rotation and output translation vector which converts the model coordinate system to the camera coordinate system.

cv2.solvePnP(objectPoints, imagePoints, cameraMatrix, distCoeffs[, rvec[, tvec[, useExtrinsicGuess[, flags]]]]) → retval, rvec, tvec

In your case the imagePoints will be the corners of the chessboard so it would look something like:

ret, rvec, tvec = cv2.solvePnP(objpoints, corners, mtx, dist)

With the returned translation vector you can calculate the distance from the camera to the chessboard. The output translation from solvePnP is in the same units as specified in objectPoints.

Finally, you can compute the real distance from the tvec as the euclidean distance:

d = math.sqrt(tx*tx + ty*ty + tz*tz).

OpenCV - How to get real world distance from a 2D image using a chessboard as reference

2 Answers