Warping Perspective using arbitary rotation angle

Question

I have an image of a chessboard taken at some angle. Now I want to warp perspective so the chessboard image look again as if was taken directly from above.

I know that I can try to use 'findHomography' between matched points but I wanted to avoid it and use e.g. rotation data from mobile sensors to build homography matrix on my own. I calibrated my camera to get intrinsic parameters. Then lets say the following image has been taken at ~60degrees angle around x-axis. I thought that all I have to do is to multiply camera matrix with rotation matrix to obtain homography matrix. I tried to use the following code but looks like I'm not understanding something correctly because it doesn't work as expected (result image completely black or white.

enter image description here

import cv2
import numpy as np
import math 



camera_matrix = np.array([[ 5.7415988502105745e+02, 0., 2.3986181527877352e+02],
                           [0., 5.7473682183375217e+02, 3.1723734404756237e+02], 
                           [0., 0., 1.]])

distortion_coefficients = np.array([ 1.8662919398453856e-01, -7.9649812697463640e-01,
   1.8178068172317731e-03, -2.4296638847737923e-03,
   7.0519002388825025e-01 ])

theta = math.radians(60)

rotx = np.array([[1, 0, 0],
               [0, math.cos(theta), -math.sin(theta)],
               [0, math.sin(theta), math.cos(theta)]])   



homography = np.dot(camera_matrix, rotx)


im = cv2.imread('data/chess1.jpg')
gray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)

im_warped = cv2.warpPerspective(gray, homography, (480, 640), flags=cv2.WARP_INVERSE_MAP)
cv2.imshow('image', im_warped)
cv2.waitKey()
pass

I also have distortion_coefficients after calibration. How can those be incorporated into the code to improve results?

rwong rwong · Accepted Answer · 2014-11-26T17:08:35

This answer is awfully late by several years, but here it is ...

(Disclaimer: my use of terminology in this answer may be imprecise or incorrect. Please do look up on this topic from other more credible sources.)

Remember:

Because you only have one image (view), you can only compute 2D homography (perspective correspondence between one 2D view and another 2D view), not the full 3D homography.
Because of that, the nice intuitive understanding of the 3D homography (rotation matrix, translation matrix, focal distance, etc.) are not available to you.
What we say is that with 2D homography you cannot factorize the 3x3 matrix into those nice intuitive components like 3D homography does.
You have one matrix - (which is the product of several matrices unknown to you) - and that is it.

However,

OpenCV provides a getPerspectiveTransform function which solves the 3x3 perspective matrix (using homogenous coordinate system) for a 2D homography between two planar quadrilaterals.

Link to documentation

To use this function,

Find the four corners of the chessboard on the image. These will be your source coordinates.
Supply four rectangle corners of your choice. These will be your destination coordinates.
Pass the source coordinates and destination coordinates into the getPerspectiveTransform to generate a 3x3 matrix that is able to dewarp your chessboard to an upright rectangle.

Notes to remember:

Mind the ordering of the four corners.
- If the source coordinates are picked in clockwise order, the destination also needs to be picked in clockwise order.
- Likewise, if counter-clockwise order is used, do it consistently.
- Likewise, if z-order (top left, top right, bottom left, bottom right) is used, do it consistently.
- Failure to order the corners consistently will generate a matrix that executes the point-to-point correspondence exactly (mathematically speaking), but will not generate a usable output image.
The aspect ratio of the destination rectangle can be chosen arbitrarily. In fact, it is not possible to deduce the "original aspect ratio" of the object in world coordinates, because "this is 2D homography, not 3D".

Warping Perspective using arbitary rotation angle

3 Answers