Get homography matrix from rotation or orientation matrix

Question

I am getting rotation matrix and orientations (Euler angles) using a sensor in an android device. I want to use these in opencv for affine transformation. affine transformation uses homography matrix to do its job. My question is how to convert rotation matrix or orientation array to homography matrix that is usable in affine transformation?

Android code to get rotation matrix and orientation:

final float[] rotationMatrix = new float[9];
SensorManager.getRotationMatrix(rotationMatrix, null, accelerometerReading, magnetometerReading);

final float[] orientationAngles = new float[3];
SensorManager.getOrientation(rotationMatrix, orientationAngles);

opencv code to affine transform:

homographtMatrix = ... # to calc from rotation matrix or orientation angls
warped = cv2.warpPerspective(img, homographtMatrix, (cols, 600))

Sample rotation matrix:

[
    [-0.39098227, -0.24775778, 0.8864249], 
    [0.9200034, -0.07699536, 0.38427263], 
    [-0.026955934, 0.96575755, 0.2580418]
]

Sample euler angles:

[1.3097044  0.0269592  1.97264932]

Image going to affine transform:

Desired transform (Cuts from left doesn't matter i can fix it):

Then I will tile a floor in a segmented image.

how do you want to warp a 2d image from a 3d rotation? What's the assumption? — Micka
I want to replace 3d rotated image with a segmented image. Now I only need homography matrix from rotation matrix or orientation. @Micka — Amir Fo
if you are able to construct the 3D object rotation (R) from those angles, the image homography will be something like K^-1 * R * K with K = camera intrinsics. Have to read/try which of the sides has to be inverted. — Micka
sorry, dont have a python script. Feel free to implement it yourself and add the answer. — Micka

Amir Fo Amir Fo · Accepted Answer · 2021-05-14T12:48:15

Homography matrix is a matrix to project a point in the world that is 3d (Pw) to a point in an image plane (Px). For example:

Px = H . Pw

This can be done by an original equation:

Px = K . (R . Pw + t)

Which K is the camera intrinsic parameters function that is created by a number called focal (f) length and a point called principal point (c) that is the center of the image most of the time.

K = [
      [f, 0, cx],
      [0, f, cy],
      [0, 0, 1 ],
]

And R is a 3d rotaion matrix. To explain this, We assume that the point in the world has z = 0 (lays on z plane). To move it to camera coordinates a rotation matrix should be used. z = 0 causes the removal of a column of the R matrix. The combination of K, one column removed R ant t that is the distance of point coordinates from camera coordinates is called homography matrix.

TL;DR

Mathematics concepts are boring to understand. I found an implementation of camera calibration from github that makes projections easy:

import cv2
from vcam import vcam, meshGen

img = cv2.imread("chess.png")

c1.focus = 390
c1.sx = 101/100
c1.sy = 101/100
c1.KpCoeff[0] = 0
c1.KpCoeff[1] = 0
c1.KpCoeff[2] = 0
c1.KpCoeff[3] = 0

c1.set_tvec(0, 0, -500)
c1.set_rvec(0, 0, -180)

pts2d = c1.project(pts3d)
map_x, map_y = c1.getMaps(pts2d)
output = cv2.remap(img, map_x, map_y, interpolation=cv2.INTER_LINEAR)

It has an interesting GUI to test parameters that how they effect the output.

Get homography matrix from rotation or orientation matrix

1 Answers

TL;DR