Finding most suitable Rotation and Translation from Homography decomposition

Question

I'm trying to find the rotation and translation from Homography function. First I compute the corresponding feature points and using findHomography() I computed the Homography Matrix. Then, using decomposeHomographyMat(), I got four rotation and translations results.

The code I used is below:

Mat frame_1, frame_2;


frame_1 = imread("img1.jpg", IMREAD_GRAYSCALE);
frame_2 = imread("img2.jpg", IMREAD_GRAYSCALE);

vector<KeyPoint> keypts_1, keypts_2;
Mat desc1, desc2;

Ptr<Feature2D> ORB = ORB::create(100    );
ORB->detectAndCompute(frame_1, noArray(), keypts_1, desc1);
ORB->detectAndCompute(frame_2, noArray(), keypts_2, desc2);

Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("BruteForce-Hamming");
vector<DMatch> matches;
matcher->match(desc1, desc2, matches);

vector<Point2f>leftPts, rightPts;
    for (size_t i = 0; i < matches.size(); i++)
    {
        //queryIdx is the left Image
        leftPts.push_back(keypts_1[matches[i].queryIdx].pt);

        //trainIdx is the right Image
        rightPts.push_back(keypts_2[matches[i].trainIdx].pt);
    }

Mat cameraMatrix = (Mat1d(3, 3) << 706.4034, 0, 277.2018, 0, 707.9991, 250.6182, 0, 0, 1);
Mat H = findHomography(leftPts, rightPts);
vector<Mat> R, t, n;
decomposeHomographyMat(H, cameraMatrix, R, t, n);

Now what is the right rotation and translation, at least the most suitable. I even checked if the rotation is valid using the below function, and found all are valid.

bool isRotationMatrix(Mat &R)
{
    Mat Rt;
    transpose(R, Rt);
    Mat shouldBeIdentity = Rt * R;
    Mat I = Mat::eye(3, 3, shouldBeIdentity.type());

    return  norm(I, shouldBeIdentity) < 1e-6;
}

Please some one suggest me, what value should I use. And is the resultant translation is a scaled value, which can be used directly, unlike the Essential Matrix decomposition case? I highly appreciate if someone can guide me on finding this.

Thanking You!

Take one or few matching pair of features and apply 4 transformations to check which is transformation is more suitable. — nayab
Thank You @nayab, but will be helpful if you can suggest me a small of code to check it. I feel like, all the results are projecting to the same point. — Althaf

Ankit Ankit · Accepted Answer · 2017-12-18T09:42:05

I used the function 'filterHomographyDecompByVisibleRefpoints' by Vaesper. You can check the code here.

You just need to input the Rotation matrix, normals obtained from decomposeHomographyMat and the point correspondences used to obtain the homograpy matrix. The above function will return 2 possible solutions. You can see more on this idea in the answer by Ebya here.

After getting the 2 possible solutions, you will have to somehow make some checks depending on your case in order to find the right solution. Since I used identity matrix for camera matrix, the translation values obtained are in pixels and will need to be scaled using some external sensor. In my case, the distance between the camera and planar object is fixed in the z-axis, therefore, I just calculated what 1 pixel represents in world units.

Finding most suitable Rotation and Translation from Homography decomposition

2 Answers