Calculating 3D world point

1
votes

I am beginner in OpenCV and I want to find 3D point from 2D (projected on image) and known two out of three world coordinates of a point.

What I have:

  1. calibrated camera (known: matrix of intrinsic parameters, vector of distortion coefficients)
  2. rotation vector and translation vector
  3. (u, v) - coordinates of a point on image
  4. two known world coordinates of a point and one unknown

Problem variant #1 Known: X, Y Unknown: Z

Problem variant #2 Known: X, Z Unknown: Y

Problem variant #3 Known: Y, Z Unknown: X

How can I find third unknown coordinate? Is it possible?

1
pythonopencv3dcomputer-vision
I assume that rotation vector and translation vector are the camera pose, the homogeneous transformation that allows to transform a point in the world coordinate/frame to the camera coordinate/frame. X, Y and Z are expressed in the world coordinate/frame and not in the camera coordinate/frame. Is it correct? – Catree
If so, I think #2 and #3 should be possible: transform Z_w to Z_c using rvec and tvec, transform u and v to the normalized camera frame (x, y) using the intrinsic parameters and then multiply x or y by Z. You should get X_c or Y_c and to get X_w or Y_w, you would have to multiply by the inverse homogeneous transformation using rvec and tvec. I don't know for #1. – Catree
from distortion, intrinsics and extrinsics you can compute a 3D ray from the camera center through the pixel. If you follow that ray you will eventually find the point that has your 2 desired coordinate values and then you know the 3rd one, too. – Micka

1 Answers

0
votes

Yes it is possible. Consider the simple case of absent nonlinear distortion. Let Ki be the inverse of the camera matrix, and camera center at the world origin (i.e. no rotation nor translation). Let p=(u, v, 1) be the homogeneous pixel coordinate. Then the ray through the pixel is:

s * P = Ki * p

where s > 0 is an unknown scale. But s * P = [X, Y, Z], so if you know any one of X, Y or Z you can solve for s and find the missing coordinates.

For non-zero roto-translation, replace Ki with the inverse of the projection matrix. For non-zero distortion, replace the simple multiplication by Ki with the complete reprojection equation.