OpenCV project 3D coordinates to 2d camera coordinates

6
votes

I have the 3D-world coordinates of an object and I want to get its coordinates in the camera-2D-plane. I have already calibrated the camera using cv::calibrateCamera, so that I have the camera matrix and distortionCoeffs.

For projecting the 3D-point to 2d-camera-coordinates, I use cv::projectPoints. Documentation says:

void projectPoints(InputArray objectPoints, InputArray rvec, InputArray tvec, InputArray cameraMatrix, InputArray distCoeffs, OutputArray imagePoints, OutputArray jacobian=noArray(), double aspectRatio=0 )

How do I get rvec/ tvec and is my approach right?calibrateCameragives me rvecs and tvecs, but they are for each input chessboard-image and I guess, they are the rotation and translation of the chessboard and not of the camera.

1
c++opencv3dcamera
Well, they describe the rotation and translation of the camera relative to the chess board. If you know the rotation and translation of the chessboard relative to your world coordinate system you can easily compute the extrinsics of your camera in the world coordinate system. – sietschie
But why do I need this translation/ rotation matrices? Should it work with only the camera (=projection?) matrix? – black
The camera matrix only contains the intrinsic parameters like focal length and not the extrinsic parameters which are the rotation and translation of the camera. – sietschie
Ok, I understand this. In the projection matrix p, there are both (intrinsic and extrinsic) parameters, aren't there? So theoretically, I would just have to multiply p with the homogeneous 3D-coordinates in order to get the 2D-homogeneous coordinates of my point, right? Is there a OpenCV function for this? – black
If you have such a matrix you don't need a function to do this. Just multiply your point with the matrix and divide the result by its third entry. Of course this does not take distortion into account. But for this to work you need the a transformation from your world coordinate system to your camera coordinate system. And you stated so for you only have transformations to the checker boards. – sietschie

1 Answers

2
votes

As I have the projection matrix (P), I can calculate the 2D coordinates (x) in the camera plane like this from the 3D-coordinates X (using homogeneous coordinates):

x (u,v,w) = P * X (x1,x2,x3,w)

I just have a problem with the rectification, so that I do not get the exact projection to my 2d image. See here: OpenCV stereo vision 3D coordinates to 2D camera-plane projection different than triangulating 2D points to 3D