How to form a projection matrix in OpenCV and Matlab calibration tool box?

Question

The projection matrix in camera calibration is a 3x4 matrix

P = camera matrix * [R|t]

However in the OpenCV Documentation, it didn't mention if the R and t are formed by the rvec and tvec, the calibration target object space to the camera space transformation, or they are formed by the camera world space orientation rodrigues(rvec).t() and - rodrigues(rvec).t() * tvec'. The wikipedia gives a more detailed explaination.

T is the position of the origin of the world coordinate system expressed in coordinates of the camera-centered coordinate system. T is often mistakenly considered the position of the camera.

But I still kind of confused, so what exactly the extrinsic parameters ? Is it the T above, or the extrinsic parameters should be the camera position and orientation in the world space, and the R and t in the projection matrix formula should not be the extrinsic parameters ?

I'd like to confirm what exactly the R and t in the projection matrix are ?

Answer 1

Basically the extrinsic parameters are used to transform homogeneous world coordinates into camera coordinates. Afterwards the intrinsic parameters are used to map points in camera coordinates onto the image plane, which leads to pixel coordinates.

Wikipedia quotes R and T and they are equivalent to the R and t of your projection matrix P. R basically represents the orientation of the camera coordinate system and t is the origin of the world coordinate system in camera coordinates.

However, if you want to make the camera centre C (in world coordinates) explicit you have to rewrite your projection matrix:

P = K[R|-RC].

As you can see t=-RC, but actually there is no difference. Both combinations are extrinsic parameters.

For more details, Multiple View Geometry in Computer Vision is kind of the bible regarding this field.

I hope I answered your question. Do no hesitate to ask if something was not clear or if I misunderstood you.