Why is the opengl projection of a model not giving the exact same results as the opencv projection?

Question

I'm working on a project where I use cv::solvePnP to get the camera pose and use this pose to project a 3D model in an OpenGL window.

I used this link to go from opencv intrinsic and extrinsic matrices to opengl mvp matrices.

The projection seems to be fine just when I zoom on the object, it is slightly bigger with opengl as shown in the image : on the left is the object seen with the camera and on the right is what opengl projection gives me.

I thought I would have the exact same projection. Is it because of the distortion field or am I using the wrong projection matrix in opengl?

Answer 1

The source image has a barrel lens distortion. This is a non-linear distortion of the image that cannot be described by a projection matrix. OpenGL, on the other hand, uses a rectilinear projection: one that projects straight lines onto straight lines. So it cannot account for the lens distortion in the source image.

To solve this properly you need to render the image using a rectilinear projection and then distort it using the lens distortion coefficients, which you should be able to retrieve from OpenCV intrinsic camera parameters. This would require rendering to a texture and then sampling from that texture at the correct coordinates. See Distortion on Wikipedia for details.