How to get the accuracy of the calibration in millimeters not in pixels?

Question

I'd like to know the accuracy of the camera's calibration status in millimeter units. In other words, what does the X pixels error (the Root Mean Square error) in the 2D image correspond to the distance in millimeters in the 3D object coordinate system?

In OpenCV, I can get the accuracy of the calibration based on the Root Mean Square error (pixel units) as a result of calibrateCamera function. Or, I can manually calculate by reprojecting the object points to the image with projectPoints and by comparing them with the current image points.

For example, let's say I got the RMS error around 2.0. Does this mean 5mm or 10mm difference? How to calculate the conversion from pixels in 2d to millimeters in 3d? Note that the ArUco marker is always placed horizontally and I know the size of the marker.

Answer 1

To calculate your projection 2d error you have to use your camera sensor phisical dimensions to determine pixel width and height.

There is no simple way to determine 3d mm error. Are you using single or multi camera setup? If you are using single camera you cant determine real 3d coordinates based on their projection without depth map.

In multicamera system there is some workaround. You can measure distances between corners detected on both cameras (in 3d) and compare it to marker dimension or try to fit plane to whole board.