Need help to determine quality of stereo camera calibration and setting up calibration matrix

Question

I am playing around with two cheap webcams(0.3Mpx), trying to make a stereo camera setup that can measure distances and dimensions of objects. But I am struggling to find out If my stereo camera calibration Is properly done and how to move on to set up the camera calibration matrix?

What I have done so far Is to make a camera rig, where the camera's are set 30 cm apart tilted slightly Inwards so the images more or less overlap each other. I put the calibration checkerboard(A4 size) from Matlab approximately 50 cm away from the cameras. Then i proceeded to take 20 image pairs in different angles and positions via a Matlab script that I made, and feed these to the stereo camera calibration app In Matlab. This is the result i get:

Extrinsic parameter visualisation - This Is the Image that makes me feel like there Is something wrong with my calibration, as It looks like the cameras are not apart:

Pixel mean error reprojections - Here I have removed some of the Image pairs with higher reprojecion errors:

Standard Errors of Estimated Stereo Camera Parameters ----------------------

Camera 1 Intrinsics ------------------- 
Focal length (pixels): [ 829.8567 +/- 4.6262 831.1900 +/- 4.4714 ] 
Principal point (pixels):[ 437.7113 +/- 1.8180 254.0188 +/- 1.9227 ] 
Skew: [ 1.7063 +/- 0.3069 ] 
Radial distortion: [ 0.0024 +/- 0.0067 -0.1042 +/- 0.0281 ] 
Tangential distortion: [ 0.0036 +/- 0.0006 0.0100 +/- 0.0007 ]

Camera 1 Extrinsics ------------------- 
Rotation vectors: 
[ -0.0289 +/- 0.0020 0.1813 +/- 0.0025 -0.0132 +/- 0.0005 ] [ -0.2268 +/- 0.0030 -0.3439 +/- 0.0026 -0.1029 +/- 0.0004 ] [ -0.0209 +/- 0.0024 -0.0018 +/- 0.0024 -0.0438 +/- 0.0003 ] [ 0.0962 +/- 0.0032 -0.2916 +/- 0.0034 -1.8484 +/- 0.0004 ] [ -0.0042 +/- 0.0021 0.1804 +/- 0.0027 -0.0219 +/- 0.0006 ] [ -0.2726 +/- 0.0039 0.1822 +/- 0.0041 2.8569 +/- 0.0007 ] [ 0.0499 +/- 0.0020 0.1646 +/- 0.0024 -0.0207 +/- 0.0004 ] [ -0.2745 +/- 0.0024 0.0080 +/- 0.0022 -1.5557 +/- 0.0004 ] [ -0.2607 +/- 0.0024 0.0325 +/- 0.0022 -1.5807 +/- 0.0004 ] [ -0.2684 +/- 0.0028 0.0384 +/- 0.0025 -1.5800 +/- 0.0006 ]

Translation vectors (mm): 
[ -150.5696 +/- 1.0617 -46.2962 +/- 1.1548 491.0520 +/- 2.7303 ] [ -214.8622 +/- 1.1726 -40.0423 +/- 1.2625 528.0044 +/- 2.9924 ] [ -164.9147 +/- 1.0168 -69.9880 +/- 1.1023 465.4540 +/- 2.6645 ] [ -37.2595 +/- 1.4144 59.4185 +/- 1.4939 643.6316 +/- 3.5056 ] [ -203.2476 +/- 1.0742 -42.6796 +/- 1.1813 493.9661 +/- 2.7924 ] [ 109.4280 +/- 1.4785 -69.4211 +/- 1.5662 678.8234 +/- 3.6783 ] [ -130.7829 +/- 1.0151 -42.2859 +/- 1.1012 468.7133 +/- 2.6260 ] [ -73.5988 +/- 1.0368 64.6519 +/- 1.1120 473.5281 +/- 2.6423 ] [ -43.1916 +/- 1.0432 73.5025 +/- 1.1112 474.7453 +/- 2.6190 ] [ -204.2203 +/- 1.1513 64.6115 +/- 1.2599 524.6072 +/- 3.0409 ]

Camera 2 Intrinsics ------------------- 
Focal length (pixels): [ 829.8567 +/- 4.6262 831.1901 +/- 4.4714 ] 
Principal point (pixels):[ 437.7113 +/- 1.8180 254.0188 +/- 1.9227 ] 
Skew: [ 1.7063 +/- 0.3069 ] 
Radial distortion: [ 0.0024 +/- 0.0067 -0.1042 +/- 0.0281 ] 
Tangential distortion: [ 0.0036 +/- 0.0006 0.0100 +/- 0.0007 ]

Position And Orientation of Camera 2 Relative to Camera 1 ----------------------Rotation of camera 2: [ 0.0000 +/- NaN 0.0000 +/- NaN 0.0000 +/- NaN ] Translation of camera 2 (mm): [ -0.0000 +/- 0.0696 -0.0000 +/- 0.0759 0.0000 +/- 0.4605 ]

If anyone are able to help me get a little bit further with this project, It would be greatly appreciated :-) have butted my head against the wall for a while now

It's strange that checker board planes look all in parallel. Did you tilt it with various angles while capturing the 20 images? — hiroki
I did, but tried to keep It to less than 45 degrees angles as specified by the matlab tutorial. But also the Images with tilt usually get very high reprojection errors, and following matlab guidelines I have removed them from the calibration set. Have tried many times now, even with more angled pictures without any success. Is It possible the cheap optics or the low resolution(VGA - 640x480) of the cameras makes it very hard to calibrate? — Balder
Do you think the Extrinsics visualization result is valid? When I look at the Rotation vectors values, they have some variance in x and y component but in the visualization, they only have rotation along z axis. I think mean reprojection errors less than 1 pixel is fine at this stage. Calibration algorithm needs planes to be rotated with at least 3 (or more) different angles. — hiroki
I am still In the learning phase, so I have no Idea If the results are valid or not. That Is what I am trying to learn, hopefully someone with experience could try to show me how to understand the results — Balder

Paolo Ferraiuoli Paolo Ferraiuoli · Accepted Answer · 2017-01-26T13:51:34

In order to check the accuracy of cameras calibration you can reconstruct a known distance between two points of an object placed in different positions within the field of view of the stereo-cameras and see if the result is right.

If you have used the Matlab Stereo Camera Calibrator application and saved the results as mat file, you can use the following script to compute the Euclidean distance between two points and compare with the known geometry.

%Pick two points in the Left Image
cam1 = uigetfile('*', 'Select Left Image');
imshow(cam1)
[x y] = ginput(2);% 2 is the number of points
x_left_image = vertcat(x',y');
close all

%Pick the same two points in the Right Image
cam2 = uigetfile('*', 'Select Right Image');
imshow(cam2)
[x y] = ginput(2);% 2 is the number of points
x_right_image = vertcat(x',y');
close all


load calibrationSession.mat %load the stereo calibration data .mat
[Xc_1_left, error] = triangulate(x_left_image',x_right_image',calibrationSession.CameraParameters);

Xc_1_left = Xc_1_left';

X = cell(size(Xc_1_left,2),1);
Y = cell(size(Xc_1_left,2),1);
Z = cell(size(Xc_1_left,2),1);

for n = 1:size(Xc_1_left,2);

    X{n,1} = Xc_1_left(1,n);
    Y{n,1} = Xc_1_left(2,n);
    Z{n,1} = Xc_1_left(3,n);

end

distance = cell(size(Xc_1_left,2)-1,1);

for i = 1:size(Xc_1_left,2)-1;

    distance{i,1} = sqrt((X{i+1} - X{i})^2 + (Y{i+1} - Y{i})^2 + (Z{i+1} -Z{i})^2)

end

P.S. Remember to put both the calibration data (.mat) and the the two images in the Current Folder.

Need help to determine quality of stereo camera calibration and setting up calibration matrix

1 Answers