stereo reconstruction of point cloud based on rectified images

Question

I have a pair of matched 2D features extracted from rectified stereo image. Using cvPerspectiveTransform function in OpenCV, I attempted to reconstruct those features in 3D. The result is not consistent with the actual object dimension in real world. I realize there is a function in Matlab calibration toolbox that converts 2D stereo features into 3D point cloud. Nevertheless, the features are lifted from original images.

If I want to work with rectified images, is it possible to reconstruct the 3D locations based on 2D feature locations and disparity information.

Answer 1

If you know the focal length (f) and the baseline width (b, the distance of the projection axis of both cameras) as well as the disparity (d) in a rectified stereo image pair, you can calculate the distance (Z) with the following formula:

Z = f*(b/d);

This follows from the following equations:

x_l = f*(X/Z);  // projecting a 3D point onto the left image
x_r = f*((X+b)/Z);  // projecting the same 3D point onto the right image
d = x_r - x_l = f * (b/Z); // calculating the disparity

Solving the last equation for Z should lead to the formula given above.