Getting real depth from disparity map

Question

I want to get real distance of an object from stereo camera. I am using OpenCV example code as given in Learning OpenCV O'Reilly book. After getting disparity map I want to use the formula:

distance = focal_length * baseline distance / disparity

The problem is :

1. I am getting negative values of disparities. How to convert these values so that they might be used in actual depth calculation ?

2. In above formula focal length and baseline distance are in mm (returned by reprojection matrix) whereas disparity will be in pixels. So the result will be in mm^2/pixel. How to convert disparity value from pixel to mm.

Answer 1

You need to perform a camera calibration step first, to get a proper parameters, know as camera matrix.

One you have these values, you can perform a proper computation of the disparity map with the corrected values (the one got from the camera calibration step, know as remapping or undistortion) and then, to obtain the real depth (in mm or meters), you can finally do:

depth = baseline * focal / disparity

Answer 2

you can use CVs stereo correspondence functions, such as Stereo Block Matching or Semi Global Block Matching. This will give you a disparity map for the entire image which can be transformed to 3D points using the Q matrix (cv::reprojectImageTo3D).

Answer 3

There are two problems here:

1. The units. Assuming that your cameras are calibrated you have the focal length in pixels, so the unit of disparity is in mm or any other metric unit. See my answer to the same question here.

2. Disparity cannot be negative if the 3D point is in front of both cameras that are parallel to each other. If the axes are converged or the cameras are uncalibrated it may happen. See this question for details.