OpenCV calculate disparity in image coordinate

Question

I am trying to calculate disparity (difference of pixels) in left and right images. Assume the images are rectified (row aligned) and the principle point is known and indicated as cx, cy (note principle point is different for left and right cameras).

Say we have a pixel in left image x_l has corresponding pixel in right image x_r. If we use a image coordinate whose origin is cx and cy, then the disparity of two pixels is simply as: disp = x_l - x_r

However, if a image coordinate that has origin at top left of the image (i.e. defined in OpenCV), do I need to consider this offset (cx) when calculating the disparity? Or can I still use disp = x_l - x_r to get the correct disparity?

It may seems a bit stupid question but I do feel confused. Thanks.

Calculating a disparity for a single pixel in both images will not give reliable and accurate results. One pixel error in your images may result in half a meter error in depth perception. So I'd strongly suggest to use BM/SGBM algorithms from opencv as they are based on sophisticated algorithms. if you know the coordinates of the object in the scene you could provide them to the StereoBM class as ROI (region of interest) to decrease the computation costs. But calculating disparity as a simple triangulation will not give you accuracy. — A_P

BConic BConic · Accepted Answer · 2014-09-12T19:04:26

It really depends on the way you use disparity to calculate depth.

The projection matrices of two rectified images (with coordinate origin at the top left of the image) are of the following form:

      [ fx   0  cx_l  0 ]
P_l = [  0  fy    cy  0 ]
      [  0   0     1  0 ]

      [ fx   0  cx_r  -B ]
P_r = [  0  fy    cy   0 ]
      [  0   0     1   0 ]

where B is the baseline and fx the common horizontal focal length. If you project in both images the same 3D point M = [X; Y; Z; 1] you obtain the following relation:

u_l-u_r = fx*B/Z + cx_l-cx_r

where P_l*M = Z*[u_l; v; 1] and P_r*M = Z*[u_r; v; 1]. This relation can be refactored as

Z = fx * B / (u_l-u_r-cx_l+cx_r)
  = fx * B / (disp_1-cx_l+cx_r)
  = fx * B / disp_2

with

disp_1 = u_l-u_r
disp_2 = u_l-u_r-cx_l+cx_r

Hence, you can choose to compute disparity as you prefer, as long as you take the two different principle points into account when you use disparity to calculate depth.

OpenCV calculate disparity in image coordinate

1 Answers