bad distance results using stereo camera

Question

I'm trying to measure distance in real time from stereo pair to a person detected in the scene. First i calibrated both cameras separately with a 9x6 checkerboard (square size of 59 mm) and i obtained a rms error between 0.15 and 0.19 for both cameras. Using the obtained parameters i calibrated the stereo pair and the rms error was 0.36. Later, I rectified, undistort and remap the stereo pair giving me this result: rectified and undistorted stereo

Done that, I computed stereo correspondence using stereoSGBM. That's how i did:

Mat imgDisp= Mat(frame1.cols, frame1.rows,CV_16S);

cvtColor(frame1, frame1, CV_BGR2GRAY);
cvtColor(frame2, frame2, CV_BGR2GRAY);

//parameters for stereoSGBM
  stereo.SADWindowSize = 3;
  stereo.numberOfDisparities = 144;
  stereo.preFilterCap = 63;
  stereo.minDisparity = -39;
  stereo.uniquenessRatio = 10;
  stereo.speckleWindowSize = 100;
  stereo.speckleRange = 32;
  stereo.disp12MaxDiff = 1;
  stereo.fullDP = false;
  stereo.P1 = 216;
  stereo.P2 = 864;

double minVal; double maxVal;
minMaxLoc(imgDisp, &minVal, &maxVal);

return imgDisp;

I attached the result from stereoSGBM here: disparity map.

For detect a person in the scene I used hog + svm (the default people dectector) and tracked that person with optical flow (cvCalcOpticalFlowPyrLK()). Using the disparity map obtained in the stereo correspondence process i obtained the disparity for each corner tracked from one person as follow:

int x= cornersA[k].x;
int y= cornersA[k].y;

short pixVal= mapaDisp.at<short>(y,x); 
float dispFeatures= pixVal/ 16.0f;

with the disparity for each corner tracked for one person in the scene I computed the maxim disparity and computed the depth in that pixel using the formula ((focal*baseline)/disp):

float Disp =maxDisp_v[p];

cout<< "max disp"<< Disp<<endl;
float d = ((double)(879.85* 64.32)/(double)(Disp))/10; //distance in cms.

** for focal length I calculated the average between fx and fy obtained in the cameras matrix [3x3] parameters:

CM1: [9.0472706037497187e+02 0. 3.7829164759284492e+02
0. 8.4576999835299739e+02 1.8649783393160138e+02 0. 0. 1.]

CM2: [9.1390904648169953e+02 0. 3.5700689147467887e+02 0. 8.5514555697053311e+02 2.1723345133656409e+02 0. 0. 1.]

so fx camera1: 904.7; fy camera1: 845.7; fx camera2: 913.9; fy camera2: 855.1

** The result of T[0,0] matrix matched with the baseline that I measure manuallly so I assumed that's correct baseline.

**due to the square size of checkerboard is in mm i assumed that baseline must be in the same unit, that's why I'm put 64.32 mm in baseline.

The result of distance is aprox. 55 cms but the real distance is 300 cms. I have checked many times but the measured distance is still incorrect: distanceResult

Help me please!, I have no idea what i'm doing wrong.

*** I'm using opencv 2.4.9 in osx system.

hagor hagor · Accepted Answer · 2018-05-29T11:03:35

I think you are making a mistake with units: focal length is provided in pixels, baseline is provided in cm disparity is provided in pixels. Right? According to formula you have pix*cm/pix = cm. But you devide it by 10 and get dm. So you have the distance around 55dm which is twice bigger then 300. Which is not a bad case for you approach.

bad distance results using stereo camera

2 Answers