Converting Kinect depth image to Real world coordinate

Question

I'm working with the kinect, using OpenNI 2.x, c++, OpenCV.

I am able to get the kinect depth streaming and obtain a grey-scale cv::Mat. just to show how it is defined:

cv::Mat m_depthImage;
m_depthImage= cvCreateImage(cvSize(640, 480), 8, 1);

I suppose that the closest value is represented by "0" and the farthest by "255".

After that, I make a conversion between depth coordinates to World coordinates. I do it element by element of the cv::Mat grey-scale matrix, and i collect data in PointsWorld[640*480]. In order to display these data, I adjust the scale in order to collect value in a 2000x2000x2000 matrix.

cv::Point3f depthPoint;
cv::Point3f PointsWorld[640*480];
for (int j=0;j<m_depthImage.rows;j++)
    {
        for(int i=0;i<m_depthImage.cols; i++)
        {
            depthPoint.x = (float) i;
            depthPoint.y = (float) j;
            depthPoint.z = (float) m_depthImage.at<unsigned char>(j, i);

            if (depthPoint.z!=255)
            {
                  openni::CoordinateConverter::convertDepthToWorld(*m_depth,depthPoint.x,depthPoint.y,depthPoint.z, &wx,&wy,&wz);
                wx = wx*7,2464; //138->1000
                if (wx<-999)    wx = -999;
                if (wx>999)     wx = 999;

                wy = wy*7,2464; //111->1000 with 9,009
                if (wy<-999)    wy = -999;
                if (wy>999)     wy = 999;

                wz=wz*7,8431;   //255->2000
                if (wz>1999)        wy = 1999;

                Xsp = P-floor(wx);
                Ysp = P+floor(wy);
                Zsp = 2*P-floor(wz);

                PointsWorld[k].x = Xsp;
                PointsWorld[k].y = Ysp;
                PointsWorld[k].z = Zsp;

                k++;
            }
        }
    }

But i'm sure that doing that do not allow me to have a comprehension of the real distance between points. An x,y,z coordinate what will mean? There is a way to know the real distance between points, to know how much far is, for example, the grey value of the matrix "255"? and the wx,wy,wz what they are for?

Why are you representing the depth from 0 to 255? are you doing this to display the depth? or you need it to obtain distances between point? if it is the second one you should use any double/float value or even better use other libraries like PCL. If you did the scale to be able to save the image my recommendation is that you should use other way to save them, like yml/xml (opencv filestorage) or .pcd files (PCL) — api55

George Profenza George Profenza · Accepted Answer · 2014-04-29T14:20:45

If you have OpenCV built with OpenNI support you should be able to do something like:

int ptcnt;
cv::Mat real;
cv::Point3f PointsWorld[640*480];
if( capture.retrieve(real, CV_CAP_OPENNI_POINT_CLOUD_MAP)){
    for (int j=0;j<m_depthImage.rows;j++)
    {
        for(int i=0;i<m_depthImage.cols; i++){
            PointsWorld[ptcnt] = real.at<cv::Vec3f>(i,j);
            ptcnt++;
        }
    }
}

Converting Kinect depth image to Real world coordinate

1 Answers