Manually Feed Point Cloud Data into Kinect Fusion Toolkit

Question

I'm trying to manually feed in data to Kinect Fusion's AlignPointCloud functionality. In the Kinect SDK Toolkit 1.8, there's an example of how to use Kinect Fusion. I'm using the toolkit provided in the samples to try to use Fusion to try to align two point clouds; however, I can't seem to get the AlignPointCloud method to ever converge successfully.

I'm sure that there is something I'm misunderstanding about how to copy data into the FusionPointCloudImageFrame.

What I'm currently trying to do (A trivial case, simply matching together two planes) fails:

float[] arr1 = new float[80 * 60 * 6];
float[] arr2 = new float[80 * 60 * 6];
for (int y = 0; y < 60; y++) {
  for (int x = 0; x < 80; x++) {
    int ind = y * 80 + x;

    arr1[ind] = x / .1f; // X coordinate
    arr1[ind + 1] = y / .1f; // Y coordinate
    arr1[ind + 2] = 1; // Z coordinate
    // Normals
    arr1[ind + 3] = 0;
    arr1[ind + 4] = 0;
    arr1[ind + 5] = 1;

    arr2[ind] = x / .1f; // X coordinate
    arr2[ind + 1] = y / .1f; // Y coordinate
    arr2[ind + 2] = 2; // Z coordinate
    // Normals
    arr1[ind + 3] = 0;
    arr1[ind + 4] = 0;
    arr1[ind + 5] = 1;
  }
}
FusionPointCloudImageFrame pcl1 = new FusionPointCloudImageFrame(80, 60);
FusionPointCloudImageFrame pcl2 = new FusionPointCloudImageFrame(80, 60);

pcl1.CopyPixelDataFrom(arr1);
pcl2.CopyPixelDataFrom(arr2);

Matrix4 m = Matrix4.Identity;
bool success = FusionDepthProcessor.AlignPointClouds(pcl1, pcl2, 7, null, ref m);
// Does not converge, m is identity regardless of what it was before

What am I doing incorrectly, or what do I need to change to manually feed in data to match two point clouds? Also, can someone please explain to me the significance of a point cloud having a width and height? Each point has an x,y, and z value and doesn't need to be in any ordered way afaik, so why do we need to provide a width or height? If I'm reading the data from a .obj (wavefront) file, how can I determine the width and height?

Thanks!

Toby Sharp Toby Sharp · Accepted Answer · 2013-11-18T10:33:12

At a quick glance, this example probably fails to converge because there is no texture to "fix" these planes: they can just slide around and match equally well. Try using some less trivial test data.

The "point cloud" is essentially just a depth image, so it can have a width and height as with a frame that originated from the Kinect camera.

If you are rendering from a mesh, you can choose an appropriate width and height, e.g. 640x480.

Manually Feed Point Cloud Data into Kinect Fusion Toolkit

2 Answers