Transformation between point clouds

1
votes

I hope to find some hints where to start with a problem I am dealing with. I am using a Kinect sensor to capture 3d point clouds. I created a 3d object detector which is already working.

Here my task:

Lets say I have a point cloud 1. I detected a object in cloud A and I know the centroid position of my object (x1,y1,z1). Now I move my sensor around a path and create new clouds (e.g. cloud 2). In that cloud 2 I see the same object but e.g. from the side, where the object detection is not working fine.

I would like to transform the detected object form cloud 1 to cloud 2, to get the centroid also in cloud 2. For me it sound like I need a matrix (Translation, Rotation) to transform point from 1 to 2.

And ideas how I could solve my problem? Maybe ICP? Are there better solutions?

THX!

2
kinecttransformationpoint-cloud-librarypoint-clouds
To clarify: you do not know the path the sensor has taken, or have any estimate of it, am I right? - D.J.Duff
I am not sure. I move the kinect in the room by hand - user2576458
So I understand the detection works in frame A but not B. Can you track the object while the sensor is moving? That may solve it. Otherwise, try matching shape features from the original found object to the new image (does the object have many features?) - change in viewpoint needs to be small enough though. This would be helped if you had an estimate of the transformation. ICP might work if you have a good estimate of the transformation and the viewpoint has not changed too much. After that it gets more interesting ;) - D.J.Duff

2 Answers

1
votes

In general, this task is called registration. It relies on having a good estimation of which points in cloud 1 correspond to which clouds in point 2 (more specifically, which given a point in cloud 1, which point in cloud 2 represents the same location on the detected object). There's a good overview in the PCL library documentation

If you have such a correspondence, you're in luck and you can directly compute a rotation and translation as demonstrated here.

If not, you'll need to estimate that correspondence. ICP does that for approximately aligned point clouds, but if your point clouds are not already fairly well aligned, you may want to start by estimating "key points" (such as book corners, distinct colors, etc) in your point clouds, computing a rotation and translation as above, and then performing ICP. As D.J.Duff mentioned, ICP works better in practice on point clouds that are already approximately aligned because it estimates correspondences using one of two metrics, minimal point-to-point distance or minimal point to plane distance, according to wikipedia, the latter works better in practice, but it does involve estimating normals, which can be tricky. If the correspondences are far off, the transforms likely will be as well.

0
votes

I think what you were asking about was in particular to the Kinect Sensor and the API Microsoft released for it.

If you are not planning to do reconstruction, you can look into the AlignPointClouds function in Sensor Fusion namespace. This should take care of it automatically, in methods similar to the answer given by @pnhgiol.

On the other hand, if you are looking at doing reconstruction as well as point cloud transforms, the Reconstruction class is what you are looking for. All of which can be found out about, here.