Organized point cloud from stereo

Question

I am working with disparity maps (1024 x 768) obtained via stereo and I am able to get point clouds with XYZRGB pcl::Points. However not all pixels from the disparity map are valid depth hence there will never be 1024x768 = 786432 XYZRGB points. Fortunately I am able to save the point clouds unorganized (i.e. height=1). Unfortunately, some normal estimation methods etc, are tailored for organized pointclouds. How can I create organised pointclouds from this ?

Answer 1

I believe that this is not possible.

First of all unorganized point cloud (PC) is just list of points in random order written in file On the other hand organized PC carries information of in which order orginal points were obtained by depth camera and some other information. This information is stored in lets call it grid.

Once you destroy this grid omiting some points theres no algorithm that can put it back together as it originally was

You can use other methods which provides PCL that doesnt take OPC as an argument. Result will be same as if you would use organized point cloud only little bit slower (depends on size of your input cloud)

Answer 2

I assume that you do have the calibration parameters that are necessary to transform the image points and their depth into 3D points, right? In this case, you simply create a 2D point cloud and do the following for each pixel of the disparity map:

If the point is valid:
    set the corresponding point in the point cloud to the 3D point
else:
    set the corresponding point in the cloud to NaN (i.e. a 3D point with NaN as coordinates)