How to visualize kinect point cloud in real time with open3d

Question

What i want to do is visualize the data streamed from my xbox360 kinect in real time.

Essentially I am creating a point cloud from the data and then visualizing it. The code I have is working but it is really slow.

Basically this code adds a point cloud once it is received from the kinect. Whenever another point cloud is recieved, the previous one is removed and a new one is added.

Is there a better way to do this? Something that is much more responsive with higher frame rates

mat = rendering.Material()  
        mat.base_color = [
            0,
            0,
            0, 1.0
        ]
        mat.shader = "defaultLit"
        pcl = o3d.geometry.PointCloud()
# This line recieves the data from the kinect in the format [x,y,z,r,g,b]
        pcl.points = o3d.utility.Vector3dVector(kinect.streamCloud())
        self.scene.scene.remove_geometry("kinect")
        self.scene.scene.add_geometry("kinect", pcl, mat)

This is the code for streaming data from the kinect

def streamCloud():
    depth = freenect.sync_get_depth()
    pcl = np.zeros(shape=(307200,3))
    c = 0
    for i in range(480):
        for j in range(640):
            z = depth[0][i,j]
            #z = 1.0 / (d[i,j] * -0.0030711016 + 3.3309495161)
            #z = depth[0][i,j].astype(np.uint8)
            #x = (i - cx) * z / fx
            x = j
            y = i
            #y = (j - cy) * z / fy
            pcl[c] = [x,y,z]
            c = c+1
    return pcl

Rubens Benevides Rubens Benevides · Accepted Answer · 2021-02-23T03:37:56

Kinect generates about 300.000 points in every frame, too much data to draw. At 30 FPS this is 9.000.000 points in one second. The first thing you can do is downsample the cloud, you can use cloud.uniform_down_sample(every_k_points) to take points at every k points of the cloud. Or you can modify your read function, just change the loop of i and j to take pixels at every 10:

for i in range(0,480,10):

for j in range(0,640,10):

How to visualize kinect point cloud in real time with open3d

1 Answers