VTK - calculate point cloud for a renderered scene

Question

I render a triangulated mesh (polydata) using VTK renderer. I want to get a 3D point cloud of the rendered scene (with respect to the camera).

I found a similar snippet where do the similar thing in the PCL library which uses VTK as its visualisation tool.

I implement the PCL function in my code and I managed to get same results (3D point cloud). However, I found an issue that I don't know where to start to solve:

• For a point that should have valid depth value in the z-buffer, 1 (I believe it is a default set value means NOT a valid point) is returned.

I tried to play with the clippingplane (also known as nearz/farz), but nothing improved.

UPDATE: Issue solved, it was because I forget VTK defined image coordinate differently than the traditional one. In VTK, origin is bottom-left instead of top-left.

Answer 1

Renderer will transform display <--> world coordinates. Look at methods of class vtkViewport: SetDisplayPoint(), DisplayToWorld(), GetWorldPoint(). This API uses homogeneous coordinates. Look at the source of any vtk widget -- during interaction the display coordinates are transformed to world coordinates.

But if you are trying to get coordinates of a point lying on a surface, you really want a vtkPointPicker or vtkCellPicker. Look at examples and tests of these classes.

Answer 2

I have been working on that same problem. I am using vtk's python bindings but I think you will be interested in looking at the method RequestData() in this question that I recently posted. My question is about pipeline execution and not about projecting points.

mirni's answer definitely works, but if you are doing that for every pixel you will do an extra two matrix creations, a matrix multiply, and a matrix multiply for each pixel.

Doing it the way I did in my method saved about 16 seconds on a 300x300 window.