VTK efficiency of vtkRenderWindow::GetZbufferData and vtkWindowToImageFilter::Update

Question

I have a snippet that converts vtk (off screen) rendering to 1)Point cloud; 2)Color image. The implementation is correct, it just the speed/efficiency is an issue.

At the beginning of every iteration, I update my rendering by calling:

renderWin->Render ();

For point cloud, I get the depth using following line and then convert it to point cloud (code not posted).

float *depth = new float[width * height];
renderWin->GetZbufferData (0, 0, width - 1, height - 1, &(depth[0]));

For color image, I use vtkWindowToImageFilter to get current color rendered image:

windowToImageFilter->Modified();    // Must have this to get updated rendered image
windowToImageFilter->Update();  // this line takes a lot of time
render_img_vtk = windowToImageFilter->GetOutput();

Above program is run in the same thread sequentially. The renderWindow size is about 1000x1000. There is not a lot of polydata needs to be rendered. VTK was compiled with OpenGL2 support.

Issue: This code only runs about 15-20Hz, when I disabled/comment the windowToImageFilter part (vtkWindowToImageFilter::Update() takes a lot of time), the framerate goes to about 30Hz. When I disabled/comment vtkRenderWindow::GetZbufferData, it goes up to 50Hz (which is how fast I call my loop and update the rendering).

I had a quick look of the VTK source file of these two function, I see it copy data using GL command. I am not sure how can I speed this up.

Update: After some search, I found that the glReadPixels function called in the GetZbufferData causes delay as it try to synchronize the data. Please see this post: OpenGL read pixels faster than glReadPixels. In this post, it is suggested that PBO should be used. VTK has a class vtkPixelBufferObject but no example can be found for using it to avoid blocking the pipeline when do glReadPixels()

So how can I do this within the VTK pipeline?

Lucas Chavez Lucas Chavez · Accepted Answer · 2016-02-23T16:00:23

My answer is just about the GetZbufferData portion.

vtkOpenGLRenderWindow already uses glReadPixels with little overhead from what I can tell. here

What happens after that I believe can introduce overhead. Main thing to note is that vtkOpenGLRenderWindow has 3 method overloads for GetZbufferData. You are using the method overload with the same signature as the one used in vtkWindowToImageFilter here

I believe you are copying that part of the implementation in vtkWindowToImageFilter, which makes total sense. What do you do with float pointer depthBuffer after you get it? Looking at the vtkWindowToImageFilter implementation, I see that they have a for loop that calls memcpy here. I believe their memcpy has to be in a for loop in order to deal with spacing, because of the variables inIncrY and outIncrY. For your situation you should only have to call memcpy once then free the array pointed to by depthBuffer. Unless you are just using the pointer. Then you have to think about who has to delete that float array, because it was created with new.

I think the better option is to use the method with this signature: int GetZbufferData( int x1, int y1, int x2, int y2, vtkFloatArray* z )

In python that looks likes this:

import vtk
# create render pipeline (not shown)
# define image bounds (not shown)
vfa = vtk.vtkFloatArray()
ib = image_bounds
render_window.GetZbufferData(ib[0], ib[1], ib[2], ib[3], vfa)

Major benefit is that the pointer for the vtkFloatArray gets handed straight to glReadPixels. Also, vtk will take of garbage collection of the vtkFloatArray if you create it with vtkSmartPointer (not needed in Python)

My python implementation is running at about 150Hz on a single pass. On a 640x480 render window.

edit: Running at 150Hz

VTK efficiency of vtkRenderWindow::GetZbufferData and vtkWindowToImageFilter::Update

1 Answers