How can kernel execution time increase with better occupancy?

Question

A kernel is using 54 registers per thread with an occupancy of 0.33. It has an if statement in it for debugging purpose:

if ( -1 == val ) // val is INT32
{
    printf( "Invalid value!\n" );
}

If I comment out this if statement, the kernel uses 33 registers per thread and the occupancy also improves to 0.5. But, the strange part is that this new leaner kernel actually takes 15% more time to execute! All other code and launch parameters (blocks, threads) are constant. Can anyone explain how this performance degradation could happen?

I am using CUDA 4.0 on a GTX 580, with code compiled to CUDA capability 2.0.

First, are you sure the kernel is running slower? Specifically, how did you measure the time? And what are your launch parameters? — harrism
I know of a train which runs between a mine and a port. The train uses more fuel travelling empty from the port to the mine than it does travelling full of ore from the mine to the port. Which begs the question why? And the answer is in something I haven't told you. This question is much the same - the answer is clearly in something that you haven't told us. It will be very difficult to answer without a lot more information than you have provided so far. — talonmies
HarrisM: I am looking at the time for this kernel from CUDA profiler, so I guess this cannot be wrong. I am launching with <<< 512, 256 >>> over data that has 1 million elements. — Ashwin Nanjappa
The answer to the train problem was that the mine is in a mountain range, and the port is at sea level. So the train goes downhill from the mine and uphill from the port. How about this question, any more insights to offer? — talonmies
Talonmies: Yes. If I reduce the number of threads per block from 128 to 32, the kernel behaves properly. That is, there is no odd slowdown or speedup from a single if statement. With 32 threads, the kernel is actually 30% faster than any time before. Does this offer any clue? — Ashwin Nanjappa

Greg Smith Greg Smith · Accepted Answer · 2012-03-01T04:02:38

Ashwin, Can you provide additional details about your algorithm, launch parameters, ... The Visual Profiler v4.1 and Parallel Nsight both have additional metrics that may help identify you bottleneck.

Theoretical occupancy is an abstract measurement of how well the SM can hide latency. Increasing occupancy above a certain amount may not improve performance and may impact performance. For example, as you increase the occupancy the number of L1 bytes/threads is decreasing. One item to look at would be cache hitrate.

How can kernel execution time increase with better occupancy?

1 Answers