What happen if a CUDA kernel is called from multiple pthreads simultaneously?

Question

I have a CUDA kernel that do my hard work, but I also have some hard work that need to be done in the CPU (calculations with two positions of the same array) that I could not write in CUDA (because CUDA threads are not synchronous, I need to perform a hard work on a position X of an array and after do z[x] = y[x] - y[x - 1], where y is the array resultant of a CUDA kernel where each thread works on one position of this array and z is another array storing the result). So I'm doing this in the CPU.

I have several CPU threads to do the CPU side work, but each one is calling a CUDA kernel passing some data. My question is: what happens on the GPU side when multiple CPU threads are making GPU calls? Would be better if I do the CUDA kernel call once and then create multiple CPU threads to do the CPU side work?

Answer 1

Kernel calls are queued and executed one by one in single stream.

However you can specify stream during kernel execution - then CUDA operations in different streams may run concurrently and operations from different streams may be interleaved. Default stream is 0.

See: CUDA Streams and Concurrency

Things are similar when different processes use the same card.

Also remember that kernels are executed asynchronously from CPU stuff.

Answer 2

On CUDA 4.0 and later, multiple threads can share the same CUDA context, hence no need for cuPush/PopContext anymore. You just need to call cudaSetDevice for each thread. Then, mentioned @dzonder, you can run multple kernels simulatenously from different threads with streams.