What does the nVIDIA CUDA driver do exactly?

2
votes

What does Nvidia CUDA driver do exactly? from the perspective of using CUDA. The driver passes the kernel code, with the execution configuration (#threads, #blocks)... and what else?

I saw some post that the driver should be aware of the number of available SMs. But isn't that unnecessary ? Once the kernel is passed to GPU, the GPU scheduler just needs to spread the work to available SMs...

1
cudadrivernvidia

1 Answers

9
votes

The GPU isn't a fully autonomous device, it requires a lot of help from the host driver to do even the simplest things. As I understand it the driver contains at least:

  • JIT compiler/optimizer (PTX assembly code can be compiled by the driver at runtime, the driver will also recompile code to match the execution architecture of the device if required and possible)
  • Device memory management
  • Host memory management (DMA transfer buffers, pinned and mapped host memory, unified addressing model)
  • Context and runtime support (so code/heap/stack/printf buffer memory management), dynamic symbol management, streams, etc
  • Kernel "grid level" scheduler (includes managing multiple simultaneous kernels on architectures that support it)
  • Compute mode management
  • Display driver interop (for DirectX and OpenGL resource sharing)

That probably represents the bare minimum that is required to get some userland device code onto a GPU and running via the host side APIs.