CUDA driver version is insufficient for CUDA runtime version

Question

I got the message:

"cutilCheckMsg() CUTIL CUDA error :
kernel launch failure : CUDA driver
version is insufficient for CUDA
runtime version."

While trying to run an example source code. Also happens for the function cutilSafeCall.

I am using:

• Windows 7 64bits
• Visual studio 2008
• CUDA developer driver, toolkit, and SDK 3.1
• Emulation mode

Answer 1

I saw the same at runtime with the latest driver on Mac OS 10.6.

cudaError_t error = cudaGetDevice(&device);
printf("%s\n", cudaGetErrorString(error));

I went back to the developer site, downloaded the driver again and now it runs. http://developer.nvidia.com/object/cuda_3_1_downloads.html#MacOS

Answer 2

You need to ensure that your driver version matches or exceeds your CUDA Toolkit version.

For 2.3 you need a 190.x driver, for 3.0 you need 195.x and for 3.1 you need 256.x (actually anything up to the next multiple of five is ok, e.g. 258.x for 3.1).

You can check your driver version by either running the deviceQueryDrv SDK sample or go into the NVIDIA Control Panel and choose System Information.

Download an updated driver from www.nvidia.com/drivers.

Answer 3

You can either download the latest driver OR use an older toolkit version to compile your code.

Answer 4

CUDA driver version is insufficient for CUDA runtime version: means your GPU can`t been manipulated by the CUDA runtime API, so you need to update your driver.

Answer 5

My cent,

with Linux/Unix this error may be related to the selected GPU mode (Performance/Power Saving Mode), when you select (with nvidia-settings utiliy) the integrated Intel GPU and you execute the deviceQuery script... you get this error:

-> CUDA driver version is insufficient for CUDA runtime version

But this error is misleading, by selecting back the NVIDIA(Performance mode) with nvidia-settings utility the problem disappears.

It is not a version problem.

Regards

P.s: "Power Saving Mode" tells Optimus to activate the CPU integrated Intel GPU

Answer 6

Counterintuitively, this error also happens if libcuda.so is not found, even when versions reported by nvidia-smi match perfectly. This library is part of nvidia-drivers package (On CentOS: nvidia-driver-latest-cuda-libs, on Gentoo x11-drivers/nvidia-drivers). It is possible to have the CUDA Tookit with nvcc and libcudart installed and building your app fine, but the drivers part not installed, causing this error.

To diagnose whether this is the reason, use strace:

strace -f -e trace=file ./your_cuda_app

and check for open calls to libcuda.so*, at least one of them should return with a success code, like so:

4928  open("/lib64/libcuda.so.1", O_RDONLY|O_CLOEXEC) = 3 

Answer 7

I also had similar problem, updated my graphic driver but the problem still remained. I finally decided to remove Cuda 9.2 and install Cuda 8, it solved my issue.

Answer 8

This is what has worked for me :

1. Installing TensorFlow version 1.14, installing keras
2. Install CUDA 10.0 from https://developer.nvidia.com/cuda-10.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exenetwork for windows
3. Download the compatible CuDNN i.e 7.4.2 from https://developer.nvidia.com/rdp/cudnn-archive
4. set the following in the path variable cudnn-10.0-windows10-x64-v7.4.2.24\cuda\bin cudnn-10.0-windows10-x64-v7.4.2.24\cuda\include cudnn-10.0-windows10-x64-v7.4.2.24\cuda\lib\x64
5. Restart the Pc

Answer 9

Maybe it is related to the TBB lib: Error OpenCV with CUDA using TBB for multiple GPUs

Try rebuilding it making sure you passed the following parameters to CMake (assuming you already installed "tbb" and "tbb-devel" packages:

-D WITH_TBB=YES -D TBB_INCLUDE_DIRS=/usr/include/tbb