CUDA(GPU) as OpenCV backend

8
votes

Am trying to use CUDA as backend for dnn module provided in opencv-4.1.1 , I have build opencv with CUDA enabled, nvidia drivers and CUDA is properly placed on system, here am using manjaro as development platform.

am trying to load pre-trained YOLOv3 weights using cv2.dnn module,

net = cv2.dnn.readNetFromDarknet('yolov3.cfg', 'yolov3.weights')

But it uses CPU as defualt inference engine, but am trying to use GPU as backend IE, from offecial opencv doc, i found following 

DNN_TARGET_CPU  
DNN_TARGET_OPENCL   
DNN_TARGET_OPENCL_FP16  
DNN_TARGET_MYRIAD       
DNN_TARGET_FPGA

as target backend, but no support for direct GPU inference, so what's point of compiling opencv with CUDA if it doen't make use of GPU as underlying inferencing engine,

in order to make sure processes are runing on GPU I posted output of nvidia-smi ,

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.26       Driver Version: 430.26       CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 710      Off  | 00000000:01:00.0 N/A |                  N/A |
| 40%   40C    P0    N/A /  N/A |    598MiB /  1998MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0                    Not Supported                                       |
+-----------------------------------------------------------------------------+

the process id for GPU utilization is 0, clearly means that no process utilizing GPU, any guidance will be appreciated.

1
pythonc++opencv
OpenCV can also make use of the GPU outside the DNN module, i.e. the Transparent API. So there is a reason to compile with cuda support. Also, what happens if you use OpenCL as backend? This should make use of the GPU aswell. - Timo
@Timo But that can't be used for production inferencing, as the requirement to run the model in higher fps, CUDA does it better than opencl, so can you elaborate more on Transparent API. - U.Swap
I see. I'm not used to the DNN module so I cannot help you in that regard. The Transparent API is a transparent wrapper (hence the name) for most OpenCV functions that allows them to be executed on the GPU. I just said that to point out that there is a use for cuda in OpenCV. But this has nothing to do with the DNN module. - Timo
At the moment, there is no CUDA support for DNN inferencing. Just use your preferred DL library (Tensorflow, PyTorch, MxNet, Chainer, ...) if you need good performance on GPU. - Catree
@Timo The OpenCL backend is insanely slow on CUDA GPUs. The OpenCL backend does not support all layers and hence, it the inference process involves switching between the OpenCL and CPU backends (as a fallback). This works great if you are using the integrated graphics but will be extremely slow for devices which do not share the main memory. The cost of the memory transfer between the CUDA device and the CPU defeats most of the potential speedups that could be obtained. - Yashas

1 Answers

5
votes

At the moment CUDA support for DNN module way in progress under a GSOC task so there is no official release yet. You can check its repo here to see progress.

Edit: Looks CUDA backend integration is completed and included in the release version 4.2.0, you can check the change logs here.