NVENC session limitations

Question

I am doing H264 encoding using the hardware encoder present in NVIDIA GPUs (NVENC). I get the correct output but I am unable to launch more than 140 encoding streams.(One stream performs one video encoding)

The error I am getting is "NV_ENC_OUT_OF_MEMORY".The memory usage for the application when the error happens is much less than total available: 600MB (320X240) 2.5GB(640X480) 6GB(1280X710) out of 8GB in Quadro M5000).

For bitstream allocation, NV_ENC_CREATE_BITSTREAM_BUFFER has size more than a single frame size and memoryHeap type is CACHED.

Here is the test machine configuration: Debian Linux 8.5 Intel Xeon E5 @2.60 GHz Nvidia Quadro M5000 CUDA 7.5, driver 352.39

Any suggestion/direction towards solution would be appreciated. Thanks for your time.

Answer 1

NVENC runs on a discrete silicon IP core (SIP) block in the GPU, and not on the GPU's shaders. That SIP block has hard-coded limits (set in firmware) to ensure that these who need more than the artificial limit buy Tesla-grade hardware.

Marketing shills by NVIDIA.

You can override this limit by using Keylase's nvidia patcher here: https://github.com/keylase/nvidia-patch

Also, see this on session limitations: https://developer.nvidia.com/video-encode-decode-gpu-support-matrix