I have successfully implemented a video player using ffmpeg. I am now trying to use hardware decoding but I'm facing a couple issues. I found a post that I followed as a starting point here: https://ffmpeg.org/pipermail/libav-user/2014-August/007323.html
I have updated the code that setup the necessary stuff for the decoder. The updated code is available here: https://drive.google.com/file/d/0B5ufHdoDzA4ieVk5UVpxcDNzRHc/view?usp=sharing
And this is how I'm using it to initialize the decoder:
// Prepare the decoding context
AVCodec *codec = nullptr;
_codecContext = _avFormatContext->streams[_streamIndex]->codec;
if ((codec = avcodec_find_decoder(_codecContext->codec_id)) == 0)
{
std::cout << "Unsupported video codec!" << std::endl;
return false;
}
_codecContext->thread_count = 1; // Multithreading is apparently not compatible with hardware decoding
InputStream *ist = new InputStream();
ist->hwaccel_id = HWACCEL_AUTO;
ist->hwaccel_device = "dxva2";
ist->dec = codec;
ist->dec_ctx = _codecContext;
_codecContext->coded_width = _width;
_codecContext->coded_height = _height;
_codecContext->opaque = ist;
dxva2_init(_codecContext);
_codecContext->get_buffer2 = ist->hwaccel_get_buffer;
_codecContext->get_format = GetHwFormat;
_codecContext->thread_safe_callbacks = 1;
if (avcodec_open2(_codecContext, codec, nullptr) < 0)
{
std::cout << "Video codec open error" << std::endl;
return false;
}
And here is the definition of GetHwFormat referenced above:
AVPixelFormat GetHwFormat(AVCodecContext *s, const AVPixelFormat *pix_fmts)
{
InputStream* ist = (InputStream*)s->opaque;
ist->active_hwaccel_id = HWACCEL_DXVA2;
ist->hwaccel_pix_fmt = AV_PIX_FMT_DXVA2_VLD;
return ist->hwaccel_pix_fmt;
}
When I open an mp4 (encoded in h264) video that is HD resolution or less, everything seems to be working fine. However, as soon as I try higher resolution videos like 3840x2160, I get the following errors repeatedly:
Failed to execute: 0x80070057
Hardware accelerator failed to decode picture
I also start getting the following errors after a few seconds:
co located POCs unavailable
And the video is not displayed properly: I get a lot of artifacts all over the video and it is lagging. I checked the first error in the ffmpeg source code. It seems that IDirectXVideoDecoder_Execute fails because of an invalid parameter. Since this is happening withing ffmpeg, there must be something that I'm missing but I can't figure out what. The only relevant post that I found with this error was because of multithreading but I set the thread_count to 1 before opening the codec.
This issue is happening on my main computer which has the following specs:
- i7-4790 CPU @ 3.6GHz
- RAM 16 GB
- Intel HD Graphics 4600
- Windows 8.1
The same issue is not happening on my second computer which has the following specs:
- i7 4510U @ 2GHz
- RAM 8 GB
- NVIDIA GeForce GTX 750Ti
- Windows 10
If I use DXVAChecker on my main computer, it says that my graphics card supports DXVA2 for H264_VLD_*, and I can see that the calls to the Microsoft API are being made (DXVA2_DecodeDeviceCreated, DXVA2_DecodeDeviceBeginFrame, DXVA2_DecodeDeviceGetBuffer, DXVA2_DecodeDeviceExecute, DXVA2_DecodeDeviceEndFrame) while my video is playing.
I also don't see any increase of GPU usage (on either computer) between the version with hardware decoding and the version without; however, I do see a decrease in CPU usage (not as much as I was expecting though). This is also very strange.
Note that I tried both the Windows release available on the FFmpeg website, and a version that I compiled with --enable-dxva2. I have searched a lot already but I was unable to find what I'm doing wrong.
Hopefully, someone can help me, or maybe point me to a better example?