Consumer-Producer threads with DirectX desktop frames

Question

I'm writing a DirectX application with two threads:

Producer thread grabs desktop frames with DirectX (as in the Desktop Duplication DirectX sample)

IDXGIResource* DesktopResource = nullptr;
ID3D11Texture2D *m_AcquiredDesktopImage = nullptr;
HRESULT hr = m_DeskDupl->AcquireNextFrame(500, &FrameInfo, &DesktopResource);
hr = DesktopResource->QueryInterface(__uuidof(ID3D11Texture2D), reinterpret_cast<void **>(&m_AcquiredDesktopImage));
DesktopResource->Release();
// The texture pointer I'm interested is m_AcquiredDesktopImage

Consumer thread performs image processing operations on the GPU.

To avoid copies I'd like to keep everything on the GPU as much as possible. From ReleaseFrame's documentation I kinda get that I should call ReleaseFrame on the desktop duplication interface as soon as I'm done processing the frame.

My question: should I copy the m_AcquiredDesktopImage texture into another one and call ReleaseFrame as soon as the copy is finished and return that new texture to the producer thread for processing or can I just get away with returning the m_AcquiredDesktopImage texture pointer to the consumer thread? Is this a copy of the framebuffer texture or is it the framebuffer texture and I might generate a data race by returning it?

Which one is the correct way to handle a producer of grabbed frames and a consumer of GPU textures?

Varrak Varrak · Accepted Answer · 2018-05-14T15:55:10

The MSDN documentation on ReleaseFrame is a little convoluted. It specifically states you need to release the current frame before processing the next one, and that the surface state is "invalid" after release, which would indicate it is either not a copy, or not a copy that your process owns (which would yield the same effective result). It also states you should delay the call to ReleaseFrame until right before you call AcquireNextFrame for performance reasons, which can make for some interesting timing issues, especially with the threading model you're using.

I think you'd be better off making a copy (so ReleaseFrame from the previous capture, AcquireNextFrame, CopyResource). Unless you're using fences you don't have any guarantees the GPU will be consuming the resource before your producer thread has called ReleaseFrame, which could give you undefined results. And if you are using fences, and the AcquireNextFrame call is delayed until the GPU has finished consuming the previous frame's data, you'll introduce stalls and lose a lot of the benefits of the CPU being able to run ahead of the GPU.

I'm curious why you're going with this threading model, when the work is done on the GPU. I suspect it makes life a little more complicated. Although making a copy of the texture would remove a lot of those complications.

Consumer-Producer threads with DirectX desktop frames

2 Answers