OpenGL Hooking -- Rendering to an arbitrarily-sized FBO

Question

So I'm trying to make an OpenGL application render at a higher resolution than it normally would. I've already created a shared library that hooks most of the relevant GLX/OpenGL functions. Here's my current approach (at a high-level):

When my hooked SwapBuffers() is called
    Unbind my FBO
    Call the (original/unhooked) SwapBuffers()
    Bind my FBO
    Set the viewport to (0, 0, HIGH_RES_X, HIGH_RES_Y)
    Set the scissor region to (0, 0, HIGH_RES_X, HIGH_RES_Y)
    return

This approach doesn't seem to work for (most) applications. I suspect that is because some applications perform texture lookups (for screen-space operations) by dividing glFragCoord.xy by a uniform that represents their screen resolution (to convert from screen space to texture coordinates).

If resizing the output isn't possible, I wonder if it is possible to obtain the contents drawn onto the default framebuffer (i.e both the color and the depth buffer) without using glReadPixels. Ideally there would be a way to access this data in the form of a texture (so it's already on the GPU). I've heard things about Pixel Buffer Objects -- would using one of these prevent a pipeline stall?

This approach isn't going to work for any application that renders to its own framebuffers. They have to bind them every frame, and thus set the viewport and whatever else as they see fit. — Nicol Bolas
My original approach also hooked glBindFramebuffer so that it would replace calls to glBindFramebuffer(GL_FRAMEBUFFER, 0) to bind my framebuffer instead. Other glBindFramebuffer calls would be unaffected. I'd also hook glViewport to make sure that the rendering resolution is always what I want, assuming that a different FBO isn't bound. — AnimatedRNG
That too is broken. The default framebuffer does not behave like a normal FBO. For example, the glDrawBuffers enumerators are completely different. — Nicol Bolas

AnimatedRNG AnimatedRNG · Accepted Answer · 2018-01-03T22:02:56

The technique I proposed actually works for a handful of applications, but most of the time it just doesn't work.

If you just want to extract the color/depth buffer, you can either use a pool of PBOs with glReadPixels or use glBlitFramebuffer. The former is not an option if latency is a concern; the latter works fairly well (see this example from the reshade project)

OpenGL Hooking -- Rendering to an arbitrarily-sized FBO

1 Answers