OpenGL sprite scene, using z-buffer with perspective projection and without distortion. Possible?

Question

I'm working on some sprite based games. For z-ordering sprites there are basically the alternatives
1) draw in correct order - can be troublesome for batching
2) use orthographic projection and depth testing - can be troublesome with translucency

I am favoring 2) however I would like to keep a perspective projection to allow some easy 3d animations. Cards being flipped in 3d etc. If I do that, the z-buffer differences may/will result in scaling of the sprites, since the objects are at different distances to the camera.
I thought about scaling the sprites based on their distance to undo the projection. But then i will get interference with 3d animated objects on top of the then "non-flat" 2d scene.

I guess the sane way is, to go for an orthographic 2d scene and do rare 3d animations in a second step with "cloned" objects on top of it. But maybe someone has a different idea?

mikkokoos answer led to a solution. Via an additional depth value the vertex shader can adjust the z coords of all vertices in clip space to push them to different depth layers. The real 3d coords can stay at z=0. Changing in clip space eliminates the perspective problems, so perspective projection can be used in the scene.
To allow for different camera angles than just perpendicular to the sprite plane, instead of pushing the sprite vertices to fixed layers they should be adjusted by "layerOffset * layerIdx". So for every layer a vertex gets pushed one step closer to the camera.

Note: With fixed layers it can be very tricky to find out the correct clip space coordinates, since the xy plane is not necessarily in the center.

Open questions: Resolution of clip space? Smallest coord difference in clip space? Possible interference between pushed sprites and real 3d elements? To prevent plane sprites from overdrawing 3d objects, all non-sprite-plane-objects should be pushed by the maximum used layer offset. Calculating the layer offset from available space in front of the camera would probably be a good idea.

Achieved: Sprites are organized on a 2d plane in a perspective projected 3d scene. Sprites are occluded based on fragment depth, calculated in the shaders. Drawing can still be done in texture batches.

Note2: I still consider doing two batched draw runs. 1st one for 2d sprites with adjusted depth, 2nd one for sprites or objects that are not "attached" to the sprite plane. Clearing depth buffer inbetween and adding an invisible sprite plane for clipping. (if no objects behind the stage are allowed)
Pics:
http://www.imagebam.com/image/109f6e356918267
http://www.imagebam.com/image/a2085f356918275

you can write the depth yourself after the projection. opengl.org/sdk/docs/man/html/gl_FragDepth.xhtml To clarify: Render your objects with a perspective projection all at the same depth. After that use gl_FragDepth to position your objects in the right order. — dari
Damn, my mistake. I should have mentioned OpenGL ES 2.0. Afaik there is no gl_FragDepth support. Sorry! Should have said that earlier. Thank you anyway! — J-S
Use perspective projection with glPolygonOffset instead. opengl.org/sdk/docs/man/html/glPolygonOffset.xhtml — dari
I thought about that. But it would break apart the batched rendering calls. I would have to set the offset per sprite instance. — J-S

ovikoomikko ovikoomikko · Accepted Answer · 2014-10-08T13:07:38

You could use the perspective projection to render all sprites and have a uniform float "depth" for each which will substitute the gl_Position.z at the end of the vertex shader:

gl_Position.z=(u_zOffset-1.0)*gl_Position.w;

This will "flatten" the mesh, so if your meshes are more complex and depending on the used DepthFunc, you might want to have a range instead. (visible range is from -1 to 1):

gl_Position.z=((1.0+(gl_Position.z/gl_Position.w))*u_zRange+u_zOffset-1.0)*gl_Position.w;

Where u_zRange is 2f/(number_of_layers) and u_zOffset is u_zRange*layer_id. Layer 0 is the closest.

EDIT: Summary: Put everything through the MVP matrix at the same distance from the camera, then modify the Z value in the vertex shader to order them on the screen.

OpenGL sprite scene, using z-buffer with perspective projection and without distortion. Possible?

2 Answers