Clip to mesh depth in Unity

2
votes

I'm writing a portal renderer in Unity, for a VR game. The idea is that a complex mesh can be used as a 'window' into an alternate world. I have set things up to render the portals with three ordered shaders currently:

  1. The first shader renders portals to the stencil buffer.
  2. The second shader is a modified standard shader, that renders a world within the stencil mask.
  3. The third shader renders portals to the depth buffer, so that the 'outside' world doesn't render inside the portal in a later pass.

The problem is that objects inside the portal occasionally show up in front of the portal when they shouldn't. I can't think of any way to clip them to the portal easily. If the portals were flat, I could use oblique near-plane clipping. For a complex mesh, though, I thought maybe I could leverage a second depth buffer, and only draw on z-fail for the world behind the portal?

I'm not sure how to go about creating a second depth buffer though, or if there's a better way.

Any thoughts?

1
unity3dshader
Wouldn't it be possible to have a second camera rendering in a texture what you want to see through the portal. Then you just apply this texture to the portal object. This would allow you not to use custom shaders. I don't know if this is acceptable for your use case - Basile Perrenoud
Thanks for your comment! I think the clipping might be a problem either way, right? (For stencil vs. render texture, for VR, I was thinking the stencil buffer might be more efficient because it’ll use single-pass stereo.) - Chris Nolet

1 Answers

1
votes

It may be possible to create a second depth buffer with the new scriptable render pipeline in Unity 2018.1.

In the mean time, you can clip to a mesh by adding two additional shaders and tweaking the modified standard shader a bit:

  1. Do a depth pre-pass on objects inside the portal.
  2. Render the portal itself with ZTest Greater. This will clip the depth buffer to the portal.
  3. Add ZTest Equal to the Forward and Forward_Delta passes for the modified standard shader. This will ensure that fragments in front of the portal don't get rendered.

Combined with the three steps in the question, the total pipeline from end-to-end is:

  1. Render portal to the stencil buffer.
  2. Do a depth pre-pass on objects inside the portal.
  3. Render the portal itself with ZTest Greater. This will clip the depth buffer to the portal.
  4. Use a modified standard shader to render a world within the stencil mask. Add ZTest Equal to the Forward and Forward_Delta passes for the modified standard shader. This will ensure that fragments in front of the portal don't get rendered.
  5. Render portals to the depth buffer again, but with ZTest LEqual, so that the 'outside' world doesn't render inside the portal in a later pass.

Note: Each step requires a separate material, with increasing Queue values for each material, to make sure everything happens in the correct order.

You'll also want to render each portal with a different camera / layer culling mask, so that objects from different worlds don't shadow each-other.