Apply a Shader on RealityKit Scene

Question

Is there a way to apply a full screen shader to a RealityKit scene (Swift/iOS)?

I know I can do this with SCNTechnique in SceneKit/ARSCNView, but I'm dealing with an ARView.

Is there a way in RealityKit one can apply a vertex/fragment shader?
or a way to get the rendered scene as a texture?

With an ARSCNView I can apply a shader to run over the output of a scene, just like: https://medium.com/ar-tips-and-tricks/arkit-theory-an-introduction-to-scntechniques-710e024bc91e

But with RealityKit, I'm dealing with an ARView which doesn't have a SCNTechnique property

Thanks :-)

Hi @Chris! What full screen shader do you mean exactly? Environment shader? — Andy Fedoroff
Hi Andy, Anything at the minute, I've been trawling the docs, but can't seem to find a reference to apply ANY shader — Chris
Sorry))) Shader to what? To environment or to a model? What do you mean saying full screen shader? — Andy Fedoroff
In a ARSCNView I can apply a SCNTechnique over the full screen, as if I'm dealing with a texture. but in RealityKit, I'm dealing with an ARView — Chris
Apologies for using the wrong terminology, is there something similar in ARView? — Chris

Andy Fedoroff Andy Fedoroff · Accepted Answer · 2020-06-09T12:15:25

In RealityKit 1.0 and 2.0 you can apply background color/image, HDR for image based lighting/reflections (IBL) and audio properties. You can apply environmental image via skybox(:_) type method. Here's a link to official documentation.

static func skybox(_ resource: EnvironmentResource) -> ARView.Environment.Background

This method uses EnvironmentResource classthat contains background and lighting information for your scene. You can read about it here and in my story on Medium.

Now in RealityKit 2.0 you are capable to apply a vertex/fragment shader (via CustomMaterial) and get the rendered scene as a texture (like you could do in SceneKit+Metal).

Apply a Shader on RealityKit Scene

1 Answers