ARKit SceneKit Metal Integration

1
votes

I am trying to integrate the creation of objects in ARKit with SceneKit and Metal. There are objects I can easily create with SeneKit and others with Metal, so I'd like to be able to use both side by side.

For example creating an ArKit project with the sceneKit template places a space ship as a demo through SceneKit Nodes. On the other hand using the Metal template creates a cube in ARKit with a tap using an anchor.

How would I be able to use both in the same project? Creating a cube through Metal and a spaceship with SceneKit, for example.

Another example, there is a MTKView that I want to wrap as an object or layer it on top of an object in SceneKit, and I also have other objects that I am creating with PBR in SceneKit, I want to have these side by side in ARKit. If possible.

Maybe with this? : https://developer.apple.com/documentation/scenekit/scnprogram

I think this, if possible would be very useful for the growing ARKit community.

Thanks!

2
scenekitmetalarkit
Could you elaborate on what you mean by "there is a MTKView that I want to wrap as an object or layer it on top of an object in SceneKit"? – mnuages
Take a look at this article by Marius at metal kit.org: metalkit.org/2017/07/29/using-arkit-with-metal.html – Simon Gladman

2 Answers

5
votes

You should be able to use SceneKit and Metal content in your scene at the same time - you can use SCNSceneRendererDelegate for that by having access to the renderer and it's currentRenderCommandEncoder property (https://developer.apple.com/documentation/scenekit/scnscenerendererdelegate):

  • renderer(_:willRenderScene:atTime:) - for rendering Metal content before SceneKit's
  • renderer(_:didRenderScene:atTime:) - for rendering Metal content after SceneKit's

Or use SCNNodeRendererDelegate to replace the content of the node with Metal content (https://developer.apple.com/documentation/scenekit/scnnoderendererdelegate):

  • renderNode(_:renderer:arguments:)
0
votes

How would I be able to use both in the same project?

You have to choose wether you want to build your app with Metal or SceneKit, both have they ups and downs. I think there a two approaches you could take:

  • Use SceneKit. Since SceneKit uses Metal you can enhance and change shaders or the program. Maybe start reading the documentation about using Metal in SceneKit. Maybe you can do what you want to do in Metal while still using SceneKits abstractions where possible.

  • Use Metal. If using Metal in SceneKit as Apple intended it is not enough you have to build everything in Metal. To load complex models you can use Apples ModelIO to load 3D content into metal.