iOS - How to add sceneKit node objects on top of the front facing camera. Not ArKit

1
votes

I'm trying to create some animations to be seen in the front facing camera in iOS, with SceneKit. Something similar to what ArKit does on the back camera. Not using iPhone X nor ArKit (ArKit does not work on the front facing camera). Right now I'm not able to combine the camera and the sceneKit scene for both to appear. I can see either the front camera view from:

AVCaptureDevice.default(.builtInWideAngleCamera, for: AVMediaType.video, position: .front)

or I can see the scene view.

It should look like the front facing camera live as a backdrop and all the nodes in the scene appearing in the front on top of that.

Another way to think of it is that I want the background of the scene to by a live feed from the front facing camera.

How would I do this?

1
iosscenekit

1 Answers

1
votes

SCNScene.background is an instance of SCNMaterialProperty which can take an AVCaptureDevice instance as its contents. The online documentation doesn't reflect that addition to iOS 11 yet, but it's mentioned in the SceneKit headers as well as their WWDC'17 presentation.

// Setup background video
let captureDevice: AVCaptureDevice = ...
scene.background.contents = captureDevice