I am developing an Augmented Reality app using the Vuforia SDK. I am trying to use AVCaptureVideoPreviewLayer and SceneKit for application rendering instead of raw OpenGL calls provided by Vuforia sample code.
I got the AVCaptureVideoPreviewLayer and SceneView working without Vuforia, i.e. I managed to draw 3D scene on top of camera video background. The code is at: https://github.com/lge88/scenekit-test0/blob/master/scenekit-test0/GameViewController.swift#L74-L85:
func initViews() {
let rootView = self.view
let scnView = createSceneView()
let scene = createScene()
scnView.scene = scene
let videoView = createVideoView()
rootView.addSubview(videoView)
rootView.addSubview(scnView)
}
The implementation can be summarized as:
- Create a UIView called videoView.
- Initialize an AVCaptureVideoPreviewLayer, and add it as a sublayer of videoView.
- Create a SCNView called scnView and initialize the scene o scnView.
- Add both videoView and scnView to the root UIView.
Currently I am trying to integrate Augmented Reality feature, GameViewController.swift#L68-L71:
initViews()
animateScene()
initControls()
ARServer(size:viewFrame.size, done: initARCallback)
ARServer is a class that takes care of the Vuforia initialization, its implementation is taken from Vuforia ImageTargets sample code. The tracker is working, it can successfully track the targets of the sample dataset.
However the AVCaptureVideoPreviewLayer rendering doesn't work correctly, the area of the video rendering area is resized, and the video layer is not updating, it shows a static image captured when the tracker camera started. Here is how it looks from a ipad screenshot: https://github.com/lge88/scenekit-test0/blob/master/video-preview-layer-issue.png