Recording video from ARKit

Question

I would like to add a feature for users to record their ARKit experience. I'm taking the capturedImage of the ARFrame supplied by session(_ session: ARSession, didUpdate frame: ARFrame) and concatenating them into a video.

Unfortunately, ARFrame.capturedImage shows the frame of video captured by the camera, but doesn't include nodes placed by ARKit.

Is there any way to capture video coming from an ARSCNView?

I've tried this library, but it has major bugs (no shadows, large stutter at beginning of recording). I'd also like to not use ReplayKit for this project.

Here is what I'm using to turn ARFrame.capturedImage into a UIImage, and subsequently, a video.

extension UIImage {
    convenience init(pixelBuffer: CVPixelBuffer) {
        let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
        let size = CGSize(width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))

        let tempContext = CIContext()
        let image = tempContext.createCGImage(ciImage, from: CGRect(origin: CGPoint.zero, size: size))!

        // This assumes we're using an iPhone in portrait.
        self.init(cgImage: image, scale: 1, orientation: .right)
    }
}

Answer 1

I actually found a library to do this. It's SceneKitVideoRecorder.

I don't fully understand how it works yet, but the important code is located in SceneKitVideoRecorder.swift.

Answer 2

You can try to use ReplayKit. I use ReplayKit in my AR App to record when I place a model to the scene and more.

Try this my snippet:

import ReplayKit

class YourController: UIViewController, RPPreviewViewControllerDelegate { 

@IBAction func shotVideo(_ sender: UIButton) {

    print("Video")

    if !isRecording {

        startRecording()

    } else {

        stopRecording()
    }
}

func startRecording() {


    guard recorder.isAvailable else {

        print("The recording isn't available now.")

        return
    }

    recorder.isMicrophoneEnabled = false

    recorder.startRecording{ [unowned self] (error) in

        guard error == nil else {
            print("Trouble with starting this recording.")
            return
        }

        print("Recording started with success.")

        self.isRecording = true

    }

}

func stopRecording() {

    recorder.stopRecording { (preview, error) in

        print("The recording is stopped")

        guard preview != nil else {
            print("The preview controller isn't available.")
            return
        }

        let alert = UIAlertController(title: "End Recording", message: "Want to edit or delete this recording?", preferredStyle: .alert)

        let deleteAction = UIAlertAction(title: "Delete", style: .destructive, handler: { (action: UIAlertAction) in
            self.recorder.discardRecording(handler: { () -> Void in
                print("Recording suffessfully deleted.")
            })
        })

        let editAction = UIAlertAction(title: "Edit", style: .default, handler: { (action: UIAlertAction) -> Void in
            preview?.previewControllerDelegate = self
            self.present(preview!, animated: true, completion: nil)
        })

        alert.addAction(editAction)
        alert.addAction(deleteAction)

        self.present(alert, animated: true, completion: nil)

        self.isRecording = false

    }

}

// RPPreviewViewControllerDelegate

func previewControllerDidFinish(_ previewController: RPPreviewViewController) {
    dismiss(animated: true)
}
}

This is my code, I hope I have been helpful :).