How can I record a video and synchronize it with an AVPlayer?

Question

I need to record a video and show a video with an AVPlayer at the same time. The result needs to be synchronized: if the video I'm showing is a metronome, or someone clapping, the recorded video and the player should click or clap at the same time.

I play the AVPlayer using preroll and setRate, with a delegate in the camera controller.

Using AVCaptureMovieFileOutput, I've tried calling setRate on the player once fileOutput(output:didStartRecordingTo) is called, but the videos end up desynchronized, with the recorded video going behind the player.

Using an AssetWriter, I've tried calling setRate on the player once captureOutput(captureOutput:sampleBuffer) is called, and the first buffer is appended, but the result is the same.

Is there any other way to do this?

EDIT: Adding some code to show what I'm doing:

Camera with AVCaptureMovieFileOutput:

func startRecordingVideo() {
        guard let movieFileOutput = self.movieFileOutput else {
            return
        }

        sessionQueue.async {
            if !movieFileOutput.isRecording {

                let movieFileOutputConnection = movieFileOutput.connection(with: .video)

                movieFileOutput.setOutputSettings([AVVideoCodecKey: AVVideoCodecH264], for: movieFileOutputConnection!)

                let outputFileName = NSUUID().uuidString
                let outputFilePath = (NSTemporaryDirectory() as NSString).appendingPathComponent((outputFileName as NSString).appendingPathExtension("mov")!)
                movieFileOutput.startRecording(to: URL(fileURLWithPath: outputFilePath), recordingDelegate: self)
            } else {
                movieFileOutput.stopRecording()
            }
        }
    }

    func fileOutput(_ output: AVCaptureFileOutput, didStartRecordingTo fileURL: URL, from connections: [AVCaptureConnection]) {
        cameraDelegate?.startedRecording()
    }

Implementing delegate's startRecording in the viewController:

private func setRateForAll() {
        let hostTime = CMClockGetTime(CMClockGetHostTimeClock())
        activePlayers.forEach {
            $0.setRate(atHostTime: hostTime)
        }
    }

Once the active player or players end, I call stopRecording on the camera, and show the resulting recorded video in another player, along with the player or players I've been showing.

I've upvoted the question, sending it back to "0", because I think it is a good one, but I'm going to guess the downvote was because you described the code, but you didn't provide a snippet. I know you probably feel that is unnecessary, but for this type of Apple provided framework, there are often things that get spotted. — benc
Thank you! I've added some code, I don't know if it will be enough. There is a lot of lines regarding the configurations of session and inputs/outputs, I don't know if showing all those lines is relevant to the question. — Francisco

Francisco Francisco · Accepted Answer · 2020-05-06T20:25:57

I ended up using the recording session master clock as the hostTime for the setRate method. The desynchronization is almost imperceptible now!

In fileOutput(output:didStartRecordingTo):

cameraDelegate?.startedRecording(clock: CMClockGetTime(self.session.masterClock!))

In the view with the players:

private func setRateForAll(clock: CMTime) {
        activePlayers.forEach {
            $0.setRate(atHostTime: clock)
        }
}

How can I record a video and synchronize it with an AVPlayer?

1 Answers