Black frames in AVMutableComposition

9
votes

This question is quite related to AVMutableComposition - Blank/Black frame between videos assets but as I am not using an AVAssetExportSession the answers doesn't fit my problem.

I'm using an AVMutableComposition to create a video composition and I'm reading it using an AVAssetReader (I need to have the frame data, I can't use a AVPlayer) but I often have black frames between my video chunks (there is no glitch noticeable in the audio).

I create my Composition as 

AVMutableCompositionTrack *compositionVideoTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
AVMutableCompositionTrack *compositionAudioTrack = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];

NSMutableArray* durationList = [NSMutableArray array];
NSMutableArray* videoList= [NSMutableArray array];
NSMutableArray* audioList= [NSMutableArray array];

for (NSInteger i = 0; i < [clips count]; i++)
{
    AVURLAsset *myasset = [clips objectAtIndex:i];
    AVAssetTrack *clipVideoTrack = [[myasset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
    [videoList addObject:clipVideoTrack];

    AVAssetTrack *clipAudioTrack = [[myasset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
    [audioList addObject:clipAudioTrack];

    CMTime clipDuration = [myasset duration];
    CMTimeRange clipRange = CMTimeRangeMake(kCMTimeZero, clipDuration);
    [durationList addObject:[NSValue valueWithCMTimeRange:clipRange]];
}

[compositionVideoTrack insertTimeRanges:durationList ofTracks:videoList atTime:kCMTimeZero error:nil];
[compositionAudioTrack insertTimeRanges:durationList ofTracks:audioList atTime:kCMTimeZero error:nil];

I also tried to insert each track manually in my composition but I have the same phenomenon.

Thanks

3
iosavfoundationavmutablecomposition
Did you ever solve this issue? I'm getting a similar problem. – elprl
@elprl I had a look at my code and it stayed pretty much the same. I think we "fixed" the issue by tweaking (key frame interval, framerate, ...) the videos format input (we can control it). That was a long time ago and I can't say for sure what parametter improved the situation. – chub

3 Answers

25
votes

I managed to solve this problem after many hours of testing. There are two things I needed to change. 1) add the 'precise' option when creating the asset.

 NSDictionary *options = @{AVURLAssetPreferPreciseDurationAndTimingKey:@YES};
 AVURLAsset *videoAsset3 = [AVURLAsset URLAssetWithURL:clip3URL options:options];

2) don't use

CMTime duration3 = [videoAsset3 duration];

use instead

AVAssetTrack *videoAsset3Track = [[videoAsset3 tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
CMTime duration3 = videoAsset3Track.timeRange.duration;

I found this out after I set the AVPlayer background color to Blue, then I noticed blue frames appearing, so the problem was to do with timings. Once I changed to above settings, the different videos aligned up fine when using:

 AVMutableCompositionTrack *videoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo
                                                                        preferredTrackID:kCMPersistentTrackID_Invalid];

[videoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, duration3)
                        ofTrack:videoAsset3Track
                         atTime:kCMTimeZero error:&error];
0
votes

Try creating two video tracks, and then alternate between the two when adding the clips.

0
votes

calculate Last CMTime is very important to insertTimeRange: for new AVAsset I had the same problem and solve it with:

 func addChunk(media: AVAsset) throws {
    let duration = self.mutableComposition.tracks.last?.timeRange.end
    try mutableComposition.insertTimeRange(CMTimeRange(start: .zero, duration: media.duration), of: media, at: duration ?? .zero)
}