AVFoundation UIImage behind video track

Question

I'm currently rendering a video track that is smaller than the output size which is working fine. I want to draw a UIImage into the background so that the video is on top with the image showing in the area where the video isn't. I've tried using CoreAnimation Layers along with videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayer:inLayer: but layers below the video layer don't seem to show through (ones above show just fine) - just black or whatever background color I set on the AVMutableVideoCompositionInstruction object. I've also tried setting that background color to [UIColor clearColor].CGColor but it just comes through as black.

Anyone done something similar and have suggestions?

CALayer *parentLayer = [CALayer layer];
CALayer *videoLayer = [CALayer layer];
CALayer *backgroundLayer = [CALayer layer];
backgroundLayer.frame = rect;
parentLayer.frame = rect;
videoLayer.frame = rect;
videoLayer.backgroundColor = [UIColor clearColor].CGColor;
backgroundLayer.backgroundColor = [UIColor purpleColor].CGColor;
[parentLayer addSublayer:backgroundLayer];
[parentLayer addSublayer:videoLayer];

mainCompositionInst.animationTool = [AVVideoCompositionCoreAnimationTool videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayer:videoLayer inLayer:parentLayer];

Answer 1

After trying several things finally I got a way to work it. To make it work need to use a blank video/audio track. Then add background image an overlay to this blank video layer. Then export it and combine the original asset(video) and exported asset(asset) and export the final asset(video).Hope it will help you.

Add overlay

- (void)addOverlayImage:(UIImage *)overlayImage ToVideo:(AVMutableVideoComposition *)composition inSize:(CGSize)size {
    // 1 - set up the overlay
    CALayer *overlayLayer = [CALayer layer];

    [overlayLayer setContents:(id)[overlayImage CGImage]];
    overlayLayer.frame = CGRectMake(0, 0, size.width, size.height);
    [overlayLayer setMasksToBounds:YES];

    // 2 - set up the parent layer
    CALayer *parentLayer = [CALayer layer];
    CALayer *videoLayer = [CALayer layer];
    parentLayer.frame = CGRectMake(0, 0, size.width, size.height);
    videoLayer.frame = CGRectMake(0, 0, size.width, size.height);
    [parentLayer addSublayer:videoLayer];
    [parentLayer addSublayer:overlayLayer];

    // 3 - apply magic
    composition.animationTool = [AVVideoCompositionCoreAnimationTool
                                 videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayer:videoLayer inLayer:parentLayer];
}

- (void)getBackgroundVideoAssetWithcompletion:(void (^)(AVAsset *bgAsset))completionBlock {

    NSString *path = [[NSBundle mainBundle] pathForResource:@"blank_video" ofType:@"mp4"];
    NSURL *trackUrl = [NSURL fileURLWithPath:path];
    AVAsset *asset = [AVAsset assetWithURL:trackUrl];
    AVAssetTrack *track = [[asset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];

    CMTimeRange range = CMTimeRangeMake(kCMTimeZero, [asset duration]);
    AVMutableComposition* mixComposition = [AVMutableComposition composition];

    AVMutableCompositionTrack *compositionVideoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
    [compositionVideoTrack insertTimeRange:range ofTrack:[[asset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] atTime:kCMTimeZero error:nil];


    CGAffineTransform videoTransform = track.preferredTransform;
    CGSize naturalSize = CGSizeApplyAffineTransform(track.naturalSize, videoTransform);
    naturalSize = CGSizeMake(fabs(naturalSize.width), fabs(naturalSize.height));


    AVMutableVideoComposition *composition = [AVMutableVideoComposition videoCompositionWithPropertiesOfAsset:asset];
    UIImage *img = [self imageWithImage:[UIImage imageNamed:@"white_image"] convertToSize:naturalSize];
    [self addOverlayImage:img ToVideo:composition inSize:naturalSize];


    AVMutableVideoCompositionInstruction *instruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
    instruction.timeRange = range;
    composition.instructions = @[instruction];


    AVAssetExportSession *_assetExport = [[AVAssetExportSession alloc] initWithAsset:mixComposition presetName:AVAssetExportPresetMediumQuality];
    _assetExport.videoComposition = composition;



    NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:[NSString stringWithFormat:@"exported-%d.mov", arc4random() % 100000]];
    unlink([exportPath UTF8String]);
    NSURL *exportUrl = [NSURL fileURLWithPath:exportPath];

    _assetExport.outputFileType = AVFileTypeQuickTimeMovie;
    _assetExport.outputURL = exportUrl;
    _assetExport.shouldOptimizeForNetworkUse = YES;

    [_assetExport exportAsynchronouslyWithCompletionHandler:^{

        switch (_assetExport.status) {

            case AVAssetExportSessionStatusFailed:
                 break;

            case AVAssetExportSessionStatusExporting:
                break;

            case AVAssetExportSessionStatusCompleted:{

                dispatch_async(dispatch_get_main_queue(), ^{
                    NSLog(@"Successful!!!");
                    AVAsset *finalAsset = [AVAsset assetWithURL:_assetExport.outputURL];
                    completionBlock(finalAsset);
                });
            }
                break;

            default:
                break;
        }
    }];
}

Now there is a video asset with an overlay image. Only thing is remain to combine original video and the exported video asset. Exported asset should be bottom layer and original should be top layer.