Is it possible to use both CIFilter and GPUImage filter on AVMutableComposition?

Question

I have an AVMutableComposition with two tracks (one for video and audio each) that combines multiple videos (as segments: AVCompositionTrackSegment) for playback. I have to apply filters on selected videos and not on the whole composition (lets say on 2nd and 4th segment when I have total 4 segments). For example if user selects to apply a filter on video 2 then I apply that filter for the respective duration of composition using AVMutableVideoComposition(asset: AVAsset, applyingCIFiltersWithHandler: (AVAsynchronousCIImageFilteringRequest) -> Void). It is working fine.

Now I want to use some custom filters, created using GPUImage library. So my question is does GPUImage has an API to apply filters on a selected segment or for a specific duration of AVMutableComposition? So that I can use both CIFilter for built-in and GPUImage for custom filters.

Answer 1

Yes that's where you can use AVVideoCompositing ,In a videocomposition you can use custom compositor where you can define every track as a track id and then by this trackId you will get frameBuffer and you can do anything with that framebuffer like you can use gpuImage for custom filters or you can you Metal for creating custom filters. There is a sample project made by apple engineers in this topic you can see it . Links for reference: https://developer.apple.com/documentation/avfoundation/avvideocompositing?language=objc https://github.com/ElfSundae/AVDemo/tree/master/AVCustomEdit