Is it possible to cache HLS segments with AVPlayer?

Question

Root Problem​

Our video buffers a lot when seeking in iOS. It buffers quite a bit more than our web player which saves copies of the already watched segments in temp storage.

​Desired Solution​

Caching the video segments locally on the device's disk. We're fine with caching a single quality and always replaying it.

​Blocker​

We can't find a way to perform caching within AVFoundation/AVPlayer.

What We've Tried

2 ways to intercept networking requests with AVPlayer.

1. Conforming to ​AVAssetResourceLoaderDelegate ​and handling the loading of the media manually

Doesn't work with HLS. You can load the m3u8 files by implementing AVAssetResourceLoaderDelegate​, which allows you to pass authentication or to decrypt the response, however the .ts files can't be loaded. Here's the code we tried: https://gist.github.com/nathanhillyer/84e46152d7c4c88183b6

2. Implementing a NSURLProtocol ​to capture requests for .ts files.

AVURLAsset actually avoids being intercepted. Somehow the network requests just don't get captured. (No clue why)

Answer 1

Let's start with really good news - iOS 10 and up - gives this out of the box. No more need for hacks soon. More details can be found in the following WWDC16 session about whats new in HTTP Live Streaming: https://developer.apple.com/videos/play/wwdc2016/504/

Now back to the current state of things - iOS 9 and lower: With AVPlayer, no. But you can cache HLS segments via a local HTTP server and play the local stream with AVPlayer.

AVPlayer and AVAsset don't contain the necessary information when dealing with HLS playback (It behaves differently than a MP4 static file for example).

TL;DR - You need to use HTTP requests to get the segments and serve them using a local HTTP Server.

A few companies, including the one I'm working for, are using this strategy.

Use a connection to download the segments at the quality you want, rebuild the manifest and flatten it all into one directory and one quality and then use a local http server inside the app to serve it to AVPlayer (AVPlayer can only play HLS streams served over HTTP - not from file assets).

There are edge cases, such as, buffering if you want to play and download in one run, rebuilding the m3u8 manifest correctly, and different AVPlayer states with disk reading.

I've found this out from first hand knowledge, both having such a system in production for 5 years and other video products in the App Store that use the same solution - in total serving many users.

This is also the best solution we've found for android.

Answer 2

Actually, we can get AVPlayer to play a video from network but if you want to to cache the downloaded data to play it locally, with AVPlayer that seems impossible now.

Fortunately, there are a great API is the resourceLoader object in AVURLAsset, which you can provide controlled access to a remote audio file to AVPlayer. This works like a local HTTP proxy but without all the hassles.

You can find more detail on https://gist.github.com/anonymous/83a93746d1ea52e9d23f

Answer 3

Starting with iOS 10, you can use AVFoundation to download and store HLS movies on users' devices while they have access to a fast, reliable network, and watch them later without a network connection.

AVAssetDownloadURLSession

This wwdc2016/504/ session talks about Offline HLS. It is about downloading and persisting assets using AVAssetDownloadURLSession, which is a subclass of URLSession, and here is used to manage AVAssetDownloadTasks. The APIs mentioned in this session are available after iOS10.

AVAggregateAssetDownloadTask

wwdc2017/504 session introduced AVAggregateAssetDownloadTask in iOS11.

An AVAssetDownloadTask used for downloading multiple AVMediaSelections for a single AVAsset, under the umbrella of a single download task.

Apple provides an example project for using AVFoundation to Play and Persist HTTP Live Streams. Demo doc. The demo project uses AVAggregateAssetDownloadTask

AVAssetDownloadStorageManager

/wwdc2017/504 also introduced a new API, AVAssetDownloadStorageManager, to manage the policy for automatic purging of downloaded AVAssets.

• Expiration date
• Priority (important, default)

// Get the singleton
let storageManager = AVAssetDownloadStorageManager.shared()
// Set the policy
let newPolicy = AVMutableAssetDownloadStorageManagementPolicy() 
newPolicy.expirationDate = myExpiryDate
newPolicy.priority = .important 
storageManager.setStorageManagementPolicy(newPolicy, forURL: myDownloadStorageURL)

• Working with HTTP Live Streaming

Answer 4

About NSURLProtocol: As I understood, it make own requests, so your custom tags/fields/marks will be removed.

I've made it other way: redirect segments requests to some custom url scheme and just check the scheme in protocol's canInitWithRequest method.

This way it works just fine. (spent a week to figure whole hls-processing thing out...)