Retrofit + OkHTTP - response cache not working

Question

I know there has been a lot of similar questions, but I have read them all and none of them really helped.

So, here is my problem:

I am using retrofit + okhttp to fetch some data from API and I'd like to cache them. Unfortunately, I don't have admin access to the API server so I can't modify headers returned by the server. (currently, server returns Cache-control: private)

So I decided to use okhttp header spoofing to insert appropriate cache headers. Sadly, no matter what I do, caching doesn't seem to work.

I initialise the api service like this:

int cacheSize = 10 * 1024 * 1024; // 10 MiB
File cacheFile = new File(context.getCacheDir(), "thumbs");
final Cache cache = new Cache(cacheFile, cacheSize);

OkHttpClient client = new OkHttpClient();
client.setCache(cache);
client.interceptors().add(new Interceptor() {
    @Override
    public Response intercept(Chain chain) throws IOException {
        Response originalResponse = chain.proceed(chain.request());
        return originalResponse.newBuilder()
                .removeHeader("Access-Control-Allow-Origin")
                .removeHeader("Vary")
                .removeHeader("Age")
                .removeHeader("Via")
                .removeHeader("C3-Request")
                .removeHeader("C3-Domain")
                .removeHeader("C3-Date")
                .removeHeader("C3-Hostname")
                .removeHeader("C3-Cache-Control")
                .removeHeader("X-Varnish-back")
                .removeHeader("X-Varnish")
                .removeHeader("X-Cache")
                .removeHeader("X-Cache-Hit")
                .removeHeader("X-Varnish-front")
                .removeHeader("Connection")
                .removeHeader("Accept-Ranges")
                .removeHeader("Transfer-Encoding")
                .header("Cache-Control", "public, max-age=60")
              //.header("Expires", "Mon, 27 Apr 2015 08:15:14 GMT")
                .build();
    }
});

RestAdapter restAdapter = new RestAdapter.Builder()
    .setEndpoint(API_ROOT)
    .setLogLevel(RestAdapter.LogLevel.HEADERS_AND_ARGS)
    .setClient(new OkClient(client))
    .setConverter(new SimpleXMLConverter(false))
    .setRequestInterceptor(new RequestInterceptor() {
        @Override
        public void intercept(RequestFacade request) {
            if (Network.isConnected(context)) {
                int maxAge = 60; // read from cache for 2 minutes
                request.addHeader("Cache-Control", "public, max-age=" + maxAge);
            } else {
                int maxStale = 60 * 60 * 24 * 28; // tolerate 4-weeks stale
                request.addHeader("Cache-Control",
                    "public, only-if-cached, max-stale=" + maxStale);
            }
        }
    })
    .build();
api = restAdapter.create(ApiService.class);

Of course, it's not necessary to remove all these headers, but I wanted to make the response as clean as possible to rule out some interference from these extra headers.

As you can see, I tried to also spoof Expires and Date header (I tried removing them, setting them so that there is exactly max-age differnece between them and also setting Expires far into future). I also experimented with various Cache-control values, but no luck.

I made sure the cacheFile exists, isDirectory and is writeable by the application.

These are the request and response headers as logged directly by retrofit:

Request:
Cache-Control: public, max-age=60
---> END HTTP (no body)

Response:
Date: Mon, 27 Apr 2015 08:41:10 GMT
Server: Apache/2.2.22 (Ubuntu)
Expires: Mon, 27 Apr 2015 08:46:10 GMT
Content-Type: text/xml; charset=UTF-8
OkHttp-Selected-Protocol: http/1.1
OkHttp-Sent-Millis: 1430124070000
OkHttp-Received-Millis: 1430124070040
Cache-Control: public, max-age=60
<--- END HTTP (-1-byte body)
<--- BODY: ...

And, finally one strange incident: At some point, the cache worked for a few minutes. I was getting reasonable hit counts, even offline requests returned cached values. (It happened while using the exact setting posted here) But when I restarted the app, everything was back to "normal" (constant hit count 0).

Co if anyone has any idea what could be the problem here, I'd be really glad for any help :)

Answer 1

Use networkInterceptors() instead of interceptors(). That in combination with your strategy of removing any headers that are somewhat related to caching will work. That's the short answer.

When you use interceptors to change headers it does not make any adjustments before CacheStrategy.isCacheable() is called. It's worthwhile to look at the CacheStrategy and CacheControl classes to see how OKHttp handles cache-related headers. It's also worthwhile to do ctrl+f "cache" on http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

I am not sure if the networkInterceptors() and interceptors() documentation is just unclear or if there is a bug. Once I look into that more, I will update this answer.

Answer 2

One more thing to add here, Apart from Brendan Weinstein's answer just to confirm OkHttp3 cache will not work with post requests.

Answer 3

After a full day, I found that my offline caching was not working just because I was using POST in the API type. The moment I changed it to GET, it worked!

@GET("/ws/audioInactive.php")
Call<List<GetAudioEntity>> getAudios();

My entire Retrofit class.

import android.util.Log;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.limnet.iatia.App;
import com.limnet.iatia.netio.entity.registration.APIInterfaceProviderIMPL;

import java.io.File;
import java.io.IOException;
import java.util.concurrent.TimeUnit;

import okhttp3.Cache;
import okhttp3.CacheControl;
import okhttp3.Interceptor;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.Response;
import okhttp3.logging.HttpLoggingInterceptor;
import retrofit2.Retrofit;
import retrofit2.converter.gson.GsonConverterFactory;

public class RHTRetroClient {

    public static final String BASE_URL = "https://abc.pro";
    private static Retrofit retrofit = null;
    private static RHTRetroClient mInstance;

    private static final long cacheSize = 10 * 1024 * 1024; // 10 MB
    public static final String HEADER_CACHE_CONTROL = "Cache-Control";
    public static final String HEADER_PRAGMA = "Pragma";


    private RHTRetroClient() {
        Gson gson = new GsonBuilder()
                .setLenient()
                .create();
        HttpLoggingInterceptor interceptor = new HttpLoggingInterceptor();

        Cache cache = new Cache(new File(App.getAppContext().getCacheDir(), "soundbites"),cacheSize);

        OkHttpClient client = new OkHttpClient.Builder()
                .cache(cache)
                .addInterceptor(httpLoggingInterceptor()) // used if network off OR on
                .addNetworkInterceptor(networkInterceptor()) // only used when network is on
                .addInterceptor(offlineInterceptor())
                .build();

        interceptor.setLevel(HttpLoggingInterceptor.Level.BODY);

        retrofit = new Retrofit.Builder()
                .baseUrl(BASE_URL)
                .client(client)
                .addConverterFactory(GsonConverterFactory.create(gson))
                .build();
    }

    /**
     * This interceptor will be called both if the network is available and if the network is not available
     *
     * @return
     */
    private static Interceptor offlineInterceptor() {
        return new Interceptor() {
            @Override
            public Response intercept(Chain chain) throws IOException {
                Log.d("rht", "offline interceptor: called.");
                Request request = chain.request();

                // prevent caching when network is on. For that we use the "networkInterceptor"
                if (!App.hasNetwork()) {
                    CacheControl cacheControl = new CacheControl.Builder()
                            .maxStale(7, TimeUnit.DAYS)
                            .build();

                    request = request.newBuilder()
                            .removeHeader(HEADER_PRAGMA)
                            .removeHeader(HEADER_CACHE_CONTROL)
                            .cacheControl(cacheControl)
                            .build();
                }

                return chain.proceed(request);
            }
        };
    }

    /**
     * This interceptor will be called ONLY if the network is available
     *
     * @return
     */
    private static Interceptor networkInterceptor() {
        return new Interceptor() {
            @Override
            public Response intercept(Chain chain) throws IOException {
                Log.d("rht", "network interceptor: called.");

                Response response = chain.proceed(chain.request());

                CacheControl cacheControl = new CacheControl.Builder()
                        .maxAge(5, TimeUnit.SECONDS)
                        .build();

                return response.newBuilder()
                        .removeHeader(HEADER_PRAGMA)
                        .removeHeader(HEADER_CACHE_CONTROL)
                        .header(HEADER_CACHE_CONTROL, cacheControl.toString())
                        .build();
            }
        };
    }

    private static HttpLoggingInterceptor httpLoggingInterceptor() {
        HttpLoggingInterceptor httpLoggingInterceptor =
                new HttpLoggingInterceptor(new HttpLoggingInterceptor.Logger() {
                    @Override
                    public void log(String message) {
                        Log.d("rht", "log: http log: " + message);
                    }
                });
        httpLoggingInterceptor.setLevel(HttpLoggingInterceptor.Level.BODY);
        return httpLoggingInterceptor;
    }

    public static synchronized RHTRetroClient getInstance() {
        if (mInstance == null) {
            mInstance = new RHTRetroClient();
        }
        return mInstance;
    }

    public APIInterfaceProviderIMPL getAPIInterfaceProvider() {
        return retrofit.create(APIInterfaceProviderIMPL.class);
    }

}

Answer 4

Check if there is a Pragma header in your response. Caching with max-age will not work if Pragma: no-cache header is present.

If it does have Pragma header, remove it by doing the following in your Interceptor:

override fun intercept(chain: Interceptor.Chain): Response {
    val cacheControl = CacheControl.Builder()
        .maxAge(1, TimeUnit.MINUTES)
        .build()

    return originalResponse.newBuilder()
        .header("Cache-Control", cacheControl.toString())
        .removeHeader("Pragma") // Caching doesnt work if this header is not removed
        .build()
}