Can Firebase Hosting Serve Cached Data from Cloud Functions?

Question

Let's say I have a database of 100,000 pieces of content inside of firestore. Each piece of content is unlikely to change more than once per month. My single page app, using firebase hosting, uses a function to retrieve the content from firestore, render it to HTML, and return it to the browser.

It's a waste of my firestore quotas and starts to add up to a lot of money if I'm routinely going through this process for content that is not that dynamic.

How can that piece of content be saved as a static .com/path/path/contentpage.html file to be served whenever that exact path and query are requested, rather than going through the firestore / functions process every time?

My goal is to improve speed, and reduce unnecessary firestore requests, knowing each read costs money.

Thanks!

Answer 1

When you use Firebase Hosting on top of Cloud Functions for Firebase, Hosting can act as an edge-cached layer on top of the responses from your HTTPS functions. You can read about that integration in the documentation. In particular, read the section managing cache behavior:

The main tool you'll use to manage cache is the Cache-Control header. By setting it, you can communicate both to the browser and the CDN how long your content should be cached. In your function, you set Cache-Control like so:

res.set('Cache-Control', 'public, max-age=300, s-maxage=600');

Answer 2

On top of setting the Cache-Control header, you can utilize the benefits of global variables setup in your Cloud Functions instances, see Cloud Functions Tips where they mention "Use global variables to reuse objects in future invocations".

With that idea, I am able to use the npm package treasury (yes I did develop this but that is unrelated to the fact that it happens to work with this use case in Cloud Functions - also I use it in production if it makes you feel better).

Example that will utilize the "Memory" adapter of Treasury to store data as long as the variable treasury exists, which lives and dies with the Cloud Function instance:

const functions = require('firebase-functions');
const tauist = require('tauist');
const Treasury = require('treasury');
const cors = require('cors')({
    origin: true
});

// note that memory adapter uses MS not S for ttl
const treasury = new Treasury({
    ttl: tauist.ms.thirtyMinutes
});

function getAnimalData({name}) {
    // replace this with your actual Promise code doing the "real work" that you want to cache
    return new Promise();
}

exports.animal = functions.https.onRequest((req, res) => {
    if (req.method !== 'GET') {
        return res.status(403).send('Forbidden');
    }

    // Enable CORS using the `cors` express middleware.
    return cors(req, res, () => {
        // Reading ticker symbol from URL query parameter.
        let name = (req.query.name || '').trim();
        console.log('animal name:', name);

        if (!name) {
            return res.status(400).send('Bad Request');
        }

        res.set('Cache-Control', `public, max-age=${tauist.s.thirtyMinutes}, s-maxage=${tauist.s.thirtyMinutes}`);
        treasury.invest(getAnimalData, {name})
            .then((data) => res.status(200).send(data))
            .catch((error) => {
                console.error('error caught:', error);
                res.status(500).send('Internal Server Error');
            });

    });
});

Answer 3

This is a good use-case for JAMstack style architecture where you pre-render your pages, and at build-time load the data you need. You can think of pre-rendered static site builders as just another form of caching. In this case, given your expectation of once-per month updates, it doesn't even really make that much sense to be server-rendering your pages at runtime at all.

When your data changes, just rebuild your site. Gastby (in the react world) is setup for this and has many different data sources that plug in to the builder, including a plugin for firestore.

Netlify is a static-site host that has webooks to trigger a rebuild. You can use a firebase cloud function triggered by events to various firestore collections/documents that pings Netlify to run "build" upon inserts/updates/deletes.

Not only is this cheaper, it is less complex than running run-time servers, and provides highest end-user performance as static pages load quickest.

Answer 4

If you need to cache some very expensive results or maybe coming from another API you can use fireStore as a "provider". My operations went from ~30 s to ~1s. You will reduce the usage on your outbound quota

Here is a sample code i created a method in my service:

             ...
    import {db} from "../database";
    const tauist = require('tauist');

    ... 

    /**
     * @param name must be uniq
     * @param fallback
     * @param parameters
     * @param namespace
     * @param ttl milliseconds
     */
    static async getCache(name: string, fallback: (...a: any) => Promise<any>, parameters: any[], ttl: number = tauist.s.thirtyDays, namespace: string = 'api') {
        let response = {};
        const parms = parameters || [];
        const now = new Date();
        const collectionRef = db.collection(`cache-${namespace}-${parms.join('-')}`);
        const documentRef = collectionRef.doc(name);
        try {
            const cacheSnapshot = await documentRef.get();
            if (cacheSnapshot.exists) {
                const cache = cacheSnapshot.data();
                if (new Date(new Date((cache as Cache).created).getTime() + ttl) < now) {
                    throw new Error();
                } else {
                    response = (cache as Cache).data;
                }
                // Using cache
            } else {
                throw new Error();
            }
        } catch (e) {
            // Cache created
            response = await fallback(...parms);
            await documentRef.set({
                data: response,
                created: new Date(new Date().getTime() + ttl)
            })
        }

        return response;
    }

how i use it:

import {INTERNAL_SERVER_ERROR, OK, UNPROCESSABLE_ENTITY} from "http-status-codes";
...
Service.getCache('getHugeData',  AWSService.getHugeData, [name, simple])
                .then((data: any) => {
                    res.status(OK).json({
                        data
                    });
                })
                .catch((error: any) => {
                    console.log(error);
                    res.status(UNPROCESSABLE_ENTITY).json(error);
                });