Best practice for creating multiple lightweight Google Cloud Functions?

Question

The way it seems Google Cloud Functions works is by:

your modules go inside a functions directory
that functions directory then contains a package.json file to contain the shared dependencies across all your modules
modules can contain many exported functions per module
google cloud functions and firebase functions have different opinions how to handle exported functions in modules
- for gcp, it seems to work like this: the module is uploaded, and then you specify via the web interface or command line which exported method should be called from the loaded module
- for firebase, it seems to work like this: the listener methods from firebase-functions return the handler, but also attaches to that handler the trigger meta-data. The firebase-tools cli app then requires your code locally, grabs the exported functions, then creates cloud functions for each exported method based on its attached meta-data from the firebase-functions stuff. As such, if you put all your cloud functions in the same module, then it will deploy that module multiple times for each cloud function, and for each cloud function the entire module is loaded, and then the specific exported function is called.
if you configure a exported function to be a http trigger, it uses an undefined version of express.js, and an amorphous amount and order of bundled middlewares

This is strange as:

say even if the modules one.js and two.js require different packages at runtime, the shared package.json between them means that their startup time will be slower than if done individually as they will both need to install all the dependencies of the package rather than just their own
if you have several exported functions inside index.js, such as hi() and hello(), then the hi cloud function will also have the hello() function loaded in memory despite not using it, as well as the hello cloud function will have hi() in memory despite not using it, as for both the resulting cloud function will still use the same index.js file, loading everything inside that module into memory even if other parts aren't needed

As such, what is the best practice for making sure your cloud functions run optimally with the lightest runtime footprint possible? As it seems the design decisions by Google mean that the more cloud functions you make, then the more junk gets bundled with each cloud function, slowing them down and costing more.

As an side, it seems to me that this would have been a better approach for google: Each cloud function should have its own directory, and in each directory there is a package.json file and a index.js file. The index.js file then does a module.exports = function(...args){} or a export default function(...args){}.

This way the architecture aligns with how one expects cloud functions to operate - being that a cloud function represents a single function - rather than a cloud function being the installation of the shared dependencies between your all your cloud functions, then the loading of a module that can contain multiple cloud functions but only one is used, then the execution of only one function out of that loaded module.

Funnily enough, Azure Functions seems be designed exactly the way I expect cloud functions to operate: https://docs.microsoft.com/en-us/azure/azure-functions/functions-reference-node

Simplest approach is to export one function in the index.js file, list only the necessary (top-level) dependencies in your package.json and deploy that cloud function. My thinking is that each cloud function executes in its own sandbox, using only the resources that it needs. Resources can be shared but that does not require a cloud function to load them if they are not needed. — TheAddonDepot

Callam Callam · Accepted Answer · 2017-10-04T19:32:37

Rather than exporting each Cloud Function individually, we can export all of them at once by requiring a file that export's every file in a specific directory and its subdirectories.

'use strict';

const admin = require('firebase-admin');
const functions = require('firebase-functions');
const logging = require('@google-cloud/logging')();
const stripe = require('stripe')(functions.config().stripe.token);

admin.initializeApp(functions.config().firebase);

module.exports = require('./exports')({
    admin, functions, logging, stripe
});

We can create a folder for each provider, e.g. auth, database, https, in which we can organise the associated resource's events such as auth.user.onCreate or database.ref.onWrite.

By structuring our files this way with an event in each file, we can search for the function files and use the file path to dynamically create the Cloud Function's name and export it.

e.g. functions/exports/auth/user/onCreate.f.js -> onCreateAuthUser

'use strict';

module.exports = (app) => {

    const glob = require('glob');

    glob.sync('./**/*.f.js', { cwd: __dirname }).forEach(file => {

        const only = process.env.FUNCTION_NAME;
        const name = concoctFunctionName(file);

        if (only === undefined || only === name) {
            exports[name] = require(file)(app);
        }
    });

    return exports;
}

function concoctFunctionName(file) {

    const camel = require('camelcase');
    const split = file.split('/');
    const event = split.pop().split('.')[0];
    const snake = `${event}_${split.join('_')}`;

    return camel(snake);
}

Finally, our function files can use the argument passed to access commonly required services such as Firebase Admin and Functions, and even Stripe.

'use strict';

module.exports = ({ admin, functions, stripe }) => {

    return functions.auth.user().onCreate(event => {

        const { email, uid } = event.data;
        const ref = admin.database.ref.child(`user-customer/${uid}`);

        return stripe.customers.create({ 
            email, metadata: { uid } 
        }).then(customer => {
            return ref.set(customer);
        });
    });
}

Best practice for creating multiple lightweight Google Cloud Functions?

2 Answers