GCP Nodejs8 Cloud Function - Synchronous PubSub publish

Question

I'm struggling with a javascript/Nodejs8 Google Cloud Function to publish payloads to Google PubSub.

So I have a Cloud Function triggered by HTTP requests and the request body is then published to a pubsub topic (configured for pull mode).

Here is my code:

const {PubSub} = require('@google-cloud/pubsub');
const pubsub = new PubSub();
const topic = pubsub.topic('my-fancy-topic');

function formatPubSubMessage(reqObj){
    // the body is pure text
    return Buffer.from(reqObj.body);
};

exports.entryPoint = function validate(req, res) {

topic.publish(formatPubSubMessage(req)).then((messageId) => {
            console.log("sent pubsub message with id :: " + messageId)
        });

res.status(200).json({"res":"OK"});
};

My issue is that the cloud function finishes executing before the pubsub message being published (in logs, the log "Function execution took X ms, finished with status code: 200" shows up around 30 or 40 seconds before the my pubsub log. I also had several times a log with "Ignoring exception from a finished function" and I dont get my pubsub log)

I'm not a javascript or nodejs specialist and I don't master javascript promises neither but I was wondering if I could make the publish synchronous. I'm thinking as well that I might be doing something wrong here !

Thank you in advance for your help.

Kolban Kolban · Accepted Answer · 2019-10-09T14:20:27

In your logic, your callback / event handler function is being called when the HTTP message arrives. You then execute a publish() function. Executing a publish is an asynchronous activity. This means that it make take some time for the publish to complete and since JavaScript (intrinsically) doesn't want to block, it returns immediately with a promise that you can then use to be notified when the asynchronous work has completed. Immediately after executing the publish() your logic executes a res.status(....) which sends a response to the HTTP request and that is indeed the end of the flow request from the HTTP client. The asynchronous publish is still cooking and when it itself completes, then the callback for the publish occurs and you log a response.

Unfortunately, this is not a good practice as documented by Google here ...

https://cloud.google.com/functions/docs/bestpractices/tips#do_not_start_background_activities

In this last story, the function you call validate will still end prior to the publish being completed. If you want to block while the publish() executes (effectively making it synchronous), you can use the JavaScript await key word. Loosely, something like:

try {
   let messageId = await topic.publish(....);
   console.log(...);
catch(e) {
 ...
}

You will also need to flag the functions as being async. For example:

exports.entryPoint = async function validate(req, res) {
   ...

See: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function

You can also simply return a Promise from the function and the callback function will not be considered resolved until the Promise as a whole is resolved.

The bottom line is to study Promises in depth.

GCP Nodejs8 Cloud Function - Synchronous PubSub publish

2 Answers