CloudFunctions with Pubsub : long delay to publish

Question

I have an issue with Pubsub. I have one function and one topic. When I publish on the topic, it takes 2 minutes before publishing the message. Is that normal?

The problem is not present on local (with webstrom / Google Cloud SQK for windows). Publish time on local: 240ms Publish time on google cloud console: 2minutes

Below my code :

var topicName = "projects/XXXXXXXXXXXXXX/topics/ps_checkNiveauBacTampon";
const publisher = pubSubClient.topic(topicName, {
    batching: {
        maxMessages: 100,
        maxMilliseconds: 1,
    },
});

exports.helloPubSub = function(event, context) => {
    var jsonPub = {idBassin: 1};
    var payload = Buffer.from(JSON.stringify(jsonPub));

    // Publishes the message as a string, e.g. "Hello, world!" or JSON.stringify(someObject)
    //const dataBuffer = Buffer.from(JSON.stringify(jsonPub));


    try {
        console.log('In try, before publish');
        const messageId = await publisher.publish(payload);
        console.log('In try, after publish');
    } catch (error) {
        console.error(`Received error while publishing: ${error.message}`);
        process.exitCode = 1;
    }
    console.log('After try');
}

Below the google cloud log :

I've added the Cloud Functions tag to this question in case the issue here is related to Cloud Functions. A 2 minute publish time is unusual, especially since you have set maxMilliseconds to 1 in your batching settings. — Lauren
@guillaumeblaquiere I'm located at Reunion Island. The cloud functions is on europe-west1 — Soufiane Mohamed
It's possible your Cloud Function is resource-constrained, if it's performing any significant computations. Can you try using a higher tier of CPU/Memory for your Function and seeing if that improves performance? If it does not help, please open a case with Google Cloud support so they can help you with your particular Function and Pub/Sub topic. — Lauren

Felix L. Felix L. · Accepted Answer · 2021-05-27T14:45:42

Publishing latency could be rooted at both Pub/Sub or Cloud Functions end.

Regarding Pub/Sub, low throughputs can lead to a cold cache issue. This happens because Pub/Sub was designed for low-latency and high-throughput. Another possibility is that there are implementation issues. You could try decreasing maxMessages and see if this fixes the issue.
Regarding Cloud Functions, it could be the issue that Lauren already said. If the function experiences high CPU or RAM usage, it could cause unexpected delays publishing topics on Pub/Sub. As you state that everything works fine when running on local, maybe you were using higher hardware specifications. If this is the case, you can try increasing CPU and RAM.

CloudFunctions with Pubsub : long delay to publish

1 Answers