11
votes

Basically I have created a cloud function(written a Node.js code) which will trigger on the message of cloud pubsub topic and will load that data to Bigquery table.

A message in a topic gets deleted after reading it by cloud function. I understand that subscriber internally sends acknowledgement and result of that, message gets deleted from topic.

I want to control the acknowledgement sent to publisher. How can it be achieved, didn't find any document on this.

3
Are you doing this manually or are you using one of Googels client libraries? The client libraries are designed to return the ack themselves I think your going to have to see if there is a way of turning that off. - DaImTo

3 Answers

6
votes

Google Cloud Functions does not allow you to control the acknowledgement of the Cloud Pub/Sub message. Upon completion of the function, the message is acknowledged for the subscription. If you want finer-grained control over acknowledgements, then you will need to use Google Cloud Pub/Sub directly. There is a Node.js client library.

Just some clarifying notes on acknowledgements: Acknowledging a message for a single subscription doesn't mean the message is deleted for the topic, only for the subscription. Other independent subscriptions will still receive the message and have to acknowledge it. This is also independent of the acknowledgement sent to the publisher. When a Google Cloud Pub/Sub message is published, the publish call is acknowledged (i.e., a response is sent to the publisher) once Google Cloud Pub/Sub has saved the message and guarantees it will be delivered to subscriptions. This is one of the main advantages of an asynchronous message delivery system: receiving the message from the publisher (and acknowledging the publish) is independent of delivering the message via a subscription (which is separately acknowledged by the subscriber).

0
votes

The only way I have found to reliably control what messages get acknowledged and don't in a cloud function is by using the REST Service APIs.

This is because the node.js pubsub client services acknowledgements and manages connections in the background. This is clearly forbidden in a cloud function.

However, the REST API's are fairly easy to use, and give fine grain control over what messages get acknowledged.

0
votes

If I understand correctly; you made a pub/sub topic and placed a cloud function within the same project as this topic. The cloud function is deployed with a google.pubsub.topic.publish trigger for the specified topic.

Since using a queue/topic, producer and consumer operate independently of each other. This enables a loosely coupled architecture, which has its own advantages and disadvantages.

If the publisher publishes a message to the topic, it gets confirmation that the message is sent to the topic successfully. Otherwise your code will give an exception (connection refused, forbidden, etc). For Node.js and other languages, there are pub/sub client sdk's which you can use to publish messages fairly easy.

When the message is on the topic, it will go to the subscribers, which can be push or pull subscriptions. At this point, acknowledgement is getting important. Google pub/sub, as do other queues/topics, are designed with guaranteed delivery. This means if a message could not be delivered, it will try again after some (configurable) time, until the total lifetime is exceeded (default is 7 days)

When using a pull subscription and want to let the topic know that you successfully received the message you would need something like this in Node.js:

message.ack();


When using a push subscription to an API or a HTTP cloud function, you would need to return a custom http code. Pub/sub expects a succes status code (e.g. 200 or 204):

res.status(204); 
res.send('OK');