Google Cloud Functions can't use Pub/Sub dead-letter topic

3
votes

It seems GCP Cloud Functions always ACK Pubsub PUSH messages when invoked, even if function crashes or fails.

This prevents the use of the new Pub/Sub "dead-letter" topic feature, which required message delivery to fail before forwarding it to a different topic. Presently, messages never fail.

Is there a way to utilize "dead-letter topic" or pubsub re-delivery with Cloud Functions? Apart for setting the "retry" flag on the function itself, that doesn't solve this issue.

2
google-cloud-platformgoogle-cloud-functionsgoogle-cloud-pubsub
You mean that you have a push subscription? And the function error code is 400, 500 or empty in case of crash? – guillaume blaquiere
Yes @guillaumeblaquiere, the Cloud Function is registered as a push subscription (google-defined when setting function trigger to pubsub topic). The error code is 500 I believe, but as it's a "background" type func with pubsub trigger the error is not defined and returned from the function itself, but from google invoking wrapper (as opposed to http-type functions where the user controls return code) – Nihil

2 Answers

1
votes

A push subscription impies 2 things:

  • create your function in --trigger-http mode
  • create a push PUSH subscription that use the Cloud Functions URL.

Here you have created a function in --trigger-topic mode, it's a background function.

You have here what you have to do for marking the function as failed.

Share your code if you need more help

0
votes

One way that this can be handled (until GCP starts supporting manual acknowledgement for cloud function) is by re-queueing to the same topic with the retry count incremented in case of failure. You can store the retry counts in the payload itself like:

{
  "retry_count": 1,
  "data": {...}
}

Also to avoid the a faulty message being retried permanently, you can configure it to push to the dead-letter instead if the retry attempts > x.

This only solved the problem if the topi has only one subscriber though.