Ack Deadline, Message Retention Duration, Dead Lettering and Retry Policy in GCP

Question

I have a few questions related to the above topics in GCP. It would be of great help if someone could explain them in detail. Thank you. I have gone through a few docs but I couldn't find concise answers to them.

My understanding:

1. Ack Deadline: Example, if this feature is set for 10 secs, then it waits for the subscriber to ack the message within 10 secs else after 10 secs it redelivers the message.

question 1: In the case of a push subscriber, pubsub service redelivers/pushes the message to the subscriber again after waiting for 10 secs for the ack deadline to end. In the case of pull message, the subscriber tries to pull the message for the 1st time, as soon as he pulls, 10secs ack deadline clock starts, so during this time if the subscriber tries to pull the message again, will they not receive the messages as the queue will be closed for 10secs?

2. Message Retention Duration: it is by default set for 7 days. All the messages which got delivered to the subscriber but not ack'ed by the subscriber, after certain retry attempts like for example 5, after 5 retries they stay in the topic for 7 days and after 7 days it gets deleted.

Question 2: But will the subscriber be getting these messages in each pull they do on the topic, even after max retries?

3. Dead lettering: Dead letter topic is a topic which you can create to forward the bad/erroneous from the main topic to the dead letter topic.

Question 3: Bad messages here, do they mean the messages which cannot be delivered by the pubsub service to the subscribers or the messages which the subscribers are not able to ack. But in the second case where the subscribers are not able to ack. It can also mean that messages might be good but the subscriber is not acking them. In this case, as message retention is set for 7 days, will they stay in the same topic or if the dead letter is created by the subscription, will it be the responsibility of the pubsub service to forward the messages to the dead letter topic?

4. Retry Policy: There are two options here 1. retry immediately: which when selected the pubsub servcie retries to deliver the message immediately to the subscriber if the subscriber doesn't ack the message before the ack deadline. Second option: Retry using Exponential Backoff: which when selected the pubsub service tries to give a delay before redelivering the message to the subscriber and the max delay it can do is maximum exponential backoff. Question 4: Let us take an example here: Say I set the ack deadline to 10 secs. And set the retry policy to minimum exponential backoff to 30 secs and maximum to 600 seconds. So in this case if the subscriber pulls the message for the first time but doesn't acknowledge it, the ack deadline clock starts and lets say it ends, then if the subscriber pulls it the second time does the pubsub service wait for another 30 seconds (minimum exponential backoff) before it tries to redeliver the message?

Thank you.

Answer 1

We recommend using our supported client libraries, which should extend the ack deadlines automatically. In general terms, please find the answers below:

so during this time if the subscriber tries to pull the message again, will they not receive the messages as the queue will be closed for 10secs

You will not receive the same message again in that time period. You can still pull other messages from the backlog. Please note this is a best effort feature and at times you might receive the message again within the ack deadline.

But will the subscriber be getting these messages in each pull they do on the topic, even after max retries

Where are you configuring the max retries? Cloud Pub/Sub only provides this max delivery attempts as part of dead letter queues. If you have dead letter queues configured, the message will be relayed to the dead letter topic after certain number of retries and removed from the parent subscription. Otherwise, the message will stay on the parent subscription and Cloud Pub/Sub will attempt to deliver it when possible.

Bad messages here, do they mean the messages which cannot be delivered by the pubsub service to the subscribers or the messages which the subscribers are not able to ack. But in the second case where the subscribers are not able to ack. It can also mean that messages might be good but the subscriber is not acking them. In this case, as message retention is set for 7 days, will they stay in the same topic or if the dead letter is created by the subscription, will it be the responsibility of the pubsub service to forward the messages to the dead letter topic?

Bad messages in this context will be the messages which the subscribers are not able to ack. If a subscriber does not want to dead letter a message even if they are not able to process them, they should not be using dead letter queues. Dead letter queues are ideal if you are okay with the message being removed from the subscription in failure scenarios. Messages that expire the 7 day retention window are not moved to the dead letter topic systematically.

Say I set the ack deadline to 10 secs. And set the retry policy to minimum exponential backoff to 30 secs and maximum to 600 seconds. So in this case if the subscriber pulls the message for the first time but doesn't acknowledge it, the ack deadline clock starts and lets say it ends, then if the subscriber pulls it the second time does the pubsub service wait for another 30 seconds (minimum exponential backoff) before it tries to redeliver the message?

That is correct. This answer might be useful as well: GCP PubSub retry backoff timing.

I hope this was helpful.