How to implement the "locked" feature in AWS/SQS when using Google Cloud Pub/Sub?

Question

When you want to implement a producer/consumer pattern on top of Google Cloud Pub/Sub, you would expect each message can only be processed by one consumer. But Google Cloud Pub/Sub would send each message to all the subscribers.

But the AWS/SQS has the following feature to easy guarantee this:

When a message is received, it becomes “locked” while being processed. This keeps other computers from processing the message simultaneously. If the message processing fails, the lock will expire and the message will be available again. In the case where the application needs more time for processing, the “lock” timeout can be changed dynamically via the ChangeMessageVisibility operation.

How to implement the producer/consumer pattern on top of Google Cloud Pub/Sub? Or there is other alternative Google Cloud product?

Answer 1

Google Cloud Pub/Sub has a similar mechanism. Keep in mind that not every subscriber will receive every message, every subscription will receive every message. The distinction is important. There can be multiple subscribers for a single subscription, pulling messages simultaneously and processing different messages for the same subscription simultaneously.

Within a single subscription, when a message is sent to a subscriber, it will not be sent to another subscriber for that subscription for the ack deadline specified at subscription creation time (default value is 10 seconds). The subscriber that receives the message has until the ack deadline to acknowledge the message or extend the ack deadline, which allows it to take longer to process the message without the message being delivered to other subscribers. Extending the ack deadline is handled automatically in some of the client libraries, e.g., Java and Go, while others require the explicit modification of the ack deadline, e.g., Python. The ack deadline behaves the same way as SQS's visibility timeout.

See message acknowledgement deadline for more information.