GCP Pub/Sub, can you replay old messages on a new Subscription if there is already an active Subscription

Question

In GCP Pub/Sub I have a topic and a Subscription1 and started have publishing messages.

Can I add another subscription Subscription2, and can I replay old messages that were published before the Subscription2 is created, would it allow it? (Kafka allows it)

At what point would I loose access to messages (with in the retention period)?, deleting all the Subscriptions? Keeping at least one active Subscription would allow PubSub to add new Subscriptions and replay old messages?

Can I also increase the retention period beyond 7 days on the topic?

Edit: The messages will be persisted to DB, but I am more interested in a pub/sub architecture

Answer 1

Messages published to a topic are available to subscriptions taken only prior to those messages being published. If you publish a message at 1pm and take a subscription at 2pm, there is no possibility that any messages published prior to to 2pm will be available for that subscription.

I think of Pub/Sub as a transport and delivery technology as opposed to thinking about it as a storage repository. If you think you may need messages that were previously published before a subscription was taken, consider a design pattern where you add a special subscriber that takes each message published and inserts it into a datastore for subsequent retrieval.

If you now find that you need historical messages, you can query that store for historical items while taking a subscription for new items that will subsequently arrive.

Answer 2

It would not be good to rely on being able to replay messages in Subscription2 based on the fact that Subscription1 exists. There is no guarantee provided around messages published prior to the existence of Subscription2. The only exception is that if you were to capture a snapshot on Subscription1 and then seek to that snapshot on Subscription2, then you would see the messages published prior to Subscription2's existence. However, you would only see the messages no older than seven days. Therefore, if the snapshot were more than seven days old, you would see no older messages.

There is no way to extend the retention period beyond 7 days. In general, Kolban's assessment matches the goals of Cloud Pub/Sub: it is a reliable message transport service. The goal of message retention is to ensure that if subscribers are down for a period of time, they will get messages that were sent in that time once they come back up. The same for snapshot and seek: these are particularly useful for the case of restoring from a bad deployment of a subscriber that might have acked messages it should not have. Using seek allows one to replay those messages so they can be processed properly once the subscriber is fixed.