Is transaction really required in a distributed counter?

Question

According to firestore documentation:

a transaction is a set of read and write operations on one or more documents.

Also:

Transactions will fail when the client is offline.

Now the limitation in firestore is that:

In Cloud Firestore, you can only update a single document about once per second, which might be too low for some high-traffic applications.

So using cloud functions and running transactions to increment/decrement counters when the traffic is high will fail.

So they have discussed to use the approach of distributed counters.

According to the algorithm of distrbuted counter:

• create shards
• choose a shard randomly
• run a transaction to increment/decrement the counter
• get all the shards and aggregate the result to show the value of a counter

Scenerio:
consider you have a counter which is to be updated when a document is added and that counter is being displayed in the UI. Now for good UX, I cannot block the UI when network is offline. So I must allow creation/updation of documents even when client is offline and sync these changes once client is online so that everyone else listening to these changes receive the correct value of the counter.

Now transactions fail when the client is offline. So my question for best user experience (even when offline) is:

1. Do you really require a transaction to increment a counter? I know transactions ensure that writes are atomic and are either successful/unsuccessful and prevent partial writes. But what's the point when they fail offline? I was thinking maybe write them to local cache and sync it once the network is back online.

2. Should this be done via client sdks of via cloud functions?

Answer 1

Do you really require a transaction to increment a counter?

Definitely yes! Because we are creating apps that can be used in a multi user environment, transactions are mandatory, so we can provide consistent data.

But what's the point when they fail offline?

When there is a loss of network connectivity (there is no network connection on user device), transactions are not supported for offline use. This is because a transaction absolutely requires round trip communications with server in order to ensure that the code inside the transaction completes successfully. So, transactions can only execute when you are online.

Should this be done via client sdks of via cloud functions?

Please note, that the Firestore SDK for Android has a local cache that's enabled by default. According to the official documentation regarding Firestore offline persistence:

For Android and iOS, offline persistence is enabled by default. To disable persistence, set the PersistenceEnabled option to false.

So all read operations will come from cache if there are no updates on the server. So Firestore provides this feature for handle offline data.

You can also write a function in Cloud Function that will increment the counter while a new document is added or to decrement the conter while a document is deleted.

I also recommend you to take a look:

• How to count the number of documents under a collection in Firestore?

So you may also consider using Firebase realtime database for that. Cloud Firestore and Firebase realtime database work very well together.

Edit:

It allows one to upvote the answer even when the device is offline. After the network is online, it syncs to the server and the counter is updated. Is there a way i can do this in firestore when the device is offline.

This is also happening by default. So if the user tries to add/delete documents while offline, every operation is added to a queue. Once the user regains the connection, every change that is made while offline, will be updated on Firebase servers. With other words, all queries will be commited on the server.

Cloud fnctions are triggered only when the change is received and that can only happen when the device is online.

Yes, that correct. Once the device regains the network connection, the document is added/deleted from the database, moment in which the function fires and increases/decreases the counter.

Edit2:

Suppose I have made around 100 operations offline, will that not put a load on the cloud functions when the device comes online? What's your thought on this?

When offline, pending writes that have not yet been synced to the server are held in a queue. If you do too many write operations without going online to sync them, that queue will grow fast and it will not slow down only the write operations it will also slow down your read operations. So I suggest use this database for its online capabilities.

Regarding Cloud Functions for those 100 offline operations, there will be no issues. Firebase servers work very well with concurent operations.