Erratic behaviour of Firebase Cloud Function with Firestore

0
votes

I am experiencing some erratic behaviour of one and the same cloud function. Here is the scenario:

I have a file with around 223000 small database records (around 512 Bytes each) that I divide up into JSON files with an array of 5000 records each and upload them to Firebase Storage via an iOS app.

The upload triggers a cloud functions that reads and parses the JSON file, divides the 5000 records into 10 packages of 500 records and sends them to a Firestore database using batch writes. The next batch write is only triggered after an successfull completion of the prior batched write. If a batched write returns an error, the promise that handles the batched writes fails.

After each successful batch write there is also an update to a document with a transaction to update a record count as well as an update of a branch in a Firebase database.

The 223000 records are correctly divided into 45 files by my app and also correctly uploaded. One file is approximately 2 MBytes of size. The respective cloud function also gets triggered 45 times. But:

  • 31 times the cloud function runs normally as expected and ends after around 30 seconds
  • 7 times the cloud function times out after 60 seconds

  • 5 times the cloud function ends with this error:

    Error: 10 ABORTED: Too much contention on these documents. Please try again. at Object.exports.createStatusError (/user_code/node_modules/firebase-admin/node_modules/grpc/src/common.js:87:15) at Object.onReceiveStatus (/user_code/node_modules/firebase-admin/node_modules/grpc/src/client_interceptors.js:1188:28) at InterceptingListener._callNext (/user_code/node_modules/firebase-admin/node_modules/grpc/src/client_interceptors.js:564:42) at InterceptingListener.onReceiveStatus (/user_code/node_modules/firebase-admin/node_modules/grpc/src/client_interceptors.js:614:8) at callback (/user_code/node_modules/firebase-admin/node_modules/grpc/src/client_interceptors.js:841:24)

  • The database "records" are very similar, so I they are probably not at fault here.

Any ideas? Thanks!

1
google-cloud-firestoregoogle-cloud-functions

1 Answers

2
votes

  1. If there is variance in the amount of time that your function requires to finish, you should increase the timeout of the function in the cloud console to suit the expectations. Default timeout is 60s, which is obviously not long enough here.

  2. The error message is telling you that you're writing too frequently to a single document. Firestore has a limit of 1 write per second per document, and will eventually reject burst writes that exceed the limit. Counters are difficult in Firestore, and may be solved by sharding your counter, or using some other mechanism to retain the count.