How to update very large firestore collection

Question

I need to add a field to each document of a very large collection in Firestore via the admin sdk for node.js. Some of the documents already have the field so I need to check for its value before setting/updating it. The collection is around 150k documents. Even trying to get the documents with the code below times out.

  const documents = await db.collection('collectionName').get()

Is there a special way to handle very large collections?

Answer 1

You should use pagination in order to avoid loading all the documents into memory with a single query. This will let you process documents in batches by specifying the last document snapshot you saw in the prior query in order to make a new query to get the next page of data.

You will want to place a reasonable limit on each query to make sure you're getting a reasonable number of documents in each page.