Paginate with Firestore

Question

I want to paginate my query with Firestore because I have a very large collection (more than 20k documents growing). By pagination I mean displaying an array of numbers to my user, if he clicks on one, it redirects him to a specific chunk of my collection and only fetches this chunk when he clicks. For instance, if I have 100 documents and I want to display 10 per page, I have an array of numbers from 1 to 10. Each time the user clicks, it loads 10 products.

I have implemented infinite scrolling with ease thanks to the startAfter() method with this doc on query cursors.

However, trying to follow the same documentation for pagination I just don't get it. I also tried to follow this tutorial.

This is my code so far:

let filteredProductsCollection = new Promise(function(resolve, reject) {
      productsCollection
        .orderBy("productName")
        .where(
          req.query.where.field,
          req.query.where.comparison,
          req.query.where.value
        )
        .startAfter(
          req.query.lastProduct
        )
        .limit(req.query.limit)
        .get()
        .then(collection => resolve(collection))
        .catch(err => reject(err));
}

Then I take the promise and save the last document

.then(snapshot => {
  let snapshots = {
    last
  };
  snapshot.forEach(doc => {
    snapshots[doc.id] = {
      ...doc.data(),
      id: doc.id
    };
  });
  res.status(200).send(data)
})

This fetch the next chunk of products considering the current chunk (saved in lastProduct), allowing me to make an infinite scrolling that is working perfectly.

My main problem is that I don't have any clue how to jump from that to a pagination feature. Two main issues:

I don't know how many documents I have in total in my query so I can't display an array of numbers to my user
Even if I had this array, how would I retrieve a specific chunk that is not directly next to the previous chunk that has been loaded? Example, how to query page 8 if my user is currently on page 1?

It seems that all the tutorials are directly aimed to describe how to do pagination but I can't get it. Is my definition of pagination wrong? Or are they calling this feature pagination when it is really a 'get next chunk of data' feature...?

Frank van Puffelen Frank van Puffelen · Accepted Answer · 2019-04-21T20:56:58

I don't know how many documents I have in total in my query so I can't display an array of numbers to my user

There is no count query in the Firestore API. If you need to know the total number of documents, you'll need to either load all documents, or keep a counter when you perform updates. Since pagination is meant to prevent loading all documents, I'd have a look at the documentation on distributed counters.

Even if I had this array, how would I retrieve a specific chunk that is not directly next to the previous chunk that has been loaded? Example, how to query page 8 if my user is currently on page 1?

The Firestore API in general is not ideal for offset-based pagination. It lends itself much more to an infinite-scroll approach to on-demand data loading. If you want to implement pagination, you could consider using a server-side SDK, which do support offset-based queries. But even these are probably not what you need, since you get charged for all documents that the offset 'skips'.

The only workaround I know is to keep a list of the key (and possibly the other fields you sorted on) for each page you've read. Or to keep a separate collection with all the keys (as you can fit quite a few of those in the 1MB limit on document size). You might also want to have a look at the FirestorePagingAdapter implementation of FirebaseUI, which implements pagination already.

Paginate with Firestore

1 Answers