Firebase Cloud Functions causing a huge spike in Firestore Reads?

Question

I wrote a Firebase cloud function to get data from external api and read/write to Firestore. I ran the function locally with 'firebase serve' several times. Then I noticed for each hour ~500 Firestore API Calls were made, even when the function is not invoked. And Firestore Writes and Deletes remains 0, only Reads increased.

The spike in reads stopped only when I shut down my computer so I believe the cloud function is the cause.

const functions = require('firebase-functions');
const admin = require('firebase-admin');
const axios = require('axios');

admin.initializeApp();

exports.someFunc = functions.https.onRequest((request, response) => {
  const url = 'https://someurl.com';

  const getData = async url => {
    try {
      const response = await axios.get(url);
      const data = response.data;
      if (data.status === 'ok') {
        data.articles.forEach(element => {
          if (element.author) {
            element.author = element.author.replace(/\//g, ',');

            var authorRef = admin
              .firestore()
              .collection('authors')
              .doc(element.author);

            authorRef
              .get()
              .then(doc => {
                if (!doc.exists) {
                  authorRef.set({
                    sources: [element.source.name]
                  });
                } else {
                  var sources = doc.data().sources;
                  if (!sources.includes(element.source.name)) {
                    sources.push(element.source.name);
                    authorRef.update({
                      sources: sources
                    });
                  }
                }
                return null;
              })
              .catch(error => {
                console.log('Error getting document:', error);
              });
          }

          if (element.source.name) {
            element.source.name = element.source.name.replace(/\//g, ',');

            var sourceRef = admin
              .firestore()
              .collection('sources')
              .doc(element.source.name);

            sourceRef
              .get()
              .then(doc => {
                if (!doc.exists) {
                  var sourceUrl = 'https://' + element.url.split('/')[2];
                  sourceRef.set({
                    sourceUrl: sourceUrl
                  });
                }
                return null;
              })
              .catch(error => {
                console.log('Error getting document:', error);
              });
          }

          admin
            .firestore()
            .collection('articles')
            .add(element);
        });
      }
    } catch (error) {
      console.log(error);
    }
  };

  getData(url);
  response.send('Success!');
});

'authors', 'sources' and 'articles' collection only have at most 20 documents each so there must be something repeatedly calling the Firestore that I could not found.

do you also have a cloud function with Db triggers? and can you post your cloud function usage details — Mahi Tej Gvp
@MahiTejGvp no that's the only function I have. Here are the details imgur.com/a/9vC8oQP — Lan Vu
It shows exactly the number of times I invoked it. Since I deployed the function it only shows 1. — Lan Vu

Mahi Tej Gvp Mahi Tej Gvp · Accepted Answer · 2019-04-12T07:40:22

Although this looks like a pricing issue I think this could be an issue of Cloud Functions Retries moreover your usage numbers are not matching to the code because you have Set and Update Methods Which would make at least one write. You might want to check if your functions are executing successfully.

In case you want to understand Firestore pricing this might help.

Firebase Firestore reads are based on many parameters such as

Number of Documents Requested
Real-time updates to the collection if you are listening to changes
Firebase Firestore Security Rules (Every time your rules have to be evaluated)
Number of pages requested if pagination is used

You can find a detailed explanation here

and a few more examples here

Firebase Cloud Functions causing a huge spike in Firestore Reads?

1 Answers