Firebase realtime database transaction and cloud functions

Question

I have a list in the Firebase realtime database, structured as follows:

list: {
  change: {
    rate: integer,
    last: Date
  },
  lock: string,
  elements: {
    ...
  },
  size: integer
}

Clients only add elements to the list. There is a cloud function triggered by onCreate event on elements' child. It transactionaly increments list.size in the database and then performs calculations on elements possibly involving removing all or some children. And there is another cloud function triggered by onDelete event on elements' children which only transactionaly decrements list.size in the database. The issue is that the counter becomes inaccurate after some time. Sometimes it shows more elements that there are in the list, sometimes less. Below is the [adapted] code:

exports.element_add = functions.database.ref('/list/elements/{element}).onCreate(
  (snapshot, context) => {
    const elements = snapshot.ref.parent;

    return new Promise(
      (resolve, reject) => {
        const start = new Date().getTime();

        async.waterfall(
          [
            // read and write to list.change
            // ...
            (next) => {
              // increment list size
              _changeSize(elements.parent, +1, 1, next)
            },
            // acquire lock if required and do calculations on list elements
            // ...
          ],
          // finally
          function (e) {
            // release the lock
            // ...
            // handle errors
            if (e)
              return reject(e);

            resolve();
          }
        );
      });
  });


exports.element_remove = functions.database.ref('/list/elements/{element}).onDelete(
  (snapshot, context) => {
    const elements = snapshot.ref.parent;

    return new Promise(
      function (resolve, reject) {
        _changeSize(elements.parent, -1, 1,
          function (e) {
            if (e)
              return reject(e);

            resolve();
          }
        )
      }
    )
  });

function _changeSize(list, delta, attempt, done) {
  list.child('size').transaction(
    // update
    (size) => {
      if (size === null)
        return null;

      return size + delta;
    },
    // finally
    (e, committed, snapshot) {
      if (e) {
        console.error(`size transaction error: ${e}, retrying in 1 sec`);

        // try again
        return setTimeout(_changeSize, 1000, list, delta, attempt + 1, done);
      }

      console.log(committed ? `new list size is ${snapshot.val()}` : 'list size was not updated');

      // return new size if committed
      done(null, committed ? snapshot.val() : undefined);
    },
    // do not apply locally
    false
  );
}

Except for the two cloud functions above no other entity is working with list.size. The question is why is the value of list.size gets out of sync with list.elements?

Update 2020-02-26: Apparently some of onCreate/onDelete events is being received more than once! We've observed them arriving even 3 times spread between instances of the cloud function. Is it a normal behavior or something we're doing wrong?

You mention a Realtime Database transaction. How do you execute the transaction? Why don't you use the standard transaction mechanism (firebase.google.com/docs/database/web/…)? — Renaud Tarnec
Transaction is actually used in _changeSize(). Please look at the first line there. — faraway

Renaud Tarnec Renaud Tarnec · Accepted Answer · 2020-01-30T11:20:15

You have to use a Realtime Database transaction.

As follows for incrementing:

exports.element_add = functions.database.ref('/list/elements/{element}').onCreate(
  (snapshot, context) => {

    const sizeRef = admin
    .database()
    .ref('/list/size');

    return sizeRef
      .transaction(current_value => {
        return (current_value || 0) + 1;
      });

});

and as follows for decrementing:

exports.element_delete = functions.database.ref('/list/elements/{element}').onDelete(
  (snapshot, context) => {

    const sizeRef = admin
    .database()
    .ref('/list/size');

    return sizeRef
      .transaction(current_value => {
        if (current_value > 0) {
            return current_value - 1;
        } else  {
           //Normally we should never reach this line...
           return null;
        }
      });

});

Firebase realtime database transaction and cloud functions

1 Answers