Saturday, 21 August 2021

Firestore Concurrent Transactions Freezing

As background, I have a Firestore collection, where each document has a dollar amount and a time. I am creating a function that releases a desired dollar amount from this collection, starting with the oldest documents.

For example, a function call to release $150 will iterate through the collection, removing dollar amounts from the collection until a total of $150 is removed. I do this using a recursive function which 1) finds the oldest dollar amount, 2) removes the inputted amount (i.e. $150) from that number, or deletes the number if the inputted amount is greater than the number, and 3) recurs if there is still a remaining amount to be removed. I use a Firestore transaction for step (2), as there is the possibility this collection will be changed by multiple users at the same time.

The code below updates the collection correctly. However, it takes an extremely long time if it is called while an earlier instance is already running: if I call it once, then call it again before the first call is completed, it freezes and has potential to take 20-30 minutes instead of the usual 1-5 seconds. I am not sure what is causing this freezing.

 function releaseAmountFromStack(amount) {
  return new Promise((resolve, reject) => {
    let db = admin.firestore();
    let stackRef = db.collection("stack");

    stackRef.orderBy("expirationTime", "asc").limit(1)
      .get().then((querySnapshot) => {
        if(querySnapshot.empty) {
          return reject("None left in stack");
        }

        let itemToRelease = querySnapshot.docs[0];

        releaseItem(itemToRelease.ref, amount)
        .then((actualReleaseAmount) => {
          // If there is still more to release, trigger the next recursion
          // If the full amount has been released, return it
          if (amount > actualReleaseAmount) {
            releaseAmountFromStack(amount-actualReleaseAmount)
            .then((nextActualReleaseAmount) => {
              return resolve(actualReleaseAmount + nextActualReleaseAmount);
            })
            .catch(() => {
              return resolve(actualReleaseAmount);
            });
          } else {
            return resolve(actualReleaseAmount);
          }
        });
    });
  });
}

function releaseItem(itemRef, amountToRelease) {
  let db = admin.firestore();
  return db.runTransaction((transaction) => {
    // Get the item again, so that it is part of the transaction, as
    // it is not possible to use a transaction with the original query
    return transaction.get(itemRef).then((itemDoc) => {
      let itemAmount = itemDoc.data().amount;
      let actualReleaseAmount = Math.min(amountToRelease, itemAmount);

      // If item is exhausted, delete it. Else, update amount
      if (actualReleaseAmount >= itemAmount) {
        transaction.delete(itemDoc.ref);
      } else {
        transaction.set(itemDoc.ref, {
          amount: admin.firestore.FieldValue.increment(-1*Number(actualReleaseAmount)),
        }, {merge: true});
      }
      return actualReleaseAmount;
      });
  });
}

Here are some useful facts from the debug process so far. Thank you so much.

  • During the freeze, it does not trigger a breakpoint on any of these lines of code. Only when the freeze finishes is a breakpoint triggered. This indicates the delay is not caused by looping through my code (if it were, a breakpoint should be triggered)
  • The function eventually works as intended in that it releases the correct amount, it just takes a very long time. It will generally freeze, then execute, then freeze, then execute, and so forth until the process is completed
  • Firestore usage stats show the function executes hundreds of reads and writes, even if it only needs to (and I would expect) it to iterate a few dozen times to release the requisite amount from the collection


from Firestore Concurrent Transactions Freezing

No comments:

Post a Comment