Confusion about objectify transactions retrying

Question

The objectify documentation states this about transactions:

Work MUST be idempotent. A variety of conditions, including ConcurrentModificationException, can cause a transaction to retry. If you need to limit the number of times a transaction can be tried, use transactNew(int, Work).

However, the google datastore documentation states:

The Datastore API does not automatically retry transactions, but you can add your own logic to retry them, for instance to handle conflicts when another request updates the same entity at the same time.

These statements seem contradictory. Are objectify transactions really retried? Just to be safe I am using transactNew(1, Work) to make sure it runs only once, but what is going on under the hood and why?

The google documentation states that one use of transactions is to do things like transfer money from one account to another. If transactions are retrying then that would not work because by nature transferring money is not idempotent. In this case, using transactNew(1, Work) is the correct thing to do right? Basically, I am looking to do a non-idempotent transaction safely.

Answer 1

Objectify will retry on CME. There is some question as to whether you can get a CME when the transaction actually commits - once upon a time it was documented that this was possible, but Google may have eliminated that.

Nevertheless, the "right way" to ensure a (say) bank transfer completes is not to limit the retries.

1. Create a transaction id outside of the retry. Just some unique value.
2. Start your transaction, attempt to load that transaction id. Does it exist? Your transaction is already complete.
3. If it didn't exist, create your transaction object (with the id) and do the debit + credit.

This ends up being pretty much standard behavior for any bank-like ledger; you create a transaction record along with the debit+credit. If you create a transaction record, it is easy to enforce idempotence.

Answer 2

You are looking at 2 different client libraries:

• the objectify one, which appears to be including automatic retries, customizable
• the plain datastore one which does not include such retries, you'd have to take care of the retries yourself

The money transfer problem isn't necessarily not idempotent, in the sense that it can be made idempotent using transactions. The key is to include both account modifications inside the same transaction, as shown in the datastore client example:

void transferFunds(Key fromKey, Key toKey, long amount) {
  Transaction txn = datastore.newTransaction();
  try {
    List<Entity> entities = txn.fetch(fromKey, toKey);
    Entity from = entities.get(0);
    Entity updatedFrom =
        Entity.newBuilder(from).set("balance", from.getLong("balance") - amount).build();
    Entity to = entities.get(1);
    Entity updatedTo = Entity.newBuilder(to).set("balance", to.getLong("balance") + amount)
        .build();
    txn.put(updatedFrom, updatedTo);
    txn.commit();
  } finally {
    if (txn.isActive()) {
      txn.rollback();
    }
  }
}

This way either both accounts are updated, or none of them is - if the transaction fails all changes are either not committed or rolled back.

FWIW, to verify my (ndb-based) transaction retry logic and idempotence I placed the transactions (with relevant debug messages) in push task queue handlers and triggered multiple tasks simultaneously to cause conflicts. The request and app logs were fairly sufficient for the verifications.