Concurrently modifying NDB entities inside transactions not working as expected

0
votes

My understanding is that ndb.transactional is used to ensure a function is working on the most current data. I tested the following code in the Interactive Console of my local Google App Engine development server:

from google.appengine.ext import ndb

class UserModel(ndb.Model):

    level = ndb.IntegerProperty(default=0)

@ndb.transactional(retries=0)
def inc_user_lvl(user_key, recurse=True):
    user = user_key.get()
    print(user.level)
    user.level += 1
    if recurse:
        inc_user_lvl(user_key, recurse=False)
    user.put()

user_key = UserModel().put()
inc_user_lvl(user_key)
user = user_key.get()
print(user.level)

The docs say

There is a limit (default 3) to the number of retries attempted; if the transaction still does not succeed, NDB raises TransactionFailedError.

In this case the number of retries is 0 so I expected the user's level to be incremented to 1 and a TransactionFailedError to be raised.

Instead the function calls both succeeded and the second call to inc_user_lvl operated on a user with level 1 (before the first call had put the entity). The user's level was 2 after both calls finished. Why is this the case?

1
pythongoogle-app-enginetransactionsapp-engine-ndb

1 Answers

2
votes

Code written in transactions will use the context cache (thread specific)

From the docs:

Transaction behavior and NDB's caching behavior can combine to confuse you if you don't know what's going on. If you modify an entity inside a transaction but have not yet committed the transaction, then NDB's context cache has the modified value but the underlying datastore still has the unmodified value.

This means that the second call to inc_user_lvl will pull the UserModel entity from the context cache instead of pinging the datastore. You can get around this by setting _use_cache = False on the ndb Model. E.g.

class UserModel(ndb.Model):
    _use_cache = False
    level = ndb.IntegerProperty(default=0)

So now the user's level is 1 after the function calls but there is no Exception raised...

Writes in a transaction don't affect subsequent reads

For some reason this isn't mentioned in the ndb docs. You have to look at superseded version (db) docs:

This consistent snapshot view also extends to reads after writes inside transactions. Unlike with most databases, queries and gets inside a Cloud Datastore transaction do not see the results of previous writes inside that transaction. Specifically, if an entity is modified or deleted within a transaction, a query or get returns the original version of the entity as of the beginning of the transaction, or nothing if the entity did not exist then.

This means that, because the second call to inc_user_lvl is in the transaction of the first call, getting the user entity will return the user as it was at the start of the transaction.

You can use the kwarg propagation=ndb.TransactionOptions.INDEPENDENT in ndb.transactional to start a separate transaction. See the docs for a ful list of options for transactions.

@ndb.transactional(retries=0, propagation=ndb.TransactionOptions.INDEPENDENT)
def inc_user_lvl(user_key, recurse=True):
    user = user_key.get()
    user.level += 1
    if recurse:
        inc_user_lvl(user_key, recurse=False)
    user.put()

This now raises the expected TransactionFailedError.