Datastore Entity read (keys_only) inside ndb transactions

Question

I have a question regarding the datastore entity reads inside a ndb transaction.

I know that when we read an entity inside an ndb transaction, that specific entity gets locked and no other thread can put/update/write the same entity because it will result in a contention error. That totally makes sense.

However, what happens when we read only the key of an entity instead of the whole entity itself inside the transaction? This can be done by passing keys_only flag as True in ndb.query().fetch() In that case, will the entity again get locked?

Answer 1

The Datastore documentation for Transaction Locks says:

Read-write transactions use reader/writer locks to enforce isolation and serializability.

And it does not mention any situation with the specifics of the use of keys_only during transactions. So I would assume that the same applies to that situation, which does make sense if you consider that you are still making a read never the less, you are just ignoring the data.

That being said, maybe this is something that could be improved into Datastore, or even made clear in the documentation. If you wish, you could consider opening a Feature Request for Google to implement that by following this link.

Answer 2

In general, it's better to think of transactions in terms of their guarantee - serializability - rather than their implementation details - in this case, read/write locks. The implementation details (how queries are executed, locking granularity, exactly what gets locked, etc) can potentially change at any time, while the guarantee will not change.

For this specific question, and assuming the current implementation of Firestore in Datastore Mode: to ensure serializability, a keys-only query in a transaction T1 locks the ranges of index entries examined by the query. If such a query returned a key K for entity E, then an attempt to delete E in a different transaction T2 must remove all of E's index entries, including the one in the range locked by the query. So in this example T1 and T2 require the same locks and one of the two transactions will be delayed or aborted.

Note that there are other ways for T2 to conflict with T1: it could also be creating a new entity that would match T1's query (which would require writing an index entry in the range locked by T1's query).

Finally, if T2 were to update (rather than delete) E in a way that does require any updates to index entries in the range examined by T1's query (e.g., if the query is something like 'select * from X where a = 5' and the update to E does not change the value of it's 'a' property) then T1 and T2 will not conflict (this is an optimisation - behaviour would still be correct if these two transactions did conflict, and in fact for "Datastore Native" databases they can conflict).