Has the transaction behavior changed when a conflict occurred in firestore datastore?

2
votes

I created new Google Cloud Platform project and Datastore.

Datastore was created as "Firestore in Datastore mode".

But, I think Firestore Datastore and Old Datastore behave differently if Conflict occurred.

e.g Following case.

procA: -> enter transaction -> get -> put -----------------> exit transaction
procB: -----> enter transaction -> get -> put -> exit transaction

Old Datastore;

  • procB is Completed and data is updated.
  • procA Conflict occured and data is rollbacked.

Firestore in Datastore mode;

  • procB is waited before exit transaction until procA is completed.Then Conflict occured.
  • procA Completed and data is updated.

Is it spec? I cannot find document on Google Cloud Platform documentation.

2
google-cloud-platformgoogle-cloud-firestoregoogle-cloud-datastore
@DanCornilescu The post you talked me is about "Conflict occure in spite of entityGr isn't same Gr in ndb".I think it is another topic.My result is using google-client library.Conflict is not occure if entityGr is not sameGr. – stack_user
Exactly. There shouldn't be a conflict, yet reported behavior appears as if it there would be one. The explanation, once found, might apply to your case as well. – Dan Cornilescu
Are you using the python2 ndb client or cloud ndb? – Dan Cornilescu
Im using python client library.And GUI GCP console seems same behavior, I think.(get & put & sleep within transaction from client library, Then GUI edit is waiting.Old datastore is not wait.) – stack_user
Good hint from GUI waiting - I think this indicates the behaviour change is from the datastore side, not the client side, so unrelated to the other post. – Dan Cornilescu

2 Answers

2
votes

I've been giving it some thought and I think the change may actually be intentional.

In the old behaviour that you describe basically the shorter transaction, even if it starts after the longer does, is successful, preempting the longer one and causing it to fail and be re-tried. Effectively this give priority to the shorter transactions.

But imagine that you have a peak of activity with a bunch of shorter transactions - they will keep preempting the longer one(s) which will keep being re-tried until eventually reaching the maximum retries limit and failing permanently. Also increasing the datastore contention in the process, due to the retries. I actually hit such scenario in my transaction-heavy app, I had to adjust my algorithms to work around it.

By contrast the new behaviour gives all transactions a fair chance of success, regardless of their duration or the level of activity - no priority handling. It's true, at a price - the shorter transactions started after the longer ones and overlapping them will take overall longer. IMHO the new behaviour is preferable to the older one.

-1
votes

I would recommend you to take a look at the documentation Transactions and batched writes. On this documentation you will be able to find more information and examples on how to perform transactions with Firestore.

On it, you will find more clarification on the get(), set(),update(), and delete() operations.

I can highlight the following from the documentation for you, that is very important for you to notice when working with transactions:

  • Read operations must come before write operations.
  • A function calling a transaction (transaction function) might run more than once if a concurrent edit affects a document that the transaction reads.
  • Transaction functions should not directly modify application state.
  • Transactions will fail when the client is offline.

Let me know if the information helped you!