Google Datastore - Not Seeing 1 Write per Second per Entity Group Limitation

Question

I've read a lot about strong vs eventual consistency, using ancestor / entity groups, and the 1 write per second per entity group limitation of Google Datastore.

However, in my testing I have never hit the exception Too much contention on these datastore entities. please try again. and am trying to understand whether I'm misunderstanding these concepts or missing a piece of the puzzle.

I'm creating entities like so:

func usersKey(c appengine.Context) *datastore.Key {
    return datastore.NewKey(c, "User", "default_users", 0, nil)
}

func (a *UserDS) UserCreateOrUpdate(c appengine.Context, user models.User) error {

    key := datastore.NewKey(c, "User", user.UserId, 0, usersKey(c))
    _, err := datastore.Put(c, key, &user)

    return err
}

And then reading them with datastore.Get. I know I won't have issues reading since I'm doing a lookup by key, but if I have a high volume of users creating and updating their information, I would theoretically hit the max of 1 write per second constantly.

To test this, I attempted to create 25 users at once (using the above methods, no batching), yet I don't log any exceptions, which this post implies I should: Google App Engine HRD - what if I exceed the 1 write per second limit for writing to the entity group?

What am I missing? Does the contention only apply to querying, is 25 not a high enough volume, or am I missing something else entirely?

Answer 1

From the documentation:

Writes to a single entity group are serialized by the App Engine datastore, and thus there's a limit on how quickly you can update one entity group. In general, this works out to somewhere between 1 and 5 updates per second; a good guideline is that you should consider rearchitecting if you expect an entity group to have to sustain more than one update per second for an extended period.

Note the words "extended period". 1 update per second is basically a minimum guaranteed throughput. At any given moment you may be able to achieve a significantly higher levels, but Google is warning you not to architect for those levels to be always available.

Answer 2

The limitation is per entity group, that means you could create as many users as you need without limitation (that's where scaling shines), as long as they don't share the same ancestor.

Things change once you start using the user key as the ancestor of other entities, making them part of the same group and thus having a limit on how many changes you can make to it per second.

Btw this is a generalization, most likely you will be able to make ~5 changes per second, this limitation exist because of the transactional properties of an entity group, so there's some kind of table with changes that must be executed sequentially, so you have to lock, and thus there's limited throughput.

Still, rule of thumb is thinking you can only do 1 per second to force yourself think about how to work under this conditions.

And like mentioned, this is only relevant when you update the database, gets and queries should scale as needed.

Answer 3

I don't think you're missing anything here. Previously, I had seen the same limitations when writing to the same entity group but recently (this week, in fact) I have not seen the delays. I'm willing to suggest that Google has solved this problem, and I'm hoping that someone will prove me correct.