1
votes

It is well-known that the AppEngine datastore is built on top of Bigtable, which is intrinsically sorted by key. It is also known (somewhat) how the keys are generated by the AppEngine datastore: by "combining" the app id, entity kind, instance path, and unique instance identifier (perhaps through concatenation, see here).

What is not clear is whether a transformation is done on that unique instance identifier before it is stored such as would make sequential keys non-sequential in storage (e.g. if I specify key_name="Test", is "Test" just concatenated at the end of the key without transformation?) Of course it makes sense to preserve the app-ids, entity kinds, and paths as-is to take advantage of locality/key-sorting in the Bigtable (Google's other primary storage technology, F1, works similarly with hierarchical keys), but I don't know about the unique instance identifier.

Can I rely upon key_names being preserved as-is in AppEngine's datastore?

2

2 Answers

4
votes

Keys are composed using a special Protocol Buffer serialization that preserves the natural order of the fields it encodes. That means that yes, two entities with the same kind and parent will have their keys ordered by key name.

Note, though, that the sort order has entity type and parent key first, though, so two entities of different types, or of the same type but with different parent entities, will not appear sequentially even if their keys are sequential.

0
votes

In addition to what @Nick explained:

If you use auto-generated numeric IDs, the legacy system used to be semi-increasing (IDs were assigned in increasing blocks), but with the new system they are pretty scattered.