Why does the Google App Engine NDB datastore have both "—" and "null" for unkown data?

Question

I recently updated an entity model to include some extra properties, and noticed something odd. For properties that have never been written, the Datastore query page shows a "—", but for ones that I've explicitly set to None in Python, it shows "null".

In SQL, both of those cases would be null. When I query an entity that has both types of unknown properties, they both read as None, which fits with that idea.

So why does the NDB datastore viewer differentiate between "never written" and "set to None", if I can't differentiate between them programatically?

Answer 1

You have to specifically set the value to NULL, otherwise it will not be stored in the Datastore and you see it as missing in the Datastore viewer.

This is an important distinction. NULL values can be indexed, so you can retrieve a list of entities where date of birth, for example, is null. On the other hand, if you do not set a date of birth when it is unknown, there is no way to retrieve a list of entities with date of birth property missing - you'll have to iterate over all entities to find them.

Another distinction is that NULL values take space in the Datastore, while missing values do not.