App Engine JDO datastore query returning stale data – 2 hours old

Question

I'm seeing hours-old stale data in JDO queries, and queries from the App Engine Datastore Viewer control panel:

. . .

Compare that to the following, on the entity edit screen (which presumably uses an ancestor query):

. . .

I changed that value from . to --none-- over an hour ago! I have no idea why this is happening and it's infuriating!

According to Google's GAE JDO documentation, queries should be returning values with strong consistency:

To override the read policy for a single query, call its addExtension() method:

Query q = pm.newQuery(Person.class); q.addExtension("datanucleus.appengine.datastoreReadConsistency", "EVENTUAL");

The possible values are "EVENTUAL" and "STRONG"; the default is "STRONG", unless set otherwise in the configuration file jdoconfig.xml.

Well, here's my jdoconfig.xml - no mention of datastoreReadConsistency so it must be using STRONG right?

<?xml version="1.0" encoding="utf-8"?>
<jdoconfig xmlns="http://java.sun.com/xml/ns/jdo/jdoconfig"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://java.sun.com/xml/ns/jdo/jdoconfig">
    <persistence-manager-factory name="transactions-optional">
        <property name="javax.jdo.PersistenceManagerFactoryClass" value="org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory"/>
        <property name="javax.jdo.option.ConnectionURL" value="appengine"/>
        <property name="javax.jdo.option.NontransactionalRead" value="true"/>
        <property name="javax.jdo.option.NontransactionalWrite" value="true"/>
        <property name="javax.jdo.option.RetainValues" value="true"/>

        <property name="datanucleus.appengine.autoCreateDatastoreTxns" value="true"/>
        <property name="datanucleus.appengine.datastoreEnableXGTransactions" value="true"/>
    </persistence-manager-factory>
</jdoconfig>

What am I supposed to do about this? It's driving me crazy!

Answer 1

Unfortunately, it appears that documentation is wrong or just a little out of date.

When you are using High Replication Datastore, you have to make a trade off between strong and eventually consistent operations. Global queries, like the queries in your question, are eventually consistent.

You noticed this when eventually consistency was particularly painful, but it is by no means unexpected.

For queries that need strong consistency, you need to guarantee this with ancestor queries.

I would recommend this article about eventual consistency.