Datastore & Memcache consistency - Google App Engine & Objectify

Question

I'm having a pretty tough problem ensuring consistency in the datastore. We're trying to do a sync job to BigQuery every 1 minute (cron) and are relying on the Datastore to store a timestamp for when the previous sync was completed.

We are still seeing eventual consistency when the object is loaded and I'm getting to hair-tearing time..

Both the Settings & ParentClass classes are stored in the Datastore as singletons, I.e. only one exists.

@Entity
public class Settings {

    @Parent
    private Key<ParentClass> parent = ParentClass.getKey();

    @Id
    private Long id = 123L;

    ...

    public Settings save(){
        ofy().cache(false).consistency(ReadPolicy.Consistency.STRONG).save().entity(this).now();
        return this;
    }

    public static Settings get(){
        Settings settings = ofy().cache(false).consistency(ReadPolicy.Consistency.STRONG).load().key(Key.create(ParentClass.getKey(), Settings.class, 123L)).now();
        if (settings == null) settings = create();
        return settings;
    }

    private static Settings create(){
        return new Settings().save();
    }

}

Does anyone know what's causing this being eventually consistent?

Edit: web.xml extract:

<filter>
    <filter-name>ObjectifyFilter</filter-name>
    <filter-class>com.googlecode.objectify.ObjectifyFilter</filter-class>
</filter>
<filter-mapping>
    <filter-name>ObjectifyFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

<filter>
    <filter-name>asyncCacheFilter</filter-name>
    <filter-class>com.googlecode.objectify.cache.AsyncCacheFilter</filter-class>
</filter>
<filter-mapping>
    <filter-name>asyncCacheFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

I saw that post there stackoverflow.com/questions/22269411/… Could you be having similar issues? Mind posting your web.xml? — Patrice
Since the get() method is doing a fetch-by-key, it should always have strong consistency. — tx802
@Patrice, I saw that post too, but alas.. Added the objectify filters to the post.. — kengjoran
@tx802 - I agree, which is why I'm completely puzzled by this problem.. I have even renamed the Settings class to Setting after optimising as much as possible, but the problem still persists, sometimes even getting hour old objects which just does not make sense.. — kengjoran
Also worth noting that this persists through flushing the memcache.. — kengjoran

kengjoran kengjoran · Accepted Answer · 2015-02-22T13:35:35

The problem was the internal session cache of Objectify.

Initially resolved by manually clearing the cache using ofy().clear() prior to the first load() and then improved further by upgrading to Objectify 5.1.5.

Datastore & Memcache consistency - Google App Engine & Objectify

1 Answers