GAE/J-JDO select by entity type returns outdated result

Question

Investigating a problem for a while now. So asking the community. My aim is to fetch all entities of specific kind and they all need to be up to date. (not taking into account any transaction isolation for now - can be dirty read)

I use GAE/J-JDO for that (use of tx.begin() is just a precaution):

    PersistenceManager pm = PMF.get().getPersistenceManager();
    Transaction tx = pm.currentTransaction();
    Query q = pm.newQuery(TokenJdo.class);
    List<TokenJdo> tokenList = (List<TokenJdo>) q.execute();
    Collection<TokenJdo> res = Collections.unmodifiableCollection(tokenList);
    tx.commit();
    pm.close();
    for (TokenJdo t : tokenList) {
        log.info(t.getToken() + " - " + t.getCounter());
    }

Doing this gives me outdated entities (I request a list, go to AppEngine Console and modify one entity, request a list once again). So I wrote a following debug code using DatastoreService directly.

    DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
    com.google.appengine.api.datastore.Query gaeQuery = new com.google.appengine.api.datastore.Query(TokenJdo.class.getSimpleName());
    PreparedQuery pq = ds.prepare(gaeQuery);
    List<Entity> list = pq.asList(FetchOptions.Builder.withDefaults());
    for (Entity i : list) {
        log.info(i.getKey() + " - " + i.getProperties());
    }

Executing both pieces of code in one request gives following output and shows that data via JDO is outdated compared to what DatastoreService returns:

I 2013-03-20 12:31:16.950 com.serverside.bl.TokenProviderJdo getAll: dummy - -999
I 2013-03-20 12:31:16.950 com.serverside.bl.TokenProviderJdo getAll: asd - 0
I 2013-03-20 12:31:16.951 com.serverside.bl.TokenProviderJdo getAll: dummy-aaa - -111
I 2013-03-20 12:31:16.957 com.serverside.bl.TokenProviderJdo logDSEntities: TokenJdo(7003) - {token=dummy, counter=-999}
I 2013-03-20 12:31:16.957 com.serverside.bl.TokenProviderJdo logDSEntities: TokenJdo(12001) - {token=asd, counter=0}
I 2013-03-20 12:31:16.957 com.serverside.bl.TokenProviderJdo logDSEntities: TokenJdo(13001) - {token=dummy-aaa, counter=-222}

Apparently (or as it looks to me) JDO uses some kind of internal cache (L1 or L2, though I didn't enable any explicitly). How do I make JDO do a 'clean' read?

Muhammad Gelbana Muhammad Gelbana · Accepted Answer · 2013-03-21T00:52:02

To stop datanucleus from caching anything you should disable both caching levels:

datanucleus.cache.level1.type=none
datanucleus.cache.level2.mode=none

Or you can disable caching per Query or per PersistenceManager, just set the mentioned keys using the Query's or PersistenceManager's methods.

On the other hand, you should detach you objects when needing to use them after looking them up. I'm not quite sure about the technical reason for this since the PersistenceManager should be closed before returning from the method using it, or so I learned, but after disabling all caching, you have to fully detach the looked up object before using it.

GAE/J-JDO select by entity type returns outdated result

1 Answers