JDOPersistenceManagerFactory is taking forever to create PersistencManager

Question

I've created an app on appengine, but am running into an issue. It seems to me that the PersistenceManagerFactory is creating a new connection to the datastore on each request. It is common for my datastore reads to take 4 to 5 seconds for only about 10 objects! (Its a get all entity level query).

After looking at appstats, it looks like the requests are sitting there for about 3 seconds before making the call to the datastore. The actual datastore time is small, 4-10ms.

I then changed my get all query to use the low level DatastoreService api. I create the service once, and use it in the get all method. This pretty much erased the wait time (except for the first time the method is called). So I'm guessing that the PMF is getting a new connection to the datastore for each read! This seems crazy.

Does anyone know how to minimize the spin up time of a PersistenceManager instance?
jdoconfig.xml

   <persistence-manager-factory name="transactions-optional">
       <property name="javax.jdo.PersistenceManagerFactoryClass"
           value="org.datanucleus.api.jdo.JDOPersistenceManagerFactory"/>
       <property name="javax.jdo.option.ConnectionURL" value="appengine"/>
       <property name="javax.jdo.option.NontransactionalRead" value="true"/>
       <property name="javax.jdo.option.NontransactionalWrite" value="true"/>
       <property name="javax.jdo.option.RetainValues" value="true"/>
       <property name="datanucleus.appengine.autoCreateDatastoreTxns" value="true"/>
       <property name="datanucleus.appengine.singletonPMFForName" value="true"/>
   </persistence-manager-factory>

getAll method (using jdo)

    @Override
    public ArrayList<CrudObject> getAll(String className) {
        List results = null;
        PersistenceManager pm = pmf.getPersistenceManager();
        try{
            Query q = pm.newQuery(Class.forName(className));
            results = (List)q.execute();
            return new ArrayList<CrudObject>(pm.detachCopyAll(results));
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        }finally
        {
            pm.close();
        }
        return null;
      }

A PMF doesn't have a connection. A PM has a connection. The create of a PM is trivial. A connection is typically allocated on the first data operation on a PM. The log would tell you what is happening (at DEBUG level), or maybe what is happening up until the datastore taking time — DataNucleus
I'll have to experiment with a few different ways of writing my data service. I was creating a new PersistenceManager and closing it for each read. I was under the impression (from the Google examples) that this was the best practice. I'll put on debug, and try using only one PersistenceManager for the service. — aglassman
Creating and closing a PM is clearly a reasonable usage (as per all DataNucleus docs). Maybe you're hitting AppEngine problems like in groups.google.com/forum/?fromgroups=#!topic/google-appengine/… — DataNucleus
Thanks for the link to the topic. I do believe now after testing, that my delays were due to the latency issue they describe in the forum. I was just unlucky enough for the issues to coincide with my first deploy to app-engine! I'm closing this question. — aglassman

aglassman aglassman · Accepted Answer · 2013-02-04T16:06:59

The issue was due to a latency issue on the AppEngine Datastore platform. The issue was temporary, and has been fixed. The issue just happened to coincide with my first deploy to AppEngine using JDO, so I figured it was my code. I retested today, and it is running much faster.

JDOPersistenceManagerFactory is taking forever to create PersistencManager

1 Answers