ConcurrentModificationException when reading from GAE Datastore using JDO

2
votes

I am querying the GAE datastore to retrieve a list of entities (using a dev gae with sdk 1.8.3) :

public List<OT> getAll() {
    PersistenceManager pm = PMF.get().getPersistenceManager();
    Query q = pm.newQuery();
    q.setClass(OT.class);
    try {
        return (List<OT>) q.execute();
    }
    finally {
        q.closeAll();
        pm.close();
    }
}

This is always throwing a ConcurrentModificationException on the pm.close() line :

java.util.ConcurrentModificationException
    at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)
    at java.util.ArrayList$Itr.next(ArrayList.java:791)
    at com.google.appengine.datanucleus.DatastoreConnectionFactoryImpl$DatastoreManagedConnection.close(DatastoreConnectionFactoryImpl.java:146)
    at org.datanucleus.store.connection.ConnectionManagerImpl.closeAllConnections(ConnectionManagerImpl.java:181)
    at org.datanucleus.store.AbstractStoreManager$1.preClose(AbstractStoreManager.java:260)
    at org.datanucleus.ObjectManagerImpl.close(ObjectManagerImpl.java:1112)
    at org.datanucleus.api.jdo.JDOPersistenceManager.internalClose(JDOPersistenceManager.java:359)
    at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.releasePersistenceManager(JDOPersistenceManagerFactory.java:1106)
    at org.datanucleus.api.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java:343)

I am using the following dependencies:

    <dependency>
        <groupId>com.google.appengine.orm</groupId>
        <artifactId>datanucleus-appengine</artifactId>
        <version>2.0.1.1</version>
    </dependency>
    <dependency>
        <groupId>org.datanucleus</groupId>
        <artifactId>datanucleus-core</artifactId>
        <version>3.1.3</version>
    </dependency>
    <dependency>
        <groupId>org.datanucleus</groupId>
        <artifactId>datanucleus-api-jdo</artifactId>
        <version>3.1.3</version>
    </dependency>
    <dependency>
        <groupId>javax.jdo</groupId>
        <artifactId>jdo-api</artifactId>
        <version>[3.0, 4.0)</version>
    </dependency>
    <dependency>
        <groupId>org.datanucleus</groupId>
        <artifactId>datanucleus-rdbms</artifactId>
        <version>3.1.3</version>
        <scope>runtime</scope>
    </dependency>

and the following version of the DN enhancer:

    <dependency>
        <groupId>org.datanucleus</groupId>
        <artifactId>datanucleus-enhancer</artifactId>
        <version>3.1.1</version>
    </dependency>

I am not doing any modifications to any objects from the collection, or to the collection itself. As a matter of fact, the errors occures even when the collection is empty... Any idea what I might be doing wrong?

Additional debug details I have entered debug mode on the pm.close() statement and found out that while iterating on the listeners, one of them is removing himself from the list of listeners:

447468239@qtp-1694835335-71@11277, prio=5, in group 'main', status: 'RUNNING'
  at org.datanucleus.store.connection.ConnectionManagerImpl$1.managedConnectionPostClose(ConnectionManagerImpl.java:247)
  at com.google.appengine.datanucleus.DatastoreConnectionFactoryImpl$DatastoreManagedConnection.close(DatastoreConnectionFactoryImpl.java:147)
  at org.datanucleus.store.connection.ConnectionManagerImpl.closeAllConnections(ConnectionManagerImpl.java:181)
  at org.datanucleus.store.AbstractStoreManager$1.preClose(AbstractStoreManager.java:260)
  at org.datanucleus.ObjectManagerImpl.close(ObjectManagerImpl.java:1112)
  at org.datanucleus.api.jdo.JDOPersistenceManager.internalClose(JDOPersistenceManager.java:359)
  at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.releasePersistenceManager(JDOPersistenceManagerFactory.java:1106)
  at org.datanucleus.api.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java:343)
  at net.mycrub.jacasse.persistence.dao.GenericDAO.getAll(GenericDAO.java:43)

Code at ConnectionManagerImpl.java:247

                    // Remove this listener
                    mconn.removeListener(this);

This is not a listener I have added myself. It was actually a DN listener added during the Query.closeAll() statement. Should I keep either of these 2 "close" statements, are they redundant?

2
javagoogle-app-enginegoogle-cloud-datastorejdodatanucleus
Kind of hard to tell what's going on with the excerpt you've posted. Typically ConcurrentModificationExceptions occur any time an arraylist has more than one iterator iterating over it, regardless of whether they actually modify anything in it. - blgt
Thanks for answering. The thing is it is occuring before I could even try to iterate. - dotvav

2 Answers

1
votes

Have you tried something like this?

public List<OT> getAll()
{
  List<OT> liResult = null;

  PersistenceManager pm = PMF.get().getPersistenceManager();
  Query q = pm.newQuery();
  try
  {
    q.setClass(OT.class);
    liResult = (List<OT>)q.execute();
  }
  finally
  {
    try
    {
      q.closeAll();
    }
    finally
    {
      pm.close();
    }
  }

  return liResult;
}
0
votes

The DatastoreManagedConnection#close() method is simply

public void close() {
    for (ManagedConnectionResourceListener listener : listeners) {
        listener.managedConnectionPreClose();
    }
    // nothing to actually close
    for (ManagedConnectionResourceListener listener : listeners) {
        listener.managedConnectionPostClose();
    }
}

The ConcurrentModificationException cannot come from here directly as nothing in this code modifies the listeners List. Is it possible you have other code running in parallel and adding or removing from listeners, calling addListener or removeListener?