Unable to get ID of newly-created JDO persistent entity using GAE/J DataNucleus plug-in version 2.1.2

Question

My problem

I am porting my application from version 1.x to 2.0 of the DataNucleus plug-in for GAE/J using the new 1.7.5 GAE/J SDK. This changes my JDO version from 2.3 to 3.0.1. My persistent entity class has a primary key of type encoded string, along with read-only access to the object’s numeric ID. Each instance is the sole member of its entity group (children and parent are linked by numeric ID only).

Previously, I have been able to create and persist a new MyEntity instance and then immediately access its numeric ID to store in the parent MyEntity instance’s list of child IDs.

Now I find that the new instance’s numeric ID is not available immediately after persistence – even though it is generated and stored and is available later.

My question

Is there anything I can do to restore access to the numeric ID immediately following object creation and persistence?

"jdoconfig.xml" configuration extract

<persistence-manager-factory name="big-table">
  <property
   name="javax.jdo.PersistenceManagerFactoryClass"
   value="org.datanucleus.api.jdo.JDOPersistenceManagerFactory"
  />
  <property name="datanucleus.DetachAllOnCommit" value="true"/>
  <property name="javax.jdo.option.NontransactionalRead" value="true"/>
  <property name="javax.jdo.option.NontransactionalWrite" value="true"/>
  <property
   name="datanucleus.appengine.autoCreateDatastoreTxns"
   value="true"
  />
  [...]
</persistence-manager-factory>

Persistent entity class code extract

@PersistenceCapable(identityType = IdentityType.APPLICATION, detachable = "true")
public class MyEntity implements Serializable
{
  private static final long serialVersionUID = 1L;

  // No setter for this read-only data member
  @PrimaryKey
  @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
  @Extension(vendorName="datanucleus", key="gae.encoded-pk", value="true")
  private String sEncodedKey;

  // No setter for this read-only data member
  @Persistent
  @Extension(vendorName="datanucleus", key="gae.pk-id", value="true")
  private Long loID;

  @Persistent
  private Long loParentID;

  //
  // Other persistent data members
  //

  public Long getID()
  {
    return loID;
  }

  //
  // Other getters and setters
  //
}

Persistence code including 3 logging points

/**
 * Create a new entity.
 * @param loParentID
 *   The ID of the entity,
 *   a new child of which is to be created.
 * @param sChildName
 *   The name of the new child to be created.
 * @return
 *   The created entity child,
 *   or <code>null</code> if the operation was carried out unsuccessfully.
 */
public static MyEntity createEntityChild(Long loParentID, String sChildName)
{
  MyEntity meResult = null;
  MyEntity mePersistedChild = null;

  PersistenceManagerFactory pmf =
   DataExchange.getPersistenceManagerFactory();    // My own method
  PersistenceManager pm = pmf.getPersistenceManager();
  Transaction tx = pm.currentTransaction();
  try
  {
    tx.begin();

    MyEntity meChild = new MyEntity();
    meChild.setParentID(loParentID);
    meChild.setName(sChildName);
    meChild.setActive(true);
    mePersistedChild = pm.makePersistent(meChild);

    // "Touch" data member not in the default fetch group
    ArrayList<Long> liChildIDs = mePersistedChild.getChildIDs();
    if (liChildIDs != null)
      liChildIDs.size();

    if (mePersistedChild != null)
      g_logger.log(Level.FINE, String.format(
       "Pre-commit: mePersistedChild.getID() = %d,"
       + " mePersistedChild.getEncodedKey() = \"%s\".",
       mePersistedChild.getID(), mePersistedChild.getEncodedKey()));

    tx.commit();

    if (mePersistedChild != null)
      g_logger.log(Level.FINE, String.format(
       "Post-commit: mePersistedChild.getID() = %d,"
       + " mePersistedChild.getEncodedKey() = \"%s\".",
       mePersistedChild.getID(), mePersistedChild.getEncodedKey()));
  }
  finally
  {
    try
    {
      if (tx.isActive())    // Because of an exception, say
        tx.rollback();
    }
    finally
    {
      pm.close();
    }
  }

  if (mePersistedChild != null)
    g_logger.log(Level.FINE, String.format(
     "Post-pm-close: mePersistedChild.getID() = %d,"
     + " mePersistedChild.getEncodedKey() = \"%s\".",
     mePersistedChild.getID(), mePersistedChild.getEncodedKey()));

  [...]

  return meResult;
}

Dev server logging output

24-Feb-2013 13:28:02 [...].MyEntityBusiness createMyEntityChild
FINE: Pre-commit: mePersistedChild.getID() = null, mePersistedChild.getEncodedKey() = "agttYXJrZXQtdHJlZXISCxIMSXRlbUNhdGVnb3J5GAUM".

24-Feb-2013 13:28:03 [...].MyEntityBusiness createMyEntityChild
FINE: Post-commit: mePersistedChild.getID() = null, mePersistedChild.getEncodedKey() = "agttYXJrZXQtdHJlZXISCxIMSXRlbUNhdGVnb3J5GAUM".

24-Feb-2013 13:28:03 [...].MyEntityBusiness createMyEntityChild
FINE: Post-pm-close: mePersistedChild.getID() = null, mePersistedChild.getEncodedKey() = "agttYXJrZXQtdHJlZXISCxIMSXRlbUNhdGVnb3J5GAUM".

24-Feb-2013 13:28:07 com.google.appengine.api.datastore.dev.LocalDatastoreService$PersistDatastore persist
INFO: Time to persist datastore: 141 ms

JDO enhancement version verification

The build process succeeded with output fragment:

datanucleusenhancer:
09:33:00,531 (main) INFO  [DataNucleus.Enhancer] - DataNucleus Enhancer for API "JDO"
09:33:01,125 (main) INFO  [DataNucleus.Enhancer] - DataNucleus Enhancer (version 3.1.1) : Enhancement of classes
DataNucleus Enhancer (version 3.1.1) : Enhancement of classes
09:33:03,281 (main) INFO  [DataNucleus.Enhancer] - Writing class file "[Path]\MyEntity.class" with enhanced definition
[... (N entries in all)]
09:33:04,046 (main) INFO  [DataNucleus.Enhancer] - DataNucleus Enhancer completed with success for [N] classes. Timings : input=1922 ms, enhance=984 ms, total=2906 ms. Consult the log for full details
DataNucleus Enhancer completed with success for [N] classes. Timings : input=1922 ms, enhance=984 ms, total=2906 ms. Consult the log for full details

Software environment

• Web server: Google App Engine for Java version 1.7.5
• Web framework: Apache Wicket 6.5.0
• Java version: 1.6.0_39; Java HotSpot(TM) Client VM 20.14-b01
• GAE/J DataNucleus plug-in version: 2.1.2
• DataNucleus Access Platform version: 3.1.3
• JDO version: 3.0.1
• Operating system: Microsoft Windows XP version 5.1 running on x86
• IDE: NetBeans 7.2 (build 201207171143)

Answer 1

The GAE JDO plugin only ever sets a "gae.pk-id"/"gae.pk-name" field when it reads in a field marked with that from the datastore (just do a search in SVN trunk, FetchFieldManager is the only place where it's loaded - it doesn't set it when it does a PUT). No idea what it did in 1.x, but all of GAE's own tests pass in 2.x as they did in 1.x. But then that "feature" isn't standard JDO anyway, so of little interest to me.

JDO provides lifecycle listener and you could easily enough set up a postStore callback and set some field in your object in that (and not be reliant on AppEngine-specific "features").

Answer 2

Inspired by @DataNucleus's comments, I have made a work-around in a vaguely similar spirit. The work-around set out below works for me, but I find that my underlying issue remains.

All persistent entities that use a (read-only) numeric ID of an encoded key string will need to have their getID() method changing to use the work-around.

Java code

I amend my ID getter method (given earlier) to as follows:

public Long getID()
{
  Long loResult = DataExchange.getIDFromEKSIfIDIsNull(loID, sEncodedKey);
  return loResult;
}

My DataExchange class has the new method:

import com.google.appengine.api.datastore.Key;
import com.google.appengine.api.datastore.KeyFactory;

/**
 * Get the ID supplied, or get it from the encoded key string if the ID is
 * <code>null</code>.
 * <br/>
 * This method is necessary since JDO version 3.0.1 introduces a long delay
 * between entity first persistence and ID availability using the DataNucleus
 * GAE primary key ID plug-in.
 * @param loID
 *   The persistent entity ID.
 *   This may be <code>null</code> if the entity has been persisted for the
 *   first time but its generation is delayed (a big hello to JDO version
 *   3.0.1).
 * @param sEncodedKey
 *   The persistent entity encoded key string.
 *   This should be not <code>null</code> if the entity has been persisted.
 * @return
 *   <ul>
 *     <li>
 *       If the persistent entity ID supplied is not <code>null</code>
 *       then return it
 *     </li>
 *     <li>
 *       else if the encoded key string is not <code>null</code> then extract
 *       the ID and return it
 *     </li>
 *     <li>
 *       else return <code>null</code>.
 *     </li>
 *   </ul>
 */
public static Long getIDFromEKSIfIDIsNull(Long loID, String sEncodedKey)
{
  Long loResult = null;

  if (loID != null)
    loResult = loID;
  else if (sEncodedKey != null)
  {
    Key key = KeyFactory.stringToKey(sEncodedKey);
    if (key != null)
    {
      long loIDFromEKS = key.getId();
      loResult = Long.valueOf(loIDFromEKS);
    }
  }

  return loResult;
}