App Engine Objectify not creating index on base class field

Question

Let's say I have a simple Entity hierarchy setup where a base (abstract) class has common fields:

@Entity
public abstract class Base {
    @Id
    Long  mId;

    @Index
    Long  mVal;
}

Now I have a subclass (both in Java and in Objectify):

@Subclass(index=true)
public class Concrete extends Base {
    @Index  mOtherVal;
}

When Concrete is saved, it is properly creating the entries in the datastore for all of the fields (mId, mVal and mOtherVal). However, if I attempt to run a query against Concrete entries with a filter on mVal, app engine is complaining that there is no index:

List<Concrete> result = 
    OfyService.ofy().load().type(Concrete.class).filter("mVal > ", 10).list();

I see an exception log like this:

com.google.api.server.spi.SystemService invokeServiceMethod: cause={0} com.google.appengine.api.datastore.DatastoreNeedIndexException: no matching index found. The suggested index for this query is: <datastore-index kind="Post" ancestor="false" source="manual"> <property name="^i" direction="asc"/> <property name="mVal" direction="asc"/> </datastore-index>

I have not created a datastore-indexes.xml intentionally, as the Objectify docs state that indexes on fields are created on the fly. So my question: is this a known limitation of Objectify, or have I done something incorrectly?

To my knowledge, it should be possible but I can see why a composite index is required. Because you're filtering only the subclass (held in the ^i property), the query needs a composite index for the class information (^i) and mVal. My guess is that the Objectify docs are referring to single property indexes, but I'll defer to stickfigure on that! — tx802
Thanks, that was my concern. The docs are a bit vague with regards to this type of composite index. I'll try injecting the composite in the datastore-indexes.xml and see if it changes the behavior. — Larry Schiefer
@LarrySchiefer you don't necessarly need to add it yourself. A good idea to make sure all indexes you need is to run all your queries on your devserver. That'll populate "datastore-indexes.xml" itself (see my full answer here : stackoverflow.com/questions/28424157/… — Patrice
@LarrySchiefer very welcome :) composite indexes are not easy to adapt to. I prefer to not take any chances, run my devserver to create everything, and then looking in datastore-indexes.xml and remove the super big indices that I don't need to be fast (to make use of the zig-zag merge) — Patrice

stickfigure stickfigure · Accepted Answer · 2015-03-06T08:29:15

As you have probably figured out from the comments on the question, you need a multiproperty index in datastore-indexes.xml.

I hope it's fairly obvious why this is necessary - you are requesting a query where mVal is constrained with an inequality and the discriminator index (implemented by Objectify as a property ^i) includes a specific value. If you want to be a little more clever, you can query for ofy().load().type(Base.class).filter("mVal > ", 10) and you won't need the extra index because the simple single-property index suffices.

Multiproperty indexes (maintained by GAE in datastore-indexes.xml) are very different from the single property indexes maintained by Objectify. This is explained at length in the GAE docs.

App Engine Objectify not creating index on base class field

1 Answers