no matching index found - GAE

Question

I'm deploying a little backend with some methods. One of them makes a simple query to retrieve a list of objects. This is the method:

@ApiMethod(path = "getMessagesByCity", name = "getMessagesByCity", httpMethod = ApiMethod.HttpMethod.POST)
    public MessageResponse getMessagesByCity(@Named("City_id") Long city) {
        MessageResponse response = new MessageResponse();
        List<Message> message = ofy().load().type(Message.class).filter("city", city).list();
        response.response = 200;
        return response;
    }

And this is the Message class:

@Entity
public class Message {
    @Id 
    private Long id;
    private String name;
    @Index
    private Long city;
    ...
}

I've read a lot of posts and all of them are mentioning that probably is caused because datastore-indexes.xml are not being updated automatically. However, Google doc says this (https://cloud.google.com/appengine/docs/standard/python/config/indexconfig):

Every Cloud Datastore query made by an application needs a corresponding index. Indexes for simple queries, such as queries over a single property, are created automatically.

So,following that, I think that index related files are not necessary for me.

If I execute the method "getMessagesByCity" with the simple query:

List<Message> message = ofy().load().type(Message.class).filter("city", city).list();

The backend returns me an error 503 with this log message:

"com.google.appengine.api.datastore.DatastoreNeedIndexException: no matching index found. An index is missing but we are unable to tell you which one due to a bug in the App Engine SDK. If your query only contains equality filters you most likely need a composite index on all the properties referenced in those filters."

Any idea? How can I solve it?

Answer 1

You need to upload index configs in, so Datastore will start to accept your queries with custom projections with this command.

gcloud app deploy index.yaml

See https://cloud.google.com/datastore/docs/concepts/indexes for more information about Datastore queries handling and indexes.

Answer 2

Every time you use a new datastore query in your code with a different set of filters / orders etc. your index.yaml should automatically update, (might need to run that logic at least once in the local dev server for it to add the new index to the file)

On local dev, the first time you hit it, it should work, HOWEVER when deploying new indexes, there is a lag time before it becomes available in production / on the appspot server. We have run into this a lot, and from the google console you can actually see if its in progress or not, by going to Datastore > Indexes (https://console.cloud.google.com/datastore/indexes) for the project in question.

If all indexes have a green tick and the issue persists then this is not the issue, and can debug further, however if some have spinners next to them this means this index is still being made and cannot be used until its finished.

If this is your problem, you can avoid it in the future by deploying the index.yaml first through gcloud and then only deploying your application.

alternatively make sure you have run the new method / function on your local, and make sure that the index.yaml did in-fact get changed, if you use Git or something the file should have popped up as modified after the local server ran the function / method.