Dynamic indexing in Go app engine datastore

Question

I am working with Google App Engine with GoLang. I am using datastore to keep data. On my app the no of requests on datastore was high, so explored on the solutions and implement sharding.

For each entity, I kept 5 shards and while inserting I got a no between 0-4 and insert in the entityname_no. While reading the data, I read it from all the four entities and process.

The problem I faced is I had two get queries, in the first query I had two filters(String, bool) and in the second query(String, bool and date) I had three filters. The first query is running fine but in second query it is expecting the index for all the 5 entities where in the index.yaml the index was related to the entity only.

Is there a way to add dynamic indexing without udating the index.yaml every time?

Answer 1

Unfortunately, at this moment there is no way to add dynamic indexes. The documentations says [1]: “ These indexes are updated whenever an entity changes, so the results can be returned quickly when the app makes a query. To do this, the datastore needs to know in advance which queries the application will make. ”, So the indexes needs to be set previously and aren’t dynamic.

[1] https://cloud.google.com/appengine/docs/flexible/go/configuring-datastore-indexes-with-index-yaml