GAE datastore index limit

Question

1st, my application, in quota details, the number of indexes get reach to 200 of 200. but when i count in Datastore Indexes, its over 200 indexes. so why i can add more than 200 indexes ?

2nd, in https://cloud.google.com/appengine/docs/quotas Free Default Daily Limit of number of indexes is 200* ( *. not a daily limit but a total limit)

So it's mean when you pay for it you will add more than 200 indexes? is that true ?

3rd, my code : ModelMeta meta = ModelMeta.get(); List modelList = Datastore.query(meta).filter(meta.fieldA.equal("0")) .filter(meta.fieldB.equal("1").asList(); Event i defined index for its or not, every things run OK. i dont understand why?

thanks for help.

Answer 1

1. Indexes on individual properties do not count towards the limit. You don't have to define them anywhere - if a property is indexed, you can use it in filters and you can use it to sort query results.

2. You only need to define composite indexes, and App Engine Java SDK generates them automatically, by the way. I don't remember the last time I had to add an index manually, but we do a lot of testing locally and that helps to auto-generate all the indexes.

3. If you need more than 200 composite indexes, you are almost certainly doing something wrong. 200 is supposed to be a crazy limit that no one ever reaches, even with the most complex data models.

To give you an example, I have an app with 20 different modules, almost a hundred different kinds of entities in very complex relationships, and the total number of entity properties in the thousands. App Engine console shows 24 indexes for this app.

Note that there are many situations where it's cheaper and faster to retrieve all entities and then filter them out, rather than creating many composite indexes. The reason for this is simple: indexes can easily take much more space than the data itself, and the write costs for entities is directly proportionate to the number of indexed properties and composite indexes. Thus, 200 composite indexes on top of individual indexes for each indexed property will most certainly be a major cost and performance problem. There are also other ways to avoid unnecessary indexes by carefully designing your data models.