Google App Engine query optimization

Question

I have a Google App Engine datastore that could have several million records in it and I'm trying to figure out the best way to do a query where I need get records back that match several Strings.

For example, say I have the following Model:

String name String level Int score

I need to return all the records for a given "level" that also match a list of "names". There might be only 1 or 2 names in the name list, but there could be 100.

It's basically a list of high scores ("score") for players ("name") for a given level ("level"). I want to find all the scores for a given "level" for a list of players by "name" to build a high score list that include just your friends.

I could just loop over the list of "names" and do a query for each their high scores for that level, but I don't know if this is the best way. In SQL I could construct a single (complex) query to do this.

Given the size of the datastore, I want to make sure I'm not wasting time running python code that should be done by the query or vise-versa.

The "level" needs to be a String, not an Int since they are not numbered levesl but rather level names, but I don't know if that matters.

Answer 1

players = Player.all().filter('level =', level).order('score')  

names = [name1, name2, name3, ...]

players = [p for p in players if p.name in names]

for player in players:
    print name, print score

is this what you want?
...or am i simplifying too much?

Answer 2

No you can not do that in one pass.

You will have to either query the friends for the level one by one

or

make a friends score entity for each level. Each time the score changes check which friends list he belongs to and update all their lists. Then its just a matter or retrieving that list.

the first one will be slow and the second costly unless optimized.

Answer 3

You can use IN filter operator to match property against a list of values (user names):

scores = Scores.all().filter('level ==', level).filter('user IN', user_list)

Note that under the hood this performs as much queries as there are users in user_list.