Google App Engine Projection Costs more than 1

Question

I am trying to reduce the datastore cost by using Projection. I have read that a Projection Query costs only 1 Read Operation but in my case the Projection cost goes more than 1. Here is the code:

      Query<Finders> q = ofy().load().type(Finders.class).project("Password","Country");
      for(Finders finder:q)
      {
          resp.getWriter().println(finder.getCountry()+" "+finder.getPassword());
      }

On executing this, the q object contains 6 items and to retrieve these 6 items it takes 6 Read operations as shown in Appstats. Can anyone tell me what's wrong over here ?

I know nothing about objectify, but I believe you can either fetch the results as a list or set a chunking (batching) rate in the query. You may find that iterating fetches each result as individual operations. — Tim Hoffman
@TimHoffman My main task is to fetch all items with a single read operation.. so that's why I wanted to use projection query. — Nayan Jain

Alex Martelli Alex Martelli · Accepted Answer · 2015-12-18T03:57:57

To read all items (with a single read operation if they will all fit) call .list() on the query, to get a List<Finders>. You chose to iterate on the query instead, and that's quite likely to not rely on a single, possibly huge read from the datastore, but parcel things out more.

Where projections enter the picture is quite different: if you have entities with many fields, or some fields that are very large, and in a certain case you know you need only a certain subset of fields (esp. if it's one not requiring "some fields that are very large"), then a projection is a very wise idea because it avoids reading stuff you don't need.

That makes it more likely that a certain fetch of (e.g) 10 entities will take a single datastore read -- there are byte limits on how much can come from a single datastore read, so, if by carefully picking and choosing the fields you actually require, you're reading only (say) 10k per entity, rather than (say) 500k per entity, then clearly you may well need fewer reads from the datastore.

But if you don't do one big massive read with .list(), but an entity-by-entity read by iteration, then most likely you'll still get multiple reads -- essentially, by iterating, you've said you want that!-)

Google App Engine Projection Costs more than 1

1 Answers