Google ndb and memcache

0
votes

I am working on a Google app engine application which uses the Python ndb datastore APIs. Since I am trying to optimize the application I am wondering about caching data instead of querying the datastore as described in this link http://blog.initlabs.com/post/16359268329/how-i-reduced-google-app-engine-costs-by-75.

Anyway as far as I understand NDB already transparently uses memcache https://cloud.google.com/appengine/docs/python/ndb/

I see a high amount of read operations and I can't see why as the data are not frequently changed.

is it pointless to explicitly use memcache when using ndb?

Update

Although I refresh the same page I always see something like this in the appstat

 @0ms memcache.Get real=6ms api=0ms cost=0 billed_ops=[]
 @7ms datastore_v3.Get real=12ms api=0ms cost=0 billed_ops=[]
 @22ms datastore_v3.RunQuery real=16ms api=0ms cost=0 billed_ops=[]
 @41ms datastore_v3.RunQuery real=12ms api=0ms cost=0 billed_ops=[]
 @92ms datastore_v3.RunQuery real=71ms api=0ms cost=0 billed_ops=[]

which makes me think the memcache.Get (first Row) failed. Am I right? how can it be?

1
pythongoogle-app-enginegoogle-cloud-datastoregoogle-data-api
how do you see the read ops? what is the response time of those? - Rafael Barros
I see the percentage. I am the only user by now and it reaches 10% in few hours. Anyway I am introducing AppStats to have a bettere measurement - lowcoupling
Try doing some profiling with this: cloud.google.com/appengine/docs/python/tools/appstats - Rafael Barros
Hi, see my update. Thank you - lowcoupling
Fantastic! Can you share the code that provokes that log? is there any call to memcache that you explicitly make? - Rafael Barros

1 Answers

1
votes

I'm unclear on what you're asking, so let me rephrase your question just to be on the safe side :

you're asking why, if you put your data into memcache, can the memcache get fail? Well it's probably because the memcache flushed by itself, as it can randomly do.

If you see that EVERY time, then maybe something is wrong indeed. But the way I see it you made a datastore get that wasn't present in the memcache at first.