Using Objectify in long-running background tasks

0
votes

I am planning to use Objectify to talk to Cloud Datastore from GAE Flex. The app is going to be running quite a few background threads talking to Datastore regarding which I have a couple of questions.

  • I am not planning to use any memcache setup and since these threads are going to be running for a long time, I dont want the Session cache to fill up either. I could not find a way to set ofy() to never cache locally too and the only option seems to be to run a clear() operation periodically. Is there a better way to avoid these caches?

  • As I see it, we need to wrap any such invocations of ofy() in a run() block to perform the cleanup. I wanted to confirm that this was the only way to use it outside requests scope and there was no in-built support for these longer contexts.

Thanks

1
google-app-enginegoogle-cloud-datastoreobjectify
if you are not planning to use any cache, why are you worried that the session cache gets full? anyways it is a best effort, free service level - Ggrimaldo
There is a specific mention about this cache being full causing memory issues - it was internally a Map of key-value which means if the same map is being maintained for the entire lifecycle, it would indeed cause issues. I could periodically run a clear() but seems a hack and unnecessary work to do, Hence my question around how to avoid it in the first place given that I dont plan to use it. - noobNeverything

1 Answers

1
votes

You are correct. ObjectifyService.run() is the way to run requests outside of the ObjectifyFilter.

There is not currently any way to disable the session cache. The session cache is pretty deeply woven into the fabric of Objectify in order to get sane behavior for @Load operations. It's not impossible, it just hasn't risen to the top of the priority queue.

The best way to iterate large quantities of your datastore without hitting memory issues is to iterate specifying an explicit chunk() size and then clear() after processing that number of items. If you use Guava's Iterators.partition(), this is pretty much a one-liner.