GAE datastore how to poll for new items

Question

I use python, ndb and the datastore. My model ("Event") has a property:

created = ndb.DateTimeProperty(auto_now_add=True).

Events gets saved every now and then, sometimes several within one second.

I want to "poll for new events", without getting the same Event twice, and get an empty result if there aren't any new Events. However, polling again might give me new events.

I have seen Cursors, - but I don't know if they can be used somehow to poll for new Events, after having reached the end if the first query? The "next_cursor" is None when I've reached the (current) end of the data.

Keeping the last received "created" DateTime-property and use that for getting the next batch works, but that's only using a resolution of seconds, so the ordering might get screwed up..

Must I create my own transactional, incrementing counter in Event for this?

Bogdan.Nourescu Bogdan.Nourescu · Accepted Answer · 2015-01-30T12:12:06

Yes, using cursors is a valid option. Even tho this link is from the Java documentation, it's valid for python also. The second paragraph is what you are looking for:

An interesting application of cursors is to monitor entities for unseen changes. If the app sets a timestamp property with the current date and time every time an entity changes, the app can use a query sorted by the timestamp property, ascending, with a Datastore cursor to check when entities are moved to the end of the result list. If an entity's timestamp is updated, the query with the cursor returns the updated entity. If no entities were updated since the last time the query was performed, no results are returned, and the cursor does not move.

GAE datastore how to poll for new items

2 Answers