Query cursor for app engine in python using query iterator

Question

I want to use pagination not by calling,

greets, next_curs, more = Greeting.query().fetch_page(10, start_cursor=curs)

but by using query iterators. Because that way I can use tasklets to club together all consecutive 'gets' as described at, https://developers.google.com/appengine/docs/python/ndb/async#tasklets

Now in my code things looks like,

class Book(ndb.Model):
    author = ndb.KeyProperty('a', kind=User, required=True)

author_key = ndb.Key(urlsafe=user_id)
books_query=Book.query(Book.author == author_key)
@ndb.tasklet
def callback(book):
     author = yield book.author.get_async()
     raise ndb.Return(book, author)

q_option = {'limit' : 2, 'produce_cursors' : True }
outputs = books_query.map(callback, **q_option)

Not sure where things are going wrong but if I try to do,

books_query.iter().cursor_before()

or

books_query.iter().cursor_after()

It fails with following trace,

raise datastore_errors.BadArgumentError('There is no cursor currently')
BadArgumentError: There is no cursor currently

Any elaborate example of using query iterators and tasklets with cursors would be really helpful to wrap my head around this.

I am suspecting that your map function has something to do with it. developers.google.com/appengine/docs/python/ndb/… — Jimmy Kane
@JimmyKane I get that feeling too, but not able to put my finger on anything specific that's causing the issue. — Harshal Patil
Your problem isn't with your map function. You haven't constructed a cursor any where. Thats why you get the "There is no cursor currently". The map() call supports all query options including a cursor. — Tim Hoffman
@TimHoffman I am not sure if I should be constructing the cursor for the very first page that I am trying to fetch. Shouldn't this api be smart enough to know that if you get 'None' as 'start_cursor' it should just get the results specified by 'limit' and generate cursor for the second page? — Harshal Patil
No because you may not want/need a cursor. Follow the examples for using cursors and construct one and supply one when you create the query. — Tim Hoffman

Josh Josh · Accepted Answer · 2014-01-31T20:36:05

books_query.iter().cursor_after() relies on .iter(**q_options), which creates a QueryIterator with its own parameters.

The problem is that there is no way to access the internal iterator used by .map().

See Guido's answer to a related question on the subject: https://stackoverflow.com/a/14150081/2380615

Query cursor for app engine in python using query iterator

1 Answers