next() method in python: is the definition correct?

Question

with open('some.txt', 'r') as input_file:
    #read the file starting from the second line.
    for line in input_file:
        print line

The code is self-explanatory. However, I didn't know why it worked just by reading the document about the next() method as it was my first time using the method. The official document describes the method as:

next(iterator[, default])

Retrieve the next item from the iterator by calling its next() method. If default is given, it is returned if the iterator is exhausted, otherwise StopIteration is raised.

Isn't the explanation missing the part that when this method is executed, the iterator is moved to the second element of the object, on which the method is called?

You're right that it doesn't explicitly say that the notion of "next" is updated after each retrieval, but as user2357112's answer below suggests, that meaning is supposed to be part of "next". "Next" means "the one after the last one you got". — BrenBarn

abarnert abarnert · Accepted Answer · 2013-09-11T00:51:20

What you're talking about is not really up to the next function, but rather up to how each iterator implements its next method. And, while many iterators do "advance" when you call next, they don't have to.

The prototypical iterator is a virtual "pointer" to a position in some iterable, and their next method returns the value the iterator is currently pointing to, and advances it to point to the next value in the iterable.

For example, consider the iterator returned by calling iter on a list. The first call to next(i) returns element 0, then the next call returns element 1, and so on. You could simulate its behavior like this:

class ListIterator(object):
    def __init__(self, lst):
        self.lst = lst
        self.idx = 0
    def next(self):
        try:
            value = self.lst[self.idx]
        except IndexError:
            raise StopIteration
        self.idx += 1
        return value

And obviously that's intended to be the usual case, hence the name next.

But you can easily create an iterator that doesn't work this way—and not just as some pathological type to prove a point, but actual useful iterators.

For example, look at itertools.repeat. The iterator returned by itertools.repeat(10) just gives you 10 each time you call next on it. You could think of it as an iterator over a list of infinite length whose elements are all 10, but that's obviously not how it really works. It actually works like this:

class RepeatIterator(object):
    def __init__(self, value):
        self.value = value
    def next(self):
        return self.value

Other iterators work by computing values on the fly, pulling buffers off a socket, etc. You can always find a way to think of any iterator as a pointer to a position within some kind of virtual sequence, just like that virtual infinite list of 10s, but it's often a stretch, and in those cases it's more helpful to just think about what's really happening—calling the next method.

By the way, this is a lot easier to talk about in Python 3, where you have a next function that calls a __next__ method, rather than a next method. And of course Python 3's way follows the same pattern as ever other special method—iter(i) calls i.__iter__(), getattr(obj, attr) calls obj.__getattr__(attr), and next(i) calls i.__next__().

next() method in python: is the definition correct?

2 Answers