Why does Python skip elements when I modify a list while iterating over it?

Question

I'm currently developing a program in python and I just noticed that something was wrong with the foreach loop in the language, or maybe the list structure. I'll just give a generic example of my problem to simplify, since I get the same erroneous behavior on both my program and my generic example:

x = [1,2,2,2,2]

for i in x:
    x.remove(i)

print x        

Well, the problem here is simple, I though that this code was supposed to remove all elements from a list. Well, the problem is that after it's execution, I always get 2 remaining elements in the list.

What am I doing wrong? Thanks for all the help in advance.

Edit: I don't want to empty a list, this is just an example...

Answer 1

This is a well-documented behaviour in Python, that you aren't supposed to modify the list being iterated through. Try this instead:

for i in x[:]:
    x.remove(i)

The [:] returns a "slice" of x, which happens to contain all its elements, and is thus effectively a copy of x.

Answer 2

When you delete an element, and the for-loop incs to the next index, you then skip an element.

Do it backwards. Or please state your real problem.

Answer 3

I think, broadly speaking, that when you write:

for x in lst:
    # loop body goes here

under the hood, python is doing something like this:

i = 0
while i < len(lst):
    x = lst[i]
    # loop body goes here
    i += 1

If you insert lst.remove(x) for the loop body, perhaps then you'll be able to see why you get the result you do?

Essentially, python uses a moving pointer to traverse the list. The pointer starts by pointing at the first element. Then you remove the first element, thus making the second element the new first element. Then the pointer move to the new second – previously third – element. And so on. (it might be clearer if you use [1,2,3,4,5] instead of [1,2,2,2,2] as your sample list)

Answer 4

Why don't you just use:

x = []

It's probably because you're changing the same array that you're iterating over.

Try Chris-Jester Young's answer if you want to clear the array your way.

Answer 5

I know this is an old post with an accepted answer but for those that may still come along...

A few previous answers have indicated it's a bad idea to change an iterable during iteration. But as a way to highlight what is happening...

>>> x=[1,2,3,4,5]
>>> for i in x:
...     print i, x.index(i)
...     x.remove(i)
...     print x
...
1 0
[2, 3, 4, 5]
3 1
[2, 4, 5]
5 2
[2, 4]

Hopefully the visual helps clarify.

Answer 6

I agree with John Fouhy regarding the break condition. Traversing a copy of the list works for the remove() method, as Chris Jester-Young suggested. But if one needs to pop() specific items, then iterating in reverse works, as Erik mentioned, in which case the operation can be done in place. For example:

def r_enumerate(iterable):
    """enumerator for reverse iteration of an iterable"""
    enum = enumerate(reversed(iterable))
    last = len(iterable)-1
    return ((last - i, x) for i,x in enum)

x = [1,2,3,4,5]
y = []
for i,v in r_enumerate(x):
    if v != 3:
        y.append(x.pop(i))
    print 'i=%d, v=%d, x=%s, y=%s' %(i,v,x,y)

or with xrange:

x = [1,2,3,4,5]
y = []
for i in xrange(len(x)-1,-1,-1):
    if x[i] != 3:
        y.append(x.pop(i))
    print 'i=%d, x=%s, y=%s' %(i,x,y)