iterator - Python generator behaviour

Question

import itertools
ws=[]
subs=[]
set_subs=[]
for i in xrange(int(raw_input())):
    S=raw_input()
    l=len(S)
    subs.append(S[i:j+1] for i in xrange(l) for j in xrange(i,l))

input:

2
aab
aac

now both subs[0] and subs[1] give me same result.

print list(subs[0])
>>>['a','aa','aac','a','ac','c']
print list(subs[1])
>>>['a','aa','aac','a','ac','c']

whereas list(subs[0]) should have been ['a','aa','aab','a','ab','b']

I vaguely understand why this is happening. What do I do to make subs[0] and subs[1] actually different.

NOTE: changing the line

subs.append(S[i:j+1] for i in xrange(l) for j in xrange(i,l))

with

subs.append([S[i:j+1] for i in xrange(l) for j in xrange(i,l)])

is not an option

just an advise: you shouldn't reuse i as variable in your generator expression. It can lead to confusion. — mata

Janne Karila Janne Karila · Accepted Answer · 2012-05-24T08:36:17

http://docs.python.org/reference/expressions.html#generator-expressions

Variables used in the generator expression are evaluated lazily when the __next__() method is called for generator object (in the same fashion as normal generators). However, the leftmost for clause is immediately evaluated, so that an error produced by it can be seen before any other possible error in the code that handles the generator expression. Subsequent for clauses cannot be evaluated immediately since they may depend on the previous for loop.

S[i:j+1] is evaluated when you execute the generator, and at that point S has the latest value.

You can use a normal generator instead. Now ss is local to subgen:

import itertools

def subgen(ss):
    l=len(ss)
    for i in xrange(l):
        for j in xrange(i,l):
            yield ss[i:j+1]

subs=[]
for i in xrange(int(raw_input())):
    S=raw_input()
    subs.append(subgen(S))

iterator - Python generator behaviour

2 Answers