operators - Is the behaviour of Python's list += iterable documented anywhere?

28
votes

It would appear that in Python, list += x works for any iterable x:

In [6]: l = []

In [7]: l += [1]

In [8]: l += (2, 3)

In [9]: l += xrange(5)

In [10]: l
Out[10]: [1, 2, 3, 0, 1, 2, 3, 4]

Is this behaviour documented anywhere?

To contrast this with list + x, the latter only works if x is also a list. This is spelled out in the documentation.

4
pythonoperatorscontainers
I'll look for some documentation to back it up, but I believe in the case of a list the += operator mimics extend. I'll see if I can find something to confirm this. - RocketDonkey
python.org/dev/peps/pep-0203 - Ashwini Chaudhary
@AshwiniChaudhary: I actually looked at the PEP before posting the question, but didn't find anything specifically about += and lists. Is there some part I am overlooking? - NPE
@NPE may be this p.boxnet.eu/16970, also from the same PEP: The i' in __iadd__' stands for in-place, and if you call the module dis` on += then you'll see that it is in-place add only. - Ashwini Chaudhary
The closest I have found is a remark in the __iadd__ documentation that " These methods should attempt to do the operation in-place (modifying self) [...]". - user395760

4 Answers

18
votes

From Guido van Rossum:

It works the same way as .extend() except that it also returns self. I can't find docs explaining this. :-(

Here is the relevant source code taken from listobject.c:

list_inplace_concat(PyListObject *self, PyObject *other)
{
     PyObject *result;

     result = listextend(self, other);
     if (result == NULL)
         return result;
     Py_DECREF(result);
     Py_INCREF(self);
     return (PyObject *)self;
}

I've raised a bug report to have the documentation fixed: http://bugs.python.org/issue16701

5
votes

It is now documented in Python 3.4+ and Python 2.7:

4.6.3. Mutable Sequence Types

The operations in the following table are defined on mutable sequence types. The collections.abc.MutableSequence ABC is provided to make it easier to correctly implement these operations on custom sequence types.

[Below] s is an instance of a mutable sequence type, t is any iterable object and x is an arbitrary object that meets any type and value restrictions imposed by s (for example, bytearray only accepts integers that meet the value restriction 0 <= x <= 255).

s.extend(t) or s += t

extends s with the contents of t (for the most part the same as s[len(s):len(s)] = t)

So it is now documented that for any mutable sequence type s, s += t is synonymous with s.extend(t).

4
votes

No (Guido confirms; thanks to Ashwini Chaudhary). The behaviour of += for sequences in general is underspecified. I conclude that it is not required by the specification that x + y where x is a list, and y some other iterable be an error (so other implementations could choose to allow it), and that other implementations could restrict += to require homogenous operands.

However, the reasons not to do this are obvious: python in general tries to do the right thing with operands, rather than requiring rigid type equality. The real mystery is why heterogenous addition is not allowed with lists.

Update: I've never really thought about the nonhomogenous addition problem, largely because itertools.chain is pretty much a complete solution to the problem.

Comments from those more familiar with Python's internals are welcome to explain why addition is required to be homogenous. (Question here: Why must Python list addition be homogenous?)

-1
votes

For the performance freaks out there, yes, += is a tiny bit faster than extend:

>>> from timeit import repeat
>>> min(repeat('a.extend((1,2,3,4,5,6,7))', 'a=[]'))
0.23489440699995612
>>> min(repeat('e((1,2,3,4,5,6,7))', 'a=[]; e = a.extend'))
0.2214308570000867
>>> min(repeat('a+=(1,2,3,4,5,6,7)', 'a=[]'))
0.21909333300027356

And here is how it compares to append:

>>> min(repeat('a.append(1)', 'a=[]'))
0.062107428999297554
>>> min(repeat('p(1)', 'a=[]; p = a.append'))
0.04968810399986978
>>> min(repeat('a+=(1,)', 'a=[]'))
0.0501599309991434

(Testing on Python 3.7 64-bit, Windows)