Can literals in Python be overridden?

Question

Couldn't find a way to phrase the title better, feel free to correct.

I'm pretty new to Python, currently experimenting with the language.. I've noticed that all built-ins types cannot be extended with other members.. I'd like for example to add an each method to the list type, but that would be impossible. I realize that it's designed that way for efficiency reasons, and that most of the built-ins types are implemented in C.

Well, one why I found to override this behavior is be defining a new class, which extends list but otherwise does nothing. Then I can assign the variable list to that new class, and each time I would like to instantiate a new list, I'd use the list constructor, like it would have been used to create the original list type.

class MyList(list):
    def each(self, func):
       for item in self:
           func(item)
list = MyList

my_list = list((1,2,3,4))
my_list.each(lambda x: print(x))

Output:

The idea can be generalize of course by defining a method that gets a built it type and returns a class that extends that type. Further more, the original list variable can be saved in another variable to keep an access to it.

Only problem I'm facing right now, is that when you instantiate a list by its literal form, (i.e. [1,2,3,4]), it will still use the original list constructor (or does it?). Is there a way to override this behavior? If the answer is no, do you know of some other way of enabling the user to extend the built-ins types? (just like javascript allows extending built-ins prototypes).

I find this limitation of built-ins (being unable to add members to them) one of Python's drawbacks, making it inconsistent with other user-defined types... Overall I really love the language, and I really don't understand why this limitation is REALLY necessary.

Zen of Python - "Explicit is better than implicit." Basically, it's probably not going to be obvious to other people working on your code that you have overridden such a basic feature of the language, so it's better to be explicit when doing things like that. — SethMMorton
No, you cannot override any built-in literal syntax; literals exist for convenience, there is no literal syntax for custom types at all. That does add clarity; you now always know what type literal syntax produces. — Martijn Pieters♦
While there is a module designed to allow patching built-in types, it's a bad idea. — user2357112 supports Monica
@RB14 Because Python is already dynamically typed, and if the modules you were importing were able to override and add members to built in types willy nilly, you would have no warning, and would be left scratching your head at unexpected behavior. There is no way for a person who has added a method to list in a module to advertise this to anyone who uses this code in the future. I don't see what the problem would be with using your own type which can take a plain old list or generator as a constructor argument. You can use something like mylist([1,2,3,4]) while being concise. — Asad Saeeduddin
The real reason is that Python isn't Ruby (or any other language). — SethMMorton

Veedrac Veedrac · Accepted Answer · 2013-09-29T21:24:07

This is a conscientious choice from Python.

Firstly, with regards to patching inbuilt type, this is primarily a design decision and only secondarily an optimization. I have learnt from much lurking on the Python Mailing List that monkey patching on builtin types, although enjoyable for small scripts, serves no good purpose in anything larger.

Libraries, for one, make certain assumptions about types. If it were encouraged to extend default types, many libraries would end up fighting each other. It would also discourage making new types – a deque is a deque, an ordered set is an ordered set, a dictionary is a dictionary and that should be that.

Literal syntax is a particularly important point. If you cannot guarantee that [1, 2, 3] is a list, what can you guarantee? If people could change those behaviours it would have such global impacts as to destroy the stability of a lot of code. There is a reason goto and global variables are discouraged.

There is one particular hack that I am fond of, though. When you see r"hello", this seems to be an extended literal form.

So why not r[1, 2, 3]?

class ListPrefixer:
    def __init__(self, typ):
        self.typ = typ

    def __getitem__(self, args):
        return self.typ(args)

class MyList(list):
    def each(self, func):
        return MyList(func(x) for x in self)

e = ListPrefixer(MyList)

e[1, 2, 3, 4].each(lambda x: x**2)
#>>> [1, 4, 9, 16]

Finally, if you really want to do deep AST hacks, check out MacroPy.

Can literals in Python be overridden?

1 Answers