Why are Python's 'private' methods not actually private?

716
votes

Python gives us the ability to create 'private' methods and variables within a class by prepending double underscores to the name, like this: __myPrivateMethod(). How, then, can one explain this

>>> class MyClass:
...     def myPublicMethod(self):
...             print 'public method'
...     def __myPrivateMethod(self):
...             print 'this is private!!'
... 
>>> obj = MyClass()
>>> obj.myPublicMethod()
public method
>>> obj.__myPrivateMethod()
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: MyClass instance has no attribute '__myPrivateMethod'
>>> dir(obj)
['_MyClass__myPrivateMethod', '__doc__', '__module__', 'myPublicMethod']
>>> obj._MyClass__myPrivateMethod()
this is private!!

What's the deal?!

I'll explain this a little for those who didn't quite get that.

>>> class MyClass:
...     def myPublicMethod(self):
...             print 'public method'
...     def __myPrivateMethod(self):
...             print 'this is private!!'
... 
>>> obj = MyClass()

What I did there is create a class with a public method and a private method and instantiate it.

Next, I call its public method.

>>> obj.myPublicMethod()
public method

Next, I try and call its private method.

>>> obj.__myPrivateMethod()
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: MyClass instance has no attribute '__myPrivateMethod'

Everything looks good here; we're unable to call it. It is, in fact, 'private'. Well, actually it isn't. Running dir() on the object reveals a new magical method that python creates magically for all of your 'private' methods.

>>> dir(obj)
['_MyClass__myPrivateMethod', '__doc__', '__module__', 'myPublicMethod']

This new method's name is always an underscore, followed by the class name, followed by the method name.

>>> obj._MyClass__myPrivateMethod()
this is private!!

So much for encapsulation, eh?

In any case, I'd always heard Python doesn't support encapsulation, so why even try? What gives?

13
pythonpython-2.7encapsulationinformation-hiding
Same is true for Java or C# if you use reflection (which is somehow what you doing there). – 0x434D53
It was build for Unit Testing purpose, so you can use that "hack" in order to unit test the private methods of your class from outside. – waas1919
Isn't testing private methods an anti-pattern? Private methods will be used in some public method for sure else it's just unused forever. And the right way to test private methods (based on my learning so far from ThoughtWorks) is that you write tests for public methods only that covers all cases. If that works fine, you don't need to test private methods from outside at all. – Vishnu Narang
@VishnuNarang: Yeah, that's whats often teached. But As always, an almost "religious" approach of "always do this, never do that" is the only thing that "never" is good. If unit tests are "only" used for regression tests or testing public API, you don't need to test privates. But if you do unit test driven development, there are good reasons to test privat methods during development (for example when it's hard to mock certain unusual / extreme parameters through the public interface). Some languages / unit test environments don't let you do this, which IMHO is not good. – Marco Freudenberger
@MarcoFreudenberger I see your point. I do have experience in unit test driven development. Often when it becomes difficult to mock parameters, most often it's resolved by changing and improving the design. I'm yet to come across a scenario where the design is perfect and still unit testing is extremely difficult to avoid testing private methods. I'll look out for such cases. Thanks. I'd appreciate if you could maybe share one scenario off the top of your head to help me understand. – Vishnu Narang

13 Answers

631
votes

The name scrambling is used to ensure that subclasses don't accidentally override the private methods and attributes of their superclasses. It's not designed to prevent deliberate access from outside.

For example:

>>> class Foo(object):
...     def __init__(self):
...         self.__baz = 42
...     def foo(self):
...         print self.__baz
...     
>>> class Bar(Foo):
...     def __init__(self):
...         super(Bar, self).__init__()
...         self.__baz = 21
...     def bar(self):
...         print self.__baz
...
>>> x = Bar()
>>> x.foo()
42
>>> x.bar()
21
>>> print x.__dict__
{'_Bar__baz': 21, '_Foo__baz': 42}

Of course, it breaks down if two different classes have the same name.

214
votes

Example of private function

import re
import inspect

class MyClass :

    def __init__(self) :
        pass

    def private_function ( self ) :
        try :
            function_call = inspect.stack()[1][4][0].strip()

            # See if the function_call has "self." in the begining
            matched = re.match( '^self\.', function_call )
            if not matched :
                print 'This is Private Function, Go Away'
                return
        except :
            print 'This is Private Function, Go Away'
            return

        # This is the real Function, only accessible inside class #
        print 'Hey, Welcome in to function'

    def public_function ( self ) :
        # i can call private function from inside the class
        self.private_function()

### End ###
190
votes

When I first came from Java to Python I hated this. It scared me to death.

Today it might just be the one thing I love most about Python.

I love being on a platform, where people trust each other and don't feel like they need to build impenetrable walls around their code. In strongly encapsulated languages, if an API has a bug, and you have figured out what goes wrong, you may still be unable to work around it because the needed method is private. In Python the attitude is: "sure". If you think you understand the situation, perhaps you have even read it, then all we can say is "good luck!".

Remember, encapsulation is not even weakly related to "security", or keeping the kids off the lawn. It is just another pattern that should be used to make a code base easier to understand.

148
votes

From http://www.faqs.org/docs/diveintopython/fileinfo_private.html

Strictly speaking, private methods are accessible outside their class, just not easily accessible. Nothing in Python is truly private; internally, the names of private methods and attributes are mangled and unmangled on the fly to make them seem inaccessible by their given names. You can access the __parse method of the MP3FileInfo class by the name _MP3FileInfo__parse. Acknowledge that this is interesting, then promise to never, ever do it in real code. Private methods are private for a reason, but like many other things in Python, their privateness is ultimately a matter of convention, not force.

97
votes

The phrase commonly used is "we're all consenting adults here". By prepending a single underscore (don't expose) or double underscore (hide), you're telling the user of your class that you intend the member to be 'private' in some way. However, you're trusting everyone else to behave responsibly and respect that, unless they have a compelling reason not to (e.g. debuggers, code completion).

If you truly must have something that is private, then you can implement it in an extension (e.g. in C for CPython). In most cases, however, you simply learn the Pythonic way of doing things.

33
votes

It's not like you absolutly can't get around privateness of members in any language (pointer arithmetics in C++, Reflections in .NET/Java).

The point is that you get an error if you try to call the private method by accident. But if you want to shoot yourself in the foot, go ahead and do it.

Edit: You don't try to secure your stuff by OO-encapsulation, do you?

13
votes

Important note:

Any identifier of the form __name (at least two leading underscores, at most one trailing underscore) is publicly replaced with _classname__name, where classname is the current class name with a leading underscore(s) stripped.

Therefore, __name is private, while _classname__name is public.

This does not mean that you can protect your private data as it is easily accessible by changing the name of the variable.

https://docs.python.org/3/tutorial/classes.html#tut-private

Example

class Cat:
    def __init__(self, name='unnamed'):
        self.name = name
    def __print_my_name(self):
        print(self.name)
        
        
tom = Cat()
tom.__print_my_name() #Error
tom._Cat__print_my_name() #Prints name
12
votes

The class.__stuff naming convention lets the programmer know he isn't meant to access __stuff from outside. The name mangling makes it unlikely anyone will do it by accident.

True, you still can work around this, it's even easier than in other languages (which BTW also let you do this), but no Python programmer would do this if he cares about encapsulation.

12
votes

Similar behavior exists when module attribute names begin with a single underscore (e.g. _foo).

Module attributes named as such will not be copied into an importing module when using the from* method, e.g.:

from bar import *

However, this is a convention and not a language constraint. These are not private attributes; they can be referenced and manipulated by any importer. Some argue that because of this, Python can not implement true encapsulation.

12
votes

It's just one of those language design choices. On some level they are justified. They make it so you need to go pretty far out of your way to try and call the method, and if you really need it that badly, you must have a pretty good reason!

Debugging hooks and testing come to mind as possible applications, used responsibly of course.

4
votes

With Python 3.4 this is the behaviour:

>>> class Foo:
        def __init__(self):
                pass
        def __privateMethod(self):
                return 3
        def invoke(self):
                return self.__privateMethod()


>>> help(Foo)
Help on class Foo in module __main__:

class Foo(builtins.object)
 |  Methods defined here:
 |
 |  __init__(self)
 |
 |  invoke(self)
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  __dict__
 |      dictionary for instance variables (if defined)
 |
 |  __weakref__
 |      list of weak references to the object (if defined)

 >>> f = Foo()
 >>> f.invoke()
 3
 >>> f.__privateMethod()
 Traceback (most recent call last):
   File "<pyshell#47>", line 1, in <module>
     f.__privateMethod()
 AttributeError: 'Foo' object has no attribute '__privateMethod'

https://docs.python.org/3/tutorial/classes.html#tut-private

Note that the mangling rules are designed mostly to avoid accidents; it still is possible to access or modify a variable that is considered private. This can even be useful in special circumstances, such as in the debugger.

Even if the question is old I hope my snippet could be helpful.

4
votes

The most important concern about private methods and attributes is to tell developers not to call it outside the class and this is encapsulation. one may misunderstand security from encapsulation. when one deliberately uses syntax like that(bellow) you mentioned, you do not want encapsulation.

obj._MyClass__myPrivateMethod()

I have migrated from C# and at first it was weird for me too but after a while I came to the idea that only the way that Python code designers think about OOP is different.

1
votes

Why are Python's 'private' methods not actually private?

As I understand it, they can't be private. How could privacy be enforced?

The obvious answer is "private members can only be accessed through self", but that wouldn't work - self is not special in Python, it is nothing more than a commonly-used name for the first parameter of a function.