Does Python require deep knowledge of all classes in the inheritance chain?

Question

Python classes do not have the concept of public / private, so they tell us not to touch what starts with underscores if we didn't create one. But does this not require a complete knowledge of all the classes from which we inherit, directly or indirectly? Witness:

class Base(object): def __init__(self): super(Base, self).__init__() self._foo = 0 def foo(self): return self._foo + 1 class Sub(Base): def __init__(self): super(Sub, self).__init__() self._foo = None Sub().foo() 

A TypeError is TypeError occur when evaluating None + 1 . Therefore, I should know that _foo exists in the base class. To get around this, you can use __foo , which solves the problem by changing the name. It seems, if not an elegant, acceptable solution. However, what happens if Base inherits from a class (in a separate package) called Sub ? Now __foo in my Sub overrides __foo in grandparent Sub .

This means that I must know the entire inheritance chain, including all the "private" objects that everyone uses. The fact that Python is dynamically typed makes this even more difficult since there are no ads to search for. However, the worst part is probably the fact that Base can inherit from object right now, but in some future version it will switch to inheritance from Sub . It is clear that if I know that Sub inherited from, I can rename my class, however annoying it may be. But I do not see in the future.

Is this not the case when a true type of private data prevents the problem? How, in Python, can I be sure that I am not accidentally stepping on someone if these fingers can exist at some point in the future?

EDIT: I, apparently, did not specify the main question. I am familiar with the name change and the difference between blue and double underscore. The question is, how can I deal with the fact that I can run into classes whose existence I do not know right now? If my parent class (which is in the package that I did not write) starts to inherit from the class with the same name as my class, even using mangling will not help. I am mistaken in considering this as a (angular) case that true private members allow, but does Python have problems with?

EDIT: as requested, the following example is given:

parent.py file:

 class Sub(object): def __init__(self): self.__foo = 12 def foo(self): return self.__foo + 1 class Base(Sub): pass 

sub.py file:

 import parent class Sub(parent.Base): def __init__(self): super(Sub, self).__init__() self.__foo = None Sub().foo() 

Grandparent foo is called, but my __foo .

Obviously, you will not write such code yourself, but parent can be easily provided by a third party, the details of which can be changed at any time.

Answer 1

This means that I must know the whole inheritance chain.,.

Yes, you need to know the whole inheritance chain, or the documents for an object that you directly subclass must tell you what you need to know.

A subclass is an advanced function and should be handled with care.

A good example of documents indicating what should be overridden in a subclass is the threading class:

This class is an action that runs in a separate control flow. There are two ways to indicate activity: by passing the called object to the constructor or overriding the run() method in a subclass. No other methods (except the constructor) should be overridden in a subclass. In other words, only override the __init__() and run() methods of this class.

Answer 2

Use personal names (instead of protected ones), starting with a double underscore:

 class Sub(Base): def __init__(self): super(Sub, self).__init__() self.__foo = None # ^^ 

will not conflict with _foo or __foo in Base . This is because Python replaces double underscores with one underscore and class name; The following two lines are equivalent:

 class Sub(Base): def x(self): self.__foo = None # .. is the same as .. self._Sub__foo = None 

(In response to a change :) The likelihood that two classes in the class hierarchy have not only the same name, but that they both use the same property name and both use the private mangled ( __ ) form is so insignificant that in practice, it can be safely ignored (I have not heard a single case).

In theory, however, you are right that in order to officially verify the correctness of a program, the whole chain of inheritance is best known. Fortunately, formal verification usually requires a fixed set of libraries anyway.

This is in the spirit of Zen of Python , which includes

Practicality exceeds purity.

Answer 3

• The name mangling includes a class, so your Base.__foo and Sub.__foo will have different names. That was the whole reason for adding a name management function in Python. One will be _Base__foo , the other _Sub__foo .

• Many people prefer to use composition (has-a) instead of inheritance (is-a) for some of these same reasons.

Answer 4

How often do you change base classes in inheritance chains to introduce inheritance from a class with the same name as the subclass further down the chain?

Less frivolous, yes, you should know the code you are working with. In the end, you should definitely know what public names are used. Python, being python, discovers the public names used by your ancestor classes, takes pretty much the same effort as opening private ones.

During the years of programming in Python, I never found that this was practically a problem. When you specify instance variables, you should have a pretty good idea: (a) the name is common enough to be used in other contexts, and (b) the class you are writing will most likely be involved in the inheritance hierarchy with other unknowns classes. In such cases, you are more careful about the names you use; self.value not a great idea for an attribute name, and nothing looks like an Adaptor great class name.

On the contrary, I ran into difficulties with overusing double-underlined names several times. Python, which is Python, even the names "private" have access to code defined outside the class. You might think that it will always be bad practice to allow external functions to access the "private" attributes, but what about things like getattr and hasattr ? Their call may be in the class’s own code, so the class still controls access to private attributes, but they still do not work if you do not scold the name manually. If Python actually had forced private variables, you couldn’t use functions like the ones on them at all. These days, I prefer to reserve double underscore names for cases where I write something very general, such as a decorator, metaclass or mixin, that need to add a "secret attribute" to instances of (unknown) classes to which it applies.

And, of course, there is a standard dynamic language argument: the reality is that you have to thoroughly test your code in order to have a lot of excuses for claiming to be "my software." Such testing is unlikely to miss errors caused by random names. If you do not perform this testing, then many other misunderstood errors will be introduced in other ways than by random name collisions.

In summation, the lack of private variables is simply not a big deal in the idiomatic Python code in practice, and adding true private variables will cause more frequent problems in other IMHO ways.

Answer 5

As already noted, you can use name management. However, if you document your code correctly, you can stick to one underscore (or not!) - you should not have so many private variables that this will be a problem. Just say if the method uses a private variable and adds either the variable or the method name to the docstring class to alert users.

In addition, if you create unit tests, you must create tests that check invariants on members, and accordingly they must be able to detect such name collisions.

If you really want to have "private" variables, and for some reason name management does not meet your needs, you can expose your private state to another object:

 class Foo(object): class Stateholder(object): pass def __init__(self): self._state = Stateholder() self.state.private = 1 

Answer 6

Mangling comes with double underscores. Separate underscores are more "please do not do this."

You do not need to know all the details about all the parent classes (note that it is generally best to avoid deep inheritance) since you can still use dir () and help () and any other introspection you can find.