Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #109291 > unrolled thread

Re: Multiple inheritance, super() and changing signature

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Multiple inheritance, super() and changing signature Ben Finney <ben+python@benfinney.id.au> - 2016-06-01 06:01 +1000
    Re: Multiple inheritance, super() and changing signature Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-06-02 01:22 -0700
      Re: Multiple inheritance, super() and changing signature Michael Selik <michael.selik@gmail.com> - 2016-06-02 14:44 +0000
      Re: Multiple inheritance, super() and changing signature Steven D'Aprano <steve@pearwood.info> - 2016-06-03 03:36 +1000
        Re: Multiple inheritance, super() and changing signature Ian Kelly <ian.g.kelly@gmail.com> - 2016-06-02 11:55 -0600
        Re: Multiple inheritance, super() and changing signature Random832 <random832@fastmail.com> - 2016-06-02 17:18 -0400
          Re: Multiple inheritance, super() and changing signature Marko Rauhamaa <marko@pacujo.net> - 2016-06-03 01:20 +0300
          Re: Multiple inheritance, super() and changing signature Steven D'Aprano <steve@pearwood.info> - 2016-06-03 22:47 +1000

#109291 — Re: Multiple inheritance, super() and changing signature

Nagy László Zsolt <gandalf@shopzeus.com> writes:

> Today I come across this problem for the N+1st time. Here are some
> classes for the example:

Thank you for the example.

(Note that ‘__init__’ is not a constructor, because it operates on the
*already constructed* instance, and does not return anything. Python's
classes implement the constructor as ‘__new__’, and you very rarely need
to bother with that.)


For the reasons you describe (among others), participating in multiple
inheritance is tricky.

As described in numerous articles, for example
<URL:https://rhettinger.wordpress.com/2011/05/26/super-considered-super/>,
the right way to do this is:

* Obey the Liskov Substitution Principle (sub-classes should override a
  method only when they allow all the parent's behaviour as well as
  their own).

  This also means that the sub-class should pass any unknown arguments
  along to the superclass's method to handle.

* Once a parameter is handled in a method, remove it from the set and
  pass only the remaining arguments along to the superclass's method.

> class Observable:
>     """Implements the observer-observable pattern."""
>
>     def __init__(self):
>         # initialization code here...
>         super(Observable, self).__init__()

This should be:

    def __init__(self, **kwargs):
        # …
        super(Observable, self).__init__(**kwargs)

> class Widget(Observable, AppServerSessionMixin):
>     def __init__(self, appserver, session):
>         self.appserver = appserver
>         self.session = session
>         # general widget initialization code here...
>         super(Widget, self).__init__()

This should be:

    def __init__(self, appserver, session, **kwargs):
        self.appserver = appserver
        self.session = session
        # …
        super(Widget, self).__init__(**kwargs)

> class VisualWidget(Widget):
>     def __init__(self, appserver, session):
>         # some more code here for visually visible widgets...
>         super(VisualWidget, self).__init__(appserver, session)

Since this function doesn't do anything with the parameters, it doesn't
need to have its own names for them:

    def __init__(self, **kwargs):
        # …
        super(VisualWidget, self).__init__(**kwargs)

> class BaseDesktop:
>     def __init__(self, appserver, session):
>         # general code for all desktops is here...
>         super(BaseDesktop, self).__init__(appserver, session)
>
> class BootstrapWidget(VisualWidget):
>     def __init__(self, appserver, session):
>         # bootstrap specific code for all widgets (not just desktops)
>         super(BootstrapWidget, self).__init__(appserver, session)

And again:

    def __init__(self, **kwargs):
        # …
        super(BaseDesktop, self).__init__(**kwargs)

    def __init__(self, **kwargs):
        # …
        super(BootstrapWidget, self).__init__(**kwargs)

> class BootstrapDesktop(BootstrapWidget, BaseDesktop):
>     def __init__(self, appserver, session):
>         # there is NOTHING else here, it just connects bootstrap widget
> implementation with desktop methods
>         super(BootstrapDesktop, self).__init__(appserver, session)

If the ‘__init__’ method literally does nothing except call the
superclass's method, don't define it in BootstrapDesktop at all.

And likewise for the rest: pass along all remaining arguments, and don't
write a do-nothing ‘__init__’.

> Of course, I could explicitly call constructors of base classes. But
> then I cannot use super() anywhere. Even not in places that are far
> down in the class hierarchy. This results in typing all method
> parameters many times, and also unnecesary refactoring.

You avoid this by the ‘**kwargs’ idiom (no need to name arguments you
don't explicitly handle), and by not writing any ‘__init__’ if you want
the default behaviour (call the superclass's method).

> This is time consuming and error prone.

As pointed out in Raymond Hettinger's article
<URL:https://rhettinger.wordpress.com/2011/05/26/super-considered-super/>,
multiple inheritance is inherently tricky and there is an irreducible
complexity that has to appear somewhere in your code.

> In fact, I could never safely use super() in any case when the method
> signature has changed. When signatures are changed, method resolution
> must be done manually anyway.

Use ‘**kwargs’, and always pass along the remaining ‘**kwargs’
minus-any-parameters-you-already-handled, to protect your existing
classes from unnecessary signature changes.

-- 
 \     “Creativity can be a social contribution, but only in so far as |
  `\         society is free to use the results.” —Richard M. Stallman |
_o__)                                                                  |
Ben Finney

[toc] | [next] | [standalone]

#109340

On Wednesday, June 1, 2016 at 8:02:14 AM UTC+12, Ben Finney wrote:
> (Note that ‘__init__’ is not a constructor, because it operates on the
> *already constructed* instance, and does not return anything.

Believe it or not, that *is* what “constructor” means in every OO language. Technically it should be called the “initializer”, but “constructor” is the accepted term for the special method that is called to initialize a newly-allocated class instance.

> Python's classes implement the constructor as ‘__new__’, and you very rarely
> need to bother with that.)

Python’s “__new__” goes beyond the capabilities of “constructors” in conventional OO languages.

[toc] | [prev] | [next] | [standalone]

#109371

On Thu, Jun 2, 2016 at 4:26 AM Lawrence D’Oliveiro <lawrencedo99@gmail.com>
wrote:

> On Wednesday, June 1, 2016 at 8:02:14 AM UTC+12, Ben Finney wrote:
> > (Note that ‘__init__’ is not a constructor, because it operates on the
> > *already constructed* instance, and does not return anything.
>
> Believe it or not, that *is* what “constructor” means in every OO
> language. Technically it should be called the “initializer”, but
> “constructor” is the accepted term for the special method that is called to
> initialize a newly-allocated class instance.
>

Perhaps a Pythonista may have different jargon? Since we have two different
words (initializer, constructor), we may as well give them different
meanings so that they are maximally useful in conversation.

[toc] | [prev] | [next] | [standalone]

#109375

On Thu, 2 Jun 2016 06:22 pm, Lawrence D’Oliveiro wrote:

> On Wednesday, June 1, 2016 at 8:02:14 AM UTC+12, Ben Finney wrote:
>> (Note that ‘__init__’ is not a constructor, because it operates on the
>> *already constructed* instance, and does not return anything.
> 
> Believe it or not, that *is* what “constructor” means in every OO
> language. 

I don't believe it. 

C# is an OO language, and it distinguishes constructors and initialisers:

https://msdn.microsoft.com/en-us/library/bb397680.aspx

(although they don't seem to mean quite the same as what they mean in
Python).


Objective C is also an OO language, and it calls `init` the initializer:

https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSObject_Class/index.html#//apple_ref/occ/instm/NSObject/init

The documentation doesn't specifically give a name to the `alloc` method,
but since it allocates memory for the instance, "allocator" is the obvious
term.

https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSObject_Class/index.html#//apple_ref/occ/clm/NSObject/alloc

And `new` merely calls alloc, then init. (Although I'm lead to understand
that in older versions of Cocoa, `new` handled both allocation and
initialisation.)

https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSObject_Class/index.html#//apple_ref/occ/clm/NSObject/new

Smalltalk, the grand-daddy of OOP languages, doesn't even have constructors
(at least not in the modern OOP sense):

http://www.slideshare.net/SmalltalkWorld/stoop-008fun-withsmalltalkmodel
http://image.slidesharecdn.com/stoop-008-funwithsmalltalkmodel-101025144609-phpapp02/95/8-oop-smalltalk-model-3-638.jpg?cb=1422578644

Any method can be used to create an object, there is no reserved name for
such a method.

Python is an OO language, what does it say?

The glossary doesn't define *either* constructor or initialiser (or
initializer):

https://docs.python.org/3/glossary.html

and the docs for __new__ and __init__ don't refer to them by either name:

https://docs.python.org/3/reference/datamodel.html#basic-customization

The docs do refer to the "object constructor expression", but that's the
call to the class, not the special method. And the term "initialiser"
or "initializer" is frequently used by Python developers to refer to
__init__:

https://rosettacode.org/wiki/Classes#Python


> Technically it should be called the “initializer”, but 
> “constructor” is the accepted term for the special method that is called
> to initialize a newly-allocated class instance.

Not in Python circles it isn't.

But since the constructor/initialiser methods are so closely linked, many
people are satisfied to speak loosely and refer to "the constructor" as
either, unless they specifically wish to distinguish between __new__ and
__init__.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#109376

On Thu, Jun 2, 2016 at 11:36 AM, Steven D'Aprano <steve@pearwood.info> wrote:
> On Thu, 2 Jun 2016 06:22 pm, Lawrence D’Oliveiro wrote:
>
>> On Wednesday, June 1, 2016 at 8:02:14 AM UTC+12, Ben Finney wrote:
>>> (Note that ‘__init__’ is not a constructor, because it operates on the
>>> *already constructed* instance, and does not return anything.
>>
>> Believe it or not, that *is* what “constructor” means in every OO
>> language.
>
> I don't believe it.
>
> C# is an OO language, and it distinguishes constructors and initialisers:
>
> https://msdn.microsoft.com/en-us/library/bb397680.aspx
>
> (although they don't seem to mean quite the same as what they mean in
> Python).

Indeed. The "constructor" in that example is the equivalent of the
Python __init__ method. The "initializer" is not a part of the class
at all but just a syntactic sugar for creating an instance and setting
some of its properties at the same time in a single statement. It's
very similar to the C array initializer syntax, e.g.:

    int myArray[] = {1, 2, 3, 4, 5};

[toc] | [prev] | [next] | [standalone]

#109385

On Thu, Jun 2, 2016, at 13:36, Steven D'Aprano wrote:
> On Thu, 2 Jun 2016 06:22 pm, Lawrence D’Oliveiro wrote:
> 
> > On Wednesday, June 1, 2016 at 8:02:14 AM UTC+12, Ben Finney wrote:
> >> (Note that ‘__init__’ is not a constructor, because it operates on the
> >> *already constructed* instance, and does not return anything.
> > 
> > Believe it or not, that *is* what “constructor” means in every OO
> > language. 
> 
> I don't believe it. 
> 
> C# is an OO language, and it distinguishes constructors and initialisers:
> 
> https://msdn.microsoft.com/en-us/library/bb397680.aspx

The methods described on that page as "constructors" operate *precisely*
as Lawrence said.

An "initializer" as that page discusses is not a method but is a kind of
expression, with no analogue in Python, which compiles to multiple
operations and 'returns' a value to be assigned to a variable. A
hypothetical Python analogue to the C# expression 'new c() { a = 1, b =
2 }', an "initializer", might compile to the following bytecode:
LOAD_GLOBAL (c); CALL_FUNCTION 0; DUP_TOP; LOAD_CONST (1); ROT_TWO;
STORE_ATTR (a); DUP_TOP; LOAD_CONST (2); ROT_TWO; STORE_ATTR (b); i.e.
yielding into the surrounding expression context an instance of type 'c'
which has been constructed and then subsequently had two of its
attributes (in C#'s case, fields and/or properties) set (or .Add methods
called, in the case of collection initializers)

> > Technically it should be called the “initializer”, but 
> > “constructor” is the accepted term for the special method that is called
> > to initialize a newly-allocated class instance.
> 
> Not in Python circles it isn't.

I suspect that the basis on which people refuse to accept it is a
(completely incorrect) sense that "constructor" has an inherent meaning
as allocating a new object, when no-one's been able to document that it
is used *anywhere*, Python or otherwise, in that sense. (I also
inherently dislike the notion that any given language community should
get to unilaterally decide, even if there were any such agreement, what
a term means, denying us a common way to talk about concepts shared
between languages to no benefit)

Your "python circles" seem to consist of people who are ignorant of how
other languages actually work and who want to believe that python is
special [i.e. in this case that python's __init__ is somehow different
in operation from C++ or C#'s or Java's constructors] when it is not. I
think it's more or less the same crowd, and the same motivation, as the
"python doesn't have variables" folks, and should be likewise ignored.

> But since the constructor/initialiser methods are so closely linked, many
> people are satisfied to speak loosely and refer to "the constructor" as
> either, unless they specifically wish to distinguish between __new__ and
> __init__.

Where there is looseness, it comes from the fact that because __init__
is always called even if __new__ returns a pre-existing object people
often place code in __new__ which mutates the object to be returned,
acting somewhat like a traditional constructor by assigning attributes
etc.

But it is __init__ that acts like the thing that is called a constructor
in many languages, and no-one's produced a single example of a language
which uses "constructor" for something which allocates memory.

Now, where "constructor" *does* often get misused, it is for the
new-expression in other languages, and for type.__call__ in Python,
people speak as if they're "calling the constructor" (rather than
calling something which ultimately calls both __new__ and __init__). But
from a class-definition perspective, __init__ is the one and only thing
that should be called a constructor.

The fact that many languages don't have any way to override object
allocation, and therefore no analogue to __new__, also contributes to
this conclusion. In these languages, new-expressions (or the like) are
the only way to call a constructor, so people associate object
allocation with constructors.

[toc] | [prev] | [next] | [standalone]

#109386

Random832 <random832@fastmail.com>:
> But from a class-definition perspective, __init__ is the one and only
> thing that should be called a constructor.

Not arguing agaist that, but from the *user's* perspective, I see the
class itself is the constructor function:

   class C: pass
   c = C()

You could say that the class statement in Python little else than
syntactic sugar for creating an object constructor.

For example, instead of:

   import math
   class Point:
       def __init__(self, x, y):
           self.x = x
           self.y = y
       def rotate(self, angle):
           self.x, self.y = (
               self.x * math.cos(angle) - self.y * math.sin(angle),
               self.x * math.sin(angle) + self.y * math.cos(angle))

you could write:

   import math, types
   def Point(x, y):
       def rotate(angle):
           nonlocal x, y
           x, y = (
               x * math.cos(angle) - y * math.sin(angle),
               x * math.sin(angle) + y * math.cos(angle))
       return types.SimpleNamespace(rotate=rotate)

> The fact that many languages don't have any way to override object
> allocation, and therefore no analogue to __new__, also contributes to
> this conclusion.

I see no point in Python having __new__, either. In fact, I can see the
point in C++ but not in Python.


Marko

[toc] | [prev] | [next] | [standalone]

#109410

On Fri, 3 Jun 2016 07:18 am, Random832 wrote:

> On Thu, Jun 2, 2016, at 13:36, Steven D'Aprano wrote:
[...]
>> But since the constructor/initialiser methods are so closely linked, many
>> people are satisfied to speak loosely and refer to "the constructor" as
>> either, unless they specifically wish to distinguish between __new__ and
>> __init__.
> 
> Where there is looseness, it comes from the fact that because __init__
> is always called even if __new__ returns a pre-existing object 

That's not quite correct. __init__ is not always called:

    If __new__() does not return an instance of cls, then the new 
    instance’s __init__() method will not be invoked.


https://docs.python.org/3/reference/datamodel.html#object.__new__


> people 
> often place code in __new__ which mutates the object to be returned,
> acting somewhat like a traditional constructor by assigning attributes
> etc.
> 
> But it is __init__ that acts like the thing that is called a constructor
> in many languages, and no-one's produced a single example of a language
> which uses "constructor" for something which allocates memory.

Yes you have: Python. Like it or not, it is common usage to call __new__ the
constructor and __init__ the initialiser. Even if the official docs are
agnostic on the issue, it is still widespread (but not universal) in the
Python community.

I have only come across two languages that split the "constructor" into two
methods, Objective C and Python. That does make Python rather special
compared to most other OO languages.

Objective C calls the part that allocates memory "alloc" (and is presumably
known as "the allocator") and the part which initialises it "init". "init"
is explicitly and officially known as the initialiser. Here's that link
again in case you missed it.

https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSObject_Class/index.html#//apple_ref/occ/instm/NSObject/init

Even if you dismiss Python, you can't dispute that the official Apple docs
for Cocoa refer to initialiser rather than constructor.


Python also follows that pattern of splitting the "constructor" into two:
__new__ and __init__. Until Python 2.2 introduced new-style classes, it was
common to refer to __init__ as the constructor. For instance, I have the
Python Pocket Reference from Python 1.5 which describes __init__ as:

    Constructor: initialize the new instance, self.

When object and new-style classes were introduced, people wanted to
distinguish them. Since __new__ constructs the object, and __init__
initialises it (even in old-style classes __init__ is said to *initialise*
the instance), it seems natural to call __new__ the constructor and
__init__ the initialiser.

Perhaps we ought to have called them the allocator and the initialiser. If
you want to lead the push for that terminology, please be my guest.



-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web