Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #70891 > unrolled thread

Why has __new__ been implemented as a static method?

Back to article view | Back to comp.lang.python

Contents

  Why has __new__ been implemented as a static method? Jurko Gospodnetić <jurko.gospodnetic@pke.hr> - 2014-05-03 12:37 +0200
    Re: Why has __new__ been implemented as a static method? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-05-03 15:12 +0000
      Re: Why has __new__ been implemented as a static method? Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-05-04 11:21 +1200
        Re: Why has __new__ been implemented as a static method? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-05-04 03:37 +0000
          Re: Why has __new__ been implemented as a static method? Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-05-04 20:03 +1200
            Re: Why has __new__ been implemented as a static method? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-05-04 14:16 +0000
              Re: Why has __new__ been implemented as a static method? Rotwang <sg552@hotmail.co.uk> - 2014-05-04 17:24 +0100
              Re: Why has __new__ been implemented as a static method? Ian Kelly <ian.g.kelly@gmail.com> - 2014-05-04 11:05 -0600

#70891 — Why has __new__ been implemented as a static method?

   Hi all.

   I was wandering why Python implements its __new__ method as a static 
and not a class method?

   __new__ always accepts a cls parameter, which lead me to believe it 
was a class method. Also, implementing __new__ as a class method seems 
natural when thinking about __new__ as 'a method you call on a class to 
create a new instance of that class'.

   The difference between the two implementations is admittedly not that 
significant. In most cases __new__ would still need to have its cls 
parameter explicitly specified since it is most often called on a class 
and not on an instance.

   Having it implemented as a class method would allow this:

     class X:
       def __new__(cls, x):
         return super().__new__(x)

   instead of having to do this, as we do now:

     class X:
       def __new__(cls, x):
         return super().__new__(cls, x)


   Thinking about this some more - I guess having it as a static method 
allows you to use super() and still create a new 'subclass' when asked 
to instantiate a base class. Perhaps some sort of a factory pattern 
implementation like so:

     class Base:
       def __new__(cls, x):
         actual_class = get_class_for(x)
         return super().__new__(actual_class, x)

   If __new__ were a class method, the same logic could not be so easily 
implemented as you would need to call __new__ on actual_class, and so 
each actual_class would need to implement its own __new__ skipping its 
direct parent's __new__ - yuck... what a mess...

   Could that be the actual use case causing Guido to model __new__ as a 
static method? Or was it something else?


   I'm not suggesting this be changed, of course. I'm just curious as to 
whether I'm not fully understanding something in this part of Python. I 
generally find Python highly intuitive and it's not often something 
about it takes me by surprise like this. :-)

   Thanks in advance.

   Best regards,
     Jurko Gospodnetić

P.S.
   Normally, I'd chalk this issue up under 'bike-shedding', but it came 
up while teaching others about Python and so does not feel right leaving 
it as 'because that's the way it is'. :-)

[toc] | [next] | [standalone]

#70895

On Sat, 03 May 2014 12:37:24 +0200, Jurko Gospodnetić wrote:

> Hi all.
> 
>    I was wandering why Python implements its __new__ method as a static
> and not a class method?

Have you read Guido's tutorial on it?

https://www.python.org/download/releases/2.2.3/descrintro

[quote]
Factoid: __new__ is a static method, not a class method. I initially 
thought it would have to be a class method, and that's why I added the 
classmethod primitive. Unfortunately, with class methods, upcalls don't 
work right in this case, so I had to make it a static method with an 
explicit class as its first argument.
[end quote]

I'm not entirely sure what he means by "upcalls", but I believe it means 
to call the method further up (that is, closer to the base) of the 
inheritance tree.


-- 
Steven D'Aprano
http://import-that.dreamwidth.org/

[toc] | [prev] | [next] | [standalone]

#70905

Steven D'Aprano wrote:
> I'm not entirely sure what he means by "upcalls", but I believe it means 
> to call the method further up (that is, closer to the base) of the 
> inheritance tree.

I think it means this:

    def __new__(cls):
       MyBaseClass.__new__(cls)

which wouldn't work with a class method, because
MyBaseClass.__new__ would give a *bound* method
rather than an unbound one.

Python 3's version of super() seems to work with
class methods, but Python 2's doesn't (or at least
I couldn't get it to work in a brief test). Also,
I don't think super() existed at all when __new__
was invented.

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#70906

On Sun, 04 May 2014 11:21:53 +1200, Gregory Ewing wrote:

> Steven D'Aprano wrote:
>> I'm not entirely sure what he means by "upcalls", but I believe it
>> means to call the method further up (that is, closer to the base) of
>> the inheritance tree.
> 
> I think it means this:
> 
>     def __new__(cls):
>        MyBaseClass.__new__(cls)
> 
> which wouldn't work with a class method, because MyBaseClass.__new__
> would give a *bound* method rather than an unbound one.

If it were a class method, you would call it by MyBaseClass.__new__() 
rather than explicitly providing the cls argument.


> Python 3's version of super() seems to work with class methods, but
> Python 2's doesn't (or at least I couldn't get it to work in a brief
> test). 

Works for me. Perhaps you got your super() call wrong?

py> class MyDict(dict):
...     @classmethod
...     def fromkeys(cls, *args, **kwargs):
...             print "Calling overridden method."
...             return super(MyDict, cls).fromkeys(*args, **kwargs)
...
py> MyDict.fromkeys('abc')
Calling overridden method.
{'a': None, 'c': None, 'b': None}
py> MyDict().fromkeys('abc')
Calling overridden method.
{'a': None, 'c': None, 'b': None}


> Also, I don't think super() existed at all when __new__ was
> invented.

Well, I don't know about that, but super was introduced in the same 
version as new-style classes with __new__.



-- 
Steven D'Aprano
http://import-that.dreamwidth.org/

[toc] | [prev] | [next] | [standalone]

#70907

Steven D'Aprano wrote:
> If it were a class method, you would call it by MyBaseClass.__new__() 
> rather than explicitly providing the cls argument.

But that wouldn't be any good, because the base __new__
needs to receive the actual class being instantiated,
not the class that the __new__ method belongs to.

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#70911

On Sun, 04 May 2014 20:03:35 +1200, Gregory Ewing wrote:

> Steven D'Aprano wrote:
>> If it were a class method, you would call it by MyBaseClass.__new__()
>> rather than explicitly providing the cls argument.
> 
> But that wouldn't be any good, because the base __new__ needs to receive
> the actual class being instantiated, not the class that the __new__
> method belongs to.


Which is exactly what method descriptors -- whether instance methods or 
class descriptors -- can do. Here's an example, using Python 2.7:

class MyDict(dict):
    @classmethod
    def fromkeys(cls, *args, **kwargs):
        print "Called from", cls
        return super(MyDict, cls).fromkeys(*args, **kwargs)

class AnotherDict(MyDict):
    pass


And in use:

py> MyDict.fromkeys('abc')
Called from <class '__main__.MyDict'>
{'a': None, 'c': None, 'b': None}
py> AnotherDict().fromkeys('xyz')
Called from <class '__main__.AnotherDict'>
{'y': None, 'x': None, 'z': None}


In both cases, MyDict's __new__ method receives the class doing the 
calling, not the class where the method is defined.

Whatever the difficulty is with __new__, it isn't something obvious.


-- 
Steven D'Aprano
http://import-that.dreamwidth.org/

[toc] | [prev] | [next] | [standalone]

#70913

On 04/05/2014 15:16, Steven D'Aprano wrote:
> On Sun, 04 May 2014 20:03:35 +1200, Gregory Ewing wrote:
>
>> Steven D'Aprano wrote:
>>> If it were a class method, you would call it by MyBaseClass.__new__()
>>> rather than explicitly providing the cls argument.
>>
>> But that wouldn't be any good, because the base __new__ needs to receive
>> the actual class being instantiated, not the class that the __new__
>> method belongs to.
>
>
> Which is exactly what method descriptors -- whether instance methods or
> class descriptors -- can do. Here's an example, using Python 2.7:
>
> class MyDict(dict):
>      @classmethod
>      def fromkeys(cls, *args, **kwargs):
>          print "Called from", cls
>          return super(MyDict, cls).fromkeys(*args, **kwargs)
>
> class AnotherDict(MyDict):
>      pass
>
>
> And in use:
>
> py> MyDict.fromkeys('abc')
> Called from <class '__main__.MyDict'>
> {'a': None, 'c': None, 'b': None}
> py> AnotherDict().fromkeys('xyz')
> Called from <class '__main__.AnotherDict'>
> {'y': None, 'x': None, 'z': None}
>
>
> In both cases, MyDict's __new__ method receives the class doing the
> calling, not the class where the method is defined.

Yes, when a classmethod bound to a subclass or an instance is called. 
But this is irrelevant to Gregory's point:

On 04/05/2014 04:37, Steven D'Aprano wrote:
> On Sun, 04 May 2014 11:21:53 +1200, Gregory Ewing wrote:
>> Steven D'Aprano wrote:
>>> I'm not entirely sure what he means by "upcalls", but I believe it
>>> means to call the method further up (that is, closer to the base) of
>>> the inheritance tree.
>>
>> I think it means this:
>>
>>      def __new__(cls):
>>         MyBaseClass.__new__(cls)
>>
>> which wouldn't work with a class method, because MyBaseClass.__new__
>> would give a *bound* method rather than an unbound one.
>
> If it were a class method, you would call it by MyBaseClass.__new__()
> rather than explicitly providing the cls argument.


The relevant behaviour is this:

 >>> class C:
     @classmethod
     def m(cls):
         print("Called from", cls)

 >>> class D(C):
     @classmethod
     def m(cls):
         C.m()

		
 >>> C.m()
Called from <class '__main__.C'>
 >>> D.m()
Called from <class '__main__.C'>


If __new__ were a classmethod, then a call to MyBaseClass.__new__() 
within the body of MySubClass.__new__ would pass MyBaseClass to the 
underlying function, not the MySubClass. This means that

class MySubClass(MyBaseClass):
     def __new__(cls):
         return MyBaseClass.__new__()

would fail, since it would return an instance of MyBaseClass rather than 
MySubClass.

[toc] | [prev] | [next] | [standalone]

#70914

On Sun, May 4, 2014 at 8:16 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Sun, 04 May 2014 20:03:35 +1200, Gregory Ewing wrote:
>
>> Steven D'Aprano wrote:
>>> If it were a class method, you would call it by MyBaseClass.__new__()
>>> rather than explicitly providing the cls argument.
>>
>> But that wouldn't be any good, because the base __new__ needs to receive
>> the actual class being instantiated, not the class that the __new__
>> method belongs to.
>
>
> Which is exactly what method descriptors -- whether instance methods or
> class descriptors -- can do. Here's an example, using Python 2.7:
>
> class MyDict(dict):
>     @classmethod
>     def fromkeys(cls, *args, **kwargs):
>         print "Called from", cls
>         return super(MyDict, cls).fromkeys(*args, **kwargs)
>
> class AnotherDict(MyDict):
>     pass
>
>
> And in use:
>
> py> MyDict.fromkeys('abc')
> Called from <class '__main__.MyDict'>
> {'a': None, 'c': None, 'b': None}
> py> AnotherDict().fromkeys('xyz')
> Called from <class '__main__.AnotherDict'>
> {'y': None, 'x': None, 'z': None}
>
>
> In both cases, MyDict's __new__ method receives the class doing the
> calling, not the class where the method is defined.
>
> Whatever the difficulty is with __new__, it isn't something obvious.

You cheated on two counts here.  First, you're using super; I think
Guido's comment about "upcalls" in the link you posted earlier was in
reference to calls that explicitly name the name parent class, i.e.
"dict.fromkeys()", not "super(MyDict, cls).fromkeys()".

Second, you didn't override the method in AnotherDict, so
"MyDict.fromkeys" and "AnotherDict.fromkeys" refer to the same method,
the only difference being in which class is passed to the descriptor
when it is accessed.

Compare to this:

class MyDict(dict):
    @classmethod
    def fromkeys(cls, *args, **kwargs):
        print "MyDict Called from", cls
        return dict.fromkeys(*args, **kwargs)

class AnotherDict(MyDict):
    @classmethod
    def fromkeys(cls, *args, **kwargs):
        print "AnotherDict Called from", cls
        return MyDict.fromkeys(*args, **kwargs)

>>> MyDict.fromkeys('abc')
MyDict Called from <class '__main__.MyDict'>
{'a': None, 'c': None, 'b': None}
>>> AnotherDict.fromkeys('abc')
AnotherDict Called from <class '__main__.AnotherDict'>
MyDict Called from <class '__main__.MyDict'>
{'a': None, 'c': None, 'b': None}

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web