Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #7430 > unrolled thread

Recursion error in metaclass

Back to article view | Back to comp.lang.python

Contents

  Recursion error in metaclass Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-11 03:34 +0000
    Re: Recursion error in metaclass Terry Reedy <tjreedy@udel.edu> - 2011-06-11 01:33 -0400
      Re: Recursion error in metaclass Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-11 11:38 +0000
        Re: Recursion error in metaclass Terry Reedy <tjreedy@udel.edu> - 2011-06-11 15:39 -0400
        Re: Recursion error in metaclass Daniel Urban <urban.dani@gmail.com> - 2011-06-11 22:51 +0200

#7430 — Recursion error in metaclass

I have a metaclass in Python 3.1:

class MC1(type):
    @staticmethod
    def get_mro(bases):
        print('get_mro called')
        return type('K', bases, {}).__mro__[1:]
    def __new__(cls, name, bases, dict):
        mro = None
        docstring = dict.get('__doc__')
        if docstring == 'ham':
            mro = cls.get_mro(bases)
            dict['__doc__'] = "spam spam spam"
        # Create the class we want, and return it.
        K = super().__new__(cls, name, bases, dict)
        if mro:
            assert K.__mro__ == (K,) + mro
        return K


It seems to work fine:

>>> class A(metaclass=MC1):
...     pass
...
>>> class B(A):
...     'ham'
...
get_mro called
>>> assert B.__doc__ == 'spam spam spam'
>>>


But if I move the call to get_mro outside of the if block, it works fine 
at first, and then blows up with RecursionError:


class MC2(type):
    @staticmethod
    def get_mro(bases):
        print('get_mro called')
        return type('K', bases, {}).__mro__[1:]
    def __new__(cls, name, bases, dict):
        mro = None
        docstring = dict.get('__doc__')
        mro = cls.get_mro(bases)
        if docstring == 'ham':
            dict['__doc__'] = "spam spam spam"
        # Create the class we want, and return it.
        K = super().__new__(cls, name, bases, dict)
        if mro:
            assert K.__mro__ == (K,) + mro
        return K


>>> class C(metaclass=MC2):
...     pass
...
get_mro called
>>>
>>> sys.setrecursionlimit(15)
>>> class D(C):
...     'ham'
...
get_mro called
get_mro called
get_mro called
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 9, in __new__
  File "<stdin>", line 5, in get_mro
  File "<stdin>", line 9, in __new__
  File "<stdin>", line 5, in get_mro
  File "<stdin>", line 9, in __new__
  File "<stdin>", line 4, in get_mro
RuntimeError: maximum recursion depth exceeded while calling a Python 
object



I am utterly perplexed. What's going on here?




-- 
Steven

[toc] | [next] | [standalone]

#7435

On 6/10/2011 11:34 PM, Steven D'Aprano wrote:
> I have a metaclass in Python 3.1:
>
> class MC1(type):
>      @staticmethod
>      def get_mro(bases):
>          print('get_mro called')
>          return type('K', bases, {}).__mro__[1:]

The call to type figures out the proper metaclass from bases and 
forwards the call to that (or to its __new__ method). See 
Objects/typeobject.c in the source, or read the docs on metaclasses 
carefully. If the proper metaclass is MC1, ...


>      def __new__(cls, name, bases, dict):
>          mro = None
>          docstring = dict.get('__doc__')
>          if docstring == 'ham':
>              mro = cls.get_mro(bases)

and you unconditionally call get_mro again, to call this again...

>              dict['__doc__'] = "spam spam spam"
>          # Create the class we want, and return it.
>          K = super().__new__(cls, name, bases, dict)
>          if mro:
>              assert K.__mro__ == (K,) + mro
>          return K

you are in an endless loop.

Since uou do not pass dict to get_mro. it passes {} to type and MC1 and 
the test for docstring fails and the loop is broken and the empty class 
is discarded after getting its mro.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#7446

On Sat, 11 Jun 2011 01:33:25 -0400, Terry Reedy wrote:

> On 6/10/2011 11:34 PM, Steven D'Aprano wrote:
>> I have a metaclass in Python 3.1:
>>
>> class MC1(type):
>>      @staticmethod
>>      def get_mro(bases):
>>          print('get_mro called')
>>          return type('K', bases, {}).__mro__[1:]
> 
> The call to type figures out the proper metaclass from bases and
> forwards the call to that (or to its __new__ method). 
[...]
> Since uou do not pass dict to get_mro. it passes {} to type and MC1 and
> the test for docstring fails and the loop is broken and the empty class
> is discarded after getting its mro.

Thanks for the explanation. You confused me for a while talking about 
MC1, because that's the metaclass that *doesn't* raise an exception, but 
I think I see the issue now.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#7456

On 6/11/2011 7:38 AM, Steven D'Aprano wrote:
> On Sat, 11 Jun 2011 01:33:25 -0400, Terry Reedy wrote:
>
>> On 6/10/2011 11:34 PM, Steven D'Aprano wrote:
>>> I have a metaclass in Python 3.1:
>>>
>>> class MC1(type):
>>>       @staticmethod
>>>       def get_mro(bases):
>>>           print('get_mro called')
>>>           return type('K', bases, {}).__mro__[1:]
>>
>> The call to type figures out the proper metaclass from bases and
>> forwards the call to that (or to its __new__ method).
> [...]
>> Since uou do not pass dict to get_mro. it passes {} to type and MC1 and
>> the test for docstring fails and the loop is broken and the empty class
>> is discarded after getting its mro.
>
> Thanks for the explanation. You confused me for a while talking about
> MC1, because that's the metaclass that *doesn't* raise an exception, but

Sorry, you probably changed that to MC2 for the second example and I did 
not notice. The point is that when either version calls get_mro and 
type, types calls back to the same metaclass, so that unguarding the 
call to get_mro results in looping.

> I think I see the issue now.

What may not be obvious from the docs is that the metaclass calculation 
described in the doc section on class statements is carried out within 
type.__new__ (or after a possible patch, called from within that), so 
that type calls are really "a dynamic form of the class statement" even 
when another another metaclass is specified or implied. "Return a new 
type object." includes instances of type subclasses. I am not sure what 
happens with metaclasses that are not type subclasses. There is at least 
one bug report about the metaclass calculation, which is why I happen to 
have read the typeobject.__new__ code. But I have not read the 
build-class code and all the details of class creation. So I may have 
some of the details above wrong.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#7462

On Sat, Jun 11, 2011 at 21:39, Terry Reedy <tjreedy@udel.edu> wrote:
> What may not be obvious from the docs is that the metaclass calculation
> described in the doc section on class statements is carried out within
> type.__new__ (or after a possible patch, called from within that), so that
> type calls are really "a dynamic form of the class statement" even when
> another another metaclass is specified or implied. "Return a new type
> object." includes instances of type subclasses. I am not sure what happens
> with metaclasses that are not type subclasses. There is at least one bug
> report about the metaclass calculation, which is why I happen to have read
> the typeobject.__new__ code. But I have not read the build-class code and
> all the details of class creation. So I may have some of the details above
> wrong.

Just for the record, I think this is the mentioned bug:
http://bugs.python.org/issue1294232


Daniel

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web