Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #17180 > unrolled thread

AttributeError in "with" statement (3.2.2)

Back to article view | Back to comp.lang.python

Contents

  AttributeError in "with" statement (3.2.2) Steve Howell <showell30@yahoo.com> - 2011-12-13 21:42 -0800
    Re: AttributeError in "with" statement (3.2.2) Eric Snow <ericsnowcurrently@gmail.com> - 2011-12-13 23:05 -0700
    Re: AttributeError in "with" statement (3.2.2) Terry Reedy <tjreedy@udel.edu> - 2011-12-14 01:29 -0500
      Re: AttributeError in "with" statement (3.2.2) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-12-14 08:01 +0000
        Re: AttributeError in "with" statement (3.2.2) 88888 Dihedral <dihedral88888@googlemail.com> - 2011-12-14 08:08 -0800
          Re: AttributeError in "with" statement (3.2.2) 88888 Dihedral <dihedral88888@googlemail.com> - 2011-12-14 08:28 -0800
        Re: AttributeError in "with" statement (3.2.2) Steve Howell <showell30@yahoo.com> - 2011-12-14 09:16 -0800
        Re: AttributeError in "with" statement (3.2.2) Terry Reedy <tjreedy@udel.edu> - 2011-12-14 18:13 -0500
          Re: AttributeError in "with" statement (3.2.2) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-12-15 05:01 +0000
            Re: AttributeError in "with" statement (3.2.2) MRAB <python@mrabarnett.plus.com> - 2011-12-15 05:15 +0000
              Re: AttributeError in "with" statement (3.2.2) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-12-15 07:21 +0000
              Re: AttributeError in "with" statement (3.2.2) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2011-12-16 09:34 +1300
            Re: AttributeError in "with" statement (3.2.2) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-12-15 07:47 +0000
            Re: AttributeError in "with" statement (3.2.2) Steve Howell <showell30@yahoo.com> - 2011-12-15 05:35 -0800
              Re: AttributeError in "with" statement (3.2.2) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-12-16 03:34 +0000
            Re: AttributeError in "with" statement (3.2.2) Terry Reedy <tjreedy@udel.edu> - 2011-12-15 19:39 -0500
              Re: AttributeError in "with" statement (3.2.2) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-12-16 09:22 +0000
                Re: AttributeError in "with" statement (3.2.2) Terry Reedy <tjreedy@udel.edu> - 2011-12-16 17:05 -0500
                  Re: AttributeError in "with" statement (3.2.2) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-12-17 01:26 +0000
                    Re: AttributeError in "with" statement (3.2.2) Terry Reedy <tjreedy@udel.edu> - 2011-12-17 21:09 -0500
                Re: AttributeError in "with" statement (3.2.2) Ethan Furman <ethan@stoneleaf.us> - 2011-12-16 15:26 -0800
                  Re: AttributeError in "with" statement (3.2.2) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-12-17 03:05 +0000
                Re: AttributeError in "with" statement (3.2.2) Ethan Furman <ethan@stoneleaf.us> - 2011-12-16 16:34 -0800
    Re: AttributeError in "with" statement (3.2.2) Peter Otten <__peter__@web.de> - 2011-12-14 11:02 +0100
    Re: AttributeError in "with" statement (3.2.2) Eric Snow <ericsnowcurrently@gmail.com> - 2011-12-14 09:56 -0700
    Re: AttributeError in "with" statement (3.2.2) Lie Ryan <lie.1296@gmail.com> - 2011-12-15 06:14 +1100
    Re: AttributeError in "with" statement (3.2.2) Eric Snow <ericsnowcurrently@gmail.com> - 2011-12-14 12:46 -0700

Page 2 of 2 — ← Prev page 1 [2]

#17393

Terry Reedy wrote:
> On 12/16/2011 4:22 AM, Steven D'Aprano wrote:
>> On Thu, 15 Dec 2011 19:39:17 -0500, Terry Reedy wrote:
>> [...]
>>
>> After reading your post, I think I have worked out where our disagreement
>> lies: you think that bound methods and instance methods are not the same
>> thing,
> 
> Do you agree that an unbound method and a bound method are different? In 
> Python, as indicated by the glossary entry, an unspecified 'method' is 
> usually meant to be an unbound method. 

I think you two are in violent agreement as far as how Python is 
functioning, and the conflict is in the names given to the various 
pieces... I think a glossary would help (please correct me):

function:  callable code suite

method:  function that lives in a class

unbound method:  function that lives in a class

bound method:  callable wrapper around function that has been extracted 
from class that will supply the instance object to the function (note: 
Python does not save these, they are recreated at each lookup)


and here is where I think you two diverge:

instance method (Steven):  a bound method that has been saved into the 
instance __dict__ (no matter how created)

instance method (Terry):  a function that must be looked up in the class


Have I missed anything?

Honestly-trying-learn-the-distinctions-ly yours,

~Ethan~

[toc] | [prev] | [next] | [standalone]

#17401

On Fri, 16 Dec 2011 15:26:30 -0800, Ethan Furman wrote:

> Terry Reedy wrote:
>> On 12/16/2011 4:22 AM, Steven D'Aprano wrote:
[...]

> I think you two are in violent agreement as far as how Python is
> functioning, and the conflict is in the names given to the various
> pieces... I think a glossary would help (please correct me):
> 
> function:  callable code suite

Sure, why not?

Informally: a function is some object which can be called and returns a 
result. Hence we sometimes call str() and int() functions even though 
they are actually implemented as types. "Callable" would be better for 
such informal use, but newbies may not recognise the term.

To be more precise: a function is a particular type of callable object 
created by the def and lambda statements.


> method:  function that lives in a class

Informally, I'm happy with that definition, and use it myself. Under 99% 
of circumstances, it would be overly pedantic to describe spam in the 
following as anything other than a method:

class C:
    def spam(self): pass

Call it the duck-type principle: since spam quacks like a duck, we might 
as well call it a duck. If you only care about the easy cases, then spam 
is a method and we're done.

But as a formal definition, it fails on two fronts: functions that live 
inside classes are not actually methods, they are functions, and while 
methods are commonly found in classes, they can also be found in 
instances, or anywhere else for that matter.


Functions inside classes are still functions
--------------------------------------------

I said that since spam quacks like a method, we might as well call it a 
method. But it doesn't swim like a method: it's actually still a function 
with a remarkable power of mimicry.

There are places and times where the fact that spam is not actually a 
method comes through. The three most important examples in my opinion are:

  * inside the class suite, during class definition time, if you use
    or inspect spam, you will see it is a function and not a method;

  * if you bypass the getattr mechanism for looking up attributes, e.g.
    use C.__dict__['spam'] instead of C().spam, you will see that spam
    is stored in the dict as a function object, not a method object;

  * (Python 3 only) if use the standard getattr mechanism, but call it 
    on the class instead of an instance, you will get the function 
    object instead of a method.

So, to be precise: spam is actually a function. It is stored as a 
function inside C.__dict__, but when you retrieve it using C().spam 
Python automagically builds a method object wrapping spam and gives you 
that instead. This distinction is important, well, hardly ever, but on 
the rare occasion it is important, you need to know it.

One example of when it may be important: because methods are created as 
needed, you can't assume that identity checks will necessarily pass for 
methods: `inst.spam is inst.spam` may not be true.


Methods can be created outside of classes
-----------------------------------------

The normal way to get a method is by defining it inside a class. In Java, 
I believe that is the *only* way to get methods, but this is Python, and 
we have other options. Methods are first class objects, which means they 
have a type and a constructor, so if you can get hold of their type, you 
can make them by hand.

You can retrieve the method type two ways:

  * call type on a method you made the usual way: type(C().spam);

  * or import types and use types.MethodType.

Confusingly, the method type calls itself <instancemethod> but is 
exported via the types module as MethodType. Oh well.

MethodType requires two arguments, In Python 2, it accepts an optional 
third argument. Since Python 2 is the past, I'll only talk about the 
Python 3 version, where the signature might look something like this:

MethodType(callable, instance) => instancemethod object

This creates a wrapper around callable (usually, but not necessarily a 
function) and binds instance to that wrapper so the first argument 
(conventionally called self) can be automatically provided.

Another way to create methods is by calling the function object __get__ 
method by hand. That's what gets used in the normal C().spam case.

So you can create methods without putting them inside a class. One trick 
is that you can customise behaviour on a per-instance basis by overriding 
the normal method living inside the class with one which lives inside the 
instance:

class C:
    def spam(self):
        return "Spam spam spam, lovely SPAM!!!"

c = C()
c.spam = types.MethodType(lambda self: "Bloody vikings!", c)

Instance c now has a custom method which overrides C.spam.

[Aside #1: I think this was the original point of contention between 
Terry and I. For reasons I still don't understand, it seems to me that he 
denies that the per-instance method spam is actually a method, since it 
doesn't live inside the class.]

[Aside #2: this per-instance method trick may not work for double-
underscore methods like __exit__, since CPython and Jython at least skip 
the instance __dict__ when using dunder methods. In Python 2, it will 
work for classic classes, but they're gone in Python 3. If you go all the 
way back to the initial post in this thread, that was the OP's problem.]


> unbound method:  function that lives in a class

An unbound method is merely a method that doesn't know what instance to 
use, that is, it cannot automatically supply "self".

In Python 2, the easiest way to get an unbound method is by retrieving 
the method from the class instead of an instance:

  C.spam => returns method which is not bound ("unbound method")
  C().spam => returns method which is bound ("bound method")

Since unbound methods are more or less functionally identical to the 
underlying function (the only difference is they do a type-check on the 
argument), Python 3 gets rid of them and and just uses the function 
object itself.

So all the reasons for why methods are not *strictly* functions living 
inside classes apply equally to unbound methods, since unbound methods 
are merely a particular form of method.


> bound method:  callable wrapper around function that has been extracted
> from class that will supply the instance object to the function (note:
> Python does not save these, they are recreated at each lookup)

Bound methods are not *necessarily* extracted from a class, although they 
*usually* are. But wherever it comes from, a bound method knows which 
instance to provide as first argument ("self") to the function; otherwise 
we'd call it an unbound method.


> and here is where I think you two diverge:
> 
> instance method (Steven):  a bound method that has been saved into the
> instance __dict__ (no matter how created)

Certainly not! If I gave that impression, I apologize.

Every thing I have said above refers to *instance* methods, or just 
"methods" for lazy people. Instance methods get their name from the fact 
that they supply the instance as first argument to the function ("self"). 
If they instead supply the class, we call them a class method; if they 
don't supply anything, we call them a static method (much to the 
confusion of Java programmers, who use the same names for different 
things).

You can even create your own method types:

http://code.activestate.com/recipes/577030-dualmethod-descriptor/




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#17396

Ethan Furman wrote:
> Terry Reedy wrote:
>> On 12/16/2011 4:22 AM, Steven D'Aprano wrote:
>>> On Thu, 15 Dec 2011 19:39:17 -0500, Terry Reedy wrote:
>>> [...]
>>>
>>> After reading your post, I think I have worked out where our 
>>> disagreement
>>> lies: you think that bound methods and instance methods are not the same
>>> thing,
>>
>> Do you agree that an unbound method and a bound method are different? 
>> In Python, as indicated by the glossary entry, an unspecified 'method' 
>> is usually meant to be an unbound method. 
> 
> I think you two are in violent agreement as far as how Python is 
> functioning, and the conflict is in the names given to the various 
> pieces... I think a glossary would help (please correct me):
> 
> function:  callable code suite
> 
> method:  function that lives in a class
> 
> unbound method:  function that lives in a class
> 
> bound method:  callable wrapper around function that has been extracted 
> from class that will supply the instance object to the function (note: 
> Python does not save these, they are recreated at each lookup)

I think the above 'bound method' definition should be attributed to 
Terry, and Steven's follows:

bound method:  callable wrapper around any function that will accept an 
instance object as the first parameter, and the wrapper will supply said 
instance object when calling the function (and where/how function was 
created is irrelevent, as is where the wrapper is stored)
> 
> 
> and here is where I think you two diverge:
> 
> instance method (Steven):  a bound method that has been saved into the 
> instance __dict__ (no matter how created)
> 
> instance method (Terry):  a function that must be looked up in the class
> 
> 
> Have I missed anything?
> 
> Honestly-trying-learn-the-distinctions-ly yours,
> 
> ~Ethan~

[toc] | [prev] | [next] | [standalone]

#17191

Steve Howell wrote:

> I'm using Python 3.2.2, and the following program gives me an error
> that I don't understand:
> 
> class Foo:
>   pass
> 
> foo = Foo()
> foo.name = "Steve"
> 
> def add_goodbye_function(obj):
>   def goodbye():
>     print("goodbye " + obj.name)
>   obj.goodbye = goodbye
> 
> add_goodbye_function(foo)
> foo.goodbye() # outputs goodbye Steve
> foo.__exit__ = foo.goodbye
> foo.__exit__() # outputs goodbye Steve
> 
> with foo: # fails with AttributeError:  __exit__
>   print("doing stuff")
> 
> I am dynamically adding an attribute __exit__ to the variable foo,
> which works fine when I call it directly, but it fails when I try to
> use foo as the expression in the with statement.  Here is the full
> output:
> 
>> python3 with.coffee
> goodbye Steve
> goodbye Steve
> Traceback (most recent call last):
>   File "with.coffee", line 17, in <module>
>     with foo: # fails with AttributeError:
> AttributeError: __exit__

If you don't know it yet, contextlib.contextmanager offers a highlevel 
alternative:

>>> @contextmanager
... def goodbye(obj):
...     try:
...             yield obj
...     finally:
...             print("goodbye", obj.name)
...
>>> with goodbye(Foo()) as foo:
...     print("hello", foo.name)
...
hello Steve
goodbye Steve

[toc] | [prev] | [next] | [standalone]

#17224

On Tue, Dec 13, 2011 at 11:05 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
> On Tue, Dec 13, 2011 at 10:42 PM, Steve Howell <showell30@yahoo.com> wrote:
>> I'm using Python 3.2.2, and the following program gives me an error
>> that I don't understand:
>>
>> class Foo:
>>  pass
>>
>> foo = Foo()
>> foo.name = "Steve"
>>
>> def add_goodbye_function(obj):
>>  def goodbye():
>>    print("goodbye " + obj.name)
>>  obj.goodbye = goodbye
>>
>> add_goodbye_function(foo)
>> foo.goodbye() # outputs goodbye Steve
>> foo.__exit__ = foo.goodbye
>> foo.__exit__() # outputs goodbye Steve
>>
>> with foo: # fails with AttributeError:  __exit__
>>  print("doing stuff")
>>
>> I am dynamically adding an attribute __exit__ to the variable foo,
>> which works fine when I call it directly, but it fails when I try to
>> use foo as the expression in the with statement.  Here is the full
>> output:
>>
>>> python3 with.coffee
>> goodbye Steve
>> goodbye Steve
>> Traceback (most recent call last):
>>  File "with.coffee", line 17, in <module>
>>    with foo: # fails with AttributeError:
>> AttributeError: __exit__
>>
>> What am I doing wrong?
>
> That is a tricky one.
>
> As with many of the special methods (start and end with __) in Python,
> the underlying mechanism in the interpreter is directly pulling the
> function from the class object.  It does not look to the instance
> object for the function at any time.  See
> http://docs.python.org/reference/datamodel.html#special-method-lookup-for-new-style-classes.
>
> -eric

Incidently, if you add the function directly to the class, everything works out:

  class Foo(object): # or "class Foo:" under Python 3
      pass

  foo = Foo()
  foo.name = "Steve"

  def add_goodbye_function(cls):
      def goodbye(self, *args, **kwargs):
          print("goodbye " + self.name)
      cls.goodbye = goodbye

  add_goodbye_function(type(foo))
  foo.goodbye() # outputs goodbye Steve
  Foo.__exit__ = foo.goodbye
  foo.__exit__() # outputs goodbye Steve
  Foo.__enter__ = (lambda self: None)

  with foo:
      print("doing stuff")

However, perhaps a better approach would be to put the different
pieces directly into the class:

  class Foo: # python 3
      def __init__(self, name):
          self.name = name
      def goodbye(self):
          print("goodbye " + self.name)
      def __enter__(self):
          pass
      def __exit__(self, *args, **kwargs):
          self.goodbye()

  foo = Foo("Steve")
  foo.goodbye() # outputs goodbye Steve
  foo.__exit__() # outputs goodbye Steve
  with foo:
      print("doing stuff")

If you want to be more dynamic about it you can do it, but it involves
black magic.  Chances are really good that being explicit through your
class definition is the right approach.

-eric

[toc] | [prev] | [next] | [standalone]

#17229

On 12/15/2011 03:56 AM, Eric Snow wrote:
> On Tue, Dec 13, 2011 at 11:05 PM, Eric Snow<ericsnowcurrently@gmail.com>  wrote:
>
> If you want to be more dynamic about it you can do it, but it involves
> black magic.  Chances are really good that being explicit through your
> class definition is the right approach.

Note that the black spice is to use the __class__ attribute:

foo.__class__.__exit__ = foo.goodbye

[toc] | [prev] | [next] | [standalone]

#17231

On Wed, Dec 14, 2011 at 12:14 PM, Lie Ryan <lie.1296@gmail.com> wrote:
> On 12/15/2011 03:56 AM, Eric Snow wrote:
>>
>> On Tue, Dec 13, 2011 at 11:05 PM, Eric Snow<ericsnowcurrently@gmail.com>
>>  wrote:
>>
>> If you want to be more dynamic about it you can do it, but it involves
>> black magic.  Chances are really good that being explicit through your
>> class definition is the right approach.
>
>
> Note that the black spice is to use the __class__ attribute:
>
> foo.__class__.__exit__ = foo.goodbye

Good point.  'type(foo).__exit__ ...' might be even better.
Regardless, to get the dynamicism to which Steve originally referred
(make it work when added directly to the instance), you have to use a
metaclass, which is black magic[1].  However, rarely is that sort of
thing needed, so the relatively superfluous details would likely only
cloud the question at hand.

-eric

[1] I will point out that metaclasses aren't really all that bad.
They are a tool worth understanding, even if you don't use them all
the time.  Understanding them also opens up a whole new world of
understanding how Python works, particularly regarding name lookup.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web