Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #56569 > unrolled thread

closure = decorator?

Back to article view | Back to comp.lang.python

Contents

  closure = decorator? Tim <jtim.arnold@gmail.com> - 2013-10-10 06:51 -0700
    Re: closure = decorator? Chris Angelico <rosuav@gmail.com> - 2013-10-11 01:03 +1100
      Re: closure = decorator? Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-10-10 17:15 +0300
        Re: closure = decorator? Piet van Oostrum <piet@vanoostrum.org> - 2013-10-10 12:31 -0400
          Re: closure = decorator? Roy Smith <roy@panix.com> - 2013-10-10 20:04 -0400
            Re: closure = decorator? Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-10-11 10:14 +0300
              Re: closure = decorator? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-10-11 09:24 +0000
                Re: closure = decorator? Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-10-11 15:01 +0300
                  Re: closure = decorator? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-10-11 16:44 +0000
                    Re: closure = decorator? Terry Reedy <tjreedy@udel.edu> - 2013-10-11 16:51 -0400
                Re: closure = decorator? Franck Ditter <nobody@nowhere.org> - 2013-10-11 14:08 +0200
                  Re: closure = decorator? Terry Reedy <tjreedy@udel.edu> - 2013-10-11 16:55 -0400
    Re: closure = decorator? Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-10-10 17:08 +0300
    Re: closure = decorator? Peter Cacioppi <peter.cacioppi@gmail.com> - 2013-10-12 13:54 -0700
      Re: closure = decorator? Tim <jtim.arnold@gmail.com> - 2013-10-14 09:39 -0700

#56569 — closure = decorator?

I've read a couple of articles about this, but still not sure.
When someone talks about a closure in another language (I'm learning Lua on the side), is that the same concept as a decorator in Python?

It sure looks like it.
thanks,
--Tim

[toc] | [next] | [standalone]

#56572

On Fri, Oct 11, 2013 at 12:51 AM, Tim <jtim.arnold@gmail.com> wrote:
> I've read a couple of articles about this, but still not sure.
> When someone talks about a closure in another language (I'm learning Lua on the side), is that the same concept as a decorator in Python?

No, they're quite different. A decorator (let's look at function
decorators; classes can have them too, and they work pretty much the
same way) is syntactic sugar for this:

def func(args):
    blah blah blah
func = decorator(func)

You can do all sorts of things with that. Even stupid things:

>>> @print
def foo():
pass

<function foo at 0x00F5D8A0>
>>> foo is None
True

Using print as a decorator does work, though hardly usefully :)

A closure, on the other hand, is a function that has some extra context:

def outer(x):
    x += 1
    def inner():
        return x
    return inner

The function inner() "knows" its context. When it's called, it'll use
the same value for x that would have been used in the outer function,
even though it's a separate function:

>>> foo = outer(5)
>>> foo()
6

The terminology is that inner() "closes over" x, if I have that
correct (I've not been all that big in functional programming and
lambda calculus). Someone will correct me if I'm not.

It's very common for a decorator to use closures, but the two are
completely different. They're no more connected than, say, for loops
and lists. They just happen to work well together.

Closures in other languages will, as far as I know, be the same thing
as closures in Python. (And their presence and functionality in
JavaScript leaves me wondering why on earth the 'this' reference can't
be considered "closed over" in the same way. But that's hardly the
worst of the language's warts.)

ChrisA

[toc] | [prev] | [next] | [standalone]

#56576

Chris Angelico writes:

> def outer(x):
>     x += 1
>     def inner():
>         return x
>     return inner
...
> The terminology is that inner() "closes over" x, if I have that
> correct (I've not been all that big in functional programming and
> lambda calculus). Someone will correct me if I'm not.

I don't actually know for sure what the most correct terminology is,
but I like to think that a closure is a procedure (function) closed in
the environment (namespace) where it was created, so even when it is
called from another environment, it uses the one where it was born.

But that may be private to me.

[toc] | [prev] | [next] | [standalone]

#56588

Jussi Piitulainen <jpiitula@ling.helsinki.fi> writes:

> I don't actually know for sure what the most correct terminology is,
> but I like to think that a closure is a procedure (function) closed in
> the environment (namespace) where it was created, so even when it is
> called from another environment, it uses the one where it was born.

I usually say that a closure is a package, containing a function with
some additional data it needs. The data usually is in the form of name
bindings.

On the other hand, an object (instance) is a package containg data, with
one of more functions that work on this data.

So an object is more or less the dual of a closure, and in many cases
they can be used for the same purpose. In most programming languages the
difference is that closures can be called directly, whereas an object
needs to be used with a method call to do the same (this makes Java so
ugly in this area). In Python, however, you can define the __call__
method and with this they become almost identical in behaviour.
-- 
Piet van Oostrum <piet@vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

[toc] | [prev] | [next] | [standalone]

#56621

In article <m2a9ihxf3a.fsf@cochabamba.vanoostrum.org>,
 Piet van Oostrum <piet@vanoostrum.org> wrote:

> I usually say that a closure is a package, containing a function with
> some additional data it needs. The data usually is in the form of name
> bindings.

That's pretty close to the way I think about it.  The way it was 
originally described to me is, "A closure is a function bundled up with 
it's arguments".

To make a real-life analogy, let's say you're modeling a parking garage.  
I want to be able to walk up to the attendant and say, "Please bring my 
car around front at 5 O'Clock.  It's that one" (pointing to the slightly 
dented green Ford in spot 37).  So, you've got a class:

class DeliveryRequest:
   def __init__(self, spot, time):
      self.spot = spot
      self.time = time

Now, over the course of the day, the garage attendants shuffle cars 
around to make room and retrieve vehicles that packed in the back.  
Comes 5 O'Clock, what vehicle do you want the attendant to deliver to 
the front?  The one that was in spot 37 at the time you made the 
request, or the one that's in spot 37 at 5 O'Clock?

Unless you want somebody else's car (perhaps you'd like something better 
than a slightly dented Ford), you want the attendant to capture the 
current state of spot 37 and remember that until 5 O'Clock when it's 
time to go get the car, no matter where it happens to be right now.

That's a closure.

[toc] | [prev] | [next] | [standalone]

#56656

Roy Smith writes:
> In article <m2a9ihxf3a.fsf@cochabamba.vanoostrum.org>,
>  Piet van Oostrum wrote:
> 
> > I usually say that a closure is a package, containing a function
> > with some additional data it needs. The data usually is in the
> > form of name bindings.
> 
> That's pretty close to the way I think about it.  The way it was
> originally described to me is, "A closure is a function bundled up
> with it's arguments".

Really? It should be more like "a function bundled up with some other
function's arguments" and even more like "a function bundled up with
bindings for its free variables".

And the data that makes a function a closure is bindings always, by
definition, not just usually.

> To make a real-life analogy, let's say you're modeling a parking
> garage.  I want to be able to walk up to the attendant and say,
> "Please bring my car around front at 5 O'Clock.  It's that one"
> (pointing to the slightly dented green Ford in spot 37).  So, you've
> got a class:
> 
> class DeliveryRequest:
>    def __init__(self, spot, time):
>       self.spot = spot
>       self.time = time
> 
> Now, over the course of the day, the garage attendants shuffle cars
> around to make room and retrieve vehicles that packed in the back.
> Comes 5 O'Clock, what vehicle do you want the attendant to deliver
> to the front?  The one that was in spot 37 at the time you made the
> request, or the one that's in spot 37 at 5 O'Clock?
> 
> Unless you want somebody else's car (perhaps you'd like something
> better than a slightly dented Ford), you want the attendant to
> capture the current state of spot 37 and remember that until 5
> O'Clock when it's time to go get the car, no matter where it happens
> to be right now.
> 
> That's a closure.

I fail to see a closure here. I see a class. I see an implied object
that could as well be dict(spot=37, time=5). Other entities (garage
and attendants) are not made sufficiently explicit.

There's another, more widely used word that means object. There's
another, more widely used word that means state. (I'm referring to the
words "object" and "state".) I see no need to use "closure" to mean
object or state, especially when the term has a more specific
established meaning.

[toc] | [prev] | [next] | [standalone]

#56669

On Fri, 11 Oct 2013 10:14:29 +0300, Jussi Piitulainen wrote:

> Roy Smith writes:
>> In article <m2a9ihxf3a.fsf@cochabamba.vanoostrum.org>,
>>  Piet van Oostrum wrote:
>> 
>> > I usually say that a closure is a package, containing a function with
>> > some additional data it needs. The data usually is in the form of
>> > name bindings.
>> 
>> That's pretty close to the way I think about it.  The way it was
>> originally described to me is, "A closure is a function bundled up with
>> it's arguments".
> 
> Really? It should be more like "a function bundled up with some other
> function's arguments" and even more like "a function bundled up with
> bindings for its free variables".

Closures have nothing to do with *arguments*. A better definition of a 
closure is that it is a function together with a snapshot of the 
environment it was called from.

def func(arg):
    y = arg + 1
    def inner():
        return y + 1000
    return inner

f = func(1)

At this point, f is a closure. It needs to know the value of y (not the 
argument to func) in order to work, and the implementation is to store 
that information inside f.func_closure (or f.__closure__ in Python 3). 
The part of the calling environment which is saved is y:

py> f.func_closure[0].cell_contents
2


> And the data that makes a function a closure is bindings always, by
> definition, not just usually.

Its not just *any* bindings though, it is specifically bindings to 
variables in the environment from which it was called.


[...]
>> That's a closure.
> 
> I fail to see a closure here. I see a class. I see an implied object
> that could as well be dict(spot=37, time=5). Other entities (garage and
> attendants) are not made sufficiently explicit.

In general, anything you can do with a closure, you can do with an object 
explicitly recording whatever state you want. A closure is just one 
implementation of "callable object with state that can be set when you 
create it". The closure f defined above could instead be written as:

class Func:
    def __init__(self, arg):
        self.y = arg + 1
    def __call__(self):
        return self.y + 1000

f = Func(1)


Which is better? If you want to expose the value of y to the outside 
world to modify, the class solution is better. If you don't, the closure 
is better. Closures tend to be more compact, and I suspect more 
efficient, but there's nothing you can do with one you can't do with the 
other.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#56676

Steven D'Aprano writes:
> On Fri, 11 Oct 2013 10:14:29 +0300, Jussi Piitulainen wrote:
> > Roy Smith writes:
> >> In article <m2a9ihxf3a.fsf@cochabamba.vanoostrum.org>,
> >>  Piet van Oostrum wrote:
> >> 
> >> > I usually say that a closure is a package, containing a
> >> > function with some additional data it needs. The data usually
> >> > is in the form of name bindings.
> >> 
> >> That's pretty close to the way I think about it.  The way it was
> >> originally described to me is, "A closure is a function bundled
> >> up with it's arguments".
> > 
> > Really? It should be more like "a function bundled up with some
> > other function's arguments" and even more like "a function bundled
> > up with bindings for its free variables".
> 
> Closures have nothing to do with *arguments*. A better definition of
> a closure is that it is a function together with a snapshot of the
> environment it was called from.

Well, first, I was only trying to see something good in Piet's and
Roy's formulations.

Second, it's precisely not (a snapshot of) the environment where the
function is *called* from, it's (a snapshot of) the environment where
the function was *created* in. This is the whole *point*.

Third, to be even more pedantic, in the context where I think closures
originally appeared as an innovation, all local variables are bound by
a lambda. There the (non-global) free variables of a function *are*
arguments of *another* function. I can expand on this if you like, but
it will be in terms of another language, and not terribly relevant to
this discussion anyway.

> def func(arg):
>     y = arg + 1
>     def inner():
>         return y + 1000
>     return inner
> 
> f = func(1)
> 
> At this point, f is a closure. It needs to know the value of y (not
> the argument to func) in order to work, and the implementation is to
> store that information inside f.func_closure (or f.__closure__ in
> Python 3).  The part of the calling environment which is saved is y:
> 
> py> f.func_closure[0].cell_contents
> 2

Whether there is a y in the *calling* environment or not is
*irrelevant*.

   >>> (lambda y : func(1))('whatever')()
   1002

> > And the data that makes a function a closure is bindings always,
> > by definition, not just usually.
> 
> Its not just *any* bindings though, it is specifically bindings to
> variables in the environment from which it was called.

In the environment where it was created.

> [...]
> >> That's a closure.
> > 
> > I fail to see a closure here. I see a class. I see an implied
> > object that could as well be dict(spot=37, time=5). Other entities
> > (garage and attendants) are not made sufficiently explicit.
> 
> In general, anything you can do with a closure, you can do with an
> object explicitly recording whatever state you want. A closure is
> just one implementation of "callable object with state that can be
> set when you create it". The closure f defined above could instead
> be written as:
> 
> class Func:
>     def __init__(self, arg):
>         self.y = arg + 1
>     def __call__(self):
>         return self.y + 1000
> 
> f = Func(1)
> 
> Which is better? If you want to expose the value of y to the outside
> world to modify, the class solution is better. If you don't, the
> closure is better. Closures tend to be more compact, and I suspect
> more efficient, but there's nothing you can do with one you can't do
> with the other.

Sure.

[toc] | [prev] | [next] | [standalone]

#56692

On Fri, 11 Oct 2013 15:01:40 +0300, Jussi Piitulainen wrote:

> Steven D'Aprano writes:
>> Closures have nothing to do with *arguments*. A better definition of a
>> closure is that it is a function together with a snapshot of the
>> environment it was called from.
[...]
> Second, it's precisely not (a snapshot of) the environment where the
> function is *called* from, it's (a snapshot of) the environment where
> the function was *created* in. This is the whole *point*.

Ah yes, of course you are right. I actually knew that, it was a slip of 
the brain that I wrote it wrong :-(

Thanks for the correction.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#56710

On 10/11/2013 12:44 PM, Steven D'Aprano wrote:
> On Fri, 11 Oct 2013 15:01:40 +0300, Jussi Piitulainen wrote:
>
>> Steven D'Aprano writes:
>>> Closures have nothing to do with *arguments*. A better definition of a
>>> closure is that it is a function together with a snapshot of the
>>> environment it was called from.
> [...]
>> Second, it's precisely not (a snapshot of) the environment where the
>> function is *called* from, it's (a snapshot of) the environment where
>> the function was *created* in. This is the whole *point*.
>
> Ah yes, of course you are right. I actually knew that, it was a slip of
> the brain that I wrote it wrong :-(
>
> Thanks for the correction.

The closure is also not a 'snapshot' but a reference to (or preservation 
of) (relevant parts of) the environment. A snapshot of the environment 
at the time of definition would have been much easier to implement.

x = 1
def outer():
     y = 1
     def inner():
         return x + y
     y = 2
     return inner
x = 2
print(outer()())
# 4

In a sense, all user functions are closures in that they have, and have 
always had, a reference to their definition module environment -- the 
readonly .__globals__ attribute (probably .func_globals in 2.x).

This lexical, as opposed to dynamic scoping, becomes noticable when one 
import a function from another module, as for testing. Because 
.__globals__ is read-only, one must monkey-patch the module of 
definition to change the function's global (modular) environment, as 
when replacing an object it uses with a mock. I ran into this when 
testing Idle methods that use a tk message box to display a message and 
wait for input (sometimes text, always a mouse click) from a human user.

What is relatively new (and tricky) is capturing local names of 
surrounding functions while maintaining both late binding and 
independent writability for each closure. This last means that the 
following works:

def account():
     balance = 0
     def trans(amt):
         nonlocal balance
         balance += amt
         return balance
     return trans

xmasfund = account()
pettycash = account()
print(xmasfund(100))
# 100
print(pettycash(50))
# 50
print(xmasfund(-100))
# 0
print(pettycash(-25))
# 25

Closures and decorators are *really* two different subjects.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#56677

In article <5257c3dd$0$29984$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> On Fri, 11 Oct 2013 10:14:29 +0300, Jussi Piitulainen wrote:
> 
> > Roy Smith writes:
> >> In article <m2a9ihxf3a.fsf@cochabamba.vanoostrum.org>,
> >>  Piet van Oostrum wrote:
> >> 
> >> > I usually say that a closure is a package, containing a function with
> >> > some additional data it needs. The data usually is in the form of
> >> > name bindings.
> >> 
> >> That's pretty close to the way I think about it.  The way it was
> >> originally described to me is, "A closure is a function bundled up with
> >> it's arguments".
> > 
> > Really? It should be more like "a function bundled up with some other
> > function's arguments" and even more like "a function bundled up with
> > bindings for its free variables".
> 
> Closures have nothing to do with *arguments*. A better definition of a 
> closure is that it is a function together with a snapshot of the 
> environment it was called from.
> 
> def func(arg):
>     y = arg + 1
>     def inner():
>         return y + 1000
>     return inner
> 
> f = func(1)

Maybe a better example of closure would be (just for the nonlocal) :

def fib() :
    (a,b) = (0,1)
    def producer() :
        nonlocal a,b     # Python 3
        old = a
        (a,b) = (b,a+b)
        return old
    return producer

>>> f = fib()
>>> [f() for i in range(10)]
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

> At this point, f is a closure. It needs to know the value of y (not the 
> argument to func) in order to work, and the implementation is to store 
> that information inside f.func_closure (or f.__closure__ in Python 3). 
> The part of the calling environment which is saved is y

Shouldn't it be the (a,b) pair here ? But :

>>> f.__closure__[0].cell_contents    # access to what ?
55

Shouldn't cell_contents keep the current (a,b) pair, a part of the snapshot of
the creation environment (private variables of the closure) ? 
Instead it seems to returns only a (which is the next production)...

    franck

[toc] | [prev] | [next] | [standalone]

#56711

On 10/11/2013 8:08 AM, Franck Ditter wrote:
> In article <5257c3dd$0$29984$c3e8da3$5496439d@news.astraweb.com>,
>   Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
>
>> On Fri, 11 Oct 2013 10:14:29 +0300, Jussi Piitulainen wrote:
>>
>>> Roy Smith writes:
>>>> In article <m2a9ihxf3a.fsf@cochabamba.vanoostrum.org>,
>>>>   Piet van Oostrum wrote:
>>>>
>>>>> I usually say that a closure is a package, containing a function with
>>>>> some additional data it needs. The data usually is in the form of
>>>>> name bindings.
>>>>
>>>> That's pretty close to the way I think about it.  The way it was
>>>> originally described to me is, "A closure is a function bundled up with
>>>> it's arguments".
>>>
>>> Really? It should be more like "a function bundled up with some other
>>> function's arguments" and even more like "a function bundled up with
>>> bindings for its free variables".
>>
>> Closures have nothing to do with *arguments*. A better definition of a
>> closure is that it is a function together with a snapshot of the
>> environment it was called from.
>>
>> def func(arg):
>>      y = arg + 1
>>      def inner():
>>          return y + 1000
>>      return inner
>>
>> f = func(1)
>
> Maybe a better example of closure would be (just for the nonlocal) :
>
> def fib() :
>      (a,b) = (0,1)

     a,b = 0,1 is the same thing.

a and b are separate local names and are in no sense a 'pair'.

>      def producer() :
>          nonlocal a,b     # Python 3
>          old = a
>          (a,b) = (b,a+b)
>          return old
>      return producer
>
>>>> f = fib()
>>>> [f() for i in range(10)]
> [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
>
>> At this point, f is a closure. It needs to know the value of y (not the
>> argument to func) in order to work, and the implementation is to store
>> that information inside f.func_closure (or f.__closure__ in Python 3).
>> The part of the calling environment which is saved is y
>
> Shouldn't it be the (a,b) pair here ? But :
>
>>>> f.__closure__[0].cell_contents    # access to what ?
> 55
>
> Shouldn't cell_contents keep the current (a,b) pair, a part of the snapshot of
> the creation environment (private variables of the closure) ?
> Instead it seems to returns only a (which is the next production)...

Look as f.__closure__[1] (.cell_contents) for b.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#56574

Tim writes:

> I've read a couple of articles about this, but still not sure.
>
> When someone talks about a closure in another language (I'm learning
> Lua on the side), is that the same concept as a decorator in Python?
> 
> It sure looks like it.

I don't see how. Wikipedia's opening paragraph on "closure" seems good
to me - closures are a way to implement lexical scoping when functions
that have free variables are passed as arguments and returned as
values:

<http://en.wikipedia.org/wiki/Closure_(computer_science)>
# In programming languages, a closure (also lexical closure or
# function closure) is a function or reference to a function together
# with a referencing environment—a table storing a reference to each
# of the non-local variables (also called free variables or upvalues)
# of that function.[1] A closure—unlike a plain function
# pointer—allows a function to access those non-local variables even
# when invoked outside its immediate lexical scope.

There's an example in Python on that page.

[toc] | [prev] | [next] | [standalone]

#56758

On Thursday, October 10, 2013 6:51:21 AM UTC-7, Tim wrote:
> I've read a couple of articles about this, but still not sure.
> 
> When someone talks about a closure in another language (I'm learning Lua on the side), is that the same concept as a decorator in Python?
> 
> 
> 
> It sure looks like it.
> 
> thanks,
> 
> --Tim

In the proper lambda calculus, you don't have side effects. So Terry's balance example is helpful for Python, but perhaps it might be better think of it as a Pythonic extension to the lambda calculus closure. In other words, Python's closure handles cases that don't present themselves in the lambda calculus.

When you are taught about closures in a purely formal setting the example will not include a side effect nor any need for a statement like "nonlocal balance". The closed variable (here it is balance) simply remains part of the scope of the inner function and can referenced appropriately. 

This might distract the original questioner, I only mention that "closure" probably means different things to different people.
 

[toc] | [prev] | [next] | [standalone]

#56815

On Saturday, October 12, 2013 4:54:34 PM UTC-4, Peter Cacioppi wrote:
> On Thursday, October 10, 2013 6:51:21 AM UTC-7, Tim wrote:
> 
> > I've read a couple of articles about this, but still not sure.
> > When someone talks about a closure in another language (I'm learning Lua on the side), is that the same concept as a decorator in Python?
>
> > It sure looks like it.
> > 

Thanks everyone for discussing. I see that closures are completely different from decorators and now I'm not really sure how I got them confused in the first place.

I will have to stare at the replies for a while before I can say I really 'get' closures. thanks,
--Tim

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web