Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #6711 > unrolled thread

Something is rotten in Denmark...

Back to article view | Back to comp.lang.python

Contents

  Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-05-31 01:48 -0500
    Re: Something is rotten in Denmark... Chris Rebert <clp2@rebertia.com> - 2011-05-31 00:00 -0700
    Re: Something is rotten in Denmark... Ian Kelly <ian.g.kelly@gmail.com> - 2011-05-31 01:35 -0600
    Re: Something is rotten in Denmark... Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2011-05-31 13:08 +0300
      Re: Something is rotten in Denmark... Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2011-05-31 12:48 +0200
        Re: Something is rotten in Denmark... Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2011-05-31 15:15 +0300
    Re: Something is rotten in Denmark... Terry Reedy <tjreedy@udel.edu> - 2011-05-31 13:11 -0400
      Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-05-31 15:18 -0500
        Re: Something is rotten in Denmark... Ian Kelly <ian.g.kelly@gmail.com> - 2011-05-31 16:24 -0600
        Re: Something is rotten in Denmark... Terry Reedy <tjreedy@udel.edu> - 2011-05-31 19:14 -0400
          Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-05-31 19:09 -0500
            Re: Something is rotten in Denmark... Terry Reedy <tjreedy@udel.edu> - 2011-06-01 13:11 -0400
              Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-06-01 19:40 -0500
                Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-06-01 19:50 -0500
                  Re: Something is rotten in Denmark... Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-02 04:37 +0000
                Re: Something is rotten in Denmark... Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-02 05:14 +0000
                  Re: Something is rotten in Denmark... Chris Angelico <rosuav@gmail.com> - 2011-06-02 18:02 +1000
                  Re: Something is rotten in Denmark... Alain Ketterlin <alain@dpt-info.u-strasbg.fr> - 2011-06-02 13:00 +0200
                    Re: Something is rotten in Denmark... Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2011-06-02 15:51 +0300
                    Re: Something is rotten in Denmark... Terry Reedy <tjreedy@udel.edu> - 2011-06-02 15:43 -0400
                    Re: Something is rotten in Denmark... Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2011-06-03 11:43 +1200
                      Re: Something is rotten in Denmark... rusi <rustompmody@gmail.com> - 2011-06-02 19:24 -0700
                        Re: Something is rotten in Denmark... Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2011-06-03 09:17 +0300
                          Re: Something is rotten in Denmark... rusi <rustompmody@gmail.com> - 2011-06-05 03:54 -0700
                            Re: Something is rotten in Denmark... Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2011-06-05 15:03 +0300
                              Re: Something is rotten in Denmark... rusi <rustompmody@gmail.com> - 2011-06-05 05:26 -0700
                                Re: Something is rotten in Denmark... Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2011-06-05 16:10 +0300
                      Re: Something is rotten in Denmark... Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2011-06-03 10:30 +0200
                        Re: Something is rotten in Denmark... Ian Kelly <ian.g.kelly@gmail.com> - 2011-06-03 11:53 -0600
                      Re: Something is rotten in Denmark... Alain Ketterlin <alain@dpt-info.u-strasbg.fr> - 2011-06-03 12:35 +0200
                        Re: Something is rotten in Denmark... Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2011-06-03 14:07 +0300
                        Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-06-03 15:38 -0500
                        Re: Something is rotten in Denmark... Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2011-06-04 12:40 +1200
                      Re: Something is rotten in Denmark... Nobody <nobody@nowhere.com> - 2011-06-03 14:07 +0100
                  Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-06-02 10:44 -0500
                  Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-06-02 10:55 -0500
                    Re: Something is rotten in Denmark... Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-02 17:22 +0000
                      Re: Something is rotten in Denmark... Ian Kelly <ian.g.kelly@gmail.com> - 2011-06-02 11:43 -0600
                Re: Something is rotten in Denmark... Terry Reedy <tjreedy@udel.edu> - 2011-06-02 02:02 -0400
                  Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-06-02 11:02 -0500
    Re: Something is rotten in Denmark... Martin Manns <mmanns@gmx.net> - 2011-05-31 23:14 +0200
      Re: Something is rotten in Denmark... Ian Kelly <ian.g.kelly@gmail.com> - 2011-05-31 15:47 -0600
        Re: Something is rotten in Denmark... Martin Manns <mmanns@gmx.net> - 2011-06-01 02:57 +0200
      Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-05-31 16:53 -0500
        Re: Something is rotten in Denmark... harrismh777 <harrismh777@charter.net> - 2011-05-31 17:06 -0500
        Re: Something is rotten in Denmark... Chris Angelico <rosuav@gmail.com> - 2011-06-01 08:39 +1000
        Re: Something is rotten in Denmark... Chris Angelico <rosuav@gmail.com> - 2011-06-01 08:40 +1000

Page 1 of 3  [1] 2 3  Next page →

#6711 — Something is rotten in Denmark...

>>> fs=[]
>>> fs = [(lambda n: i + n) for i in range(10)]
>>> [fs[i](1) for i in range(10)]
[10, 10, 10, 10, 10, 10, 10, 10, 10, 10]         <=== not good

     ( that was a big surprise! . . . )
     ( let's try it another way . . . )


>>> fs =[]
>>> def g(i): return (lambda n: i + n)
>>> fs = [g(i) for i in range(10)]
>>> [fs[i](1) for i in range(10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

      (aaah,  that's better . . . )
      (hmmm, let's try another . . . )


>>> fs =[]
>>> for i in range(10):
	fs.append(lambda n, i=i: i + n)
>>> [fs[i](1) for i in range(10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

      (aaaah, that works too . . . )
      (... and another... )


>>> fs=[]
>>> fs = [(lambda n, i=i: i + n) for i in range(10)]
>>> [fs[i](1) for i in range(10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

      (great! . . . )
      (now then,  what's wrong with this picture ?)

      lambda?  closure?  scope?   bug?

      What is going on with the binding in the first construct... this 
seems to reduce the usefulness of lambda to a considerable extent?





kind regards,
m harris

[toc] | [next] | [standalone]

#6713

On Mon, May 30, 2011 at 11:48 PM, harrismh777 <harrismh777@charter.net> wrote:
>>>> fs=[]
>>>> fs = [(lambda n: i + n) for i in range(10)]
>>>> [fs[i](1) for i in range(10)]
>
> [10, 10, 10, 10, 10, 10, 10, 10, 10, 10]         <=== not good
>
>    ( that was a big surprise! . . . )
<snip>
>     lambda?  closure?  scope?   bug?
>
>     What is going on with the binding in the first construct... this seems
> to reduce the usefulness of lambda to a considerable extent?

http://stackoverflow.com/questions/233673/lexical-closures-in-python
(See 1st and 2nd answers)

Cheers,
Chris

[toc] | [prev] | [next] | [standalone]

#6714

On Tue, May 31, 2011 at 12:48 AM, harrismh777 <harrismh777@charter.net> wrote:
>     What is going on with the binding in the first construct... this seems
> to reduce the usefulness of lambda to a considerable extent?

I don't see why; as you've shown there are a couple of simple ways to
avoid this problem.  The trick is just to recognize that when you have
a closure around a variable, and that variable needs to change, but
the value in the closure needs to be constant, then what you really
need are two separate variables -- the cell variable needs to be
promoted to a local.  How you accomplish that is not terribly
important.

One other technique that is sometimes preferable is functools.partial, e.g.:

fs = [functools.partial(operator.add, i) for i in range(10)]

Tangentially, I'd like to point out that this line:

[fs[i](1) for i in range(10)]

is more naturally written as:

[f(1) for f in fs]

[toc] | [prev] | [next] | [standalone]

#6722

harrismh777 writes:

> >>> fs=[]
> >>> fs = [(lambda n: i + n) for i in range(10)]
> >>> [fs[i](1) for i in range(10)]
> [10, 10, 10, 10, 10, 10, 10, 10, 10, 10]         <=== not good
> 
>      ( that was a big surprise! . . . )
>      ( let's try it another way . . . )

The ten functions share the same i. The list comprehension changes the
value of that i. At the time when the functions are called, the value
is 9.

A different list comprehension mechanism could create a fresh i for
each element instead of changing the value of one i. Then each of the
functions would have a private i which would have the value it had at
the time of the creation of the closure. That is not the Python
mechanism.

The same sharing-an-i thing happens here:

>>> fs = []
>>> for i in range(4):
...    fs.append(lambda n : i + n)
... 
>>> fs[0](0)
3

And the different private-j thing happens here:

>>> gs = []
>>> for i in range(4):
...    gs.append((lambda j : lambda n : j + n)(i))
... 
>>> gs[0](0)
0

You used the lambda itself to introduce its private i in your other
examples, in (lambda n, i=i : i + n). In its i=i, the i to the left is
a different i - will be a fresh i every time the function is called, I
think - while the i to the right gets resolved to the value of the i
that the list comprehension is stepping at the time when the closure
is created.

>       What is going on with the binding in the first
> construct... this seems to reduce the usefulness of lambda to a
> considerable extent?

The lambda is doing its lambda thing exactly. The list comprehension
just updates the one i in whatever you call it that each of the ten
closures remember, and they all observe the updates, so to say.

It's a bit subtle. Using different names might help, like I used j.

[toc] | [prev] | [next] | [standalone]

#6723

Am 31.05.2011 12:08 schrieb Jussi Piitulainen:

> The same sharing-an-i thing happens here:
>
>>>> fs = []
>>>> for i in range(4):
> ...    fs.append(lambda n : i + n)
> ...
>>>> fs[0](0)
> 3
>
> And the different private-j thing happens here:
>
>>>> gs = []
>>>> for i in range(4):
> ...    gs.append((lambda j : lambda n : j + n)(i))
> ...
>>>> gs[0](0)
> 0

There is a simpler way: with

 >>>> fs = []
 >>>> for i in range(4):
 > ...    fs.append(lambda n, i=i: i + n)
 > ...

you give each lambda a different default argument.

 >>>> fs[0](0)
 > 0


Thomas

[toc] | [prev] | [next] | [standalone]

#6727

Thomas Rachel writes:
> Am 31.05.2011 12:08 schrieb Jussi Piitulainen:
> 
> > The same sharing-an-i thing happens here:
> >
> >>>> fs = []
> >>>> for i in range(4):
> > ...    fs.append(lambda n : i + n)
> > ...
> >>>> fs[0](0)
> > 3
> >
> > And the different private-j thing happens here:
> >
> >>>> gs = []
> >>>> for i in range(4):
> > ...    gs.append((lambda j : lambda n : j + n)(i))
> > ...
> >>>> gs[0](0)
> > 0
> 
> There is a simpler way: with
> 
>  >>>> fs = []
>  >>>> for i in range(4):
>  > ...    fs.append(lambda n, i=i: i + n)
>  > ...
> 
> you give each lambda a different default argument.
> 
>  >>>> fs[0](0)
>  > 0

I know, and harrismh777 knows. I find it an unnecessary distraction
when explaining why the different closures in the initial example
behave identically, but I did discuss it at the end of my post.

[toc] | [prev] | [next] | [standalone]

#6745

On 5/31/2011 2:48 AM, harrismh777 wrote:
>>>> fs=[]

Irrelevant here since you immediately rebind 'fs'.

>>>> fs = [(lambda n: i + n) for i in range(10)]
>>>> [fs[i](1) for i in range(10)]

Same as [f(1) for f in fs]

> [10, 10, 10, 10, 10, 10, 10, 10, 10, 10] <=== not good
>
> ( that was a big surprise! . . . )

You have been hypnotizeed by lambda. (lambda n: i+n) is a *constant 
expression*, so you get 10 'equal' functions. To see this better

fs = [(lambda n: i + n) for i in range(10)]
from dis import dis
for f in fs: dis(f)

   1           0 LOAD_DEREF               0 (i)
               3 LOAD_FAST                0 (n)
               6 BINARY_ADD
               7 RETURN_VALUE

   1           0 LOAD_DEREF               0 (i)
               3 LOAD_FAST                0 (n)
               6 BINARY_ADD
               7 RETURN_VALUE
...

All have the same bytecode and all retrieve the same last value of i in 
the nonlocal listcomp scope when you call them *after* the listcomp 
scope has otherwise disappeared.

Your code is equivalent to

fs = []
for i in range(10):
     fs.append(lambda n: i + n)
print([f(1) for f in fs])

which is equivalent (except for naming the functions) to

fs = []
for i in range(10):
     def f(n): return i + n
     fs.append(f)
print([f(1) for f in fs])

Does [10, 10, 10, 10, 10, 10, 10, 10, 10, 10] still surprise?
Because the def is a constant expression, we can move it out of the 
loop, and get the equivalent (except for identity)

def f(n): return i + n
fs = []
for i in range(10):
     fs.append(f)
print([f(1) for f in fs])

This in turn is equivalent to

def f(n): return i + n
fs = []
for _ in range(10):
     fs.append(f)
i=9
print([f(1) for f in fs])

which in turn is equivalent in output to

def f(n): return i + n
i = 9
print([f(1) for _ in range(10)])

Note that:

def f(n): return i+n # or f = lambda n: i+n
fs = [f for i in range(10)]
print([f(1) for f in fs])

works in 2.7, with the same output, but not in 3.2 because in 3.x, i is 
local to the list comp and the later call raises unbound global error.


> ( let's try it another way . . . )

All these other ways create 10 *different* (unequal) functions that are 
different because they have captured 10 different values of i when 
defined instead of deferring lookup of i to when they are called.

def g(i): return (lambda n: i + n)
fs = [g(i) for i in range(10)]
print([f.__closure__[0].cell_contents for f in fs])

fs = [(lambda n, i=i: i + n) for i in range(10)]
print([f.__defaults__[0] for f in fs])

# CPython 3.2 dependent code !!!

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#6755

Terry Reedy wrote:
> You have been hypnotizeed by lambda. (lambda n: i+n) is a *constant
> expression*, so you get 10 'equal' functions.


'hypnotized' indeed!  ... ok, so let me see if I get this... the lambda 
defers lookup|bind of its references until such time as the lambda is 
'called' and not at the time (as I thought) that the anonymous 
function(s) are returned?

If I'm understanding that correctly, then that means lambda is working 
as designed, and that there are very subtle nuances to be aware of. In 
my little case

    (lambda n: i + n)

    ... if the  i  goes out of scope before the anonymous function gets 
called then we have a problem... or if  i   as a reference is mutable or 
refers to a different object before the anonymous function is called 
then we have a problem?

    What I am discovering is that 'yes' I can use lambda syntactically 
where I might not be able to code a def statement; however, if I do use 
it (as in a list comprehension) then I may get unexpected results if any 
of my lambda's references go out of scope or are mutable...(?)

Question:

    What are the ramifications of making the lookup|binding happen at 
the time the anonymous function is 'returned' vs 'called'?  Has anyone 
suggested this?  Is this PEP-able....?  Are there side-effects in the 
other direction?


PS  Thanks Chris, Ian, Jussi, Thomas, Terry...  I appreciate your 
teaching and patience !



kind regards,
m harris

[toc] | [prev] | [next] | [standalone]

#6762

On Tue, May 31, 2011 at 2:18 PM, harrismh777 <harrismh777@charter.net> wrote:
> If I'm understanding that correctly, then that means lambda is working as
> designed, and that there are very subtle nuances to be aware of. In my
> little case
>
>   (lambda n: i + n)
>
>   ... if the  i  goes out of scope before the anonymous function gets called
> then we have a problem... or if  i   as a reference is mutable or refers to
> a different object before the anonymous function is called then we have a
> problem?

Actually, if i merely goes out of scope, there is no problem.  It just
creates a closure.  It's only when the i within that scope is modified
that we run into problems.

In fact, in Python 3 the scope of a list comprehension variable is the
list comprehension itself, so in your original post i was already out
of scope by the time you started calling the lambda functions.

Cheers,
Ian

[toc] | [prev] | [next] | [standalone]

#6766

On 5/31/2011 4:18 PM, harrismh777 wrote:
> Terry Reedy wrote:
>> You have been hypnotizeed by lambda. (lambda n: i+n) is a *constant
>> expression*, so you get 10 'equal' functions.

> 'hypnotized' indeed!

I say 'hypnotized' ;-) because people have posted examples almost 
exactly like the one you did, with a list of 10 (usually) lambda-defined 
functions, at least yearly for over a decade. Never have I seen the same 
puzzlement when the functions are defined with def statements instead.


What you did differently is to continue investigating and discover some 
of the alternatives for yourself. Hence you get more effort from others 
in response.

I think part of the problem with lambda is this: the body of a def 
statement is indented and nicely set apart from surrounding code. The 
body of a lambda expression is prefixed by the header but there is no 
terminator to visually indicate the end of the nested scope. I think 
there should have been. I often add unneeded parens, as you did, but 
that is not lambda specific. Generator expressions have to be set apart 
with parentheses, and other comprehensions are all fenced.

There also seems to be a bit of lambda (oc)cultism and mystique of the 
anonymous. However, if a function raises an exception, anonymity is a 
defect.

What you did differently is to continue investigating and discover some 
of the alternatives for yourself.

 > ... ok, so let me see if I get this... the lambda
> defers lookup|bind of its references until such time as the lambda is
> 'called' and not at the time (as I thought) that the anonymous
> function(s) are returned?

In Python, there are no lambda objects to be called.

A def statement, when executed, produces an instance of the user-defined 
function class. Its .__name__ attribute is the one given in the header 
of the statement.
A lambda expression, when executed, produces an instance of the 
user-defined function class. Its .__name__ attribute is  '<lambda>' (at 
least in CPython).
See the single difference? The objects produced by equivalent def 
statements and lambda expressions are otherwise exactly the same.

statement with lambda arg-list: expression

is an abbreviation for

def new-name(arg-list): return expression
statement with new-name

assuming that the lambda in in the same scope as the statement and not 
nested in another scope-creating expression

> If I'm understanding that correctly, then that means lambda is working
> as designed, and that there are very subtle nuances to be aware of.

I would say that there is no subtle nuance. Except for not providing a 
name, a lambda expression is purely an optional syntactic abbreviation.


> In my little case
>
> (lambda n: i + n)
>
> ... if the i goes out of scope before the anonymous function gets called
> then we have a problem...

There was no problem; nonlocal names and their associations with objects 
get saved (in 'cells', for CPython) association with the function than 
needs them. One of my examples showed how to retrieve them.

 > or if i as a reference is mutable

Objects are mutable, names can be rebound.

 > or refers to a different object before the anonymous function
 > is called then we have a problem?

Anonymity is completely irrelevant to all of the above. Everything is 
the same as for def f(n): return i + n. When a function is compiled, 
each name is classified as local, non-local, or global. When the 
function is called and a name is evaluated, the corresponding object 
from its scope is retrieved, if there is one, or an error is raised.

> What I am discovering is that 'yes' I can use lambda syntactically where
> I might not be able to code a def statement;

If the lambda expression is nested in an abbreviated nested scope 
(lambda or comprehension), then you have to unabbreviate the outer 
nesting before you can unabbreviate the lambda. Several example of this 
have been shown.

> however, if I do use it (as in a list comprehension) then I may get
 > unexpected results if any of my
> lambda's references go out of scope or are mutable...(?)

Nothing needed goes out of scope and all names are rebindable (not
mutable).

> What are the ramifications of making the lookup|binding happen at the
> time the anonymous function is 'returned' vs 'called'?

This is early-binding versus late-binding. Python is a late-binding 
language.

Are you asking about changing all function compilation or only when 
functions are defined with lambda expressions? The latter *would* make 
there be a 'suble nuance' that Python now lacks and does not need.

> Has anyone suggested this?

Of course, more than once, in multiple variations. All have so far been 
rejected, more than once. The most sensible ideas are for earlier 
binding of builtins to make Python run faster.

You have already discovered and been given some of the solutions. 
Another is to define a class with a __call__ statement and save the 
desired early bindings as instance data attributes. Your g(i): return 
lambda ... as well as the default arg trick are functional equivalents 
of the same idea.

In general, if you want to immediately grab and save an object, then 
explicity grab and save the object.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#6768

Terry Reedy wrote:
> This is early-binding versus late-binding. Python is a late-binding
> language.

    ok ...


> Are you asking about changing all function compilation or only when
> functions are defined with lambda expressions?

    At least lambda expressions, but (see below) any other built-in 
where 'early-binding' makes sense for intuitive results or 
performance--- possibly, not commenting.

The latter *would* make
> there be a 'suble nuance' that Python now lacks and does not need.

    How so, and why not?


>> Has anyone suggested this?

>  Of course, more than once, in multiple variations. All have so far been
> rejected, more than once. The most sensible ideas are for earlier
> binding of builtins to make Python run faster.

    At the moment I'm only speaking about my OP and that particular list 
comprehension... the thing that happened (at least for me) is that the 
intuitive sense that each 'i' somehow becomes a part of the anonymous 
function (I know, not so) is built-in. There is little to nothing 
indicating in the docs that this is not so. Again, what we have here is 
the 'i' being saved in a cell and looked up at call time (an 
implementation detail, 'late-binding') that is critical for the 
user-coder to understand. This *may* be true for other built-ins also, 
I'm not commenting on that, but it seems to me that if lambda is going 
to remain in the language at all that 'early-binding' in the lambda 
specific case would make sense;  at least make the lambda more useful 
generally.

... just saying,


kind regards,
m harris

[toc] | [prev] | [next] | [standalone]

#6805

On 5/31/2011 8:09 PM, harrismh777 wrote:

> At the moment I'm only speaking about my OP and that particular list
> comprehension... the thing that happened (at least for me) is that the
> intuitive sense that each 'i' somehow becomes a part of the anonymous
> function (I know, not so) is built-in. There is little to nothing
> indicating in the docs that this is not so

On the contrary, the docs very explicitly say that a lambda expression 
is equivalent to a def statement.

"[Lambda forms (lambda expressions)] are a shorthand to create anonymous 
functions; the expression lambda arguments: expression yields a function 
object. The unnamed object behaves like a function object defined with

def <lambda>(arguments):
     return expression"

? Again, what we have here is
> the 'i' being saved in a cell and looked up at call time (an
> implementation detail, 'late-binding') that is critical for the
> user-coder to understand.

Again, exactly the same as if the function were created with a def 
statement.

> I'm not commenting on that, but it seems to me that if lambda is going
> to remain in the language at all that 'early-binding' in the lambda
> specific case would make sense; at least make the lambda more useful
> generally.

I disagree. Currently, late-binding is the default, with early-binding 
an option through a few different mechanisms. Making early binding the 
default would *reduce* the usefulness by eliminating the late-binding 
option and would add nothing that cannot be done now.

There are some people whose 'intuition' is the opposite of yours. They 
instead want to eliminate the early-binding option of default argument 
expressions. They want to reduce flexibility in the other direction. 
Both proposals are equally bad.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#6828

Terry Reedy wrote:
>> function (I know, not so) is built-in. There is little to nothing
>> indicating in the docs that this is not so
>
> On the contrary, the docs very explicitly say that a lambda expression
> is equivalent to a def statement.

Allow me to clarify... I'm not speaking about whether the lambda is 
short-hand for def, ... that part of the docs I understand well!... no 
problems there.

The part that I don't see much about in the docs (some books, that is) 
is that the lambda lookups occur late (the lambda is evaluated at the 
time it is called). The Python docs on-line *do say* this (I found too 
late) but its one quick phrase that can be missed. So, the i in 
range(10) is sitting there at '9' by the time *any* of the ten lambdas 
get called. This is not intuitive, nor good. IMHO

Please allow me to whine a little bit, ... but the *whole point* of 
iterating is to be able to implicitly grab each iterated value as it 
flies by (by the lambda or anything else!) and there is not much point 
to having a 'late-binding' on an iterable particularly range(n).

Yes, I can explicitly grab each 'i' as it flies by with a little clever 
coding of the default value for the  lambda n, i=i: i + n  but that 
'trick' is not intuitive, nor is it clear reading. It 'works' is just 
about all one can say for it (not very elegant).

I'm not sure what the answer is, but I think all of us need to think 
through it some more. Placing lambdas in a list comprehension is just 
delicious, except for the explicit kludges we have to code to get it to 
work. I'm wondering if whether it would make some sense to put some 
'binding smarts' into the interpreter to allow for 'interpreter 
intuition' (say AI ) that would presume to understand when early vs late 
binding makes sense and apply early binding in those cases where the 
context is not ambiguous and when it is clear that an iterable is being 
passed to the constant lambda function??


kind regards,
m harris

[toc] | [prev] | [next] | [standalone]

#6830

harrismh777 wrote:
> Allow me to clarify... I'm not speaking about whether the lambda is
> short-hand for def, ... that part of the docs I understand well!... no
> problems there.

Allow me to clarify a little further...   the docs are misleading in 
that they state that the lambda can be coded (as an expression) where 
the def 'statement' cannot be coded.   Well, I know, this is speaking to 
the syntax rules not the binding rules, but the point is that it implies 
that the lambda can be used where the def cannot... and this is where 
the hypnosis takes place... we assume that something 'additional' is 
happening with the lambda that is *not* happening with the def.

And the truth is that the def (save its coding syntax) is the 'same' 
critter as the lambda. It seems, in fact, that the only difference is 
two ... that  1) the lambda does not automatically bind to a name, and 
2) the lambda is a constant expression rather than a statement.


<sigh>


thanks for listening...


m harris

[toc] | [prev] | [next] | [standalone]

#6838

On Wed, 01 Jun 2011 19:50:14 -0500, harrismh777 wrote:

> harrismh777 wrote:
>> Allow me to clarify... I'm not speaking about whether the lambda is
>> short-hand for def, ... that part of the docs I understand well!... no
>> problems there.
> 
> Allow me to clarify a little further...   the docs are misleading in
> that they state that the lambda can be coded (as an expression) where
> the def 'statement' cannot be coded.   Well, I know, this is speaking to
> the syntax rules not the binding rules, but the point is that it implies
> that the lambda can be used where the def cannot... 

And so it can.


> and this is where
> the hypnosis takes place... we assume that something 'additional' is
> happening with the lambda that is *not* happening with the def.

This is not a failure of the docs, but of your assumption. The only 
difference is that lambda is an expression , and is limited to a single 
expression. The leap from "lambda is an expression" to "...and therefore 
the thing created by lambda has 'additional' stuff beyond ordinary def 
functions" is unjustified.

Nevertheless, it does seem to be awfully common. You're hardly alone.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#6839

On Wed, 01 Jun 2011 19:40:30 -0500, harrismh777 wrote:

> The part that I don't see much about in the docs (some books, that is)
> is that the lambda lookups occur late (the lambda is evaluated at the
> time it is called). The Python docs on-line *do say* this (I found too
> late) but its one quick phrase that can be missed. So, the i in
> range(10) is sitting there at '9' by the time *any* of the ten lambdas
> get called. This is not intuitive, nor good. IMHO

I agree it's not intuitive. But where does it say that programming 
language semantics must always be intuitive? Whose intuition? Mine? 
Yours? Linus Torvalds'? Donald Knuth's? My auntie Rose's?


> Please allow me to whine a little bit, ... but the *whole point* of
> iterating is to be able to implicitly grab each iterated value as it
> flies by (by the lambda or anything else!) and there is not much point
> to having a 'late-binding' on an iterable particularly range(n).

What do you expect this code to do?

a = 42
funcs = [(lambda x: x+a) for i in range(10)]
funcs[0](1)

a = 23
funcs[0](1)


Do you agree that `a` should be late bound in this situation?

If so, why do you think that `i` should be early bound here?

funcs = [(lambda x: x+i) for i in range(10)]


Oh, the fact that it works at all in Python 2.5 is a side-effect of i 
leaking from the list comprehension:

>>> funcs = [(lambda x: x+i) for i in range(10)]
>>> del i
>>> funcs[0](1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
NameError: global name 'i' is not defined


We can see that the closure isn't created:

>>> funcs[0].func_closure is None
True


However, with a generator expression, i does not leak, a closure is 
created, but it is still late bound:

>>> funcs = list((lambda x: x+i) for i in range(10))
>>> funcs[0].func_closure
(<cell at 0xb7ed44dc: int object at 0x8121ed0>,)
>>> del i
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'i' is not defined
>>> funcs[0](1)
10
>>> funcs[1](1)
10


> Yes, I can explicitly grab each 'i' as it flies by with a little clever
> coding of the default value for the  lambda n, i=i: i + n  but that
> 'trick' is not intuitive, nor is it clear reading. It 'works' is just
> about all one can say for it (not very elegant).

It might be more clear reading if you do it this way:

funcs = [(lambda x, i=j: x+i) for j in range(10)]

Now the reader is no longer distracted by the "i=i" ugliness.


> I'm not sure what the answer is, but I think all of us need to think
> through it some more. Placing lambdas in a list comprehension is just
> delicious, except for the explicit kludges we have to code to get it to
> work. I'm wondering if whether it would make some sense to put some
> 'binding smarts' into the interpreter to allow for 'interpreter
> intuition' (say AI ) that would presume to understand when early vs late
> binding makes sense and apply early binding in those cases where the
> context is not ambiguous and when it is clear that an iterable is being
> passed to the constant lambda function??

The problem with Do What I Mean is that it so rarely Does What You Mean. 
At best it Does What Some Other Guy Imagined I'd Probably Mean In This 
Situation. Let's not go there.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#6845

On Thu, Jun 2, 2011 at 3:14 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> The problem with Do What I Mean is that it so rarely Does What You Mean.
> At best it Does What Some Other Guy Imagined I'd Probably Mean In This
> Situation. Let's not go there.

+1

One of my biggest "threats" to my coworkers goes along the lines of
"Well, go ahead, but you have to document it for the internal wiki".
If you can't document something clearly, it's probably a bad idea.

Chris Angelico

[toc] | [prev] | [next] | [standalone]

#6854

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:

>> The part that I don't see much about in the docs (some books, that is)
>> is that the lambda lookups occur late (the lambda is evaluated at the
>> time it is called). The Python docs on-line *do say* this (I found too
>> late) but its one quick phrase that can be missed. So, the i in
>> range(10) is sitting there at '9' by the time *any* of the ten lambdas
>> get called. This is not intuitive, nor good. IMHO
>
> I agree it's not intuitive. But where does it say that programming 
> language semantics must always be intuitive?

Nowhere. But going against generally accepted semantics should at least
be clearly indicated. Lambda is one of the oldest computing abstraction,
and they are at the core of any functional programming language. Adding
a quick hack to python and call it "lambda" is just abuse of terminology
(I don't say python is the only abuser, implementing lambda is
difficult; see, e.g., Apple C extension called "blocks" and their
implementation of binding).

> What do you expect this code to do?
>
> a = 42
> funcs = [(lambda x: x+a) for i in range(10)]
> funcs[0](1)
[...]
> Do you agree that `a` should be late bound in this situation?

No, absolutely, definitely not. But hey, I didn't learn programming with
python.

-- Alain.

[toc] | [prev] | [next] | [standalone]

#6860

Alain Ketterlin writes:
> Steven D'Aprano writes:
> > I agree it's not intuitive. But where does it say that programming
> > language semantics must always be intuitive?
> 
> Nowhere. But going against generally accepted semantics should at
> least be clearly indicated. Lambda is one of the oldest computing
> abstraction, and they are at the core of any functional programming
> language. Adding a quick hack to python and call it "lambda" is just
> abuse of terminology (I don't say python is the only abuser,
> implementing lambda is difficult; see, e.g., Apple C extension
> called "blocks" and their implementation of binding).

As far as I can see, and that may not be far enough, Python's lambda
expressions do implement a generally accepted semantics. It seems to
be essentially the same semantics that Scheme uses. The value of a
lambda expression is closed in the environment where it is evaluated.
When called, a function looks up the values of its free variables in
the environment where it is closed. (*)

Alonzo Church in his lambda calculus did not deal with variable
assignment at all, as far as I know - he was a logician, not a
programming language designer - and so he also did not need to talk
about _when_ things happen.

Purely functional programming languages also do not have variable
assignment and the whole issue of this thread simply cannot arise.

(*) Some Python folks insist that Python does not have variables or
variable assignment. They talk of names and rebinding of names. And
they call environments namespaces. But the important point is the
underlying reality:

    >>> k=1; f=lambda : k; k=2; f()
    2

Or the same with a closed-over variable that is only accessible
through the values of the lambda expressions at the time they are
called:

    >>> def g(n):
    ...    f1=lambda : n
    ...    n += 1
    ...    f2=lambda : n
    ...    n += 1
    ...    return f1, f2
    ... 
    >>> f,h = g(1)
    >>> f(), h()
    (3, 3)

As expected.

Too long, didn't read? A Schemer finds no fault with Python's lambda.

[toc] | [prev] | [next] | [standalone]

#6882

On 6/2/2011 7:00 AM, Alain Ketterlin wrote:

> Nowhere. But going against generally accepted semantics should at least
> be clearly indicated. Lambda is one of the oldest computing abstraction,
> and they are at the core of any functional programming language. Adding
> a quick hack to python and call it "lambda" is just abuse of terminology

Whether or not it is abuse, I agree it was a big mistake. The keyword 
should have been something like 'func': an abbreviated term for a 
construct with abbreviated capabilities. End of argument over whether 
Python's function expressions matche the semantics of lambda in other 
languages. They are definitely highly limited in the scope of what they 
can do.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | comp.lang.python

csiph-web