Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #90150 > unrolled thread

functions, optional parameters

Back to article view | Back to comp.lang.python

Contents

  functions, optional parameters Michael Welle <mwe012008@gmx.net> - 2015-05-08 13:59 +0200
    Re: functions, optional parameters Rustom Mody <rustompmody@gmail.com> - 2015-05-08 05:09 -0700
      Re: functions, optional parameters Michael Welle <mwe012008@gmx.net> - 2015-05-08 14:36 +0200
    Re: functions, optional parameters Chris Angelico <rosuav@gmail.com> - 2015-05-08 22:39 +1000
      Re: functions, optional parameters Michael Welle <mwe012008@gmx.net> - 2015-05-08 14:57 +0200
    Re: functions, optional parameters Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-09 01:24 +1000
      Re: functions, optional parameters Chris Angelico <rosuav@gmail.com> - 2015-05-09 02:02 +1000
        Re: functions, optional parameters Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-09 03:36 +1000
          Re: functions, optional parameters Chris Angelico <rosuav@gmail.com> - 2015-05-09 03:49 +1000
            Re: functions, optional parameters Mel Wilson <mwilson@the-wire.com> - 2015-05-08 18:49 +0000
            Re: functions, optional parameters Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-05-09 13:41 +1200
              Re: functions, optional parameters Chris Angelico <rosuav@gmail.com> - 2015-05-09 12:05 +1000
                Re: functions, optional parameters Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-05-09 19:27 +1200
            Re: functions, optional parameters Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-09 12:52 +1000
      Re: functions, optional parameters Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-09 03:26 +1000
      Re: functions, optional parameters Michael Welle <mwe012008@gmx.net> - 2015-05-08 17:50 +0200
        Re: functions, optional parameters Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-09 10:57 -0600
        Re: functions, optional parameters Chris Angelico <rosuav@gmail.com> - 2015-05-10 13:33 +1000
          Re: functions, optional parameters Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-10 15:20 +1000
            Re: functions, optional parameters Chris Angelico <rosuav@gmail.com> - 2015-05-10 18:59 +1000
        Re: functions, optional parameters Rustom Mody <rustompmody@gmail.com> - 2015-05-09 20:35 -0700
          Re: functions, optional parameters Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-10 15:25 +1000
        Re: functions, optional parameters Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-10 12:45 +1000
          Re: functions, optional parameters Dave Angel <davea@davea.name> - 2015-05-10 07:25 -0400
          Re: functions, optional parameters Chris Angelico <rosuav@gmail.com> - 2015-05-11 00:39 +1000
          Re: functions, optional parameters Michael Welle <mwe012008@gmx.net> - 2015-05-11 07:58 +0200
      Re: functions, optional parameters Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-08 09:48 -0600

Page 2 of 2 — ← Prev page 1 [2]

#90259

On Sunday, May 10, 2015 at 8:16:07 AM UTC+5:30, Steven D'Aprano wrote:
> I predict that the majority of the time, late binding would just be a
> pointless waste of time:
> 
> def process_string(thestr, start=0, end=None, slice=1, reverse=True):
>     pass
> 
> Why would you want 0, None, 1 and True to be re-evaluated every time?
> Admittedly it will be fast, but not as fast as evaluating them once, then
> grabbing a static default value when needed. (See below for timings.)
> 

And what is the work involved in (re)computing 0, None, 1, True??

If I write (... arg=square_root_of_grahams_number())
I would expect to pay for it.
If I write trivial defaults, then I expect trivial payment.

[toc] | [prev] | [next] | [standalone]

#90262

On Sun, 10 May 2015 01:35 pm, Rustom Mody wrote:

> On Sunday, May 10, 2015 at 8:16:07 AM UTC+5:30, Steven D'Aprano wrote:
>> I predict that the majority of the time, late binding would just be a
>> pointless waste of time:
>> 
>> def process_string(thestr, start=0, end=None, slice=1, reverse=True):
>>     pass
>> 
>> Why would you want 0, None, 1 and True to be re-evaluated every time?
>> Admittedly it will be fast, but not as fast as evaluating them once, then
>> grabbing a static default value when needed. (See below for timings.)
>> 
> 
> And what is the work involved in (re)computing 0, None, 1, True??

Re-computing a constant is about 5 times more expensive than re-using it,
according to my earlier timing tests. So if you have four of them, there
will be about 20 times more overhead due to the defaults, each and every
time you call the function.

Setting the defaults isn't the only source of overhead, but my guestimate is
that switching to late binding would probably double the overall overhead
of calling a function with one or two defaults. If your function is
expensive, that's trivial, but for small fast functions, that will be
painful. Python's slow enough without making it slower for dubious gains.


> If I write (... arg=square_root_of_grahams_number())
> I would expect to pay for it.

Sure, but only once. If you think that Graham's Number is likely to change
*wink* then you can put it into the body of the function, like any other
code you want run every time you call the function.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#90260

On Sat, 9 May 2015 01:50 am, Michael Welle wrote:

[...]
>> How about this definition:
>>
>>     default = 23
>>     def spam(eggs=default):
>>         pass
>>
>>     del default
>>
>>     print spam()
>>
>>
>> Do you expect the function call to fail because `default` doesn't exist?
> 
> If I reference an object, that isn't available in the current context, I
> want to see it fail, yes.

Well, that's an interesting response. Of course I agree with you if the
reference to default is in the code being executed:

def spam():
    value = default


that's quite normal rules for Python functions. 

Aside: note that *closures* behave differently, by design: a closure will
keep non-local values alive even if the parent function is deleted.

py> def outer():
...     default = 23
...     def closure():
...             return default
...     return closure
...
py> f = outer()
py> del outer
py> f()
23


But I don't agree with you about default parameters. Suppose we do this:

default = 23
eggs = default
# some time later
del default
print(eggs)

I trust that you agree that eggs shouldn't raise a NameError here just
because default no longer exists!

Why should that be any different just because the assignment is inside a
parameter list?

def spam(eggs=default):
    ...


One of the nice things about Python's current behaviour is that function
defaults don't behave any differently from any other name binding. Python
uses the same semantics for binding names wherever the name is without the
need for users to memorise a bunch of special rules.

Things which look similar should behave similarly.


>> My answers to those questions are all No.
>
> Different answers are possible as it seems ;).

Obviously :-)

And if Python used late binding, as some other languages do (Lisp, I think),
we would have a FAQ 

"Q: Why does my function run slowly/raise an exception when I use a default
value?"

"A: Because the default is re-evaluated every time you call the function,
not just once when you define it."


>> To me, it is not only expected,
>> but desirable that function defaults are set once, not every time the
>> function is called. This behaviour is called "early binding" of defaults.
>>
>> The opposite behaviour is called "late binding".
>>
>> If your language uses late binding, it is very inconvenient to get early
>> binding when you want it. But if your language uses early binding, it is
>> very simple to get late binding when you want it: just put the code you
>> want to run inside the body of the function:
>
> And you have to do it all the time again and again. I can't provide hard
> numbers, but I think usually I want late binding.

I'm pretty sure that you don't. You just think you do because you're
thinking of the subset of cases where you want to use a mutable default
like [], or perhaps delay looking up a global default until runtime, and
not thinking of all the times you use a default.

I predict that the majority of the time, late binding would just be a
pointless waste of time:

def process_string(thestr, start=0, end=None, slice=1, reverse=True):
    pass

Why would you want 0, None, 1 and True to be re-evaluated every time?
Admittedly it will be fast, but not as fast as evaluating them once, then
grabbing a static default value when needed. (See below for timings.)

Whether you use early or late binding, Python still has to store the
default, then retrieve it at call-time. What happens next depends on the
binding model.

With early binding, Python has the value, and can just use it directly. With
late binding, it needs to store a delayed computation object, an executable
expression if you prefer. There are two obvious ways to implement such a
thunk in Python: a code object, or a function.

thunk = compile('0', '', 'eval')  # when the function is defined
value = eval(thunk)  # when the function is called

# or

thunk = lambda: 0
value = thunk()


Both of those are considerably slower than the current behaviour:

py> from timeit import Timer
py> static = Timer("x = 0")
py> thunk = Timer("x = eval(t)", setup="t = compile('0', '', 'eval')")
py> func = Timer("x = f()", setup="f = lambda: 0")
py> min(static.repeat(repeat=7))  # Best of seven trials.
0.04563648998737335
py> min(thunk.repeat(repeat=7))
1.2324241530150175
py> min(func.repeat(repeat=7))
0.20116623677313328


It would be nice to have syntax for late binding, but given that we don't,
and only have one or the other, using early binding is much more sensible.


This is the point where some people try to suggest some sort of complicated,
fragile, DWIM heuristic where the compiler tries to guess whether the user
actually wants the default to use early or late binding, based on what the
expression looks like. "0 is an immutable int, use early binding; [] is a
mutable list, use late binding." sort of thing. Such a thing might work
well for the obvious cases, but it would be a bugger to debug and
work-around for the non-obvious cases when it guesses wrong -- and it will.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#90285

On 05/09/2015 11:33 PM, Chris Angelico wrote:
> On Sun, May 10, 2015 at 12:45 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> This is the point where some people try to suggest some sort of complicated,
>> fragile, DWIM heuristic where the compiler tries to guess whether the user
>> actually wants the default to use early or late binding, based on what the
>> expression looks like. "0 is an immutable int, use early binding; [] is a
>> mutable list, use late binding." sort of thing. Such a thing might work
>> well for the obvious cases, but it would be a bugger to debug and
>> work-around for the non-obvious cases when it guesses wrong -- and it will.
>
> What you could have is "late-binding semantics, optional early binding
> as an optimization but only in cases where the result is
> indistinguishable". That would allow common cases (int/bool/str/None
> literals) to be optimized, since there's absolutely no way for them to
> evaluate differently.
>

Except for literals, True, False and None, I can't see any way to 
optimize such a thing.  Just because the name on the right side 
references an immutable object at compile time, it doesn't follow that 
it'll still be the same object later.

Unless late binding means something very different than I understood.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#90288

On Sun, May 10, 2015 at 9:25 PM, Dave Angel <davea@davea.name> wrote:
> On 05/09/2015 11:33 PM, Chris Angelico wrote:
>> What you could have is "late-binding semantics, optional early binding
>> as an optimization but only in cases where the result is
>> indistinguishable". That would allow common cases (int/bool/str/None
>> literals) to be optimized, since there's absolutely no way for them to
>> evaluate differently.
>>
>
> Except for literals, True, False and None, I can't see any way to optimize
> such a thing.

I did specifically say "literals" :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#90346

Hello,

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:

> On Sat, 9 May 2015 01:50 am, Michael Welle wrote:
[...]
>>> How about this definition:
>>>
>>>     default = 23
>>>     def spam(eggs=default):
>>>         pass
>>>
>>>     del default
>>>
>>>     print spam()
>>>
>>>
>>> Do you expect the function call to fail because `default` doesn't exist?
>> 
>> If I reference an object, that isn't available in the current context, I
>> want to see it fail, yes.
>
> Well, that's an interesting response. Of course I agree with you if the
> reference to default is in the code being executed:
Uhhm, I change my mind ;). I overread the del statement and interpreted
the example different. I guess, I share your opinion on this example.

[due to quota limit of the news server I had to delete more text than I wished ]
> thunk = lambda: 0
> value = thunk()
>
>
> Both of those are considerably slower than the current behaviour:
>
> py> from timeit import Timer
> py> static = Timer("x = 0")
> py> thunk = Timer("x = eval(t)", setup="t = compile('0', '', 'eval')")
> py> func = Timer("x = f()", setup="f = lambda: 0")
> py> min(static.repeat(repeat=7))  # Best of seven trials.
> 0.04563648998737335
> py> min(thunk.repeat(repeat=7))
> 1.2324241530150175
> py> min(func.repeat(repeat=7))
> 0.20116623677313328
I'm not so deep into the internals of the Python interpreter, CPython in
most cases I think, but I wonder if the generated byte code for the eval
statement from above and a hypothetic function call with late binding
has to be the same? I guess eval comes with some startup costs?


[...]
> This is the point where some people try to suggest some sort of complicated,
> fragile, DWIM heuristic where the compiler tries to guess whether the user
> actually wants the default to use early or late binding, based on what the
> expression looks like. "0 is an immutable int, use early binding; [] is a
> mutable list, use late binding." sort of thing. Such a thing might work
> well for the obvious cases, but it would be a bugger to debug and
> work-around for the non-obvious cases when it guesses wrong -- and it will.
Well, we all want a mind reading compiler, that spits out the code we
have thought about, don't we ;).

Thanks for the insights in the posting, I appreciate it.

Regards
hmw

-- 
biff4emacsen - A biff-like tool for (X)Emacs
http://www.c0t0d0s0.de/biff4emacsen/biff4emacsen.html
Flood - Your friendly network packet generator
http://www.c0t0d0s0.de/flood/flood.html

[toc] | [prev] | [next] | [standalone]

#90268

[Multipart message — attachments visible in raw view] — view raw

On May 8, 2015 9:26 AM, "Steven D'Aprano" <
steve+comp.lang.python@pearwood.info> wrote:
>
> Do you think that Python will re-compile the body of the function every
time
> you call it? Setting the default is part of the process of compiling the
> function.

To be a bit pedantic, that's not accurate. The default is evaluated when
the function object is created, i.e. when the def statement is executed at
runtime, not when the underlying code object is compiled.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web