Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #90280 > unrolled thread

Calling a function is faster than not calling it?

Back to article view | Back to comp.lang.python

Contents

  Calling a function is faster than not calling it? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-10 19:58 +1000
    Re: Calling a function is faster than not calling it? Christian Gollwitzer <auriocus@gmx.de> - 2015-05-10 12:34 +0200
      Re: Calling a function is faster than not calling it? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-11 01:04 +1000
    Re: Calling a function is faster than not calling it? Peter Otten <__peter__@web.de> - 2015-05-10 12:43 +0200
      Re: Calling a function is faster than not calling it? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-11 00:49 +1000
        Re: Calling a function is faster than not calling it? Peter Otten <__peter__@web.de> - 2015-05-10 18:14 +0200
        Re: Calling a function is faster than not calling it? Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-10 10:25 -0600
    Re: Calling a function is faster than not calling it? Terry Reedy <tjreedy@udel.edu> - 2015-05-10 12:37 -0400
    Re: Calling a function is faster than not calling it? BartC <bc@freeuk.com> - 2015-05-10 22:08 +0100
      Re: Calling a function is faster than not calling it? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-11 13:58 +1000
        Re: Calling a function is faster than not calling it? BartC <bc@freeuk.com> - 2015-05-11 10:50 +0100
          Re: Calling a function is faster than not calling it? Skip Montanaro <skip.montanaro@gmail.com> - 2015-05-11 09:12 -0500
            Re: Calling a function is faster than not calling it? BartC <bc@freeuk.com> - 2015-05-11 16:01 +0100
              Re: Calling a function is faster than not calling it? Skip Montanaro <skip.montanaro@gmail.com> - 2015-05-11 10:13 -0500
        Re: Calling a function is faster than not calling it? Tony the Tiger <tony@tiger.invalid> - 2015-05-15 01:35 +0000
    Re: Calling a function is faster than not calling it? Stefan Behnel <stefan_ml@behnel.de> - 2015-05-11 08:11 +0200
    Re: Calling a function is faster than not calling it? Piet van Oostrum <piet@vanoostrum.org> - 2015-06-22 23:49 +0200

#90280 — Calling a function is faster than not calling it?

Why is calling a function faster than bypassing the function object and
evaluating the code object itself? And not by a little, but by a lot?

Here I have a file, eval_test.py:

# === cut ===
from timeit import Timer

def func():
    a = 2
    b = 3
    c = 4
    return (a+b)*(a-b)/(a*c + b*c)


code = func.__code__
assert func() == eval(code)

t1 = Timer("eval; func()", setup="from __main__ import func")
t2 = Timer("eval(code)", setup="from __main__ import code")

# Best of 10 trials.
print (min(t1.repeat(repeat=10)))
print (min(t2.repeat(repeat=10)))

# === cut ===


Note that both tests include a name lookup for eval, so that as much as
possible I am comparing the two pieces of code on an equal footing.

Here are the results I get:


[steve@ando ~]$ python2.7 eval_test.py
0.804041147232
1.74012994766
[steve@ando ~]$ python3.3 eval_test.py
0.7233301624655724
1.7154695875942707

Directly eval'ing the code object is easily more than twice as expensive
than calling the function, but calling the function has to eval the code
object. That suggests that the overhead of calling the function is
negative, which is clearly ludicrous.

I knew that calling eval() on a string was slow, as it has to parse and
compile the source code into byte code before it can evaluate it, but this
is pre-compiled and shouldn't have that overhead.

So what's going on?



-- 
Steven

[toc] | [next] | [standalone]

#90281

Am 10.05.15 um 11:58 schrieb Steven D'Aprano:
> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?
>
> Here I have a file, eval_test.py:
>
> # === cut ===
> from timeit import Timer
>
> def func():
>      a = 2
>      b = 3
>      c = 4
>      return (a+b)*(a-b)/(a*c + b*c)
>
>
> code = func.__code__
> assert func() == eval(code)
>
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")
>
> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))

what exactly does this mean, are you calling it 10 times? Are you sure 
hat is enaough to reach the granularity of your clock? A benchmark 
should last at least 100ms IMHO.

> # === cut ===
>
>
> Note that both tests include a name lookup for eval, so that as much as
> possible I am comparing the two pieces of code on an equal footing.
>
> Here are the results I get:
>
>
> [steve@ando ~]$ python2.7 eval_test.py
> 0.804041147232
> 1.74012994766
> [steve@ando ~]$ python3.3 eval_test.py
> 0.7233301624655724
> 1.7154695875942707
>
> Directly eval'ing the code object is easily more than twice as expensive
> than calling the function, but calling the function has to eval the code
> object. That suggests that the overhead of calling the function is
> negative, which is clearly ludicrous.
>
> I knew that calling eval() on a string was slow, as it has to parse and
> compile the source code into byte code before it can evaluate it, but this
> is pre-compiled and shouldn't have that overhead.

Does eval(code) lookup a,b and c in the scope of the eval, whereas in 
the func it is bound to locals and optimized out?

I've got no idea of Python's internals, just guessing.


	Christian

[toc] | [prev] | [next] | [standalone]

#90290

On Sun, 10 May 2015 08:34 pm, Christian Gollwitzer wrote:

> Am 10.05.15 um 11:58 schrieb Steven D'Aprano:
>> Why is calling a function faster than bypassing the function object and
>> evaluating the code object itself? And not by a little, but by a lot?
>>
>> Here I have a file, eval_test.py:
>>
>> # === cut ===
>> from timeit import Timer
>>
>> def func():
>>      a = 2
>>      b = 3
>>      c = 4
>>      return (a+b)*(a-b)/(a*c + b*c)
>>
>>
>> code = func.__code__
>> assert func() == eval(code)
>>
>> t1 = Timer("eval; func()", setup="from __main__ import func")
>> t2 = Timer("eval(code)", setup="from __main__ import code")
>>
>> # Best of 10 trials.
>> print (min(t1.repeat(repeat=10)))
>> print (min(t2.repeat(repeat=10)))
> 
> what exactly does this mean, are you calling it 10 times? Are you sure
> hat is enaough to reach the granularity of your clock? A benchmark
> should last at least 100ms IMHO.


No, timeit by default calls the code snippet one million times. So in the
example above, each trial calls "eval(code)" one million times, and I have
ten trials. I pick the fastest trial, and report that.

The time for the fastest trial is 1.7 seconds for one million calls to eval.


>> # === cut ===
>>
>>
>> Note that both tests include a name lookup for eval, so that as much as
>> possible I am comparing the two pieces of code on an equal footing.
>>
>> Here are the results I get:
>>
>>
>> [steve@ando ~]$ python2.7 eval_test.py
>> 0.804041147232
>> 1.74012994766
>> [steve@ando ~]$ python3.3 eval_test.py
>> 0.7233301624655724
>> 1.7154695875942707
>>
>> Directly eval'ing the code object is easily more than twice as expensive
>> than calling the function, but calling the function has to eval the code
>> object. That suggests that the overhead of calling the function is
>> negative, which is clearly ludicrous.
>>
>> I knew that calling eval() on a string was slow, as it has to parse and
>> compile the source code into byte code before it can evaluate it, but
>> this is pre-compiled and shouldn't have that overhead.
> 
> Does eval(code) lookup a,b and c in the scope of the eval, whereas in
> the func it is bound to locals and optimized out?

Obviously I don't *fully* understand the details either, otherwise I
wouldn't be asking, but as far as I understanding, in my example above:

def func():
    a = 2
    b = 3
    c = 4
    return (a+b)*(a-b)/(a*c + b*c)

the function's code object includes a reference to the three constants (2,
3, 4, also None) and the local variable look-ups just grab them from the
internal cache:

func.__code__.co_consts
=> returns (None, 2, 3, 4)

So I can set globals with the same name, then eval the code object, and
still get the right answer:

py> a, b, c = 100, 50, 25
py> func()
-0.25
py> eval(func.__code__)
-0.25



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#90282

Steven D'Aprano wrote:

> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?

> Directly eval'ing the code object is easily more than twice as expensive
> than calling the function, but calling the function has to eval the code
> object. That suggests that the overhead of calling the function is
> negative, which is clearly ludicrous.
> 
> I knew that calling eval() on a string was slow, as it has to parse and
> compile the source code into byte code before it can evaluate it, but this
> is pre-compiled and shouldn't have that overhead.
> 
> So what's going on?

A significant part of the extra time is apparently spent on stack 
inspection:

$ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}' 'f()'
10000000 loops, best of 3: 0.179 usec per loop

$ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}' 
'eval(code)'
1000000 loops, best of 3: 0.852 usec per loop

$ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}' 
'eval(code, ns)'
1000000 loops, best of 3: 0.433 usec per loop

$ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}' 'eval; 
ns; f()'
1000000 loops, best of 3: 0.263 usec per loop

[toc] | [prev] | [next] | [standalone]

#90289

On Sun, 10 May 2015 08:43 pm, Peter Otten wrote:

> A significant part of the extra time is apparently spent on stack
> inspection:

I don't know what you mean by "stack inspection", or how you come to that
conclusion.


> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
> 'f()' 10000000 loops, best of 3: 0.179 usec per loop
> 
> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
> 'eval(code)'
> 1000000 loops, best of 3: 0.852 usec per loop
> 
> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
> 'eval(code, ns)'
> 1000000 loops, best of 3: 0.433 usec per loop

Curious.

If I'm reading that correctly, supplying an explicit namespace cuts the time
by a factor of two. That surprises me, as your function doesn't do any name
lookups in the body of the function, so I would have thought that would be
irrelevant.


> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
> 'eval; ns; f()'
> 1000000 loops, best of 3: 0.263 usec per loop

So, roughly speaking, calling eval(code, ns) takes about 1.6 times as long
as calling the function; calling eval(code) without providing a namespace
takes about 3.2 times as long. That's roughly consistent with the results
I'm getting.  



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#90293

Steven D'Aprano wrote:

> On Sun, 10 May 2015 08:43 pm, Peter Otten wrote:
> 
>> A significant part of the extra time is apparently spent on stack
>> inspection:
> 
> I don't know what you mean by "stack inspection", or how you come to that
> conclusion.
> 
> 
>> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
>> 'f()' 10000000 loops, best of 3: 0.179 usec per loop
>> 
>> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
>> 'eval(code)'
>> 1000000 loops, best of 3: 0.852 usec per loop
>> 
>> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
>> 'eval(code, ns)'
>> 1000000 loops, best of 3: 0.433 usec per loop
> 
> Curious.
> 
> If I'm reading that correctly, supplying an explicit namespace cuts the
> time by a factor of two. That surprises me, as your function doesn't do
> any name lookups in the body of the function, so I would have thought that
> would be irrelevant.

I had a quick look at the implementation in bltinmodule.c, and the only 
piece of code that I could prevent from being run was:

    if (globals == Py_None) {
        globals = PyEval_GetGlobals();
        if (locals == Py_None) {
            locals = PyEval_GetLocals();
            if (locals == NULL)
                return NULL;
        }
    }

When there was an actual speed-up I also had a look at 
PyEval_GetGlobals/Locals() which in turn call 

PyEval_GetFrame() 

and

PyEvalPyFrame_FastToLocalsWithError()

whatever these do. (The first function reminded me of sys._getframe() hence 
the mention of stack inspection)

>> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
>> 'eval; ns; f()'
>> 1000000 loops, best of 3: 0.263 usec per loop
> 
> So, roughly speaking, calling eval(code, ns) takes about 1.6 times as long
> as calling the function; calling eval(code) without providing a namespace
> takes about 3.2 times as long. That's roughly consistent with the results
> I'm getting.
 

[toc] | [prev] | [next] | [standalone]

#90294

On Sun, May 10, 2015 at 10:14 AM, Peter Otten <__peter__@web.de> wrote:
> When there was an actual speed-up I also had a look at
> PyEval_GetGlobals/Locals() which in turn call
>
> PyEval_GetFrame()
>
> and
>
> PyEvalPyFrame_FastToLocalsWithError()
>
> whatever these do. (The first function reminded me of sys._getframe() hence
> the mention of stack inspection)

Based on the names, I surmise that the first one gets the top stack
frame object, and that the second one extracts the "fast" local
variables from the frame object and builds a dict of them for use by
eval.

[toc] | [prev] | [next] | [standalone]

#90296

On 5/10/2015 5:58 AM, Steven D'Aprano wrote:
> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?
>
> Here I have a file, eval_test.py:
>
> # === cut ===
> from timeit import Timer
>
> def func():
>      a = 2
>      b = 3
>      c = 4
>      return (a+b)*(a-b)/(a*c + b*c)
>
>
> code = func.__code__
> assert func() == eval(code)
>
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")

eval has 3 parameters. I believe omitting the last two results in 
globals() and locals() calls, but at least one of them.  If {} is 
passed, access to builtins is added.

> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))

Adding
g = globals()
t3 = Timer("eval(code, g)", setup="from __main__ import code, g")
print (min(t3.repeat(repeat=10)))

 >>>
0.3992733933018515
0.6967548323372563
0.49210603735894587

2/3s of the extra time disappears, but there is still extra time needed 
for processing the two extra arguments. Passing g twice has no effect

> [steve@ando ~]$ python2.7 eval_test.py
> 0.804041147232
> 1.74012994766

> Directly eval'ing the code object is easily more than twice as expensive
> than calling the function, but calling the function has to eval the code

Calling the function executes the code via its .__call__ method.
Perhaps eval wraps the code object in a function object with .__call__. 
  I don't know the comparative internals.


-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#90306

On 10/05/2015 10:58, Steven D'Aprano wrote:
> from timeit import Timer
>
> def func():
>      a = 2
>      b = 3
>      c = 4
>      return (a+b)*(a-b)/(a*c + b*c)
>
>
> code = func.__code__
> assert func() == eval(code)
>
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")
>
> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))

Maybe the overheads of using eval() are significant when calling a 
simple function such as your example.

When I made it do a bit more work:

def func():
     a = 2
     b = 3
     c = 4
     for i in range(50):
         x=(a+b)*(a-b)/(a*c + b*c)
     return (a+b)*(a-b)/(a*c + b*c)

Then the eval() call took only 3% longer rather than 100%.

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#90342

On Mon, 11 May 2015 07:08 am, BartC wrote:

> On 10/05/2015 10:58, Steven D'Aprano wrote:
>> from timeit import Timer
>>
>> def func():
>>      a = 2
>>      b = 3
>>      c = 4
>>      return (a+b)*(a-b)/(a*c + b*c)
>>
>>
>> code = func.__code__
>> assert func() == eval(code)
>>
>> t1 = Timer("eval; func()", setup="from __main__ import func")
>> t2 = Timer("eval(code)", setup="from __main__ import code")
>>
>> # Best of 10 trials.
>> print (min(t1.repeat(repeat=10)))
>> print (min(t2.repeat(repeat=10)))
> 
> Maybe the overheads of using eval() are significant when calling a
> simple function such as your example.
> 
> When I made it do a bit more work:
[...]
> Then the eval() call took only 3% longer rather than 100%.

Well duh :-)

If I use:

def func():
    time.sleep(365*24*60*60)  # Sleep for a year.


then the overhead of calling it will be immeasurably small, regardless of
how we call it. But I don't care about big expensive functions. I actually
care about the fastest way to execute a tiny piece of code, and for that,
the overhead of eval is significant.

But regardless of whether the overhead is 3% or 30%, there is an anomaly
here. Conceptually at least, calling a function does more work than
eval'ing the function's code object: the function evaluates the code
object, plus it has overhead of its own which eval'ing the code object
lacks.

Here is one possible, minimal, pseudo-code implementation of function
__call__:


def __call__(self, *args):
    ns = {}
    for i, parameter in enumerate(self.parameters):
        ns[parameter] = args[i]
    return eval(self.__code__, globals(), ns)

If I extract the call to eval, and run that alone, without needing to set up
function arguments, I should avoid the overhead of the __call__ method. We
should expect that:

    eval(self.__code__, globals(), ns)

should be faster than:

    do_something()
    eval(self.__code__, globals(), ns)

no matter how small and insignificant do_something() actually is. The
anomaly is that, according to my tests, and confirmed by others, this is
not the case. However function __call__ works, it doesn't use eval. So what
does it do?




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#90359

On 11/05/2015 04:58, Steven D'Aprano wrote:
> On Mon, 11 May 2015 07:08 am, BartC wrote:
>
>> On 10/05/2015 10:58, Steven D'Aprano wrote:

>>> def func():
>>>       a = 2
>>>       b = 3
>>>       c = 4
>>>       return (a+b)*(a-b)/(a*c + b*c)

>>> print (min(t1.repeat(repeat=10)))
>>> print (min(t2.repeat(repeat=10)))
>>
>> Maybe the overheads of using eval() are significant when calling a
>> simple function such as your example.
>>
>> When I made it do a bit more work:
> [...]
>> Then the eval() call took only 3% longer rather than 100%.
>
> Well duh :-)
>
> If I use:
>
> def func():
>      time.sleep(365*24*60*60)  # Sleep for a year.
>
>
> then the overhead of calling it will be immeasurably small, regardless of
> how we call it. But I don't care about big expensive functions. I actually
> care about the fastest way to execute a tiny piece of code, and for that,
> the overhead of eval is significant.

Your subject line is misleading. You're not comparing calling a function 
with not calling it. You're comparing different ways of calling it (if 
'calling' means somehow passing control to the body of the function, 
then eventually getting back to where you were).

You just seem surprised that using eval() to do this is slower than a 
direct call. I, knowing nothing about eval (I thought it liked string 
arguments), except that it can be used to execute the body of a 
function, am not that surprised.

  > But regardless of whether the overhead is 3% or 30%, there is an anomaly
> here. Conceptually at least, calling a function does more work than
> eval'ing the function's code object: the function evaluates the code
> object, plus it has overhead of its own which eval'ing the code object
> lacks.
>
> Here is one possible, minimal, pseudo-code implementation of function
> __call__:
>
>
> def __call__(self, *args):
>      ns = {}
>      for i, parameter in enumerate(self.parameters):
>          ns[parameter] = args[i]
>      return eval(self.__code__, globals(), ns)
>
> If I extract the call to eval, and run that alone, without needing to set up
> function arguments, I should avoid the overhead of the __call__ method. We
> should expect that:

Perhaps you're making assumptions about how a normal call is 
implemented. Maybe it's not just implemented on top of eval(). (And it's 
unlikely to use an actual call to __call__, because how would /that/ 
call be implemented; into another call to a nested __call__, etc?)

(I've no idea how Python actually does it, but performing a quick test 
in another (also interpreted) language, calling an empty function func() 
via a function pointer takes twice as long as calling it directly. 
Because it has to do a bit more work.

In that case, because there are two bytecodes to be executed for the 
call, not one, and it needs to check the number of arguments because it 
can't be done when the bytecode is generated.)

> The
> anomaly is that, according to my tests, and confirmed by others, this is
> not the case. However function __call__ works, it doesn't use eval. So what
> does it do?

Is eval() called like a normal function? Then maybe you're forgetting 
the overhead of calling a function. Whatever is saved on calling func(), 
is expended on calling eval(). And eval() has overheads of its own 
because it can't just immediately execute the code like func() can, it 
has to get to that point first. And the 'return' in the code that is 
being executed, can't (I think) return directly to your program, but has 
to first return to eval(), which then does its own return.

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#90382

[Multipart message — attachments visible in raw view] — view raw

On Mon, May 11, 2015 at 4:50 AM, BartC <bc@freeuk.com> wrote:

> You just seem surprised that using eval() to do this is slower than a
> direct call.


Well, it is surprising. Most uses of eval() are to evaluate Python
expressions in string form. That I expect to be quite slow, given the
parsing+byte compilation overhead. I wouldn't expect eval()ing a code
object to be all that different than calling the function containing the
code object.

My guess (without looking in ceval.c) is that the code path through the VM
has been tweaked heavily over the years to try and speed it up. Except for
places where that path overlaps with the path for eval(code_object), I
doubt any attention has been paid to speeding up eval. I could certainly be
way off-base here. Perhaps the path taken through the interpreter for
eval(code_object) is more-or-less a subset of the path taken for a simple
function call.

Skip

[toc] | [prev] | [next] | [standalone]

#90390

On 11/05/2015 15:12, Skip Montanaro wrote:
>
> On Mon, May 11, 2015 at 4:50 AM, BartC <bc@freeuk.com
> <mailto:bc@freeuk.com>> wrote:
>
>     You just seem surprised that using eval() to do this is slower than
>     a direct call.
>
>
> Well, it is surprising. Most uses of eval() are to evaluate Python
> expressions in string form. That I expect to be quite slow, given the
> parsing+byte compilation overhead. I wouldn't expect eval()ing a code
> object to be all that different than calling the function containing the
> code object.

func() (when the call is itself inside a function) has this bytecode:

  11           0 LOAD_GLOBAL              0 (func)
               3 CALL_FUNCTION            0
               6 POP_TOP
               7 LOAD_CONST               0 (None)
              10 RETURN_VALUE


eval(code) (when this is also inside a function, and code is local) has 
this bytecode:

  16           6 LOAD_GLOBAL              0 (eval)
               9 LOAD_FAST                0 (code)
              12 CALL_FUNCTION            1
              15 POP_TOP
              16 LOAD_CONST               0 (None)
              19 RETURN_VALUE


So eval() seems to be just as much a function call as func() is. Except:

(1) It has an extra argument ('code'), in addition to any normal 
arguments of func (0 in this case)

(2) Before it can do whatever func() does, in has to /get there/ first. 
And that can code presumably has a similarly indirect path to return to 
the point to where eval() was called. So there is extra code inside eval 
to deal with this (as well as check whether a string has been passed). 
That code must be the extra overhead.

So it seems clear to me that the eval method has to do everything that 
calling func() has to do, and then some.

(That doesn't necessarily mean it would be slower; whatever goes on 
inside could well have been made less efficient for a direct call than 
for an indirect one via eval. But the results suggest it is slower using 
eval. And it isn't really surprising.)

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#90393

On Mon, May 11, 2015 at 10:01 AM, BartC <bc@freeuk.com> wrote:
> (1) It has an extra argument ('code'), in addition to any normal arguments
> of func (0 in this case)

Which might well push execution down the unoptimized code path. Also,
ISTR that Steven's original timeit runs tacked on a standalone "eval ;
..." to the function call case, so the LOAD_GLOBAL bytecode would have
been done in the function call case as well. He was, I think, doing
what he could to make the two cases as nearly identical as he could.

Skip

[toc] | [prev] | [next] | [standalone]

#90635

On Mon, 11 May 2015 13:58:15 +1000, Steven D'Aprano wrote:

> def func():
>     time.sleep(365*24*60*60)  # Sleep for a year.

Remember not to call that 1,000,000 times.

 /Grrr
-- 
          ___                  ___
 (\_--_/)  | _ ._    _|_|_  _   |o _  _ ._
 ( 9  9 )  |(_)| |\/  |_| |(/_  ||(_|(/_|
 stripes are forever - as overripe ferrets

[toc] | [prev] | [next] | [standalone]

#90348

Steven D'Aprano schrieb am 10.05.2015 um 11:58:
> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?
> 
> Here I have a file, eval_test.py:
> 
> # === cut ===
> from timeit import Timer
> 
> def func():
>     a = 2
>     b = 3
>     c = 4
>     return (a+b)*(a-b)/(a*c + b*c)
> 
> 
> code = func.__code__
> assert func() == eval(code)
> 
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")
> 
> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))
> 
> # === cut ===
> 
> 
> Note that both tests include a name lookup for eval, so that as much as
> possible I am comparing the two pieces of code on an equal footing.
> 
> Here are the results I get:
> 
> 
> [steve@ando ~]$ python2.7 eval_test.py
> 0.804041147232
> 1.74012994766
> [steve@ando ~]$ python3.3 eval_test.py
> 0.7233301624655724
> 1.7154695875942707
> 
> Directly eval'ing the code object is easily more than twice as expensive
> than calling the function, but calling the function has to eval the code
> object.

Well, yes, but it does so directly in C code. What you are essentially
doing here is replacing a part of the fast C code path for executing a
Python function by some mostly equivalent but more general Python code. So,
you're basically replacing a function call by another function call to
eval(), plus some extra generic setup overhead.

Python functions know exactly what they have to do internally in order to
execute. eval() cannot make the same streamlined assumptions.

Stefan

[toc] | [prev] | [next] | [standalone]

#93018

Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?
>
> Here I have a file, eval_test.py:
>
> # === cut ===
> from timeit import Timer
>
> def func():
>     a = 2
>     b = 3
>     c = 4
>     return (a+b)*(a-b)/(a*c + b*c)
>
>
> code = func.__code__
> assert func() == eval(code)
>
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")
>
> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))
>
> # === cut ===
>
>
> Note that both tests include a name lookup for eval, so that as much as
> possible I am comparing the two pieces of code on an equal footing.

They are not on equal footing. The first one only looks up eval, but the second actually calls eval. The overhead of calling eval is what makes the difference. To get them on equal footing you will have to insert an eval call also in the first example.

dummy = compile("0", 'string', "eval")

t1 = Timer("eval(dummy); func()", setup="from __main__ import dummy, func")
t2 = Timer("0; eval(code)", setup="from __main__ import code")

And then you'll see that t1 is slightly slower than t2.
-- 
Piet van Oostrum <piet@vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web