Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #83706 > unrolled thread

Performance in exec environnements

Back to article view | Back to comp.lang.python

Contents

  Performance in exec environnements Jean-Baptiste Braun <jbaptiste.braun@gmail.com> - 2015-01-13 18:02 +0100
    Re: Performance in exec environnements Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-01-14 08:48 +1100
      Re: Performance in exec environnements Jean-Baptiste Braun <jbaptiste.braun@gmail.com> - 2015-01-14 10:02 +0100
        Re: Performance in exec environnements Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-01-15 12:51 +1100
      Re: Performance in exec environnements Chris Angelico <rosuav@gmail.com> - 2015-01-14 22:14 +1100
      Re: Performance in exec environnements Jean-Baptiste Braun <jbaptiste.braun@gmail.com> - 2015-01-14 14:38 +0100
      Re: Performance in exec environnements Chris Angelico <rosuav@gmail.com> - 2015-01-15 00:43 +1100
      Re: Performance in exec environnements Ian Kelly <ian.g.kelly@gmail.com> - 2015-01-14 10:11 -0700

#83706 — Performance in exec environnements

[Multipart message — attachments visible in raw view] — view raw

Hi,

I'm working on auto-generated python code with the exec function. I've done
some performance benches :

% python -m timeit '1 + 1'
10000000 loops, best of 3: 0.0229 usec per loop

% python -m timeit "exec('1 + 1')"
100000 loops, best of 3: 11.6 usec per loop

-> Maybe creating an exec environnement (I don't know how it works) takes
time. But :

% python -m timeit "1 + 1; 1 + 1"
10000000 loops, best of 3: 0.042 usec per loop

% python -m timeit "exec('1 + 1; 1 + 1')"
100000 loops, best of 3: 15.7 usec per loop

-> As if executing one more 1 + 1 would take 4 more seconds (100000
iterations) in an exec environnement.

Am I missing something or should I expect that result ? What does using
exec imply that causes such a difference ?

Jean-Baptiste Braun

[toc] | [next] | [standalone]

#83727

Jean-Baptiste Braun wrote:

> Hi,
> 
> I'm working on auto-generated python code with the exec function. I've
> done some performance benches :

[snip timing results]

> Am I missing something or should I expect that result ? What does using
> exec imply that causes such a difference ?

exec'ing a string will be slower than directly executing code.

Firstly, your initial test:

    python -m timeit "1 + 1"

 is probably biased. Python has a keyhole optimizer which does constant
folding, so that is equivalent to:

    python -m timeit "2"

which does very little work. To get a more realistic timing, try this:

    python -m timeit "n = 1; n + 1"

which will defeat the peephole optimizer.

So you have been comparing:

    2

versus

    exec('1+1')


The first case just fetches a reference to a pre-existing int object, and
then deletes the reference. That's fast.

The second case:

- creates a new string '1+1'
- does a global lookup to find the built-in exec function
- passes the string to the function
- the function then parses that string and compiles it to byte-code
- runs the keyhole optimizer over it
- and finally executes the byte code for "2", same as above.

Only the last step is the same as your earlier test case.


In my experience, the difference between running a piece of code, and
running the same code as a string passed to exec, will be *at least* a
factor of 10 slowdown.

Sometimes you can speed things up by pre-compiling code with compile(), or
with little micro-optimization tricks such as:

def f():
    for x in seq:
        exec(x)  # Looks up the global exec many times.


def f():
    ex = exec  # Look up the global once.
    for x in seq:
        ex(x)  # fast local variable lookup


But that is a micro-optimization which might shave off a few microseconds
per loop, don't expect it to give big savings.

Bigger savings come from avoiding exec. Instead, try to use factory
functions, closures, etc. If you give an example of what you are trying to
generate with exec, we may be able to offer an alternative.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#83739

[Multipart message — attachments visible in raw view] — view raw

2015-01-13 22:48 GMT+01:00 Steven D'Aprano <
steve+comp.lang.python@pearwood.info>:

> So you have been comparing:
>
>     2
>
> versus
>
>     exec('1+1')
>
>
> The first case just fetches a reference to a pre-existing int object, and
> then deletes the reference. That's fast.
>
> The second case:
>
> - creates a new string '1+1'
> - does a global lookup to find the built-in exec function
> - passes the string to the function
> - the function then parses that string and compiles it to byte-code
> - runs the keyhole optimizer over it
> - and finally executes the byte code for "2", same as above.
>
> Only the last step is the same as your earlier test case.
>
What I don't understand is the ratio between test 2 / 4 and test 1 / 3.

Let 0.0229 sec be the execution time to read a bytecode (1st test).
Executing two times that bytecode takes 0.042 sec (test 3), which looks
coherent.

Let 11.6 sec be the execution time to call the global exec, parse, do some
stuff and read a bytecode (test 2). I'm trying to understand why does doing
the same thing and reading one more bytecode is much longer (15.7 sec) in
comparison.


> Bigger savings come from avoiding exec. Instead, try to use factory
> functions, closures, etc. If you give an example of what you are trying to
> generate with exec, we may be able to offer an alternative.
>
I think I'm going to compile before exec'ing.

What I'm trying to do is to map a transformation description in a markup
langage (XSLT) in python to improve execution time. Here is a
simplification of what it looks like :

XSLT :
<title>
 <xsl:choose>
  <xsl:when test="gender='M'">
   Mr
  </xsl:when>
  <xsl:otherwise>
   Mrs
  </xsl:otherwise>
 </xsl:choose>
</title>

Generated python :
print('<title>')
if gender == 'M':
    print('Mr')
else:
    print('Mrs')
print('</title>')

[toc] | [prev] | [next] | [standalone]

#83787

Jean-Baptiste Braun wrote:

> 2015-01-13 22:48 GMT+01:00 Steven D'Aprano <
> steve+comp.lang.python@pearwood.info>:
> 
>> So you have been comparing:
>>
>>     2
>>
>> versus
>>
>>     exec('1+1')
>>
>>
>> The first case just fetches a reference to a pre-existing int object, and
>> then deletes the reference. That's fast.
>>
>> The second case:
>>
>> - creates a new string '1+1'
>> - does a global lookup to find the built-in exec function
>> - passes the string to the function
>> - the function then parses that string and compiles it to byte-code
>> - runs the keyhole optimizer over it
>> - and finally executes the byte code for "2", same as above.
>>
>> Only the last step is the same as your earlier test case.
>>
> What I don't understand is the ratio between test 2 / 4 and test 1 / 3.
> 
> Let 0.0229 sec be the execution time to read a bytecode (1st test).
> Executing two times that bytecode takes 0.042 sec (test 3), which looks
> coherent.
> 
> Let 11.6 sec be the execution time to call the global exec, parse, do some
> stuff and read a bytecode (test 2). I'm trying to understand why does
> doing the same thing and reading one more bytecode is much longer (15.7
> sec) in comparison.

I don't know. What code are you comparing? If you are comparing these:

exec('1 + 1')
exec('1 + 1; 1 + 1')

the second one has twice as much source code to parse and compile.

You can eliminate much (but not all) of the overhead of exec by pulling the
compilation step out:

[steve@ando ~]$ python -m timeit "exec('1+1')"
100000 loops, best of 3: 18.5 usec per loop
[steve@ando ~]$ python -m timeit -s \
"c = compile('1+1', '', 'exec')" "exec(c)"
1000000 loops, best of 3: 1.49 usec per loop


Obviously this only helps if you are executing the same code repeatedly.


>> Bigger savings come from avoiding exec. Instead, try to use factory
>> functions, closures, etc. If you give an example of what you are trying
>> to generate with exec, we may be able to offer an alternative.
>>
> I think I'm going to compile before exec'ing.
> 
> What I'm trying to do is to map a transformation description in a markup
> langage (XSLT) in python to improve execution time. Here is a
> simplification of what it looks like :
> 
> XSLT :
> <title>
>  <xsl:choose>
>   <xsl:when test="gender='M'">
>    Mr
>   </xsl:when>
>   <xsl:otherwise>
>    Mrs
>   </xsl:otherwise>
>  </xsl:choose>
> </title>
> 
> Generated python :
> print('<title>')
> if gender == 'M':
>     print('Mr')
> else:
>     print('Mrs')
> print('</title>')


I'm not entirely sure of the context here. If you're only executing that "if
gender ==" block once, then trying to optimize this is a waste of time. The
time you take to pre-compile will be at least as much as the time you save
from a single call. Only if you have multiple calls will it help.

Ideally, instead of exec'ing a block of code, it would be better to create a
function and call the function:

# === Don't do this! ===
while parsing XSLT:
    if tag = 'choose':
        handle_title = """print('<title>')
             if gender == 'M': print('Mr')
             else:  print('Mrs')
             print('</title>')"""

# much later...
exec(handle_title)
print "Smith"
exec(handle_title)
print "Jones"


# === This is better ===
while parsing XSLT:
    if tag = 'choose':
        def handle_title():
             print('<title>')
             if gender == 'M': print('Mr')
             else:  print('Mrs')
             print('</title>')


# much later...
handle_title()
print "Smith"
handle_title()
print "Jones"


Understand that this is just a sketch of a solution, not an actual solution.
But the important thing is that if you can build an actual function rather
than exec'ing source code, that will be much faster. If the XSLT file only
has a small number of functions such as "choose", you may be able to use
factory functions to build the functions and avoid exec.

And if not... oh well. There is nothing wrong with using exec. True, it is a
little slower, but how much data do you have to process? Engaging in
unnecessary optimization to shave the run time from 20 minutes to 19
minutes is probably a waste of effort.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#83745

On Wed, Jan 14, 2015 at 8:02 PM, Jean-Baptiste Braun
<jbaptiste.braun@gmail.com> wrote:
> What I'm trying to do is to map a transformation description in a markup
> langage (XSLT) in python to improve execution time. Here is a simplification
> of what it looks like :
>
> XSLT :
> <title>
>  <xsl:choose>
>   <xsl:when test="gender='M'">
>    Mr
>   </xsl:when>
>   <xsl:otherwise>
>    Mrs
>   </xsl:otherwise>
>  </xsl:choose>
> </title>
>
> Generated python :
> print('<title>')
> if gender == 'M':
>     print('Mr')
> else:
>     print('Mrs')
> print('</title>')

Would it be possible to do a one-off transformation of the entire XSLT
file into a Python module with a single function in it, and then every
time you need that XSLT, you import that module and call the function?
That would potentially be a lot quicker than exec(), *and* would be
much clearer - there'd be an actual file on the disk with your
generated Python code, and anyone could read it and understand what
it's doing.

ChrisA

[toc] | [prev] | [next] | [standalone]

#83752

[Multipart message — attachments visible in raw view] — view raw

2015-01-14 12:14 GMT+01:00 Chris Angelico <rosuav@gmail.com>:

> Would it be possible to do a one-off transformation of the entire XSLT
> file into a Python module with a single function in it, and then every
> time you need that XSLT, you import that module and call the function?
> That would potentially be a lot quicker than exec(), *and* would be
> much clearer - there'd be an actual file on the disk with your
> generated Python code, and anyone could read it and understand what
> it's doing.
>
I've done some tests executing compiled generated python and it doesn't
seem to be worth over processing xslt directly. What you propose could be a
solution. In fact I'm not sure yet how it will be designed.

Thank you for feedback anyway.

[toc] | [prev] | [next] | [standalone]

#83754

On Thu, Jan 15, 2015 at 12:38 AM, Jean-Baptiste Braun
<jbaptiste.braun@gmail.com> wrote:
> 2015-01-14 12:14 GMT+01:00 Chris Angelico <rosuav@gmail.com>:
>>
>> Would it be possible to do a one-off transformation of the entire XSLT
>> file into a Python module with a single function in it, and then every
>> time you need that XSLT, you import that module and call the function?
>> That would potentially be a lot quicker than exec(), *and* would be
>> much clearer - there'd be an actual file on the disk with your
>> generated Python code, and anyone could read it and understand what
>> it's doing.
>
> I've done some tests executing compiled generated python and it doesn't seem
> to be worth over processing xslt directly. What you propose could be a
> solution. In fact I'm not sure yet how it will be designed.
>
> Thank you for feedback anyway.

I don't know how often you need to re-process the same XSLT file, but
if you often run the same file (eg with different input data), then
you could create a .py file and import it, and then call it. Something
like this:

# Python code generated from some_file.xslt - DO NOT MODIFY
def process(gender):
    print('<title>')
    if gender == 'M':
        print('Mr')
    else:
        print('Mrs')
    print('</title>')


And then you'd use it thus:

import some_file
some_file.process(gender='M')

Although rather than using print(), it might be better to use yield:

# Python code generated from some_file.xslt - DO NOT MODIFY
def process(gender):
    yield '<title>'
    if gender == 'M':
        yield 'Mr'
    else:
        yield 'Mrs'
    yield'</title>'

And then usage would be:

import some_file
print("\n".join(some_file.process(gender='M')))

which gives you the flexibility of sending it somewhere other than
stdout, without monkey-patching the print function to do something
else.

ChrisA

[toc] | [prev] | [next] | [standalone]

#83769

On Wed, Jan 14, 2015 at 2:02 AM, Jean-Baptiste Braun
<jbaptiste.braun@gmail.com> wrote:
> What I don't understand is the ratio between test 2 / 4 and test 1 / 3.
>
> Let 0.0229 sec be the execution time to read a bytecode (1st test).
> Executing two times that bytecode takes 0.042 sec (test 3), which looks
> coherent.
>
> Let 11.6 sec be the execution time to call the global exec, parse, do some
> stuff and read a bytecode (test 2). I'm trying to understand why does doing
> the same thing and reading one more bytecode is much longer (15.7 sec) in
> comparison.

You've also given it twice as much code to parse and compile.

C:\Users\Ian>python -m timeit "compile('1 + 1', '', 'exec')"
100000 loops, best of 3: 16.8 usec per loop

C:\Users\Ian>python -m timeit "compile('1 + 1; 1 + 1', '', 'exec')"
10000 loops, best of 3: 22.4 usec per loop

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web