Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <54b592ac$0$12995$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.17684.1421168544.18130.python-list@python.org> <54b592ac$0$12995$c3e8da3$5496439d@news.astraweb.com>
Date: Wed, 14 Jan 2015 10:02:36 +0100
Subject: Re: Performance in exec environnements
From: Jean-Baptiste Braun <jbaptiste.braun@gmail.com>
To: python-list@python.org
Content-Type: multipart/alternative; boundary=089e010d849e32ebaa050c99003a
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.17707.1421226164.18130.python-list@python.org>
Lines: 135
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:83739

--089e010d849e32ebaa050c99003a
Content-Type: text/plain; charset=UTF-8

2015-01-13 22:48 GMT+01:00 Steven D'Aprano <
steve+comp.lang.python@pearwood.info>:

> So you have been comparing:
>
>     2
>
> versus
>
>     exec('1+1')
>
>
> The first case just fetches a reference to a pre-existing int object, and
> then deletes the reference. That's fast.
>
> The second case:
>
> - creates a new string '1+1'
> - does a global lookup to find the built-in exec function
> - passes the string to the function
> - the function then parses that string and compiles it to byte-code
> - runs the keyhole optimizer over it
> - and finally executes the byte code for "2", same as above.
>
> Only the last step is the same as your earlier test case.
>
What I don't understand is the ratio between test 2 / 4 and test 1 / 3.

Let 0.0229 sec be the execution time to read a bytecode (1st test).
Executing two times that bytecode takes 0.042 sec (test 3), which looks
coherent.

Let 11.6 sec be the execution time to call the global exec, parse, do some
stuff and read a bytecode (test 2). I'm trying to understand why does doing
the same thing and reading one more bytecode is much longer (15.7 sec) in
comparison.


> Bigger savings come from avoiding exec. Instead, try to use factory
> functions, closures, etc. If you give an example of what you are trying to
> generate with exec, we may be able to offer an alternative.
>
I think I'm going to compile before exec'ing.

What I'm trying to do is to map a transformation description in a markup
langage (XSLT) in python to improve execution time. Here is a
simplification of what it looks like :

XSLT :
<title>
 <xsl:choose>
  <xsl:when test="gender='M'">
   Mr
  </xsl:when>
  <xsl:otherwise>
   Mrs
  </xsl:otherwise>
 </xsl:choose>
</title>

Generated python :
print('<title>')
if gender == 'M':
    print('Mr')
else:
    print('Mrs')
print('</title>')

--089e010d849e32ebaa050c99003a
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">2015=
-01-13 22:48 GMT+01:00 Steven D&#39;Aprano <span dir=3D"ltr">&lt;<a href=3D=
"mailto:steve+comp.lang.python@pearwood.info" target=3D"_blank">steve+comp.=
lang.python@pearwood.info</a>&gt;</span>:<br><blockquote class=3D"gmail_quo=
te" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204=
);padding-left:1ex">
So you have been comparing:<br>
<br>
=C2=A0 =C2=A0 2<br>
<br>
versus<br>
<br>
=C2=A0 =C2=A0 exec(&#39;1+1&#39;)<br>
<br>
<br>
The first case just fetches a reference to a pre-existing int object, and<b=
r>
then deletes the reference. That&#39;s fast.<br>
<br>
The second case:<br>
<br>
- creates a new string &#39;1+1&#39;<br>
- does a global lookup to find the built-in exec function<br>
- passes the string to the function<br>
- the function then parses that string and compiles it to byte-code<br>
- runs the keyhole optimizer over it<br>
- and finally executes the byte code for &quot;2&quot;, same as above.<br>
<br>
Only the last step is the same as your earlier test case.<br></blockquote><=
div>What I don&#39;t understand is the ratio between test 2 / 4 and test 1 =
/ 3.<br><br>Let 0.0229 sec be the execution time to read a bytecode (1st te=
st). Executing two times that bytecode takes  0.042 sec (test 3), which loo=
ks coherent.<br><br>Let 11.6 sec be the execution time to call the global e=
xec, parse, do some stuff and read a bytecode (test 2). I&#39;m trying to u=
nderstand why does doing the same thing and reading one more bytecode is mu=
ch longer (15.7 sec) in comparison.<br></div><div>=C2=A0</div><blockquote c=
lass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px soli=
d rgb(204,204,204);padding-left:1ex">

Bigger savings come from avoiding exec. Instead, try to use factory<br>
functions, closures, etc. If you give an example of what you are trying to<=
br>
generate with exec, we may be able to offer an alternative.<span class=3D""=
><font color=3D"#888888"></font></span><br></blockquote><div>I think I&#39;=
m going to compile before exec&#39;ing.<br><br></div><div>What I&#39;m tryi=
ng to do is to map a transformation description in a markup langage (XSLT) =
in python to improve execution time. Here is a simplification of what it lo=
oks like :<br><br></div><div>XSLT :<br></div><div>&lt;title&gt;<br></div><d=
iv>=C2=A0&lt;xsl:choose&gt;<br></div><div>=C2=A0 &lt;xsl:when test=3D&quot;=
gender=3D&#39;M&#39;&quot;&gt;<br></div><div>=C2=A0=C2=A0 Mr<br></div><div>=
=C2=A0 &lt;/xsl:when&gt;<br></div><div>=C2=A0 &lt;xsl:otherwise&gt;<br></di=
v><div>=C2=A0=C2=A0 Mrs<br></div><div>=C2=A0 &lt;/xsl:otherwise&gt;<br></di=
v><div>=C2=A0&lt;/xsl:choose&gt;<br></div><div>&lt;/title&gt;<br></div><div=
><br></div><div>Generated python :<br></div><div>print(&#39;&lt;title&gt;&#=
39;)<br></div><div>if gender =3D=3D &#39;M&#39;:<br></div><div>=C2=A0=C2=A0=
=C2=A0 print(&#39;Mr&#39;)<br></div><div>else:<br></div><div>=C2=A0=C2=A0=
=C2=A0 print(&#39;Mrs&#39;)<br></div><div>print(&#39;&lt;/title&gt;&#39;)<b=
r></div></div></div></div>

--089e010d849e32ebaa050c99003a--