Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!feeder.news-service.com!xlned.com!feeder5.xlned.com!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Date: Sun, 31 Jul 2011 12:21:11 +0200
From: Laszlo Nagy <gandalf@shopzeus.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110617 Thunderbird/3.1.11
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: eval, exec and execfile dilemma
References: <4E346028.5000302@shopzeus.com>	<CAPTjJmpitXU6mFH1C32=f8CSzL-oDZpx-jNxb0UZAcCZqDtq8A@mail.gmail.com>	<4E34761C.7030609@stoneleaf.us>	<mailman.1677.1312064338.1164.python-list@python.org> <4e349443$0$29975$c3e8da3$5496439d@news.astraweb.com>
In-Reply-To: <4e349443$0$29975$c3e8da3$5496439d@news.astraweb.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1691.1312107740.1164.python-list@python.org>
Lines: 50
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:10623


>> 100 functions/second
>> installed to global namespace doesn't sound well.
> What on earth are you doing needing to use exec to create hundreds of
> functions??????
:-)
> Have you considered not using exec at all, and using a good old-fashioned
> factory function and closures?
>
> def factory(x):
>      def inner(param):
>          return param + x
>      return inner
>
> plusone = factory(1)
> plustwo = factory(2)
>
>
> I'm betting that this will be much faster than exec, and much more readable.
I'm working on a program that creates pivot tables from 
multi-dimensional databases. The user is able to give expressions in a 
tiny language. These expressions are then converted to Python source 
code, and compiled into functions.

The generated function is called with several different numpy arrays. In 
most cases, there are only a few functions are created (e.g. when the 
user changes the expression) and they are called many times. But 
sometimes (for example, when creating charts from the data) I have to 
generate a separate function for every fact set in the database. When 
there are many data series with lots of data in the graph, some 100 
functions needs to be generated very fast.

This cannot be done using factory functions, because the function code 
depends on the user's expression. It COULD be done in a different way: 
parsing the user's expression into an abstract syntax tree and then 
provide methods in the AST to evaluate itself. But this approach would 
use too many python method calls. By generating the function source 
code, I can reduce the number of Python method calls needed from several 
thousand to ten or so. In most cases, the user will enter an expression 
that will produce a lambda function (eval+lambda). But with more 
elaborate expressions, I cannot efficiently convert it to a lambda 
expression.

Of course the user can always write the expression in pure Python source 
code, but there are obvious problems with that...

Best,

    Laszlo