Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.albasani.net!news.stack.nl!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.001 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'python,': 0.02; 'else:': 0.03; 'cpython': 0.05; '(python': 0.07; 'pypy': 0.07; 'variables': 0.07; 'explanation': 0.09; 'optimizing': 0.09; 'subject:Does': 0.09; 'cc:addr:python-list': 0.11; 'python': 0.11; 'jan': 0.12; 'mostly': 0.14; 'question.': 0.14; 'article:': 0.16; 'ast': 0.16; 'blocks': 0.16; 'bytecode': 0.16; 'email addr:udel.edu>': 0.16; 'email name:<tjreedy': 0.16; 'exception?': 0.16; 'folding.': 0.16; 'learnt': 0.16; 'profiling,': 0.16; 'pypy.': 0.16; 'reedy': 0.16; 'subject:already': 0.16; 'transforming': 0.16; 'work.)': 0.16; 'wrote:': 0.18; 'code.': 0.18; 'obviously': 0.18; 'any,': 0.19; 'feb': 0.22; '>>>': 0.22; 'import': 0.22; 'cc:addr:python.org': 0.22; 'byte': 0.24; 'skip': 0.24; 'mon,': 0.24; 'cc:2**0': 0.24; 'source': 0.25; 'appreciated': 0.26; 'mention': 0.26; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; "doesn't": 0.30; 'moved': 0.30; 'message-id:@mail.gmail.com': 0.30; 'url:mailman': 0.30; 'code': 0.31; 'lines': 0.31; 'constant': 0.31; "d'aprano": 0.31; 'steven': 0.31; 'probably': 0.32; 'another': 0.32; 'open': 0.33; 'url:python': 0.33; 'everyone': 0.33; 'not.': 0.33; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'library.': 0.36; 'url:listinfo': 0.36; 'thanks': 0.36; 'subject:?': 0.36; 'url:org': 0.36; 'should': 0.36; 'list.': 0.37; 'easily': 0.37; 'step': 0.37; 'thank': 0.38; 'improving': 0.38; 'pm,': 0.38; 'expect': 0.39; 'realize': 0.39; 'reported': 0.39; '8bit%:6': 0.40; 'url:mail': 0.40; 'how': 0.40; 'read': 0.60; 'most': 0.60; 'impact': 0.61; 'new': 0.61; 'simple': 0.61; "you're": 0.61; 'save': 0.62; 'show': 0.63; 'kind': 0.63; 'real': 0.63; 'great': 0.65; 'temporary': 0.65; 'effectively': 0.66; 'between': 0.67; 'believe': 0.68; 'skip:w 40': 0.68; 'safe': 0.72; 'apart': 0.72; ':).': 0.84; 'benchmark': 0.84; 'percent,': 0.84; 'resulted': 0.84; 'subject:skip:o 10': 0.84 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=h/DH6o6IUBrcPdJLZBbnFB+WGrxGSUHzT5N2/EoGWoQ=; b=CewLShS9z+tYvgqQB/okvLz4k16ji5KzmCuHUGnFaKywU+B+miocWKBrwPbdW91Ja6 Rt9qvelVIFZDzhdXyI3cyC0rTlre2BERAMeVx3/9Z5MUUMuFLzahEU2g4IO72MHYoQbV qK6kBv7i/QXF3UvvQXLFU1pNRKG9bCuuJ8ZO+9pavWnSOGYoZWUTGAfp19MV5js9vcsd ytW9YKWoNyeqDntUf4dAVVpw/CliLTGAHI3iaPrxbM8uWjJ42o1FWRltPAoPSGa+UYcI FqbOK/NKYQTAv/xuANxjPYrJktNwo/dgYBOZgbYcUqin79bRWknujwD7SucLxmfDOzmk 1K8g== MIME-Version: 1.0 X-Received: by 10.140.85.35 with SMTP id m32mr37724256qgd.40.1392700265714; Mon, 17 Feb 2014 21:11:05 -0800 (PST) In-Reply-To: References: <5301cf82$0$29985$c3e8da3$5496439d@news.astraweb.com> Date: Tue, 18 Feb 2014 10:41:05 +0530 Subject: Re: Does CPython already has Peephole optimizations? From: Laxmikant Chitare To: Terry Reedy Content-Type: multipart/alternative; boundary=001a11c13ba09df7cb04f2a74cf1 Cc: python-list X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 198 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1392700274 news.xs4all.nl 2845 [2001:888:2000:d::a6]:59375 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:66632 --001a11c13ba09df7cb04f2a74cf1 Content-Type: text/plain; charset=ISO-8859-1 Thank you all for the enlightening inputs. I have learnt a lot just with this one question. Great to know about dis library. Ned, from explanation I now realize how important it is to do impact analysis. Things are not always rosy :). I have always appreciated everyone over this list. This is just another opportunity. Best regards, Laxmikant On Mon, Feb 17, 2014 at 7:21 PM, Terry Reedy wrote: > On 2/17/2014 3:59 AM, Steven D'Aprano wrote: > >> On Mon, 17 Feb 2014 13:54:25 +0530, Laxmikant Chitare wrote: >> >> I read about this article: >>> http://www.python.org/workshops/1998-11/proceedings/papers/montanaro/ >>> >> montanaro.html >> >>> >>> Just wanted to clarify whether CPython already includes these kind of >>> byte code optimizations? >>> >> > Most of the easily seen and obviously safe low-hanging fruits for the > compile step have been plucked. Note that the effect of the peephole > process would only save a few percent, if any, for real apps*. Improving > the C code invoked by bytecode has resulted in much larger gains. > > * We now have a much better benchmark suite with some real apps. This is > thanks in part to the pypy project. > > > Are all the temporary variables removed when byte code is generated? >>> >> >> You can check these things for yourself: >> >> import dis >> dis.dis(function) >> >> will show you the byte code. >> >> But in general, I would expect not. CPython (that's the Python you >> probably use) doesn't do a lot of optimization apart from some simple >> constant folding. If you're interested in optimizing Python, you should >> look at the JIT optimizing Python compiler, PyPy. >> > > For CPython, new optimization has mostly moved to AST tranformations prior > to compilation. (Python ASTs are new since Skip started the peephole work.) > I believe there are some open issues on the tracker. > > Once optimization constraint Skip did not mention is the correspondence > between source lines and blocks of bytecode, which is used by profiling, > tracing, and tracebacks. Effectively transforming > > if type(a) == types.ComplexType: > x = cmath.sin(a) > foo(x) > else: > x = math.sin(a) > foo(x) > > into > > if type(a) == types.ComplexType: > x = cmath.sin(a) > else: > x = math.sin(a) > foo(x) > > breaks the correspondence. If foo(x) raises, which original line should be > reported as the source of the exception? > > -- > Terry Jan Reedy > > -- > https://mail.python.org/mailman/listinfo/python-list > --001a11c13ba09df7cb04f2a74cf1 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thank you all for the enlightening inputs. I have lea= rnt a lot just with this one question. Great to know about dis library. Ned= , from explanation I=A0now realize how important it is to=A0do impact analy= sis. Things are not always rosy :).
=A0
I have always appreciated everyone over this list. This = is just another opportunity.
=A0
Best regards,
Laxmikant
=A0


On Mon, Feb 17, 2014 at 7:21 PM, Terry Reedy <tjreedy@udel.edu> wrote:
On 2/17/2014 3:59 AM, Steven D'Aprano wrote:
On Mon, 17 Feb 2014 13:54:25 +0530, Laxmikant Chitare wrote:

I read about this article:
http://www.python.org/workshops/1998-11/pro= ceedings/papers/montanaro/
montanaro.html

Just wanted to clarify whether CPython already includes these kind of
byte code optimizations?

Most of the easily seen and obviously safe low-hanging fruits for the compi= le step have been plucked. Note that the effect of the peephole process wou= ld only save a few percent, if any, for real apps*. Improving the C code in= voked by bytecode has resulted in much larger gains.

* We now have a much better benchmark suite with some real apps. This is th= anks in part to the pypy project.


Are all the temporary variables removed when byte code is generated?

You can check these things for yourself:

import dis
dis.dis(function)

will show you the byte code.

But in general, I would expect not. CPython (that's the Python you
probably use) doesn't do a lot of optimization apart from some simple constant folding. If you're interested in optimizing Python, you should=
look at the JIT optimizing Python compiler, PyPy.

For CPython, new optimization has mostly moved to AST tranformations prior = to compilation. (Python ASTs are new since Skip started the peephole work.)= =A0I believe there are some open issues on the tracker.

Once optimization constraint Skip did not mention is the correspondence bet= ween source lines and blocks of bytecode, which is used by profiling, traci= ng, and tracebacks. Effectively transforming

if type(a) =3D=3D types.ComplexType:
=A0 =A0 x =3D cmath.sin(a)
=A0 =A0 foo(x)
else:
=A0 =A0 x =3D math.sin(a)
=A0 =A0 foo(x)

into

if type(a) =3D=3D types.ComplexType:
=A0 =A0 x =3D cmath.sin(a)
else:
=A0 =A0 x =3D math.sin(a)
foo(x)

breaks the correspondence. If foo(x) raises, which original line should be = reported as the source of the exception?

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

--001a11c13ba09df7cb04f2a74cf1--