Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #61137

Re: squeeze out some performance

Show key headers only | View raw

On Fri, Dec 6, 2013 at 8:46 PM, Jeremy Sanders <jeremy@jeremysanders.net> wrote:
> This sort of code is probably harder to make faster in pure python. You
> could try profiling it to see where the hot spots are. Perhaps the choice of
> arrays or sets might have some speed impact.

I'd make this recommendation MUCH stronger.

Rule 1 of optimization: Don't.
Rule 2 (for experts only): Don't yet.

Once you find that your program actually is running too slowly, then
AND ONLY THEN do you start looking at tightening something up. You'll
be amazed how little you need to change; start with good clean
idiomatic code, and then if it takes too long, you tweak just a couple
of things and it's fast enough. And when you do come to the
tweaking...

Rule 3: Measure twice, cut once.
Rule 4: Actually, measure twenty times, cut once.

Profile your code to find out what's actually slow. This is very
important. Here's an example from a real application (not in Python,
it's in a semantically-similar language called Pike):

https://github.com/Rosuav/Gypsum/blob/d9907e1507c52189c83ae25f5d7be85235b616fa/window.pike

I noticed that I could saturate one CPU core by typing commands very
quickly. Okay. That gets us past the first two rules (it's a MUD
client, it should not be able to saturate one core of an i5). The code
looks roughly like this:

paint():
    for line in lines:
        if line_is_visible:
            paint_line(line)

paint_line():
    for piece_of_text in text:
        if highlighted: draw_highlighted()
        else: draw_not_highlighted()

My first guess was that the actual drawing was taking the time, since
that's a whole lot of GTK calls. But no; the actual problem was the
iteration across all lines and then finding out if they're visible or
not (possibly because it obliterates the CPU caches). Once the
scrollback got to a million lines or so, that was prohibitively
expensive. I didn't realize that until I actually profiled the code
and _measured_ where the time was being spent.

How fast does your code run? How fast do you need it to run? Lots of
optimization questions are answered by "Yaknow what, it don't even
matter", unless you're running in a tight loop, or on a
microcontroller, or something. Halving the time taken sounds great
until you see that it's currently taking 0.0001 seconds and happens in
response to user action.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

squeeze out some performance Robert Voigtländer <r.voigtlaender@gmail.com> - 2013-12-06 00:47 -0800
  Re: squeeze out some performance Jeremy Sanders <jeremy@jeremysanders.net> - 2013-12-06 10:46 +0100
  Re: squeeze out some performance Chris Angelico <rosuav@gmail.com> - 2013-12-06 23:13 +1100
    Re: squeeze out some performance Robert Voigtländer <r.voigtlaender@gmail.com> - 2013-12-06 08:29 -0800
      Re: squeeze out some performance Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-06 16:36 +0000
        Re: squeeze out some performance Robert Voigtländer <r.voigtlaender@gmail.com> - 2013-12-06 08:44 -0800
  Re: squeeze out some performance John Ladasky <john_ladasky@sbcglobal.net> - 2013-12-06 08:52 -0800
    Re: squeeze out some performance Joel Goldstick <joel.goldstick@gmail.com> - 2013-12-06 17:07 -0500
    Re: squeeze out some performance Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-06 22:38 +0000
    Re: squeeze out some performance Dan Stromberg <drsalists@gmail.com> - 2013-12-06 15:01 -0800
      Re: squeeze out some performance Robert Voigtländer <r.voigtlaender@gmail.com> - 2013-12-09 06:19 -0800
        Re: squeeze out some performance Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-09 14:52 +0000
    Re: squeeze out some performance Robin Becker <robin@reportlab.com> - 2013-12-09 15:54 +0000
    Re: squeeze out some performance Dave Angel <davea@davea.name> - 2013-12-09 15:46 -0500
    Re: squeeze out some performance Robin Becker <robin@reportlab.com> - 2013-12-10 12:54 +0000
  Re: squeeze out some performance Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-09 23:57 +0000
    Re: squeeze out some performance Robert Voigtländer <r.voigtlaender@gmail.com> - 2013-12-10 04:14 -0800

csiph-web