Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #73628 > unrolled thread

print statements and profiling a function slowed performance

Back to article view | Back to comp.lang.python

Contents

  print statements and profiling a function slowed performance CM <cmpython@gmail.com> - 2014-06-26 11:44 -0700
    Re: print statements and profiling a function slowed performance Michael Torrie <torriem@gmail.com> - 2014-06-26 13:14 -0600
      Re: print statements and profiling a function slowed performance CM <cmpython@gmail.com> - 2014-06-26 13:36 -0700
        Re: print statements and profiling a function slowed performance Chris Angelico <rosuav@gmail.com> - 2014-06-27 12:06 +1000
        Re: print statements and profiling a function slowed performance Michael Torrie <torriem@gmail.com> - 2014-06-26 19:59 -0600
        Re: print statements and profiling a function slowed performance Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-06-28 09:41 -0400
        Re: print statements and profiling a function slowed performance Chris Angelico <rosuav@gmail.com> - 2014-06-28 23:50 +1000
        Re: print statements and profiling a function slowed performance Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-06-29 02:14 -0400
        Re: print statements and profiling a function slowed performance Chris Angelico <rosuav@gmail.com> - 2014-06-29 17:14 +1000
    Re: print statements and profiling a function slowed performance Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-06-26 20:27 +0100
      Re: print statements and profiling a function slowed performance CM <cmpython@gmail.com> - 2014-06-26 13:37 -0700
        Re: print statements and profiling a function slowed performance Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-06-27 02:55 +0000
          Re: print statements and profiling a function slowed performance Chris Angelico <rosuav@gmail.com> - 2014-06-27 13:14 +1000
          Re: print statements and profiling a function slowed performance Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-06-27 16:55 +0100
          Re: print statements and profiling a function slowed performance Skip Montanaro <skip@pobox.com> - 2014-06-27 11:05 -0500
          Re: print statements and profiling a function slowed performance Ian Kelly <ian.g.kelly@gmail.com> - 2014-06-27 15:35 -0600
          Re: print statements and profiling a function slowed performance Skip Montanaro <skip@pobox.com> - 2014-06-27 19:12 -0500
          Re: print statements and profiling a function slowed performance Chris Angelico <rosuav@gmail.com> - 2014-06-28 10:29 +1000
            Re: print statements and profiling a function slowed performance Marko Rauhamaa <marko@pacujo.net> - 2014-06-28 22:20 +0300
              Re: print statements and profiling a function slowed performance Chris Angelico <rosuav@gmail.com> - 2014-06-29 09:30 +1000
              Re: print statements and profiling a function slowed performance Rustom Mody <rustompmody@gmail.com> - 2014-06-28 19:25 -0700
                Re: print statements and profiling a function slowed performance Chris Angelico <rosuav@gmail.com> - 2014-06-29 12:32 +1000
          Re: print statements and profiling a function slowed performance Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-06-28 02:06 +0100
    Re: print statements and profiling a function slowed performance Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-06-26 22:11 -0400

Page 2 of 2 — ← Prev page 1 [2]

#73720

On Sunday, June 29, 2014 12:50:19 AM UTC+5:30, Marko Rauhamaa wrote:
> Chris Angelico wrote:

> > Threading is a focus of so many myths. People who don't understand it
> > think that threads are magic pixie dust that fixes everything, or else
> > magic pixie dust that breaks everything. Or both, at the same time.
> > Neither notion is true.

> I'm in the latter camp.

> More precisely: threads have their uses but they are used far too much
> and not feared nearly enough.

Third Camp:
Since both views are true:

Chris: Threads solve a large class of important problems
Marko: Threads are horribly error prone

lets have a (bondage-and-discipline) language that supports smooth mostly
error-free threads.

Most famous recent example is Go.
More classic (and more bondage-and-discipline) are Erlang, CloudHaskell

PS. I find the success of node.js (async model) scary.
The success is a fact

[toc] | [prev] | [next] | [standalone]

#73721

On Sun, Jun 29, 2014 at 12:25 PM, Rustom Mody <rustompmody@gmail.com> wrote:
> On Sunday, June 29, 2014 12:50:19 AM UTC+5:30, Marko Rauhamaa wrote:
>> Chris Angelico wrote:
>
>> > Threading is a focus of so many myths. People who don't understand it
>> > think that threads are magic pixie dust that fixes everything, or else
>> > magic pixie dust that breaks everything. Or both, at the same time.
>> > Neither notion is true.
>
>> I'm in the latter camp.
>
>> More precisely: threads have their uses but they are used far too much
>> and not feared nearly enough.
>
> Third Camp:
> Since both views are true:
>
> Chris: Threads solve a large class of important problems
> Marko: Threads are horribly error prone

Not strictly my view. I said that both extremes ("threads fix
everything" and "threads break everything") are quite wrong; the truth
is that threading is a tool, like any other, and you have to know when
it's appropriate to use each tool.

ChrisA

[toc] | [prev] | [next] | [standalone]

#73683

On 28/06/2014 01:12, Skip Montanaro wrote:
> On Fri, Jun 27, 2014 at 4:35 PM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
>> That's got to count for
>> something, compared to a raw print that has to wait for the I/O to
>> finish.
>
> A raw print basically just tosses some bytes in a stdio buffer (at
> least in Unix-land).
>

Exactly, in Windows land you're talking chalk and cheese when comparing 
logging to printing to a command window.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

[toc] | [prev] | [next] | [standalone]

#73646

On Thu, 26 Jun 2014 11:44:29 -0700 (PDT), CM <cmpython@gmail.com> declaimed
the following:

>1. print statements were slowing down my code enough to
>really notice a particular transition. It went from about
>2-3 seconds to a bit under 1 second. What at first seemed
>unresponsive now seems almost snappy. The only difference
>was removing a lot of print statements I had used for
>debugging (Python 2.5, on a single core 1.97 Ghz machine).
>
	Lots of I/O will slow anything down -- consider; every I/O operation
will trigger an OS level process swap, since they are synchronous and
handled by the OS.

	Last fall I was collecting data for a number of candidate hardware
platforms (apologies for the vagueness, but I suspect the details would
fall under some sort of proprietary information clause at my place of
employment).

	The biggest finding was that the console (and even disk) logging of
actions* resulted in a large latency in the throughput of those hardware
platforms.

* I'm stretching to reveal that what the platforms were logging were
considered "exceptional" events by the providers, but for our application
would be "normal"; hence we were effectively flooding them with lots of
data that triggered exception processing and associated logging. Data that
didn't trigger the exception could pass with less than 1mSec delay, but
once a batch of exception data arrived the delays grew to 10mSec or more --
mostly as the log I/O was delaying processing of subsequent packets.

-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web