Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin3!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed1a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <ogmbk9170suu6tphk8mtbb9shil2l6p8gu@4ax.com>
References: <CAChsEZTGWa9Rxhwwg=b8Cnn3tx7FiaWqTJvHRn8hatxmVhOv0A@mail.gmail.com> <mailman.8969.1396839923.18130.python-list@python.org> <87d2gt4td2.fsf@elektro.pacujo.net> <7xha651yx2.fsf@ruckus.brouhaha.com> <mailman.8986.1396914468.18130.python-list@python.org> <877g70wg8p.fsf@elektro.pacujo.net> <li3hki$grv$1@ger.gmane.org> <CAPTjJmqwhb8O8vq84mMTv+-Rkc3Ff1AQDXe5cs8Y5gY02kHyNg@mail.gmail.com> <ogmbk9170suu6tphk8mtbb9shil2l6p8gu@4ax.com>
Date: Thu, 10 Apr 2014 11:05:15 +1000
Subject: Re: threading
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.9103.1397091924.18130.python-list@python.org>
Lines: 58
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:69996

On Thu, Apr 10, 2014 at 9:44 AM, Dennis Lee Bieber
<wlfraed@ix.netcom.com> wrote:
> On Wed, 9 Apr 2014 23:47:04 +1000, Chris Angelico <rosuav@gmail.com>
> declaimed the following:
>
>>won't block. You might think "Duh, how can printing to the screen
>>block?!?", but if your program's output is being piped into something
>>else, it most certainly can :) If that were writing to a remote
>
>         Heck, even if it isn't blocking per se, it may still be enough to slow
> down the whole system (over the past year I've had to characterize through
> put on some systems -- and the console logging of "exceptions"* slowed the
> overall data rate significantly)

Oh yes, definitely. Console output can be *slow*. Back in my earliest
programming days, I'd often have a program that iterated over sub-jobs
from either 0 or 1 up to some unknown top (so I can't show a
percent-done), and the obvious thing to do is (rewritten in Python):

i = 0
while stuff_to_do():
    i += 1
    print(i, end="\r")
    do_more_stuff()
print(i)

Hmm, that's really slow. I know! I'll speed this up by printing out
only once a second. That should be way faster, right? Let's see.

i = time_printed = 0
while stuff_to_do():
    i += 1
    if int(time.time()) != time_printed:
        print(i, end="\r")
        time_printed = int(time.time())
    do_more_stuff()
print(i)

And that made it... waaaay slower. Turns out clock querying (at least
on those systems) is pretty slow too, even more so than console
output. Of course, what we ended up settling on was something like
this, which *does* make sense:

i = 0
while stuff_to_do():
    i += 1
    if i & 255 == 0: print(i, end="\r")
    do_more_stuff()
print(i)

replacing 255 with any number one less than a power of two, so it'd
print out every however-many-th (in this case, every 256th), using
bitwise operations rather than division.

But yeah, console output isn't something you want when you're going
for maximum throughput. Heh.

ChrisA