Path: csiph.com!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de>
Newsgroups: comp.lang.python
Subject: Re: Unbuffered stderr in Python 3
Date: Mon, 2 Nov 2015 11:48:32 +0100
Lines: 68
Message-ID: <mailman.59.1446461339.4463.python-list@python.org>
References: <5637165b$0$1505$c3e8da3$5496439d@news.astraweb.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0
In-Reply-To: <5637165b$0$1505$c3e8da3$5496439d@news.astraweb.com>
Precedence: list
Xref: csiph.com comp.lang.python:98093

On 02.11.2015 08:52, Steven D'Aprano wrote:
> In Python 2, stderr is unbuffered.
>
> In most other environments (the shell, C...) stderr is unbuffered.
>
> It is usually considered a bad, bad thing for stderr to be buffered. What
> happens if your application is killed before the buffer fills up? The errors
> in the buffer will be lost.
>
> So how come Python 3 has line buffered stderr? And more importantly, how can
> I turn buffering off?
>

I cannot comment on your first question, but unbuffered writing to 
sys.stderr is possible just like for any other buffered file object. You 
just call its flush method after writing.

Since Python3.3, the print function has a flush keyword argument that 
accepts a boolean and lets you do just this. Rewrite your example as:

import sys, time

def test():
# Simulate a slow calculation that prints status and/or error
# messages to stderr.
for i in range(10):
_print(i, file=sys.stderr, end="", flush=True)
time.sleep(2)
print("", file=sys.stderr)

and it should do what you want.


Before Python 3.3, if I needed an unbuffered print function, I used a 
wrapper to call flush for me, so instead of:

>
> import sys, time
>
> def test():
>      # Simulate a slow calculation that prints status and/or error
>      # messages to stderr.
>      for i in range(10):
>          print(i, file=sys.stderr, end="")
>          time.sleep(2)
>      print("", file=sys.stderr)
>

I'd use (and still do this if I need compatibility with 3.2):

import sys, time

def _print (*args, **kwargs):
    file = kwargs.get('file', sys.stdout)
    print(*args, **kwargs)
    file.flush()

def test():
     # Simulate a slow calculation that prints status and/or error
     # messages to stderr.
     for i in range(10):
         _print(i, file=sys.stderr, end="")
         time.sleep(2)
     print("", file=sys.stderr)

Best,
Wolfgang