Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Tue, 26 Aug 2014 23:07:36 -0700 (PDT)
MIME-Version: 1.0
In-Reply-To: <CAPTjJmpZS=PnOrB=B_POXjR9U5ahDn0Wqip8UMmwg2HahtkWqQ@mail.gmail.com>
References: <CAPTjJmpZS=PnOrB=B_POXjR9U5ahDn0Wqip8UMmwg2HahtkWqQ@mail.gmail.com>
From: "Naoki INADA" <songofacandy@gmail.com>
To: rosuav@gmail.com
Subject: Re: Reading from sys.stdin reads the whole file in
Content-Type: multipart/alternative; boundary="----Nodemailer-0.5.0-?=_1-1409119656022"
Cc: python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.13489.1409119664.18130.python-list@python.org>
Lines: 125
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:77105

------Nodemailer-0.5.0-?=_1-1409119656022
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

I recommend Python 3.

On Python 2, iterating lines without buffering is slow, tricky and ugly.




for line in iter(sys.stdin.readline(), ''):

=C2=A0 =C2=A0 print line
=E2=80=94
Sent from Mailbox

On Wed, Aug 27, 2014 at 3:03 PM, Chris Angelico <rosuav@gmail.com> wrote:

> On Wed, Aug 27, 2014 at 3:19 PM, Steven D'Aprano <steve@pearwood.info> =
wrote:
>> When I pipe one to the other, I expect each line to be printed as they
>> arrive, but instead they all queue up and happen at once:
> You're seeing two different problems here. One is the flushing of
> stdout in out.py, as Marko mentioned, but it's easily proven that
> that's not the whole issue. Compare =22python out.py=22 and =22python
> out.py|cat=22 - the latter will demonstrate whether or not it's getting
> flushed properly (the former, where stdout is a tty, will always flush
> correctly).
> But even with that sorted, iterating over stdin has issues in Python
> 2. Here's a tweaked version of your files (note that I cut the sleeps
> to 2 seconds, but the effect is the same):
> rosuav@sikorsky:~$ cat out.py
> import time
> print(=22Hello...=22,flush=3DTrue)
> time.sleep(2)
> print(=22World!=22,flush=3DTrue)
> time.sleep(2)
> print(=22Goodbye!=22,flush=3DTrue)
> rosuav@sikorsky:~$ cat slurp.py
> from =5F=5Ffuture=5F=5F import print=5Ffunction
> import sys
> import time
> for line in sys.stdin:
>         print(time.ctime(), line)
> rosuav@sikorsky:~$ python3 out.py|python slurp.py
> Wed Aug 27 16:00:16 2014 Hello...
> Wed Aug 27 16:00:16 2014 World!
> Wed Aug 27 16:00:16 2014 Goodbye!
> rosuav@sikorsky:~$ python3 out.py|python3 slurp.py
> Wed Aug 27 16:00:19 2014 Hello...
> Wed Aug 27 16:00:21 2014 World!
> Wed Aug 27 16:00:23 2014 Goodbye!
> rosuav@sikorsky:~$
> With a Py2 consumer, there's still buffering happening. With a Py3
> consumer, it works correctly. How to control the Py2 buffering,
> though, I don't know.
> ChrisA
> --=20
> https://mail.python.org/mailman/listinfo/python-list
------Nodemailer-0.5.0-?=_1-1409119656022
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable


<span id=3D=22mailbox-conversation=22><div>I recommend Python 3.</div>
<div>On Python 2, iterating lines without buffering is slow, tricky and =
ugly.</div>
<div><br></div>
<div>for line in iter(sys.stdin.readline(), ''):</div>
<div>=C2=A0 =C2=A0 print line</div></span><div class=3D=22mailbox=5Fsignatu=
re=22>=E2=80=94<br>Sent from <a href=3D=22https://www.dropbox.=
com/mailbox=22>Mailbox</a>
</div>
<br><br><div class=3D=22gmail=5Fquote=22><p>On Wed, Aug 27, 2014 at 3:03 PM=
, Chris Angelico <span dir=3D=22ltr=22>&lt;<a href=3D=22mailto:rosuav@gmail=
.com=22 target=3D=22=5Fblank=22>rosuav@gmail.com</a>&gt;</span> =
wrote:<br></p><blockquote class=3D=22gmail=5Fquote=22 style=3D=22margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;=22><p>On Wed, Aug 27, =
2014 at 3:19 PM, Steven D'Aprano &lt;steve@pearwood.info&gt; wrote:
<br>&gt; When I pipe one to the other, I expect each line to be printed as =
they
<br>&gt; arrive, but instead they all queue up and happen at once:
<br><br>You're seeing two different problems here. One is the flushing of
<br>stdout in out.py, as Marko mentioned, but it's easily proven that
<br>that's not the whole issue. Compare =22python out.py=22 and =22python
<br>out.py|cat=22 - the latter will demonstrate whether or not it's =
getting
<br>flushed properly (the former, where stdout is a tty, will always flush
<br>correctly).
<br><br>But even with that sorted, iterating over stdin has issues in =
Python
<br>2. Here's a tweaked version of your files (note that I cut the sleeps
<br>to 2 seconds, but the effect is the same):
<br><br>rosuav@sikorsky:~$ cat out.py
<br>import time
<br><br>print(=22Hello...=22,flush=3DTrue)
<br>time.sleep(2)
<br>print(=22World!=22,flush=3DTrue)
<br>time.sleep(2)
<br>print(=22Goodbye!=22,flush=3DTrue)
<br>rosuav@sikorsky:~$ cat slurp.py
<br>from =5F=5Ffuture=5F=5F import print=5Ffunction
<br>import sys
<br>import time
<br><br>for line in sys.stdin:
<br>        print(time.ctime(), line)
<br>rosuav@sikorsky:~$ python3 out.py|python slurp.py
<br>Wed Aug 27 16:00:16 2014 Hello...
<br><br>Wed Aug 27 16:00:16 2014 World!
<br><br>Wed Aug 27 16:00:16 2014 Goodbye!
<br><br>rosuav@sikorsky:~$ python3 out.py|python3 slurp.py
<br>Wed Aug 27 16:00:19 2014 Hello...
<br><br>Wed Aug 27 16:00:21 2014 World!
<br><br>Wed Aug 27 16:00:23 2014 Goodbye!
<br><br>rosuav@sikorsky:~$
<br><br><br>With a Py2 consumer, there's still buffering happening. With a =
Py3
<br>consumer, it works correctly. How to control the Py2 buffering,
<br>though, I don't know.
<br><br>ChrisA
<br>--=20
<br>https://mail.python.org/mailman/listinfo/python-list
<br></p></blockquote></div><br>
------Nodemailer-0.5.0-?=_1-1409119656022--