Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #77105 > unrolled thread

Re: Reading from sys.stdin reads the whole file in

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Reading from sys.stdin reads the whole file in "Naoki INADA" <songofacandy@gmail.com> - 2014-08-26 23:07 -0700
    Re: Reading from sys.stdin reads the whole file in Steven D'Aprano <steve@pearwood.info> - 2014-08-27 07:33 +0000

#77105 — Re: Reading from sys.stdin reads the whole file in

[Multipart message — attachments visible in raw view] — view raw

I recommend Python 3.

On Python 2, iterating lines without buffering is slow, tricky and ugly.




for line in iter(sys.stdin.readline(), ''):

    print line
—
Sent from Mailbox

On Wed, Aug 27, 2014 at 3:03 PM, Chris Angelico <rosuav@gmail.com> wrote:

> On Wed, Aug 27, 2014 at 3:19 PM, Steven D'Aprano <steve@pearwood.info> wrote:
>> When I pipe one to the other, I expect each line to be printed as they
>> arrive, but instead they all queue up and happen at once:
> You're seeing two different problems here. One is the flushing of
> stdout in out.py, as Marko mentioned, but it's easily proven that
> that's not the whole issue. Compare "python out.py" and "python
> out.py|cat" - the latter will demonstrate whether or not it's getting
> flushed properly (the former, where stdout is a tty, will always flush
> correctly).
> But even with that sorted, iterating over stdin has issues in Python
> 2. Here's a tweaked version of your files (note that I cut the sleeps
> to 2 seconds, but the effect is the same):
> rosuav@sikorsky:~$ cat out.py
> import time
> print("Hello...",flush=True)
> time.sleep(2)
> print("World!",flush=True)
> time.sleep(2)
> print("Goodbye!",flush=True)
> rosuav@sikorsky:~$ cat slurp.py
> from __future__ import print_function
> import sys
> import time
> for line in sys.stdin:
>         print(time.ctime(), line)
> rosuav@sikorsky:~$ python3 out.py|python slurp.py
> Wed Aug 27 16:00:16 2014 Hello...
> Wed Aug 27 16:00:16 2014 World!
> Wed Aug 27 16:00:16 2014 Goodbye!
> rosuav@sikorsky:~$ python3 out.py|python3 slurp.py
> Wed Aug 27 16:00:19 2014 Hello...
> Wed Aug 27 16:00:21 2014 World!
> Wed Aug 27 16:00:23 2014 Goodbye!
> rosuav@sikorsky:~$
> With a Py2 consumer, there's still buffering happening. With a Py3
> consumer, it works correctly. How to control the Py2 buffering,
> though, I don't know.
> ChrisA
> -- 
> https://mail.python.org/mailman/listinfo/python-list

[toc] | [next] | [standalone]

#77114

On Tue, 26 Aug 2014 23:07:36 -0700, Naoki INADA wrote:

> for line in iter(sys.stdin.readline(), ''):

Thanks for that. Removing the parens after readline seems to do the trick.



-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web