Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #77104

Re: Reading from sys.stdin reads the whole file in

Show all headers | View raw

On Wed, Aug 27, 2014 at 3:19 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> When I pipe one to the other, I expect each line to be printed as they
> arrive, but instead they all queue up and happen at once:

You're seeing two different problems here. One is the flushing of
stdout in out.py, as Marko mentioned, but it's easily proven that
that's not the whole issue. Compare "python out.py" and "python
out.py|cat" - the latter will demonstrate whether or not it's getting
flushed properly (the former, where stdout is a tty, will always flush
correctly).

But even with that sorted, iterating over stdin has issues in Python
2. Here's a tweaked version of your files (note that I cut the sleeps
to 2 seconds, but the effect is the same):

rosuav@sikorsky:~$ cat out.py
import time

print("Hello...",flush=True)
time.sleep(2)
print("World!",flush=True)
time.sleep(2)
print("Goodbye!",flush=True)
rosuav@sikorsky:~$ cat slurp.py
from __future__ import print_function
import sys
import time

for line in sys.stdin:
        print(time.ctime(), line)
rosuav@sikorsky:~$ python3 out.py|python slurp.py
Wed Aug 27 16:00:16 2014 Hello...

Wed Aug 27 16:00:16 2014 World!

Wed Aug 27 16:00:16 2014 Goodbye!

rosuav@sikorsky:~$ python3 out.py|python3 slurp.py
Wed Aug 27 16:00:19 2014 Hello...

Wed Aug 27 16:00:21 2014 World!

Wed Aug 27 16:00:23 2014 Goodbye!

rosuav@sikorsky:~$


With a Py2 consumer, there's still buffering happening. With a Py3
consumer, it works correctly. How to control the Py2 buffering,
though, I don't know.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Reading from sys.stdin reads the whole file in Steven D'Aprano <steve@pearwood.info> - 2014-08-27 05:19 +0000
  Re: Reading from sys.stdin reads the whole file in Marko Rauhamaa <marko@pacujo.net> - 2014-08-27 08:29 +0300
    Re: Reading from sys.stdin reads the whole file in Marko Rauhamaa <marko@pacujo.net> - 2014-08-27 08:31 +0300
    Re: Reading from sys.stdin reads the whole file in Steven D'Aprano <steve@pearwood.info> - 2014-08-27 06:37 +0000
      Re: Reading from sys.stdin reads the whole file in Chris Angelico <rosuav@gmail.com> - 2014-08-27 16:45 +1000
      Re: Reading from sys.stdin reads the whole file in Akira Li <4kir4.1i@gmail.com> - 2014-08-29 04:02 +0400
  Re: Reading from sys.stdin reads the whole file in Chris Angelico <rosuav@gmail.com> - 2014-08-27 16:02 +1000
  Re: Reading from sys.stdin reads the whole file in Peter Otten <__peter__@web.de> - 2014-08-27 09:42 +0200
    Re: Reading from sys.stdin reads the whole file in Marko Rauhamaa <marko@pacujo.net> - 2014-08-27 11:39 +0300
      Re: Reading from sys.stdin reads the whole file in Peter Otten <__peter__@web.de> - 2014-08-27 11:31 +0200

csiph-web