Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #52845

Re: Running a command line program and reading the result as it runs

Show all headers | View raw

Ian Simcock <Ian.Simcock@Internode.on.net> writes:

> Greetings all.
>
> I'm using Python 2.7 under Windows and am trying to run a command line
> program and process the programs output as it is running. A number of
> web searches have indicated that the following code would work.
>
> import subprocess
>
> p = subprocess.Popen("D:\Python\Python27\Scripts\pip.exe list -o",
>                      stdout=subprocess.PIPE,
>                      stderr=subprocess.STDOUT,
>                      bufsize=1,
>                      universal_newlines=True,
>                      shell=False)
> for line in p.stdout:
>     print line
>
> When I use this code I can see that the Popen works, any code between
> the Popen and the for will run straight away, but as soon as it gets
> to the for and tries to read p.stdout the code blocks until the
> command line program completes, then all of the lines are returned.
>
> Does anyone know how to get the results of the program without it blocking?

When file object is used in a for loop it works like an iterator
and then it uses a hidden read-ahead buffer. 
It might cause this kind of blocking.
You can read more details here (description of method ``next``):
http://docs.python.org/lib/bltin-file-objects.html

So basically non-blocking loop might look like this:

while True:
    line = p.stdout.readline()
    if not line: break
    print line

HTH,
Rob

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Running a command line program and reading the result as it runs Ian Simcock <Ian.Simcock@Internode.on.net> - 2013-08-22 15:21 +0930
  Re: Running a command line program and reading the result as it runs Chris Angelico <rosuav@gmail.com> - 2013-08-22 16:22 +1000
    Re: Running a command line program and reading the result as it runs Ian Simcock <Ian.Simcock@Internode.on.net> - 2013-08-23 00:56 +0930
      Re: Running a command line program and reading the result as it runs Chris Angelico <rosuav@gmail.com> - 2013-08-23 01:33 +1000
        Re: Running a command line program and reading the result as it runs Ian Simcock <Ian.Simcock@Internode.on.net> - 2013-08-23 16:22 +0930
        Re: Running a command line program and reading the result as it runs Grant Edwards <invalid@invalid.invalid> - 2013-08-23 14:02 +0000
  Re: Running a command line program and reading the result as it runs Rob Wolfe <rw@smsnet.pl> - 2013-08-22 23:14 +0200
    Re: Running a command line program and reading the result as it runs Ian Simcock <Ian.Simcock@Internode.on.net> - 2013-08-23 16:31 +0930
  Re: Running a command line program and reading the result as it runs Gertjan Klein <gklein@xs4all.nl> - 2013-08-23 11:32 +0200
  Re: Running a command line program and reading the result as it runs Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2013-08-23 11:53 +0200
  Re: Running a command line program and reading the result as it runs Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2013-08-23 12:34 +0200
  RE: Running a command line program and reading the result as it runs "Joseph L. Casale" <jcasale@activenetwerx.com> - 2013-08-23 10:50 +0000
  Re: Running a command line program and reading the result as it runs Peter Otten <__peter__@web.de> - 2013-08-23 13:14 +0200
    Re: Running a command line program and reading the result as it runs Gertjan Klein <gklein@xs4all.nl> - 2013-08-23 14:03 +0200
    Re: Running a command line program and reading the result as it runs Ian Simcock <Ian.Simcock@Internode.on.net> - 2013-08-24 19:06 +0930
  Re: Running a command line program and reading the result as it runs random832@fastmail.us - 2013-08-23 12:04 -0400
  Re: Running a command line program and reading the result as it runs Peter Otten <__peter__@web.de> - 2013-08-23 18:39 +0200

csiph-web