Path: csiph.com!usenet.pasdenom.info!aioe.org!.POSTED!not-for-mail
From: Rob Wolfe <rw@smsnet.pl>
Newsgroups: comp.lang.python
Subject: Re: Running a command line program and reading the result as it runs
Date: Thu, 22 Aug 2013 23:14:26 +0200
Organization: Aioe.org NNTP Server
Lines: 41
Message-ID: <87y57to2bx.fsf@smsnet.pl>
References: <5215a6cf$0$6512$c3e8da3$5496439d@news.astraweb.com>
NNTP-Posting-Host: 2rGoQNolRk9jw3eVwM369A.user.speranza.aioe.org
Mime-Version: 1.0
Content-Type: text/plain
X-Complaints-To: abuse@aioe.org
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux)
X-Notice: Filtered by postfilter v. 0.8.2
Cancel-Lock: sha1:B9V0uLyyjUQYbAZF3fkwSA4a+7c=
Xref: csiph.com comp.lang.python:52845

Ian Simcock <Ian.Simcock@Internode.on.net> writes:

> Greetings all.
>
> I'm using Python 2.7 under Windows and am trying to run a command line
> program and process the programs output as it is running. A number of
> web searches have indicated that the following code would work.
>
> import subprocess
>
> p = subprocess.Popen("D:\Python\Python27\Scripts\pip.exe list -o",
>                      stdout=subprocess.PIPE,
>                      stderr=subprocess.STDOUT,
>                      bufsize=1,
>                      universal_newlines=True,
>                      shell=False)
> for line in p.stdout:
>     print line
>
> When I use this code I can see that the Popen works, any code between
> the Popen and the for will run straight away, but as soon as it gets
> to the for and tries to read p.stdout the code blocks until the
> command line program completes, then all of the lines are returned.
>
> Does anyone know how to get the results of the program without it blocking?

When file object is used in a for loop it works like an iterator
and then it uses a hidden read-ahead buffer. 
It might cause this kind of blocking.
You can read more details here (description of method ``next``):
http://docs.python.org/lib/bltin-file-objects.html

So basically non-blocking loop might look like this:

while True:
    line = p.stdout.readline()
    if not line: break
    print line

HTH,
Rob