Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <878uckvjoy.fsf@Equus.decebal.nl>
References: <878uckvjoy.fsf@Equus.decebal.nl>
From: Oscar Benjamin <oscar.j.benjamin@gmail.com>
Date: Tue, 19 May 2015 14:16:13 +0100
Subject: Re: Why does the first loop go wrong with Python3
To: Cecil Westerhof <Cecil@decebal.nl>
Cc: Python List <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.133.1432041396.17265.python-list@python.org>
Lines: 49
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:90848

On 19 May 2015 at 13:24, Cecil Westerhof <Cecil@decebal.nl> wrote:
> I have the following code:
>     from __future__     import division, print_function
>
>     import subprocess
>
>     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
>     for line in iter(p.stdout.readline, ''):
>         print(line.rstrip().decode('utf-8'))
>
>     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
>     for line in p.stdout.readlines():
>         print(line.rstrip().decode('utf-8'))
>
> This works in Python2. (Both give the same output.)
>
> But when I execute this in Python3, then the first loop is stuck in a
> loop where it continually prints a empty string. The second loop is
> executed correctly in Python3.
>
> In the current case it is not a problem for me, but when the output
> becomes big, the second solution will need more memory. How can I get
> the first version working in Python3?

The problem is that Python 3 carefully distinguishes between the bytes
that come when reading from the stdout of a process and text which
must be decoded from the bytes. You're using iter(f, sentinel) and
checking for a sentinel value of ''. However in Python 3 the sentinel
returned will be b''.

Consider:
$ python3
Python 3.2.3 (default, Feb 27 2014, 21:31:18)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> '' == b''
False

If you change it from '' to b'' it will work.

However the normal way to do this is to iterate over stdout directly:

     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
     for line in p.stdout:
         print(line.rstrip().decode('utf-8'))


--
Oscar