Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Peter Otten <__peter__@web.de>
Newsgroups: comp.lang.python
Subject: Re: Other difference with Perl: Python scripts in a pipe
Date: Thu, 10 Mar 2016 23:09:21 +0100
Organization: None
Lines: 67
Message-ID: <mailman.158.1457647792.15725.python-list@python.org>
References: <nbsp83$2p1$1@gioia.aioe.org> <CALwzidk+8mf-nxFHtsEH+1j8wFSLPNuBtfF64kgFoy2ztWqa1Q@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7Bit
X-Trace: news.uni-berlin.de wQQbx1sDw410CDaZ10dnNQS04/F1h24Gp3DCG89ymtJA==
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'broken': 0.03; 'subject:Python': 0.05; 'python3': 0.05; 'sys': 0.05; 'ioerror:': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'sys.stderr': 0.09; 'python': 0.10; 'thu,': 0.15; '2016': 0.16; 'cache:': 0.16; 'experiments': 0.16; 'range(20):': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:io': 0.16; 'received:plane.gmane.org': 0.16; 'received:psf.io': 0.16; 'received:t-ipconnect.de': 0.16; 'wrote:': 0.16; 'commands,': 0.22; 'pipe': 0.22; 'suppose': 0.22; 'import': 0.24; '(most': 0.24; 'script': 0.25; 'header:User-Agent:1': 0.26; 'header:X -Complaints-To:1': 0.26; 'cat': 0.29; 'doing?': 0.29; 'traceback': 0.33; 'file': 0.34; 'this?': 0.34; 'problem.': 0.35; 'to:addr :python-list': 0.36; 'pm,': 0.36; 'subject:: ': 0.37; 'received:org': 0.37; 'missing': 0.37; 'to:addr:python.org': 0.40; 'subject:with': 0.40; 'received:de': 0.40; 'your': 0.60; 'close': 0.61; 'avoid': 0.61; '2000': 0.63; 'mar': 0.65; 'refuse': 0.93
X-Injected-Via-Gmane: http://gmane.org/
X-Gmane-NNTP-Posting-Host: p57bd8195.dip0.t-ipconnect.de
User-Agent: KNode/4.13.3
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref: csiph.com comp.lang.python:104557

Ian Kelly wrote:

> On Thu, Mar 10, 2016 at 2:33 PM, Fillmore <fillmore_remove@hotmail.com>
> wrote:
>>
>> when I put a Python script in pipe with other commands, it will refuse to
>> let go silently. Any way I can avoid this?
> 
> What is your script doing? I don't see this problem.
> 
> ikelly@queso:~ $ cat somescript.py
> import sys
> 
> for i in range(20):
>     sys.stdout.write('line %d\n' % i)
> ikelly@queso:~ $ python somescript.py | head -5
> line 0
> line 1
> line 2
> line 3
> line 4
> ikelly@queso:~ $ python3 somescript.py | head -5
> line 0
> line 1
> line 2
> line 3
> line 4

I suppose you need to fill the OS-level cache:

$ cat somescript.py 
import sys

for i in range(int(sys.argv[1])):
    sys.stdout.write('line %d\n' % i)
$ python somescript.py 20 | head -n5
line 0
line 1
line 2
line 3
line 4
$ python somescript.py 200 | head -n5
line 0
line 1
line 2
line 3
line 4
$ python somescript.py 2000 | head -n5
line 0
line 1
line 2
line 3
line 4
Traceback (most recent call last):
  File "somescript.py", line 4, in <module>
    sys.stdout.write('line %d\n' % i)
IOError: [Errno 32] Broken pipe

During my experiments I even got

close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

occasionally.