Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #11099

RE: subprocess.Popen and thread module

Path csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!tudelft.nl!txtfeed1.tudelft.nl!feeder1.cambriumusenet.nl!feed.tweaknews.nl!194.109.133.84.MISMATCH!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <dannwong@cisco.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.004
X-Spam-Evidence '*H*': 0.99; '*S*': 0.00; 'subject:module': 0.04; '"""': 0.07; 'attribute.': 0.09; 'danny': 0.09; 'deprecated': 0.09; 'doc,': 0.09; 'output': 0.10; 'message-----': 0.12; "'-r',": 0.16; 'docs,': 0.16; 'email name:chris': 0.16; 'hangs': 0.16; 'hangs.': 0.16; 'like?': 0.16; 'parallel,': 0.16; 'returncode': 0.16; 'terminate.': 0.16; 'cc:addr:python-list': 0.16; 'wrote:': 0.16; 'cheers,': 0.18; 'subject:skip:s 10': 0.18; 'cc:no real name:2**0': 0.20; 'seems': 0.20; 'trying': 0.21; 'cc:2**0': 0.22; 'header:In-Reply-To:1': 0.22; 'tue,': 0.23; 'pm,': 0.24; 'aug': 0.24; "i'm": 0.27; 'all,': 0.28; 'cc:addr:python.org': 0.30; 'module': 0.30; 'threads': 0.30; 'cc:': 0.31; 'url:library': 0.31; 'chris': 0.32; 'subject:': 0.32; "what's": 0.33; 'sent:': 0.33; 'regardless': 0.34; 'external': 0.35; 'url:python': 0.36; 'data.': 0.36; 'skip:" 10': 0.36; 'thread': 0.37; 'communicate': 0.37; 'using': 0.37; 'but': 0.37; 'could': 0.38; 'from:': 0.38; 'portion': 0.38; 'some': 0.38; 'url:org': 0.38; 'subject:: ': 0.39; 'url:docs': 0.39; 'enough': 0.39; 'skip:- 10': 0.39; 'more': 0.60; 'your': 0.61; 'back': 0.62; 'waiting': 0.63; '\xe2\x80\x93': 0.65; 'august': 0.70; 'database.': 0.74; '11:48': 0.84; '11:53': 0.84; 'isolate': 0.84; 'reverted': 0.84
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=dannwong@cisco.com; l=2432; q=dns/txt; s=iport; t=1312959400; x=1314169000; h=mime-version:content-transfer-encoding:subject:date: message-id:in-reply-to:references:from:to:cc; bh=Tp/DgRRNjxg3Vlo7pJ3O1kvqAkQbYLsPeYN3qKBCVLQ=; b=OJzX7994HChoKV2ml8yU42XkPbFoKOVVtHfkD6s6umgQ+pozfAl9GMZ1 uCyEKKHWfDRA2VtT0UJ5NrJm+2EJT0ohrC74xmw85dTON/ndKnhAH+ogw Lv1vDwEEnYmYkst0LHCWrV7IUgGl93QlE0kRxWMnOlOQ9zpBJZtQJrFQM 8=;
X-IronPort-Anti-Spam-Filtered true
X-IronPort-Anti-Spam-Result AuIAAGQrQk6tJXHA/2dsb2JhbABBhEeTH451a3eBQAEBAQECARIBEA0ERQUHBAIBBgIOAwQBAQECAgYGFwECAgIBAUQJCAEBBBMIGodLBJ9eAY0rkRaBLIQKMV8Eh12QP4t4
X-IronPort-AV E=Sophos;i="4.67,349,1309737600"; d="scan'208";a="11628233"
X-MimeOLE Produced By Microsoft Exchange V6.5
Content-class urn:content-classes:message
MIME-Version 1.0
Content-Type text/plain; charset="UTF-8"
Content-Transfer-Encoding base64
Subject RE: subprocess.Popen and thread module
Date Wed, 10 Aug 2011 01:56:38 -0500
In-Reply-To <CAMZYqRRbSmdJ5e66p7Tvi7-+X_Kpf4vjVhwr=YUkjo2WkPA-rA@mail.gmail.com>
X-MS-Has-Attach
X-MS-TNEF-Correlator
Thread-Topic subprocess.Popen and thread module
Thread-Index AcxXKkAWQPLUGwBhTjCWiutEtgKtaAAABYyA
References <20110808013500.7c9e30ff44caee0f0bbf988a@johnohagan.com><D59C59453D0EDC43832490B6DADFAECF0432A191@XMB-RCD-214.cisco.com><CAMZYqRSxAzLaea6aR0DNDnc-Jv3XWSPvo6cZdv4D0hcL5eeKTg@mail.gmail.com><D59C59453D0EDC43832490B6DADFAECF0432A192@XMB-RCD-214.cisco.com> <CAMZYqRRbSmdJ5e66p7Tvi7-+X_Kpf4vjVhwr=YUkjo2WkPA-rA@mail.gmail.com>
From "Danny Wong (dannwong)" <dannwong@cisco.com>
To "Chris Rebert" <clp2@rebertia.com>
X-OriginalArrivalTime 10 Aug 2011 06:56:37.0839 (UTC) FILETIME=[A65F05F0:01CC572A]
Cc python-list@python.org
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.12
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.2090.1312959401.1164.python-list@python.org> (permalink)
Lines 32
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1312959401 news.xs4all.nl 23956 [2001:888:2000:d::a6]:35666
X-Complaints-To abuse@xs4all.nl
Xref x330-a1.tempe.blueboxinc.net comp.lang.python:11099

Show key headers only | View raw


I did read that portion of the doc, then I change it to communicate and it still hangs. So I reverted back to "wait" while launching one thread to see if I could isolate the problem, but it still hangs regardless of using "wait" or "communicate".

-----Original Message-----
From: chris@rebertia.com [mailto:chris@rebertia.com] On Behalf Of Chris Rebert
Sent: Tuesday, August 09, 2011 11:53 PM
To: Danny Wong (dannwong)
Cc: python-list@python.org
Subject: Re: subprocess.Popen and thread module

> On Tue, Aug 9, 2011 at 11:38 PM, Danny Wong (dannwong)
> <dannwong@cisco.com> wrote:
>> Hi All,
>>   I'm trying to execute some external commands from multiple database.
>> I'm using threads and subprocess.Popen ( from docs, all the popen*
>> functions are deprecated and I was told to use subprocess.Popen) to
>> execute the external commands in parallel, but the commands seems to
>> hang.
>
> What's your Popen() call look like?

On Tue, Aug 9, 2011 at 11:48 PM, Danny Wong (dannwong)
<dannwong@cisco.com> wrote:
>    try:
>        cmd_output = subprocess.Popen(['scm', 'load', '--force', '-r', nickname, '-d', directory, project], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>        status = cmd_output.wait()

Er, did you read the warning about Popen.wait() in the docs? (emphasis added):
"""
Popen.wait()
    Wait for child process to terminate. Set and return returncode attribute.
    Warning: ***This will deadlock*** when using stdout=PIPE and/or
stderr=PIPE and the child process generates enough output to a pipe
such that it blocks waiting for the OS pipe buffer to accept more
data. Use communicate() to avoid that.
"""
– http://docs.python.org/library/subprocess.html#subprocess.Popen.wait

Cheers,
Chris

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

RE: subprocess.Popen and thread module "Danny Wong (dannwong)" <dannwong@cisco.com> - 2011-08-10 01:56 -0500

csiph-web