Path: csiph.com!usenet.pasdenom.info!news.albasani.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.024 X-Spam-Evidence: '*H*': 0.95; '*S*': 0.00; 'exits': 0.07; 'subject:issue': 0.07; 'threads,': 0.07; 'executed': 0.09; 'url:github': 0.09; 'output': 0.10; 'cleanly': 0.16; 'does,': 0.16; 'pipe"': 0.16; 'looked': 0.16; 'wrote:': 0.18; 'instance': 0.18; "haven't": 0.20; 'header:In-Reply-To:1': 0.22; '(or': 0.22; 'sender:addr:gmail.com': 0.24; 'fact': 0.27; 'yield': 0.29; 'class': 0.29; 'seem': 0.29; 'case).': 0.30; 'terminate': 0.30; 'threads': 0.30; 'url:2007': 0.30; '(as': 0.31; 'like.': 0.32; 'thread': 0.32; 'fri,': 0.34; 'test': 0.34; 'lee': 0.34; 'flag': 0.34; 'url:02': 0.34; 'yet,': 0.34; 'to:addr:python-list': 0.35; 'finished': 0.35; 'received:209.85.160': 0.36; 'example,': 0.37; 'executing': 0.37; 'but': 0.37; 'charset:us-ascii': 0.37; 'received:google.com': 0.37; 'skip:" 10': 0.37; 'using': 0.37; 'received:209.85': 0.38; 'received:209.85.160.46': 0.39; 'received:209': 0.39; 'application': 0.40; 'being': 0.40; 'to:addr:python.org': 0.40; 'john': 0.61; 'your': 0.61; 'provided': 0.62; 'header:Message-Id:1': 0.62; 'subject: / ': 0.63; 'cause': 0.67; 'here:': 0.67; 'clemens': 0.84; 'flipping': 0.84 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:in-reply-to:references :x-mailer:mime-version:content-type:content-transfer-encoding; bh=0Vui1BUp7PaqcuwBZUMLhJZNbqbfqGtjbO9S1iIcTK0=; b=eglIOVJaEazVg0o1kJqXde8vT0b1wBRL7YXBMVcM6Sld1pOzcSj+Es4XBCYzfSHJU4 LpZV9NSq7mgL+JOpRZYRI5jYfySYdUEdseygIuIdLyOMat2DO9Z9TqGLeCxaTuLwfphz xE2rilNeF4ybjDmdPwXQYpnQybD3epF2k/wy5k1jlQfaf1JW6GUvjJDzZ1Rt8xQyP8p2 akN9rEE/1rk904Qy+L3C+qg7ERinUB/+DgD4S+RFnvmxOptFjAa1hzm79CM8pYY/q47t drtsyjZoESHn/6wptecP1yRVOc+ib2UDO2AYqj+o4Drpt5vys/8FkDcN5UGD32IcX8uP qXww== Sender: "John O'Hagan" Date: Sat, 17 Mar 2012 14:37:26 +1100 From: John O'Hagan To: python-list@python.org Subject: Re: Daemonization / Popen / pipe issue In-Reply-To: <4F63F2FE.1030409@leeclemens.net> References: <4F63F2FE.1030409@leeclemens.net> X-Mailer: Sylpheed 3.2.0beta6 (GTK+ 2.24.10; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 32 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1331955457 news.xs4all.nl 6888 [2001:888:2000:d::a6]:41489 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:21796 On Fri, 16 Mar 2012 22:12:14 -0400 Lee Clemens wrote: > > I have a multi-threaded application, each thread has an instance of a class > which calls Popen. The command(s) being executed (shell=True) include pipes. > The errors I have seen involve "broken pipe" and unexepected output (as > demonstrated in the test case). > > This issues only seem to occur when the application is "daemonized", using a > double-fork and os.dup2, as shown here: > http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/ > > I have provided a test-case here: https://gist.github.com/2054194 > > Please test how flipping DAEMONIZE between True and False yield different > results. I haven't looked at your test case yet, but a possible cause is the fact that the main application (or main thread) exits if it has finished executing and only daemon threads remain. This can abruptly terminate threads which may be busy, for example, communicating via a pipe. The best solution IMO is not to use daemon threads, but to give all threads a way to terminate cleanly before the main thread does, even if this means using a flag or the like. HTH, John