Path: csiph.com!usenet.pasdenom.info!gegeweb.org!usenet-fr.net!nerim.net!novso.com!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.045 X-Spam-Evidence: '*H*': 0.91; '*S*': 0.00; 'true,': 0.05; 'string': 0.09; 'debugger': 0.09; 'exit': 0.09; 'output,': 0.09; 'overwrite': 0.09; 'thread': 0.14; '__del__': 0.16; 'enough.': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'guessing': 0.16; 'stuff,': 0.16; 'termination.': 0.16; 'thread,': 0.16; 'thread?': 0.16; 'wrote:': 0.18; 'code.': 0.18; 'trying': 0.19; 'seems': 0.21; 'code,': 0.22; 'print': 0.22; 'simpler': 0.24; 'paul': 0.24; "haven't": 0.24; 'handling': 0.26; 'code:': 0.26; 'header:In-Reply-To:1': 0.27; 'appear': 0.29; 'am,': 0.29; "doesn't": 0.30; 'message-id:@mail.gmail.com': 0.30; "i'm": 0.30; 'code': 0.31; 'sep': 0.31; 'probably': 0.32; 'stuff': 0.32; 'another': 0.32; 'running': 0.33; 'fri,': 0.33; 'sense': 0.34; "i'd": 0.34; 'message.': 0.35; 'problem': 0.35; 'basic': 0.35; 'objects': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'are,': 0.36; 'method': 0.36; 'so,': 0.37; 'process,': 0.38; 'to:addr:python-list': 0.38; 'little': 0.38; 'expect': 0.39; 'sure': 0.39; 'to:addr:python.org': 0.39; 'called': 0.40; 'skip:u 10': 0.60; 'read': 0.60; 'problems.': 0.60; 'subject: / ': 0.60; 'strictly': 0.61; 'simply': 0.61; 'first': 0.61; 'show': 0.63; 'kind': 0.63; 'soon': 0.63; 'happen': 0.63; 'here': 0.66; 'close': 0.67; 'results': 0.69; 'receive': 0.70; 'therefore': 0.72; 'gotten': 0.74; 'goal': 0.75; 'death,': 0.84; 'safer': 0.84; 'subject:skip:M 10': 0.84; '2013': 0.98 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=v8/DrCwslzQsBdjPGPgECt5Nt3FJKEpmse43Ji5gWII=; b=IzqkYJgftOi0mHPHMUBK74A6cZg1f8SFQh75fOsgAX8Iv/Ck5oXEbiT6ePK5r2AZqZ 59vGrJ+9DAai/6jxeqRWRdQKO9dp+C5jTEI5wAYpOesELU1RWQCsOa0h9eXXOr6XvfdU Prlae1c5nClJYh/isGXCpr669MJc1QVgfb5Swl/v2lIz5yOaNP2a+TCwUhkFr7DPGzfw imiSkg3b3MuYotHS7IK91emPZlHg08PvuhTE/Kw4aLerMzIcMzk4ta8YXZmQU6+gRFnR /M8MBpKw5uGuGg5NZ1UB/T42FKubCuA7y537ao6ljFAPp42A4jqeiCwFkBlBJtJM8Nir GSdg== MIME-Version: 1.0 X-Received: by 10.220.199.5 with SMTP id eq5mr4373435vcb.16.1378421200686; Thu, 05 Sep 2013 15:46:40 -0700 (PDT) In-Reply-To: References: Date: Fri, 6 Sep 2013 08:46:40 +1000 Subject: Re: Multiprocessing / threading confusion From: Chris Angelico To: python-list@python.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 34 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1378421209 news.xs4all.nl 15960 [2001:888:2000:d::a6]:45278 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:53744 On Fri, Sep 6, 2013 at 5:27 AM, Paul Pittlerson wro= te: > I'm trying to understand data handling using multiprocessing and threadin= g, haven't gotten very far without running into problems. This is my code: > > > What I expect to happen is the Debugger object will receive one string at= a time, and read it from the queue. But that's not what I see the the outp= ut, the "started worker" stuff seems to print for every process, but "ticke= d" and "exited" will show up in unpredictable ways, I'm guessing they overw= rite each other and therefore will not always appear in the output. > > So I'm looking for help in trying to make sense of this kind of stuff, I = thought this was the basic functionality that Queue() would take care of un= less there is some other problem in my code. The first thing I notice is that your Debugger will quit as soon as its one-secondly poll results in no data. This may or may not be a problem for your code, but I'd classify it as code smell at best. Is your goal here to make sure Debugger doesn't stop your process from exiting? If so, a simpler and safer solution is to make it a daemon thread. The other thing I see here is your use of __del__ to print your exit message. I don't know if Thread objects are involved in reference loops, but if they are, __del__ (probably) won't be called immediately on thread termination. Your subprocesses are a little odd; they spin off another thread, then halt the first thread. Why not simply do the work in the first thread? You then treat the thread's __del__ method as the process's death, which isn't strictly true, but probably close enough. ChrisA