Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!eweka.nl!lightspeed.eweka.nl!194.109.133.87.MISMATCH!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.001 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'programmer': 0.03; 'handler': 0.05; '21,': 0.07; 'class,': 0.07; '22,': 0.09; 'exception,': 0.09; 'happen,': 0.09; 'oh,': 0.09; 'pep': 0.09; 'properly.': 0.09; 'subject:using': 0.09; 'try:': 0.09; 'wrapper': 0.09; 'python': 0.11; 'stored': 0.12; 'itself.': 0.14; 'chris,': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'hmm.': 0.16; 'notation,': 0.16; 'relevant.': 0.16; 'retains': 0.16; 'subject:skip:m 10': 0.16; 'subject:when': 0.16; 'url:peps': 0.16; 'exception': 0.16; 'thursday,': 0.16; 'appropriate': 0.16; 'wrote:': 0.18; 'properly': 0.19; "python's": 0.19; '(the': 0.22; 'issue.': 0.22; 'install': 0.23; 'case.': 0.24; "shouldn't": 0.24; 'url:dev': 0.24; 'versions': 0.24; 'looks': 0.24; 'tracker': 0.26; 'header:In-Reply-To:1': 0.27; 'chris': 0.29; 'feature': 0.29; 'am,': 0.29; 'patch': 0.29; 'thus': 0.29; 'message- id:@mail.gmail.com': 0.30; 'code': 0.31; 'exceptions': 0.31; 'url:python': 0.33; 'fri,': 0.33; 'implemented': 0.33; 'could': 0.34; 'knowledge': 0.35; 'info': 0.35; 'except': 0.35; 'something': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'next': 0.36; 'useful': 0.36; 'possible': 0.36; 'subject:?': 0.36; 'url:org': 0.36; 'performance': 0.37; 'being': 0.38; 'thank': 0.38; 'nov': 0.38; 'to:addr:python-list': 0.38; 'anything': 0.39; 'to:addr:python.org': 0.39; 'chain': 0.60; 'worry': 0.60; 'most': 0.60; 'hope': 0.61; 'john': 0.61; 'complete': 0.62; 'kind': 0.63; 'real': 0.63; 'more': 0.64; 'due': 0.66; 'it!': 0.67; 'default': 0.69; 'evaluate': 0.72; 'article': 0.77; '3.4': 0.84; 'presumably': 0.84; 'reading,': 0.84; 'stored,': 0.84; 'enhancement': 0.95; '2013': 0.98 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=IQrpwkgmVtILV7MSw3IgCGk22nNLuLXNk8q9QnQtbRw=; b=Ioi+I5Gcx2ePlJav8E62enWqWK3V/upm2sx+6I7upyu1uSjYfwauageOUrRZIb3o3F olqNs80b1jb2Roa9hGuPmZg7Az6lKMSsQZePDv1U5+pLolu5ruHnGtKq9bhkCi9pT9ip sSTv2KaC2n7Ghrq1W82KpJXhLdTzTnUBj3OwmtGOaRm8HjWz7KfTqHOH2fTj4HaJ/Tuf hMZeIp7N/uDcoBBRLM5i8tBzLjalB47cgPesRDZ+EoMgC7ud46+rOR9UCcNjYPdoak5B 24Q6kCv0au6AeZYwhOaJ28HNcXHb9f+sIx5X0cfULXioMkvKDHb2NFbDHMMdcasf3O5K c1dw== MIME-Version: 1.0 X-Received: by 10.66.11.202 with SMTP id s10mr8409162pab.86.1385067187472; Thu, 21 Nov 2013 12:53:07 -0800 (PST) In-Reply-To: <9eb71131-7ca0-4a21-a8f3-98371ee8787e@googlegroups.com> References: <9eb71131-7ca0-4a21-a8f3-98371ee8787e@googlegroups.com> Date: Fri, 22 Nov 2013 07:53:07 +1100 Subject: Re: Traceback when using multiprocessing, less than helpful? From: Chris Angelico To: python-list@python.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 42 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1385067196 news.xs4all.nl 15869 [2001:888:2000:d::a6]:45209 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:60173 On Fri, Nov 22, 2013 at 5:25 AM, John Ladasky wrote: > On Thursday, November 21, 2013 9:24:33 AM UTC-8, Chris Angelico wrote: > >> Hmm. This looks like a possible need for the 'raise from' syntax. > > Thank you, Chris, that made me feel like a REAL Python programmer -- I ju= st did some reading, and the "raise from" feature was not implemented until= Python 3! And I might actually need it! :^) > > I think that the article http://www.python.org/dev/peps/pep-3134/ is rele= vant. Reading it now. To be clear: the complete exception change is store= d in every class, it's just not being displayed? I hope that's the case. = I shouldn't have to install a "raise from" hook in multiprocessing.map_asyn= c itself. > That PEP is all about the 'raise from' notation, yes; but the exception chaining is presumably not being stored, or else you would be able to see it in the default printout. So the best solution to this is, most likely, a patch to multiprocessing to have it chain exceptions properly. I think that would be considered a bugfix, and thus back-ported to all appropriate versions (rather than a feature enhancement that goes in 3.4 or 3.5 only). What you could try is printing out the __cause__ and __context__ of the exception, to see if there's anything useful in them; if there's nothing, the next thing to try would be some kind of wrapper in your inner handler (the evaluate function) that retains additional information. Oh, something else to try: It might be that the proper exception chaining would happen, except that the info isn't traversing processes properly due to pickling or something. Can you patch your code to use threading instead of multiprocessing? That might reveal something. (Don't worry about abysmal performance at this stage.) Hopefully someone with more knowledge of Python's internals can help out, here. One way or another, I suspect this will result in a tracker issue. ChrisA