Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #60173

Re: Traceback when using multiprocessing, less than helpful?

Show key headers only | View raw

On Fri, Nov 22, 2013 at 5:25 AM, John Ladasky
<john_ladasky@sbcglobal.net> wrote:
> On Thursday, November 21, 2013 9:24:33 AM UTC-8, Chris Angelico wrote:
>
>> Hmm. This looks like a possible need for the 'raise from' syntax.
>
> Thank you, Chris, that made me feel like a REAL Python programmer -- I just did some reading, and the "raise from" feature was not implemented until Python 3!  And I might actually need it!  :^)
>
> I think that the article http://www.python.org/dev/peps/pep-3134/ is relevant.  Reading it now.  To be clear: the complete exception change is stored in every class, it's just not being displayed?  I hope that's the case.  I shouldn't have to install a "raise from" hook in multiprocessing.map_async itself.
>

That PEP is all about the 'raise from' notation, yes; but the
exception chaining is presumably not being stored, or else you would
be able to see it in the default printout. So the best solution to
this is, most likely, a patch to multiprocessing to have it chain
exceptions properly. I think that would be considered a bugfix, and
thus back-ported to all appropriate versions (rather than a feature
enhancement that goes in 3.4 or 3.5 only).

What you could try is printing out the __cause__ and __context__ of
the exception, to see if there's anything useful in them; if there's
nothing, the next thing to try would be some kind of wrapper in your
inner handler (the evaluate function) that retains additional
information.

Oh, something else to try: It might be that the proper exception
chaining would happen, except that the info isn't traversing processes
properly due to pickling or something. Can you patch your code to use
threading instead of multiprocessing? That might reveal something.
(Don't worry about abysmal performance at this stage.)

Hopefully someone with more knowledge of Python's internals can help
out, here. One way or another, I suspect this will result in a tracker
issue.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 09:01 -0800
  Re: Traceback when using multiprocessing, less than helpful? Chris Angelico <rosuav@gmail.com> - 2013-11-22 04:24 +1100
    Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 10:25 -0800
      Re: Traceback when using multiprocessing, less than helpful? Chris Angelico <rosuav@gmail.com> - 2013-11-22 07:53 +1100
        Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 13:19 -0800
          Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 13:49 -0800
            Re: Traceback when using multiprocessing, less than helpful? Ethan Furman <ethan@stoneleaf.us> - 2013-11-21 14:32 -0800
  Re: Traceback when using multiprocessing, less than helpful? Terry Reedy <tjreedy@udel.edu> - 2013-11-21 17:37 -0500
  Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 19:57 -0800
    Re: Traceback when using multiprocessing, less than helpful? Chris Angelico <rosuav@gmail.com> - 2013-11-22 15:24 +1100
      Why pickling (was: Traceback when using multiprocessing) John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-22 08:38 -0800
        Re: Why pickling (was: Traceback when using multiprocessing) Chris Angelico <rosuav@gmail.com> - 2013-11-23 10:50 +1100
    Re: Traceback when using multiprocessing, less than helpful? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-11-22 09:09 +0000

csiph-web