Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #60200

Re: Traceback when using multiprocessing, less than helpful?

Show all headers | View raw

On Fri, Nov 22, 2013 at 2:57 PM, John Ladasky
<john_ladasky@sbcglobal.net> wrote:
> or, for that matter, why data needs to be pickled to pass it between processes.

Oh, that part's easy. Let's leave the multiprocessing module out of it
for the moment; imagine you spin up two completely separate instances
of Python. Create some object in one of them; now, transfer it to the
other. How are you going to do it?

Ultimately, the operating system isn't going to give you facilities
for moving complex objects around - what you almost exclusively get is
streams of bytes (or occasionally messaged chunks with lengths, but
still of bytes). Pickling is one method of turning an object into a
stream of bytes, in such a way that it can be turned back into an
equivalent object on the other side. And therein is the problem with
exceptions; since the traceback includes references to stack frames
and such, it's not as simple as saying "Two to beam up" and hearing
the classic sound effect - somehow you need to transfer all the
appropriate information across processes.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 09:01 -0800
  Re: Traceback when using multiprocessing, less than helpful? Chris Angelico <rosuav@gmail.com> - 2013-11-22 04:24 +1100
    Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 10:25 -0800
      Re: Traceback when using multiprocessing, less than helpful? Chris Angelico <rosuav@gmail.com> - 2013-11-22 07:53 +1100
        Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 13:19 -0800
          Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 13:49 -0800
            Re: Traceback when using multiprocessing, less than helpful? Ethan Furman <ethan@stoneleaf.us> - 2013-11-21 14:32 -0800
  Re: Traceback when using multiprocessing, less than helpful? Terry Reedy <tjreedy@udel.edu> - 2013-11-21 17:37 -0500
  Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 19:57 -0800
    Re: Traceback when using multiprocessing, less than helpful? Chris Angelico <rosuav@gmail.com> - 2013-11-22 15:24 +1100
      Why pickling (was: Traceback when using multiprocessing) John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-22 08:38 -0800
        Re: Why pickling (was: Traceback when using multiprocessing) Chris Angelico <rosuav@gmail.com> - 2013-11-23 10:50 +1100
    Re: Traceback when using multiprocessing, less than helpful? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-11-22 09:09 +0000

csiph-web