Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #60253

Re: Why pickling (was: Traceback when using multiprocessing)

Show key headers only | View raw

On Sat, Nov 23, 2013 at 3:38 AM, John Ladasky
<john_ladasky@sbcglobal.net> wrote:
> On Thursday, November 21, 2013 8:24:05 PM UTC-8, Chris Angelico wrote:
>
>> Oh, that part's easy. Let's leave the multiprocessing module out of it
>> for the moment; imagine you spin up two completely separate instances
>> of Python. Create some object in one of them; now, transfer it to the
>> other. How are you going to do it?
>
> For what definition of "completely separate"?
>
> If I have two instances of the same version of the Python interpreter running on the same hardware, and the same operating system, I expect I would just copy a block of memory from one interpreter to the other, and then write some new pointers.  That kind of data sharing has to be the most common kind.  It's also the simplest.

Okay, so you copy a block of memory. Now how are you going to
guarantee that you picked up everything that object references? Python
objects frequently reference other objects:

send_me = [1.0, 2.0, 3.0]

The block of memory might have the addresses of those three floats,
but that'll be invalid in the target. Somehow you need to package up
this object and everything else you need.

Ultimately, you need some system for turning a single object reference
(a pointer, if you like) into the entire package of information needed
to recreate that object on the other side. That's what pickling is.
It's a compact (with people to fight for its compactness, there's
current discussion elsewhere about that) format that can be easily
transferred around, which refcounted blocks of memory can't.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 09:01 -0800
  Re: Traceback when using multiprocessing, less than helpful? Chris Angelico <rosuav@gmail.com> - 2013-11-22 04:24 +1100
    Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 10:25 -0800
      Re: Traceback when using multiprocessing, less than helpful? Chris Angelico <rosuav@gmail.com> - 2013-11-22 07:53 +1100
        Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 13:19 -0800
          Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 13:49 -0800
            Re: Traceback when using multiprocessing, less than helpful? Ethan Furman <ethan@stoneleaf.us> - 2013-11-21 14:32 -0800
  Re: Traceback when using multiprocessing, less than helpful? Terry Reedy <tjreedy@udel.edu> - 2013-11-21 17:37 -0500
  Re: Traceback when using multiprocessing, less than helpful? John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-21 19:57 -0800
    Re: Traceback when using multiprocessing, less than helpful? Chris Angelico <rosuav@gmail.com> - 2013-11-22 15:24 +1100
      Why pickling (was: Traceback when using multiprocessing) John Ladasky <john_ladasky@sbcglobal.net> - 2013-11-22 08:38 -0800
        Re: Why pickling (was: Traceback when using multiprocessing) Chris Angelico <rosuav@gmail.com> - 2013-11-23 10:50 +1100
    Re: Traceback when using multiprocessing, less than helpful? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-11-22 09:09 +0000

csiph-web