Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin2!goblin.stu.neva.ru!newsfeed1.swip.net!uio.no!news.tele.dk!news.tele.dk!small.news.tele.dk!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.007
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'python.': 0.02; 'interpreter': 0.05; '21,': 0.07; 'memory.': 0.07; 'compact': 0.09; 'objects:': 0.09; 'oh,': 0.09; 'subject:Why': 0.09; 'subject:using': 0.09; 'python': 0.11; '23,': 0.16; 'blocks': 0.16; "can't.": 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'other,': 0.16; 'pointer,': 0.16; 'pointers.': 0.16; 'side.': 0.16; 'subject:skip:m 10': 0.16; 'subject:when': 0.16; 'sat,': 0.16; 'thursday,': 0.16; 'wrote:': 0.18; 'discussion': 0.18; 'module': 0.19; 'memory': 0.22; 'separate': 0.22; 'header:In-Reply-To:1': 0.27; 'chris': 0.29; 'leave': 0.29; 'am,': 0.29; 'message-id:@mail.gmail.com': 0.30; 'running': 0.33; 'addresses': 0.33; 'subject: (': 0.35; 'common': 0.35; 'definition': 0.35; 'objects': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'version': 0.36; 'instances': 0.36; 'operating': 0.37; 'two': 0.37; 'easily': 0.37; 'system,': 0.38; 'nov': 0.38; 'needed': 0.38; 'to:addr:python-list': 0.38; 'expect': 0.39; 'to:addr:python.org': 0.39; 'how': 0.40; 'is.': 0.60; 'most': 0.60; 'new': 0.61; 'entire': 0.61; 'john': 0.61; 'guarantee': 0.63; 'information': 0.63; 'kind': 0.63; 'frequently': 0.68; 'invalid': 0.68; 'transferred': 0.68; 'sharing': 0.69; 'other.': 0.75; 'transfer': 0.82; 'around,': 0.84; 'that)': 0.84; 'imagine': 0.93; 'fight': 0.97; '2013': 0.98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=2HcaEUykhVg5UBe3ccl4fzVd23I5SN2EV3yjzofn1h8=; b=Bpu8XbiV0XxsehQPEQS7UzCudNf3ly1U8ez8dRy0JcQVbwuAdpemE7OXkuJAFJx2mV +x5QrXCfPdAxIKnV2HkRJcwCaC3PB3uF5vshKoscLrTT1CKoDP7wkZivntZIARljt77W 2AzflITzkLuZpQl4q3P7eEJqpF01j1FD07PUMWrYYT0/sKpBFZinQ7c6OeZE9awXCK00 /xwPMd3NGWq/RQmnN7ztCR4t4Zguio7zbqMaQEJg6Wrkm/xoTDPpJnd7CeEAdSVraB+W Ohx63lEz5Xet4dFNApb57U7L7cGvAEHQYW4FXZd96MF915qzYG7UJxOkqH94bdMVWY98 AWmw==
MIME-Version: 1.0
X-Received: by 10.68.216.132 with SMTP id oq4mr14354521pbc.50.1385164254615; Fri, 22 Nov 2013 15:50:54 -0800 (PST)
In-Reply-To: <a8bf86c1-bf44-4bcf-813e-5ad4fdedde63@googlegroups.com>
References: <e92311cb-6cc5-415a-bbf8-544c0c9c6a54@googlegroups.com> <081af7df-2330-4b8b-abbf-4707edfcc17a@googlegroups.com> <mailman.3025.1385094254.18130.python-list@python.org> <a8bf86c1-bf44-4bcf-813e-5ad4fdedde63@googlegroups.com>
Date: Sat, 23 Nov 2013 10:50:54 +1100
Subject: Re: Why pickling (was: Traceback when using multiprocessing)
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.3058.1385164257.18130.python-list@python.org>
Lines: 35
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1385164257 news.xs4all.nl 15869 [2001:888:2000:d::a6]:48567
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:60253

On Sat, Nov 23, 2013 at 3:38 AM, John Ladasky
<john_ladasky@sbcglobal.net> wrote:
> On Thursday, November 21, 2013 8:24:05 PM UTC-8, Chris Angelico wrote:
>
>> Oh, that part's easy. Let's leave the multiprocessing module out of it
>> for the moment; imagine you spin up two completely separate instances
>> of Python. Create some object in one of them; now, transfer it to the
>> other. How are you going to do it?
>
> For what definition of "completely separate"?
>
> If I have two instances of the same version of the Python interpreter run=
ning on the same hardware, and the same operating system, I expect I would =
just copy a block of memory from one interpreter to the other, and then wri=
te some new pointers.  That kind of data sharing has to be the most common =
kind.  It's also the simplest.

Okay, so you copy a block of memory. Now how are you going to
guarantee that you picked up everything that object references? Python
objects frequently reference other objects:

send_me =3D [1.0, 2.0, 3.0]

The block of memory might have the addresses of those three floats,
but that'll be invalid in the target. Somehow you need to package up
this object and everything else you need.

Ultimately, you need some system for turning a single object reference
(a pointer, if you like) into the entire package of information needed
to recreate that object on the other side. That's what pickling is.
It's a compact (with people to fight for its compactness, there's
current discussion elsewhere about that) format that can be easily
transferred around, which refcounted blocks of memory can't.

ChrisA