DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=N/DLkOcRgxWxeSsxE7A6T9ja0j8UVKZqpvlEi+5JyWmGFE6kIYkW7Q5xivNgUHqFyy LWdRKQhJK914DuUuOM3yXBWR4It0ZhkA6ZUeaami72swS0W4eSR2lNQij/NBlc+xJoge GpHeLes41LQeWAh8z5TauvXVqJy9P7SnK4MyM=
MIME-Version: 1.0
In-Reply-To: <171365925.20110603092822@bitdefender.com>
References: <171365925.20110603092822@bitdefender.com>
Date: Fri, 3 Jun 2011 16:59:20 +1000
Subject: Re: Multiprocessing.connection magic
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.2418.1307084363.9059.python-list@python.org>
Lines: 45
NNTP-Posting-Host: 82.94.164.166
Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.stben.net!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:6918

On Fri, Jun 3, 2011 at 4:28 PM, Claudiu Popa <cpopa@bitdefender.com> wrote:
> Hello guys,
> =A0 =A0 =A0While =A0working =A0at a dispatcher using
> =A0multiprocessing.connection.Listener =A0module =A0I've stumbled upon so=
me
> =A0sort =A0 =A0of =A0magic =A0trick =A0that =A0amazed =A0me. How is this =
possible and
> =A0what =A0does =A0multiprocessing =A0library doing in background for thi=
s to
> =A0work?

I'm not sure what magic trick you're referring to - is it that you can
use dissimilar versions of Python and send strange objects? I did find
that trying this in reverse (sending from 3.2, receiving with 2.7.1)
failed with "ValueError: unsupported pickle protocol: 3". That, plus
the docstring for send and recv, might suggest what's happening: the
object gets pickled.

Pickling in 2.7.1 (close enough to your 2.6 I presume):
http://docs.python.org/library/pickle.html
Pickling in 3.2:
http://docs.python.org/py3k/library/pickle.html

>From the 3.2 docs:
"The pickle serialization format is guaranteed to be backwards
compatible across Python releases."
"Protocol version 3 was added in Python 3.0. It has explicit support
for bytes and cannot be unpickled by Python 2.x pickle modules. This
is the current recommended protocol, use it whenever it is possible."
"The following types can be pickled:
...
functions defined at the top level of a module
built-in functions defined at the top level of a module
...
"

Presumably, the send() function pickles at the HIGHEST_PROTOCOL or
DEFAULT_PROTOCOL, and recv() unpickles whatever it gets. If you do the
pickling manually, you could choose to use version 2 explicitly, and
then the 2.6 other end could read it comfortably.

I don't know how effective the pickling of functions actually is.
Someone else will doubtless be able to fill that in.

Chris Angelico