Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
From: "Prasad, Ramit" <ramit.prasad@jpmorgan.com.dmarc.invalid>
To: Fredrik Tolf <fredrik@dolda2000.com>, "python-list@python.org" <python-list@python.org>
Subject: RE: Unpickling data with classes from dynamic modules
Thread-Topic: Unpickling data with classes from dynamic modules
Thread-Index: AQHOnoom7jIuiQd9r0a5cksFgOK/iZmgEmDA
Date: Wed, 21 Aug 2013 19:59:47 +0000
References: <alpine.DEB.2.10.1308211732440.18360@shack.dolda2000.com>
In-Reply-To: <alpine.DEB.2.10.1308211732440.18360@shack.dolda2000.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
content-transfer-encoding: quoted-printable
MIME-Version: 1.0
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.101.1377116305.19984.python-list@python.org>
Lines: 58
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:52791

Fredrik Tolf wrote:
> 
> Dear list,
> 
> I have a system in which I load modules dynamically every now and then
> (that is, creating a module with types.ModuleType, compiling the code for
> the module and then executing it in the module with exec()), and where I
> would wish to be able to have classes in those modules containing classes
> that could pickled, and then unpickled again.
> 
> The problem with that, currently, is of course two-fold: The dynamically
> loaded module may not be loaded at the time when the unpickling takes
> place, but even if it were, it isn't registered in sys.modules, so the
> unpickler can't find it either way. And, of course, that is detected
> already when pickling.
> 
> Is there any way to fix this, at all?
> 
> I considered trying to create subclasses of the pickler and unpickler that
> pickle a reference to a module loader and data for the particular module
> along with a class that comes from such a module, by overriding
> Pickler.save_global and Unpickler.load_global. Unfortunately, however,
> those functions aren't part of the public interface and can't be
> overridden when the C-pickle module is used instead (which, obviously, is
> what normally happens).
> 
> Is there any way around this without having to modify the pickle module
> itself?
> 
> --
> 
> Fredrik Tolf
> --

I believe rather than subclassing the pickler, you are expected to 
change the behavior from within the class via __getstate__ and __setstate__.

http://docs.python.org/2/library/pickle.html#object.__getstate__

Although, for your use case (loading unknown classes) the section on pickle=
 and 
extension types may be more appropriate.

http://docs.python.org/2/library/pickle.html#pickling-and-unpickling-extensi=
on-types

Maybe all you need to add is implementation for obj.__reduce__


~Ramit



This email is confidential and subject to important disclaimers and conditio=
ns including on offers for the purchase or sale of securities, accuracy and=
 completeness of information, viruses, confidentiality, legal privilege, and=
 legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclo=
sures/email.