Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.013
X-Spam-Evidence: '*H*': 0.97; '*S*': 0.00; 'correspond': 0.09; 'explicitely': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'subject:modules': 0.09; 'thus,': 0.09; '"global"': 0.16; 'expects': 0.16; 'implies': 0.16; 'imported.': 0.16; 'name)': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:plane.gmane.org': 0.16; 'received:t-ipconnect.de': 0.16; 'module': 0.19; 'work,': 0.20; 'input': 0.22; '(in': 0.22; 'header:User-Agent:1': 0.23; 'certainly': 0.24; 'pass': 0.26; 'skip:" 20': 0.27; 'header:X -Complaints-To:1': 0.27; 'feature': 0.29; "i'm": 0.30; 'code': 0.31; '(unless': 0.31; 'another.': 0.31; 'writes:': 0.31; 'class': 0.32; 'guess': 0.33; 'maybe': 0.34; "i'd": 0.34; 'subject:from': 0.34; 'could': 0.34; 'subject:with': 0.35; 'classes': 0.35; 'knows': 0.35; 'possible.': 0.35; 'something': 0.35; 'case,': 0.35; 'but': 0.35; 'add': 0.35; 'there': 0.35; 'maintained': 0.36; 'subject:data': 0.36; 'done': 0.36; 'charset:us-ascii': 0.36; 'similar': 0.36; 'application': 0.37; 'level': 0.37; 'implement': 0.38; 'requiring': 0.38; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'called': 0.40; 'how': 0.40; 'simply': 0.61; 'received:217': 0.63; 'information': 0.63; 'such': 0.63; 'normal.': 0.68; 'special': 0.74; 'ideas.': 0.84; 'loader.': 0.84; 'process...': 0.84; 'treats': 0.84; 'yours': 0.88
X-Injected-Via-Gmane: http://gmane.org/
To: python-list@python.org
From: dieter <dieter@handshake.de>
Subject: Re: Unpickling data with classes from dynamic modules
Date: Thu, 22 Aug 2013 08:35:29 +0200
References: <alpine.DEB.2.10.1308211732440.18360@shack.dolda2000.com> <5B80DD153D7D744689F57F4FB69AF4741867F701@SCACMX008.exchad.jpmchase.net> <alpine.DEB.2.10.1308220501470.18360@shack.dolda2000.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Gmane-NNTP-Posting-Host: pd9e097cc.dip0.t-ipconnect.de
User-Agent: Gnus/5.1008 (Gnus v5.10.8) XEmacs/21.4.22 (linux)
Cancel-Lock: sha1:jIEZqpr18gdMYdmAHWgHLmsBDF4=
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.120.1377153342.19984.python-list@python.org>
Lines: 34
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1377153342 news.xs4all.nl 16008 [2001:888:2000:d::a6]:42580
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:52815

Fredrik Tolf <fredrik@dolda2000.com> writes:

> ...
>> Maybe all you need to add is implementation for obj.__reduce__
>
> That would certainly work, and I guess I could perhaps use it as a
> work-around, but that would mean I'd have to mark every single such
> class as such in some way or another. What I'm looking for is a
> solution that could make them picklable transparently, just like
> normal.

This is not (easily) possible. In the "normal/transparent" case,
"pickle" treats the class/type part of a class/type instance
(unless it explicitely knows the "class/type" as requiring special
treatment) as a "global" (essentially a module path and a name)
and it expects that "global"s can simply be imported.


The ZODB ("Zope Object DataBase") is build on top of "pickle"
("cpickle", in fact) - and it provides for an application level
module loader. Zope is using this feature to implement
so called "ZClasses", classes the code of which are maintained
inside the ZODB.

Thus, this is a case similar to yours
(your dynamic modules correspond to code maintained in the ZODB).
This implies that there are ways to achieve something like this
(in principle) and you may look how it is done in Zope to
get some ideas. However, it is likely to be difficult.
Your "module loader" must be able to recreate your dynamic
modules - for this, it needs all the creation input data.
In the Zope case, this comes from the ZODB. You would need
to find a way to pass this information to the unpickling process...