Re: pickling instances of metaclass generated classes

Path	csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!newsfeed.eweka.nl!eweka.nl!feeder3.eweka.nl!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<python-python-list@m.gmane.org>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.005
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'definitions': 0.07; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:80.91.229.12': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'received:lo.gmane.org': 0.09; 'underlying': 0.09; 'am,': 0.12; 'enigma': 0.16; 'instances.': 0.16; 'kern': 0.16; 'metaclass': 0.16; 'metaclasses': 0.16; 'subject:classes': 0.16; 'wrote:': 0.18; 'trying': 0.21; 'maybe': 0.21; 'header:In-Reply-To:1': 0.22; 'statement': 0.23; 'defined': 0.24; 'fine': 0.24; 'code': 0.25; 'classes': 0.26; "i'm": 0.26; 'expect': 0.26; 'function': 0.27; 'van': 0.28; 'explicitly': 0.29; 'interpret': 0.29; 'weird': 0.29; 'class': 0.29; 'example': 0.29; 'url:library': 0.31; 'usually': 0.31; 'anyone': 0.31; 'header :User-Agent:1': 0.33; 'header:X-Complaints-To:1': 0.33; 'to:addr :python-list': 0.34; 'probably': 0.34; 'it.': 0.34; 'someone': 0.34; 'something': 0.35; 'unless': 0.35; 'url:python': 0.36; 'post': 0.36; 'explain': 0.36; 'minimal': 0.37; 'hello,': 0.37; 'doing': 0.38; 'using': 0.38; 'received:org': 0.38; 'some': 0.38; 'url:docs': 0.39; 'url:org': 0.39; 'help': 0.39; 'to:addr:python.org': 0.40; 'type': 0.61; 'world': 0.62; 'received:86': 0.63; 'our': 0.64; 'believe': 0.65; 'doing.': 0.73; 'why?': 0.77; 'eco': 0.91
X-Injected-Via-Gmane	http://gmane.org/
To	python-list@python.org
From	Robert Kern <robert.kern@gmail.com>
Subject	Re: pickling instances of metaclass generated classes
Date	Thu, 29 Dec 2011 11:08:29 +0000
References	<2d221d0f-458e-4821-8786-f064b44b3125@p16g2000yqd.googlegroups.com>
Mime-Version	1.0
Content-Type	text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding	7bit
X-Gmane-NNTP-Posting-Host	host86-145-112-31.range86-145.btcentralplus.com
User-Agent	Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:8.0) Gecko/20111105 Thunderbird/8.0
In-Reply-To	<2d221d0f-458e-4821-8786-f064b44b3125@p16g2000yqd.googlegroups.com>
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.12
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.4213.1325156924.27778.python-list@python.org> (permalink)
Lines	35
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1325156924 news.xs4all.nl 6897 [2001:888:2000:d::a6]:56664
X-Complaints-To	abuse@xs4all.nl
Xref	x330-a1.tempe.blueboxinc.net comp.lang.python:18165

Show key headers only | View raw

On 12/29/11 9:55 AM, lars van gemerden wrote:
> Hello,
>
> Can someone help me with the following:
>
> I am using metaclasses to make classes and these classes to make
> instances. Now I want to use multiprocessing, which needs to pickle
> these instances.
>
> Pickle cannot find the class definitions of the instances. I am trying
> to add a line to the __new__ of the metaclass to add the new class
> under the right name in the right module/place, so pickle can find
> it.
>
> Is this the right approach? Can anyone explain to me where/how to add
> these classes for pickle to find and maybe why?

Can you post some code (preferably pared down to a minimal example that fails)? 
I'm not really clear on what you are doing. I would expect that a class defined 
by a class statement would usually work fine unless if the metaclass is doing 
something particularly weird to it.

In any case, you can probably just explicitly register a reduction function for 
each type using copy_reg.pickle():

http://docs.python.org/library/copy_reg

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Thread

pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-29 01:55 -0800
  Re: pickling instances of metaclass generated classes Robert Kern <robert.kern@gmail.com> - 2011-12-29 11:08 +0000
  Re: pickling instances of metaclass generated classes Ian Kelly <ian.g.kelly@gmail.com> - 2011-12-29 12:55 -0700
    Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-30 01:50 -0800
      Re: pickling instances of metaclass generated classes Peter Otten <__peter__@web.de> - 2011-12-30 12:41 +0100
      Re: pickling instances of metaclass generated classes Peter Otten <__peter__@web.de> - 2012-01-01 13:51 +0100
    Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-30 03:16 -0800
      Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-30 07:56 -0800
        Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-30 08:51 -0800
          Re: pickling instances of metaclass generated classes Ian Kelly <ian.g.kelly@gmail.com> - 2011-12-30 10:09 -0700
  Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2012-01-03 03:43 -0800

csiph-web