Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #18190

Re: pickling instances of metaclass generated classes

Show all headers | View raw

On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <lars@rational-it.com> wrote:
> Hello,
>
> Can someone help me with the following:
>
> I am using metaclasses to make classes and these classes to make
> instances. Now I want to use multiprocessing, which needs to pickle
> these instances.
>
> Pickle cannot find the class definitions of the instances. I am trying
> to add a line to the __new__ of the metaclass to add the new class
> under the right name in the right module/place, so pickle can find
> it.
>
> Is this the right approach? Can anyone explain to me where/how to add
> these classes for pickle to find and maybe why?

It sounds like you're trying to do something like this?

>>> class MetaClass(type):
...     pass
...
>>> instance = MetaClass('<Anonymous>', (object,), {})()
>>> instance
<__main__.<Anonymous> object at 0x00CC00F0>
>>> import pickle
>>> pickle.dumps(instance)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\python27\lib\pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "c:\python27\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "c:\python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "c:\python27\lib\pickle.py", line 401, in save_reduce
    save(args)
  File "c:\python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "c:\python27\lib\pickle.py", line 562, in save_tuple
    save(element)
  File "c:\python27\lib\pickle.py", line 295, in save
    self.save_global(obj)
  File "c:\python27\lib\pickle.py", line 748, in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
it's not found as __main__.<Anonymous>


Yeah, pickle's not going to work with anonymous classes.  As you
suggest, you could dynamically add the classes to the module namespace
so that pickle.dumps will find them, but bear in mind that they will
also have to exist when calling pickle.loads, so you will need to be
able to reconstruct the same anonymous classes before unpickling later
on.

Cheers,
Ian

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-29 01:55 -0800
  Re: pickling instances of metaclass generated classes Robert Kern <robert.kern@gmail.com> - 2011-12-29 11:08 +0000
  Re: pickling instances of metaclass generated classes Ian Kelly <ian.g.kelly@gmail.com> - 2011-12-29 12:55 -0700
    Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-30 01:50 -0800
      Re: pickling instances of metaclass generated classes Peter Otten <__peter__@web.de> - 2011-12-30 12:41 +0100
      Re: pickling instances of metaclass generated classes Peter Otten <__peter__@web.de> - 2012-01-01 13:51 +0100
    Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-30 03:16 -0800
      Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-30 07:56 -0800
        Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2011-12-30 08:51 -0800
          Re: pickling instances of metaclass generated classes Ian Kelly <ian.g.kelly@gmail.com> - 2011-12-30 10:09 -0700
  Re: pickling instances of metaclass generated classes lars van gemerden <lars@rational-it.com> - 2012-01-03 03:43 -0800

csiph-web