Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #21264 > unrolled thread

pickle/unpickle class which has changed

Back to article view | Back to comp.lang.python

Contents

  pickle/unpickle class which has changed Neal Becker <ndbecker2@gmail.com> - 2012-03-06 07:34 -0500
    Re: pickle/unpickle class which has changed Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-03-06 13:55 +0000
      Re: pickle/unpickle class which has changed Peter Otten <__peter__@web.de> - 2012-03-06 15:28 +0100
      Re: pickle/unpickle class which has changed Neal Becker <ndbecker2@gmail.com> - 2012-03-06 11:29 -0500
      Re: pickle/unpickle class which has changed Peter Otten <__peter__@web.de> - 2012-03-06 18:10 +0100

#21264 — pickle/unpickle class which has changed

What happens if I pickle a class, and later unpickle it where the class now has 
added some new attributes?

[toc] | [next] | [standalone]

#21269

On Tue, 06 Mar 2012 07:34:34 -0500, Neal Becker wrote:

> What happens if I pickle a class, and later unpickle it where the class
> now has added some new attributes?

Why don't you try it?

py> import pickle
py> class C:
...     a = 23
...
py> c = C()
py> pickled = pickle.dumps(c)
py> C.b = 42  # add a new class attribute
py> d = pickle.loads(pickled)
py> d.a
23
py> d.b
42


Unless you mean something different from this, adding attributes to the 
class is perfectly fine.

But... why are you dynamically adding attributes to the class? Isn't that 
rather unusual?


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#21271

Steven D'Aprano wrote:

> On Tue, 06 Mar 2012 07:34:34 -0500, Neal Becker wrote:
> 
>> What happens if I pickle a class, and later unpickle it where the class
>> now has added some new attributes?
> 
> Why don't you try it?
> 
> py> import pickle
> py> class C:
> ...     a = 23
> ...
> py> c = C()
> py> pickled = pickle.dumps(c)
> py> C.b = 42  # add a new class attribute
> py> d = pickle.loads(pickled)
> py> d.a
> 23
> py> d.b
> 42
> 
> 
> Unless you mean something different from this, adding attributes to the
> class is perfectly fine.
> 
> But... why are you dynamically adding attributes to the class? Isn't that
> rather unusual?

The way I understand the problem is that an apparently backwards-compatible 
change like adding a third dimension to a point with an obvious default 
breaks when you restore an "old" instance in a script with the "new" 
implementation:

>>> import pickle
>>> class P(object):
...     def __init__(self, x, y):
...             self.x = x
...             self.y = y
...     def r2(self):
...             return self.x*self.x + self.y*self.y
... 
>>> p = P(2, 3)
>>> p.r2()
13
>>> s = pickle.dumps(p)
>>> class P(object):
...     def __init__(self, x, y, z=0):
...             self.x = x
...             self.y = y
...             self.z = z
...     def r2(self):
...             return self.x*self.x + self.y*self.y + self.z*self.z
... 
>>> p = P(2, 3)
>>> p.r2()
13
>>> pickle.loads(s).r2()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 7, in r2
AttributeError: 'P' object has no attribute 'z'

By default pickle doesn't invoke __init__() and updates __dict__ directly.
As pointed out in my previous post one way to fix the problem is to 
implement a __setstate__() method:

>>> class P(object):
...     def __init__(self, x, y, z=0):
...             self.x = x
...             self.y = y
...             self.z = z
...     def r2(self):
...             return self.x*self.x + self.y*self.y + self.z*self.z
...     def __setstate__(self, state):
...             self.__dict__["z"] = 42 # stupid default
...             self.__dict__.update(state)
... 
>>> pickle.loads(s).r2()
1777

This keeps working with pickles of the new implementation of P:

>>> q = P(3, 4, 5)
>>> pickle.loads(pickle.dumps(q)).r2()
50

[toc] | [prev] | [next] | [standalone]

#21275

Peter Otten wrote:

> Steven D'Aprano wrote:
> 
>> On Tue, 06 Mar 2012 07:34:34 -0500, Neal Becker wrote:
>> 
>>> What happens if I pickle a class, and later unpickle it where the class
>>> now has added some new attributes?
>> 
>> Why don't you try it?
>> 
>> py> import pickle
>> py> class C:
>> ...     a = 23
>> ...
>> py> c = C()
>> py> pickled = pickle.dumps(c)
>> py> C.b = 42  # add a new class attribute
>> py> d = pickle.loads(pickled)
>> py> d.a
>> 23
>> py> d.b
>> 42
>> 
>> 
>> Unless you mean something different from this, adding attributes to the
>> class is perfectly fine.
>> 
>> But... why are you dynamically adding attributes to the class? Isn't that
>> rather unusual?
> 
> The way I understand the problem is that an apparently backwards-compatible
> change like adding a third dimension to a point with an obvious default
> breaks when you restore an "old" instance in a script with the "new"
> implementation:
> 
>>>> import pickle
>>>> class P(object):
> ...     def __init__(self, x, y):
> ...             self.x = x
> ...             self.y = y
> ...     def r2(self):
> ...             return self.x*self.x + self.y*self.y
> ...
>>>> p = P(2, 3)
>>>> p.r2()
> 13
>>>> s = pickle.dumps(p)
>>>> class P(object):
> ...     def __init__(self, x, y, z=0):
> ...             self.x = x
> ...             self.y = y
> ...             self.z = z
> ...     def r2(self):
> ...             return self.x*self.x + self.y*self.y + self.z*self.z
> ...
>>>> p = P(2, 3)
>>>> p.r2()
> 13
>>>> pickle.loads(s).r2()
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 7, in r2
> AttributeError: 'P' object has no attribute 'z'
> 
> By default pickle doesn't invoke __init__() and updates __dict__ directly.
> As pointed out in my previous post one way to fix the problem is to
> implement a __setstate__() method:
> 
>>>> class P(object):
> ...     def __init__(self, x, y, z=0):
> ...             self.x = x
> ...             self.y = y
> ...             self.z = z
> ...     def r2(self):
> ...             return self.x*self.x + self.y*self.y + self.z*self.z
> ...     def __setstate__(self, state):
> ...             self.__dict__["z"] = 42 # stupid default
> ...             self.__dict__.update(state)
> ...
>>>> pickle.loads(s).r2()
> 1777
> 
> This keeps working with pickles of the new implementation of P:
> 
>>>> q = P(3, 4, 5)
>>>> pickle.loads(pickle.dumps(q)).r2()
> 50

So if in my new class definition there are now some new attributes, and if I did 
not add a __setstate__ to set the new attributes, I guess then when unpickled 
the instance of the class will simply lack those attributes?

[toc] | [prev] | [next] | [standalone]

#21277

Neal Becker wrote:

> Peter Otten wrote:
> 
>> Steven D'Aprano wrote:
>> 
>>> On Tue, 06 Mar 2012 07:34:34 -0500, Neal Becker wrote:
>>> 
>>>> What happens if I pickle a class, and later unpickle it where the class
>>>> now has added some new attributes?
>>> 
>>> Why don't you try it?
>>> 
>>> py> import pickle
>>> py> class C:
>>> ...     a = 23
>>> ...
>>> py> c = C()
>>> py> pickled = pickle.dumps(c)
>>> py> C.b = 42  # add a new class attribute
>>> py> d = pickle.loads(pickled)
>>> py> d.a
>>> 23
>>> py> d.b
>>> 42
>>> 
>>> 
>>> Unless you mean something different from this, adding attributes to the
>>> class is perfectly fine.
>>> 
>>> But... why are you dynamically adding attributes to the class? Isn't
>>> that rather unusual?
>> 
>> The way I understand the problem is that an apparently
>> backwards-compatible change like adding a third dimension to a point with
>> an obvious default breaks when you restore an "old" instance in a script
>> with the "new" implementation:
>> 
>>>>> import pickle
>>>>> class P(object):
>> ...     def __init__(self, x, y):
>> ...             self.x = x
>> ...             self.y = y
>> ...     def r2(self):
>> ...             return self.x*self.x + self.y*self.y
>> ...
>>>>> p = P(2, 3)
>>>>> p.r2()
>> 13
>>>>> s = pickle.dumps(p)
>>>>> class P(object):
>> ...     def __init__(self, x, y, z=0):
>> ...             self.x = x
>> ...             self.y = y
>> ...             self.z = z
>> ...     def r2(self):
>> ...             return self.x*self.x + self.y*self.y + self.z*self.z
>> ...
>>>>> p = P(2, 3)
>>>>> p.r2()
>> 13
>>>>> pickle.loads(s).r2()
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "<stdin>", line 7, in r2
>> AttributeError: 'P' object has no attribute 'z'
>> 
>> By default pickle doesn't invoke __init__() and updates __dict__
>> directly. As pointed out in my previous post one way to fix the problem
>> is to implement a __setstate__() method:
>> 
>>>>> class P(object):
>> ...     def __init__(self, x, y, z=0):
>> ...             self.x = x
>> ...             self.y = y
>> ...             self.z = z
>> ...     def r2(self):
>> ...             return self.x*self.x + self.y*self.y + self.z*self.z
>> ...     def __setstate__(self, state):
>> ...             self.__dict__["z"] = 42 # stupid default
>> ...             self.__dict__.update(state)
>> ...
>>>>> pickle.loads(s).r2()
>> 1777
>> 
>> This keeps working with pickles of the new implementation of P:
>> 
>>>>> q = P(3, 4, 5)
>>>>> pickle.loads(pickle.dumps(q)).r2()
>> 50
> 
> So if in my new class definition there are now some new attributes, and if
> I did not add a __setstate__ to set the new attributes, I guess then when
> unpickled the instance of the class will simply lack those attributes?

I don't know. If you don't trust the demo try it yourself with the actual 
code you have. Throwing in

obj = pickle.load(...)
print vars(obj)

should help.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web