Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #107468 > unrolled thread

A pickle problem!

Back to article view | Back to comp.lang.python

Contents

  A pickle problem! Paulo da Silva <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> - 2016-04-21 22:43 +0100
    Re: A pickle problem! Paulo da Silva <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> - 2016-04-22 02:52 +0100
      Re: A pickle problem! Ian Kelly <ian.g.kelly@gmail.com> - 2016-04-22 10:27 -0600
        Re: A pickle problem! Paulo da Silva <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> - 2016-04-22 21:21 +0100
          Re: A pickle problem! Ian Kelly <ian.g.kelly@gmail.com> - 2016-04-22 14:33 -0600
            Re: A pickle problem! Paulo da Silva <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> - 2016-04-22 21:48 +0100
    Re: A pickle problem! Fabien <fabien.maussion@gmail.com> - 2016-04-24 12:20 +0200

#107468 — A pickle problem!

Hi.

Why in this code fragment self.__name is not kept between pickle
dumps/loads? How to fix it?

Thanks.

import pickle
import pandas as pd
import numpy as np

class C(pd.DataFrame):
	def __init__(self,name,*a,**b):
		super(C,self).__init__(*a,**b)
		self.__name=name

	def GetName(self):
		return self.__name


dates = pd.date_range('20130101', periods=6)
c = C("FOO",np.random.randn(6,4), index=dates, columns=list('ABCD'))

cd=pickle.dumps(c,pickle.HIGHEST_PROTOCOL)

d=pickle.loads(cd)

d.GetName()

# AttributeError: 'C' object has no attribute '_C__name'

[toc] | [next] | [standalone]

#107470

Às 22:43 de 21-04-2016, Paulo da Silva escreveu:
> Hi.
> 
> Why in this code fragment self.__name is not kept between pickle
> dumps/loads? How to fix it?
> 
> Thanks.
> 
> import pickle
> import pandas as pd
> import numpy as np
> 
> class C(pd.DataFrame):
> 	def __init__(self,name,*a,**b):
> 		super(C,self).__init__(*a,**b)
> 		self.__name=name
> 
> 	def GetName(self):
> 		return self.__name
> 
# Adding this works but looks tricky!

	def __getstate__(self):
		dfstate=super(C,self).__getstate__()
		cstate=(dfstate,self.__name)
		return cstate

	def __setstate__(self,cstate):
		super(C,self).__setstate__(cstate[0])
		self.__name=cstate[1]

> 
> dates = pd.date_range('20130101', periods=6)
> c = C("FOO",np.random.randn(6,4), index=dates, columns=list('ABCD'))
> 
> cd=pickle.dumps(c,pickle.HIGHEST_PROTOCOL)
> 
> d=pickle.loads(cd)
> 
> d.GetName()
> 
> # AttributeError: 'C' object has no attribute '_C__name'
> 

[toc] | [prev] | [next] | [standalone]

#107499

On Thu, Apr 21, 2016 at 7:52 PM, Paulo da Silva
<p_s_d_a_s_i_l_v_a_ns@netcabo.pt> wrote:
> Às 22:43 de 21-04-2016, Paulo da Silva escreveu:
>> Hi.
>>
>> Why in this code fragment self.__name is not kept between pickle
>> dumps/loads? How to fix it?
>>
>> Thanks.
>>
>> import pickle
>> import pandas as pd
>> import numpy as np
>>
>> class C(pd.DataFrame):
>>       def __init__(self,name,*a,**b):
>>               super(C,self).__init__(*a,**b)
>>               self.__name=name
>>
>>       def GetName(self):
>>               return self.__name
>>
> # Adding this works but looks tricky!
>
>         def __getstate__(self):
>                 dfstate=super(C,self).__getstate__()
>                 cstate=(dfstate,self.__name)
>                 return cstate
>
>         def __setstate__(self,cstate):
>                 super(C,self).__setstate__(cstate[0])
>                 self.__name=cstate[1]

Probably this is necessary because the DataFrame class is already
customizing its pickle behavior without taking into account the
possibility of added attributes by subclasses. I think that your
solution of wrapping the state of the superclass looks fine.

[toc] | [prev] | [next] | [standalone]

#107508

Às 17:27 de 22-04-2016, Ian Kelly escreveu:
> On Thu, Apr 21, 2016 at 7:52 PM, Paulo da Silva
> <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> wrote:
>> Às 22:43 de 21-04-2016, Paulo da Silva escreveu:
...

> 
> Probably this is necessary because the DataFrame class is already
> customizing its pickle behavior without taking into account the
> possibility of added attributes by subclasses. I think that your
> solution of wrapping the state of the superclass looks fine.
> 
Thank you.

For any other vars ...
Is there a way to get the vars of only the derived class?

[toc] | [prev] | [next] | [standalone]

#107509

On Fri, Apr 22, 2016 at 2:21 PM, Paulo da Silva
<p_s_d_a_s_i_l_v_a_ns@netcabo.pt> wrote:
> Às 17:27 de 22-04-2016, Ian Kelly escreveu:
>> On Thu, Apr 21, 2016 at 7:52 PM, Paulo da Silva
>> <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> wrote:
>>> Às 22:43 de 21-04-2016, Paulo da Silva escreveu:
> ...
>
>>
>> Probably this is necessary because the DataFrame class is already
>> customizing its pickle behavior without taking into account the
>> possibility of added attributes by subclasses. I think that your
>> solution of wrapping the state of the superclass looks fine.
>>
> Thank you.
>
> For any other vars ...
> Is there a way to get the vars of only the derived class?

If they start with two underscores then you could use the name
mangling to find them. If the class name is MyClass then look for any
keys in the instance dict that start with '_MyClass__'. Otherwise no,
you'd have to list them explicitly.

[toc] | [prev] | [next] | [standalone]

#107511

Às 21:33 de 22-04-2016, Ian Kelly escreveu:
> On Fri, Apr 22, 2016 at 2:21 PM, Paulo da Silva
> <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> wrote:
...
> 
> If they start with two underscores then you could use the name
> mangling to find them. If the class name is MyClass then look for any
> keys in the instance dict that start with '_MyClass__'. Otherwise no,
> you'd have to list them explicitly.
> 
OK, thanks.

[toc] | [prev] | [next] | [standalone]

#107543

On 04/21/2016 11:43 PM, Paulo da Silva wrote:
> class C(pd.DataFrame):

Note also that subclassing pandas is not always encouraged:

http://pandas.pydata.org/pandas-docs/stable/internals.html#subclassing-pandas-data-structures

Cheers,

Fabien

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web