Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #12359 > unrolled thread

Unpickle error -- "object has no attribute ...."

Back to article view | Back to comp.lang.python

Contents

  Unpickle error -- "object has no attribute ...." luvspython <srehtvandy@gmail.com> - 2011-08-28 17:00 -0700
    Re: Unpickle error -- "object has no attribute ...." Peter Otten <__peter__@web.de> - 2011-08-29 11:02 +0200
      Re: Unpickle error -- "object has no attribute ...." luvspython <srehtvandy@gmail.com> - 2011-08-29 09:22 -0700
        Re: Unpickle error -- "object has no attribute ...." Chris Angelico <rosuav@gmail.com> - 2011-08-30 04:27 +1000
        Re: Unpickle error -- "object has no attribute ...." Peter Otten <__peter__@web.de> - 2011-08-30 16:21 +0200

#12359 — Unpickle error -- "object has no attribute ...."

I have an application that needs to keep a history of the values of
several attributes of each of many instances of many classes.  The
history-keeping logic is in a helper class, HistoryKeeper, that's
inherited by classes like Vehicle in the example below.

Pickling an instance of Vehicle works, but unpickling fails with:
 "Vehicle object has no attribute
'_orderedArgNames'"   (_orderedArgNames is an attribute in
HistoryKeeper that tells the attributes for which history must be
kept.)

During unpickling, the exception occurs at the 2nd line in the
__getattribute__ method:
        if item not in object.__getattribute__(self,
'_orderedArgNames'):

FWIW, cPickle fails the same way.

Below is a stripped-down example that fails in unpickling.

Can anyone explain why it fails and what I can do to fix it?

MANY thanks.

=========================================================

import datetime, bisect
from collections import OrderedDict



# define a class which helps keep date-history of attribute settings
in inheriting classes
class HistoryKeeper(object):
    """ Class to maintain a dated history of attribute settings in
inheriting classes.
    The initialization arguments are in an OrderedDict."""
    def __init__(self, orderedArgs):
        super(HistoryKeeper, self).__setattr__('_orderedArgNames',
orderedArgs.keys())   #remember the order of unnamed args
        for arg, value in orderedArgs.items():
            if arg != 'self':
                self.Set(arg, value)


    """ Get the current value of an attribute, optionally returning
its entire history."""
    def __getattribute__(self, item, returnHistory=False):
        value = object.__getattribute__(self, item)
        if item not in object.__getattribute__(self,
'_orderedArgNames'):
            return value         # not an attribute for which we
maintain a change history
        elif returnHistory:
            return value         # return the entire history
        else:
            return value[-1][0]  # return only the latest
value


    """ Set an attribute by appending the new value and date to
existing history of that attribute.
    Unless a setting-date is supplied, default to today.
    Set the value only if it's different than the chronological
immediately preceding value."""
    def __setattr__(self, item, value, date=None):

        # avoid history keeping if this item isn't among those
declared to require it
        if item not in self._orderedArgNames:
            super(HistoryKeeper, self).__setattr__(item, value)
        else:
            if not date:
                date = datetime.date.today()
            # if this attribute has already been set, add this value
to that history
            try:
                history = self.__getattribute__(item,
returnHistory=True)
                # if a date was supplied, ensure the history remains
in chronological order
                dates = [val[1] for val in history]
                index = bisect.bisect_right(dates, date)
                # insert this value into the history unless it doesn't
change an existing setting
                if index == 0 or history[index-1][0] != value:
                    history.insert(index, (value,date))
            except:
                history = [(value, date)]
            super(HistoryKeeper, self).__setattr__(item, history)

    def Set(self, item, value):
        self.__setattr__(item, value)




class Vehicle(HistoryKeeper):
    def __init__(self, tag, make, model):
        argDict = OrderedDict([('tag',tag),('make',make),
('model',model)])
        super(Vehicle, self).__init__(argDict)

if __name__ == "__main__":
    car = Vehicle('TAG123', 'FORD', 'Model A')

    import pickle
    pFile = open('car.pk1', 'wb')
    pickle.dump(car, pFile, -1)
    pFile.close()
    print "car pickled OK"

    pFile = open('car.pk1', 'rb')
    community = pickle.load(pFile)
    pFile.close()

[toc] | [next] | [standalone]

#12398

luvspython wrote:

> I have an application that needs to keep a history of the values of
> several attributes of each of many instances of many classes.  The
> history-keeping logic is in a helper class, HistoryKeeper, that's
> inherited by classes like Vehicle in the example below.
> 
> Pickling an instance of Vehicle works, but unpickling fails with:
>  "Vehicle object has no attribute
> '_orderedArgNames'"   (_orderedArgNames is an attribute in
> HistoryKeeper that tells the attributes for which history must be
> kept.)
> 
> During unpickling, the exception occurs at the 2nd line in the
> __getattribute__ method:
>         if item not in object.__getattribute__(self,
> '_orderedArgNames'):
> 
> FWIW, cPickle fails the same way.
> 
> Below is a stripped-down example that fails in unpickling.
> 
> Can anyone explain why it fails and what I can do to fix it?

By default unpickling an object does *not* invoke its __init__() method; 
instead it creates an instance and then updates the __dict__ attribute of 
that instance. You intercept attribute access with __getattribute__, so to 
get hold of __dict__ you need to know __dict__["_orderedArgNames"] first, i. 
e. you run into a bootstrap problem.

To fix the error I'd try special-casing "__dict__"

def __getattribute__(self, item, ...):
    if item == "__dict__":
        return super(HistoryKeeper, self).__getattribute__(item)
    ...

or making _orderedArgNames a class attribute:

class HistoryKeeper(object):
    def __init__(self, orderedArgs):
        for arg, value in orderedArgs.items():
            if arg != 'self':
                self.Set(arg, value)
    ...

class Vehicle(HistoryKeeper):
    _orderedArgNames = "tag", "make", "model"
    ...

If that doesn't work out you can write your own __reduce_ex__() method to 
customise pickling, see 
http://docs.python.org/library/pickle.html#object.__reduce_ex__

By the way, docstrings belong below the def not above:

def f():
    """Explain f() here"""

[toc] | [prev] | [next] | [standalone]

#12406

On Aug 29, 5:02 am, Peter Otten <__pete...@web.de> wrote:
> luvspython wrote:
> > I have an application that needs to keep a history of the values of
> > several attributes of each of many instances of many classes.  The
> > history-keeping logic is in a helper class, HistoryKeeper, that's
> > inherited by classes like Vehicle in the example below.
>
> > Pickling an instance of Vehicle works, but unpickling fails with:
> >  "Vehicle object has no attribute
> > '_orderedArgNames'"   (_orderedArgNames is an attribute in
> > HistoryKeeper that tells the attributes for which history must be
> > kept.)
>
> > During unpickling, the exception occurs at the 2nd line in the
> > __getattribute__ method:
> >         if item not in object.__getattribute__(self,
> > '_orderedArgNames'):
>
> > FWIW, cPickle fails the same way.
>
> > Below is a stripped-down example that fails in unpickling.
>
> > Can anyone explain why it fails and what I can do to fix it?
>
> By default unpickling an object does *not* invoke its __init__() method;
> instead it creates an instance and then updates the __dict__ attribute of
> that instance. You intercept attribute access with __getattribute__, so to
> get hold of __dict__ you need to know __dict__["_orderedArgNames"] first, i.
> e. you run into a bootstrap problem.
>
> To fix the error I'd try special-casing "__dict__"
>
> def __getattribute__(self, item, ...):
>     if item == "__dict__":
>         return super(HistoryKeeper, self).__getattribute__(item)
>     ...
>
> or making _orderedArgNames a class attribute:
>
> class HistoryKeeper(object):
>     def __init__(self, orderedArgs):
>         for arg, value in orderedArgs.items():
>             if arg != 'self':
>                 self.Set(arg, value)
>     ...
>
> class Vehicle(HistoryKeeper):
>     _orderedArgNames = "tag", "make", "model"
>     ...
>
> If that doesn't work out you can write your own __reduce_ex__() method to
> customise pickling, seehttp://docs.python.org/library/pickle.html#object.__reduce_ex__
>
> By the way, docstrings belong below the def not above:
>
> def f():
>     """Explain f() here"""- Hide quoted text -
>
> - Show quoted text -

THANK YOU!  Special-casing "__dict__" did the trick.  Not sure I'd
have ever figured that out, which leads to a different question:

I can figure out most things, though perhaps very slowly and
painfully, if I can trace through code.  I use WingIDE (love it), but
the execution
of the C code is of course hidden, which helped stymie on this
problem.  Is there another tool y'all might use and you can suggest
that deals with that problem and would have helped me with this case?
Or is one's ability to figure out this sort of problem largely
dependent on really understanding the system's internals?

[toc] | [prev] | [next] | [standalone]

#12415

On Tue, Aug 30, 2011 at 2:22 AM, luvspython <srehtvandy@gmail.com> wrote:
> I can figure out most things, though perhaps very slowly and
> painfully, if I can trace through code.  I use WingIDE (love it), but
> the execution
> of the C code is of course hidden, which helped stymie on this
> problem.  Is there another tool y'all might use and you can suggest
> that deals with that problem and would have helped me with this case?
> Or is one's ability to figure out this sort of problem largely
> dependent on really understanding the system's internals?

In terms of debugging, it's hard to ignore the old favorite stand-by:
If In Doubt, Print It Out. Pepper your code with console-output calls
and manually trace your code using those. Every other debugging tool
you'll ever use is a bonus on top of that; if you accustom yourself to
IIDPIO debugging, you'll be able to solve problems in any environment.

Python does have a number of other debugging tools, though. I'd
recommend looking at pylint, for a start. It's not technically a
debugger, but it may be of value.

ChrisA

[toc] | [prev] | [next] | [standalone]

#12439

luvspython wrote:

> THANK YOU!  Special-casing "__dict__" did the trick.  Not sure I'd
> have ever figured that out, which leads to a different question:
> 
> I can figure out most things, though perhaps very slowly and
> painfully, if I can trace through code.  I use WingIDE (love it), but
> the execution
> of the C code is of course hidden, which helped stymie on this
> problem.  Is there another tool y'all might use and you can suggest
> that deals with that problem and would have helped me with this case?
> Or is one's ability to figure out this sort of problem largely
> dependent on really understanding the system's internals?

I'm with Chris here, I make do with some understanding of Python and a 
generous amount of print statements. My other secret weapon is that I try to 
keep the complexity of my code low ;)

As Python is yet another C program you can of course step through its code 
(debug build) with an appropriate debugger, but I've never done that. My 
guess is that you wouldn't have seen the forest for the trees.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web