Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #4133 > unrolled thread

unpickling derived LogRecord in python 2.7 from python2.6

Back to article view | Back to comp.lang.python

Contents

  unpickling derived LogRecord in python 2.7 from python2.6 "ivdneut@gmail.com" <ivdneut@gmail.com> - 2011-04-27 06:56 -0700
    Re: unpickling derived LogRecord in python 2.7 from python2.6 Peter Otten <__peter__@web.de> - 2011-04-27 18:41 +0200
      Re: unpickling derived LogRecord in python 2.7 from python2.6 Vinay Sajip <vinay_sajip@yahoo.co.uk> - 2011-04-27 13:39 -0700
        Re: unpickling derived LogRecord in python 2.7 from python2.6 Peter Otten <__peter__@web.de> - 2011-04-28 09:22 +0200
          Re: unpickling derived LogRecord in python 2.7 from python2.6 "ivdneut@gmail.com" <ivdneut@gmail.com> - 2011-04-29 01:36 -0700

#4133 — unpickling derived LogRecord in python 2.7 from python2.6

Hello all,

I have a service that runs in python 2.6.4. This service sends
LogRecords to a log monitoring app on my workstation running python
2.7. The LogRecord class is derived:

class LogRecord(logging.LogRecord):

    def __init__(self, name, level, fn, lno, user, hostname, msg,
args, exc_info, func=None):

        if sys.version_info[1] > 4:
            logging.LogRecord.__init__(self, name, level, fn, lno,
msg, args, exc_info, func)
        else:
            logging.LogRecord.__init__(self, name, level, fn, lno,
msg, args, exc_info)

Now when I try to unpickle it:

record = cPickle.loads(data)

I get a TypeError exception:

TypeError: ('__init__() takes at least 8 arguments (1 given)', <class
'...gRecord'>, ())

I've searched the web and this group, but most results are old. It
worked when my workstation still ran python 2.6.

Thank you,

Ian.

[toc] | [next] | [standalone]

#4140

ivdneut@gmail.com wrote:

> I have a service that runs in python 2.6.4. This service sends
> LogRecords to a log monitoring app on my workstation running python
> 2.7. The LogRecord class is derived:
> 
> class LogRecord(logging.LogRecord):
> 
>     def __init__(self, name, level, fn, lno, user, hostname, msg,
> args, exc_info, func=None):
> 
>         if sys.version_info[1] > 4:
>             logging.LogRecord.__init__(self, name, level, fn, lno,
> msg, args, exc_info, func)
>         else:
>             logging.LogRecord.__init__(self, name, level, fn, lno,
> msg, args, exc_info)
> 
> Now when I try to unpickle it:
> 
> record = cPickle.loads(data)
> 
> I get a TypeError exception:
> 
> TypeError: ('__init__() takes at least 8 arguments (1 given)', <class
> '...gRecord'>, ())
> 
> I've searched the web and this group, but most results are old. It
> worked when my workstation still ran python 2.6.

The Problem is that as of Python 2.7 logging.LogRecord has become a newstyle 
class which is pickled/unpickled differently. I don't know if there is an 
official way to do the conversion, but here's what I've hacked up. 
The script can read pickles written with 2.6 in 2.7, but not the other way 
round.

$ cat pickle_logrec.py
import sys
import pickle
import logging

class LogRecord(logging.LogRecord):

    def __init__(self, name, level, fn, lno, user, hostname, msg, args, 
exc_info, func=None):

        if sys.version_info[1] > 4:
            logging.LogRecord.__init__(self, name, level, fn, lno, msg, 
args, exc_info, func)
        else:
            logging.LogRecord.__init__(self, name, level, fn, lno, msg, 
args, exc_info)

def makeLogRecord():
    return LogRecord(*[None]*9)

if issubclass(LogRecord, object):
    print "LogRecord is a newstyle class"
    class MyUnpickler(pickle.Unpickler):
        def find_class(self, *args):
            klass = pickle.Unpickler.find_class(self, *args)
            if klass is LogRecord:
                return makeLogRecord
            return klass
else:
    print "LogRecord is an oldstyle class"
    MyUnpickler = pickle.Unpickler

if __name__ == "__main__":
    if "--load" in sys.argv:
        print "loading"
        with open("tmp.pickle") as f:
            restored = MyUnpickler(f).load()
            print restored
    else:
        print "dumping"
        with open("tmp.pickle", "w") as f:
            f.write(pickle.dumps(LogRecord("yadda", *[None]*8)))


$ python2.6 pickle_logrec.py
LogRecord is an oldstyle class
dumping
$ python2.6 pickle_logrec.py --load
LogRecord is an oldstyle class
loading
<LogRecord: yadda, None, None, None, "None">
$ python2.7 pickle_logrec.py --load
LogRecord is a newstyle class
loading
<LogRecord: yadda, None, None, None, "None">

No warranty, use at your own risk.

[toc] | [prev] | [next] | [standalone]

#4162

On Apr 27, 5:41 pm, Peter Otten <__pete...@web.de> wrote:

> The Problem is that as of Python 2.7logging.LogRecord has become a newstyle
> class which is pickled/unpickled differently. I don't know if there is an
> official way to do the conversion, but here's what I've hacked up.
> The script can read pickles written with 2.6 in 2.7, but not the other way
> round.
> [code snipped]

I don't know about "official", but another way of doing this is to
pickle just the LogRecord's __dict__ and send that over the wire. The
logging package contains a function makeLogRecord(d) where d is a
dict.

This is the approach used by the examples in the library documentation
which pickle events for sending across a network:

http://docs.python.org/howto/logging-cookbook.html#sending-and-receiving-logging-events-across-a-network

The built-in SocketHandler pickles the LogRecord's __dict__ rather
than the LogRecord itself, precisely because of the improved
interoperability over pickling the instance directly.

Regards,

Vinay Sajip

[toc] | [prev] | [next] | [standalone]

#4211

Vinay Sajip wrote:

> On Apr 27, 5:41 pm, Peter Otten <__pete...@web.de> wrote:
> 
>> The Problem is that as of Python 2.7logging.LogRecord has become a
>> newstyle class which is pickled/unpickled differently. I don't know if
>> there is an official way to do the conversion, but here's what I've
>> hacked up. The script can read pickles written with 2.6 in 2.7, but not
>> the other way round.
>> [code snipped]
> 
> I don't know about "official", but another way of doing this is to
> pickle just the LogRecord's __dict__ and send that over the wire. The
> logging package contains a function makeLogRecord(d) where d is a
> dict.

You are right, my approach is too complicated and only needed when the OP 
cannot modify the sending script -- which is unlikely.

> This is the approach used by the examples in the library documentation
> which pickle events for sending across a network:
> 
> http://docs.python.org/howto/logging-cookbook.html#sending-and-receiving-
logging-events-across-a-network
> 
> The built-in SocketHandler pickles the LogRecord's __dict__ rather
> than the LogRecord itself, precisely because of the improved
> interoperability over pickling the instance directly.

As a minimal change ensuring that the logging.LogRecord subclass used by the 
OP is a newstyle class in 2.6 with

class LogRecord(logging.LogRecord, object):
    #...

should work, too.

[toc] | [prev] | [next] | [standalone]

#4300

On Apr 28, 9:22 am, Peter Otten <__pete...@web.de> wrote:
> Vinay Sajip wrote:
> > On Apr 27, 5:41 pm, Peter Otten <__pete...@web.de> wrote:
>
> >> The Problem is that as of Python 2.7logging.LogRecord has become a
> >> newstyle class which is pickled/unpickled differently. I don't know if
> >> there is an official way to do the conversion, but here's what I've
> >> hacked up. The script can read pickles written with 2.6 in 2.7, but not
> >> the other way round.
> >> [code snipped]
>
> > I don't know about "official", but another way of doing this is to
> > pickle just the LogRecord's __dict__ and send that over the wire. The
> > logging package contains a function makeLogRecord(d) where d is a
> > dict.
>
> You are right, my approach is too complicated and only needed when the OP
> cannot modify the sending script -- which is unlikely.
>
> > This is the approach used by the examples in the library documentation
> > which pickle events for sending across a network:
>
> >http://docs.python.org/howto/logging-cookbook.html#sending-and-receiv...
>
> logging-events-across-a-network
>
>
>
> > The built-in SocketHandler pickles the LogRecord's __dict__ rather
> > than the LogRecord itself, precisely because of the improved
> > interoperability over pickling the instance directly.
>
> As a minimal change ensuring that the logging.LogRecord subclass used by the
> OP is a newstyle class in 2.6 with
>
> class LogRecord(logging.LogRecord, object):
>     #...
>
> should work, too.

I tried this, but it didn't work. Pickling the __dict__ and then use
makeLogRecord does the trick.

Thank you very much for the excellent help,

Ian.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web