Re: enhancement request: make py3 read/write py2 pickle format

Path	csiph.com!usenet.pasdenom.info!news.redatomik.org!newsfeed.xs4all.nl!newsfeed3a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path	<jeanpierreda@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.000
X-Spam-Evidence	'H': 1.00; 'S': 0.00; 'diff': 0.05; 'ignored': 0.05; 'json': 0.07; 'dict': 0.09; 'incorrect': 0.09; 'objects.': 0.09; 'repr': 0.09; 'separately': 0.09; 'unexpected': 0.09; 'cc:addr:python-list': 0.10; 'read.': 0.13; 'wed,': 0.15; 'output': 0.15; 'weird': 0.15; '4:25': 0.16; '4:39': 0.16; 'compares': 0.16; 'encoders': 0.16; 'json,': 0.16; 'principled': 0.16; 'reedy': 0.16; 'subject:make': 0.16; 'two,': 0.16; "{'a':": 0.16; 'wrote:': 0.16; 'string': 0.17; 'comparing': 0.18; '(in': 0.18; 'examples': 0.18; 'tests': 0.18; '>>>': 0.20; 'compare': 0.20; 'cc:2**0': 0.21; 'cc:addr:python.org': 0.21; 'class,': 0.22; 'produces': 0.22; 'subject:request': 0.22; 'text,': 0.22; '2015': 0.23; '(like': 0.23; 'bigger': 0.23; 'header:In-Reply-To:1': 0.24; 'written': 0.24; 'sort': 0.25; 'moved': 0.27; 'message- id:@mail.gmail.com': 0.28; 'subject:/': 0.29; 'buffers': 0.29; 'clever': 0.29; 'inspect': 0.29; 'terry': 0.29; 'up:': 0.29; 'objects': 0.29; 'code': 0.31; "can't": 0.32; 'probably': 0.32; 'structure': 0.32; 'though,': 0.32; 'problem': 0.33; 'everyone': 0.34; 'equal': 0.34; 'received:google.com': 0.34; 'useful': 0.35; 'formats': 0.35; 'protocol': 0.35; 'strongly': 0.35; 'but': 0.36; 'being': 0.36; 'text': 0.36; 'except': 0.36; 'possible.': 0.36; 'there': 0.36; 'data.': 0.36; 'loaded': 0.36; 'two': 0.37; 'should': 0.37; 'subject:: ': 0.37; 'difference': 0.38; 'rather': 0.38; 'goes': 0.39; 'pm,': 0.39; 'test': 0.39; 'easily': 0.39; 'data': 0.40; 'where': 0.40; 'some': 0.40; "you've": 0.61; 'results.': 0.63; 'different': 0.64; 'other.': 0.64; 'between': 0.65; 'subject:read': 0.84; 'subject:write': 0.84; 'overall,': 0.93; 'regard,': 0.93
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=/zoTghtP620640K5t5AvWW8TITFHdHKvzGh1Parc6Lk=; b=n0Wd8zZe6CEuDtpAsddOA1YbzK2t2co9u4LR2a+SPR0CqZ1F/ksn3yiqx2jqhb9vYS XfpQIclJcYZm4xuyoiljIZQ2GoE/z6nCBQGvmb3355klmW6NQHaGaIPPctXF+eRhhNgl AQWkQkt74F4u2NNM5k5QVqNeztj+RbasiAZiQd4Ru30FVOwtVwsR4XLjwxaz2wi5/B81 4dcmXSkiE8LG81xTKfKk4MsfRJeOYpnKGdoE4XLy7RL+8pXTy4szyPdfScekS/ZBvXP0 /WSF2rYXGZRZy6i/t2tquC0Wbah2HZ6Y2/iVyh/To4E1+RdFIJk3dl0mIA2oH+1e2jaS eipg==
X-Received	by 10.180.100.194 with SMTP id fa2mr23947108wib.8.1433980127598; Wed, 10 Jun 2015 16:48:47 -0700 (PDT)
MIME-Version	1.0
In-Reply-To	<CABicbJL5EqxTasVMb8yALSSss5xexf=9_7i-BhC2t5a-TxRCFA@mail.gmail.com>
References	<ml7a2p$hp$1@ger.gmane.org> <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com> <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com> <mailman.325.1433891190.13271.python-list@python.org> <55778208$0$2899$e4fe514c@news2.news.xs4all.nl> <mailman.328.1433897321.13271.python-list@python.org> <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com> <mailman.332.1433911768.13271.python-list@python.org> <5578053a$0$11102$c3e8da3@news.astraweb.com> <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com> <mlah2i$f7$1@ger.gmane.org> <CABicbJL5EqxTasVMb8yALSSss5xexf=9_7i-BhC2t5a-TxRCFA@mail.gmail.com>
From	Devin Jeanpierre <jeanpierreda@gmail.com>
Date	Wed, 10 Jun 2015 16:48:07 -0700
Subject	Re: enhancement request: make py3 read/write py2 pickle format
To	Terry Reedy <tjreedy@udel.edu>
Cc	"comp.lang.python" <python-list@python.org>
Content-Type	text/plain; charset=UTF-8
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.20+
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.366.1433980134.13271.python-list@python.org> (permalink)
Lines	35
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1433980134 news.xs4all.nl 2957 [2001:888:2000:d::a6]:48186
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:92433

Show key headers only | View raw

On Wed, Jun 10, 2015 at 4:39 PM, Devin Jeanpierre
<jeanpierreda@gmail.com> wrote:
> On Wed, Jun 10, 2015 at 4:25 PM, Terry Reedy <tjreedy@udel.edu> wrote:
>> On 6/10/2015 6:10 PM, Devin Jeanpierre wrote:
>>
>>> The problem is that there are two different ways repr might write out
>>> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle
>>
>>
>> Not if one compares objects rather than string representations of objects.
>> I am strongly of the view that code and tests should be written to directly
>> compare objects as much as possible.
>
> For serialization formats that always output the same string for the
> same data (like text format protos), there is no practical difference
> between the two, except that if you're comparing text, you can easily
> supply a diff to update one to match the other.

Ugh, there's also the fiddly difference between what goes in and what
you read. A serialized data structure might contain lots of data that
is ignored by the deserializer (in protobuf), or it might contain data
which can't be loaded by the deserializer or produces weird /
incorrect results. Being able to inspect and test the serialized data
separately from the deserialized data is useful in that regard, so
that you know where the failure lies, but it's sort of fuzzy.

Some examples of where this crops up: pickles after you've moved a
class, JSON encoders that try to be clever and output invalid JSON,
protocol buffers with unexpected fields.

Overall, though, the diff thing is probably the bigger reason everyone
wants to do this sort of thing with serialized data. If you do it
right and are principled about it, I don't see a problem with it.

-- Devin

Thread

Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 09:06 +1000
  Re: enhancement request: make py3 read/write py2 pickle format Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2015-06-10 02:17 +0200
    Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-09 17:47 -0700
      Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 13:52 +1000
        Re: enhancement request: make py3 read/write py2 pickle format random832@fastmail.us - 2015-06-09 23:57 -0400
          Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 15:43 +1000
        Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 14:00 +1000
        Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-09 21:48 -0700
          Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 19:36 +1000
            Re: enhancement request: make py3 read/write py2 pickle format Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2015-06-10 19:34 +0200
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 15:10 -0700
              Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve@pearwood.info> - 2015-06-11 13:21 +1000
                Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 22:39 -0700
                Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-11 19:20 +1000
            Re: enhancement request: make py3 read/write py2 pickle format Terry Reedy <tjreedy@udel.edu> - 2015-06-10 19:25 -0400
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 16:39 -0700
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 16:48 -0700
            Re: enhancement request: make py3 read/write py2 pickle format Terry Reedy <tjreedy@udel.edu> - 2015-06-10 19:46 -0400
            Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-11 09:58 +1000
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 17:02 -0700
              Re: enhancement request: make py3 read/write py2 pickle format Marko Rauhamaa <marko@pacujo.net> - 2015-06-11 07:08 +0300
            Re: enhancement request: make py3 read/write py2 pickle format Serhiy Storchaka <storchaka@gmail.com> - 2015-06-11 14:11 +0300
    Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 11:03 +1000

csiph-web