Re: enhancement request: make py3 read/write py2 pickle format

References	(5 earlier) <mailman.328.1433897321.13271.python-list@python.org> <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com> <mailman.332.1433911768.13271.python-list@python.org> <5578053a$0$11102$c3e8da3@news.astraweb.com> <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
Date	2015-06-11 09:58 +1000
Subject	Re: enhancement request: make py3 read/write py2 pickle format
From	Chris Angelico <rosuav@gmail.com>
Newsgroups	comp.lang.python
Message-ID	<mailman.368.1433980692.13271.python-list@python.org> (permalink)

Show all headers | View raw

On Thu, Jun 11, 2015 at 8:10 AM, Devin Jeanpierre
<jeanpierreda@gmail.com> wrote:
> The problem is that there are two different ways repr might write out
> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle -- e.g.
> it's why doctest fails badly at examples involving dictionaries. Text
> format protocol buffers output everything sorted, so that you can do
> textual diffs for compatibility tests and such.

With Python's JSON module [1], you can pass sort_keys=True to
stipulate that the keys be lexically ordered, which should make the
output "canonical". Pike's Standards.JSON.encode() [2] can take a flag
value to canonicalize the output, which currently has the same effect
(sort mappings by their indices). I did a quick check for Ruby and
didn't find anything in its standard library JSON module, but knowing
Ruby, it'll be available somewhere in a gem. A web search for 'perl
json' brought up a CPAN link [4] that has a canonicalize option for
sorting by keys. So that's three out of four definite, one uncertain,
where it's pretty easy to ensure that you get byte-for-byte identical
output from a JSON encoder.

Even though failing doctests are a separate problem, it's useful to
have canonical output. Your diffs get less noisy, for instance.
Coupled with a human-readability flag (eg "indent=4" in Python,
"Standards.JSON.HUMAN_READABLE" in Pike) that splits the result over
multiple lines, it can make a pretty easy to diff file. Definitely
worth doing... and definitely worth using a JSON encoder rather than
repr().

ChrisA

[1] https://docs.python.org/3/library/json.html#json.dump
[2] http://pike.lysator.liu.se/generated/manual/modref/ex/predef_3A_3A/Standards/JSON.html
[3] http://ruby-doc.org/stdlib-2.0.0/libdoc/json/rdoc/JSON.html
[4] http://search.cpan.org/~makamaka/JSON-2.90/lib/JSON.pm#PERL_-%3E_JSON

Thread

Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 09:06 +1000
  Re: enhancement request: make py3 read/write py2 pickle format Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2015-06-10 02:17 +0200
    Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-09 17:47 -0700
      Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 13:52 +1000
        Re: enhancement request: make py3 read/write py2 pickle format random832@fastmail.us - 2015-06-09 23:57 -0400
          Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 15:43 +1000
        Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 14:00 +1000
        Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-09 21:48 -0700
          Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 19:36 +1000
            Re: enhancement request: make py3 read/write py2 pickle format Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2015-06-10 19:34 +0200
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 15:10 -0700
              Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve@pearwood.info> - 2015-06-11 13:21 +1000
                Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 22:39 -0700
                Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-11 19:20 +1000
            Re: enhancement request: make py3 read/write py2 pickle format Terry Reedy <tjreedy@udel.edu> - 2015-06-10 19:25 -0400
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 16:39 -0700
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 16:48 -0700
            Re: enhancement request: make py3 read/write py2 pickle format Terry Reedy <tjreedy@udel.edu> - 2015-06-10 19:46 -0400
            Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-11 09:58 +1000
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 17:02 -0700
              Re: enhancement request: make py3 read/write py2 pickle format Marko Rauhamaa <marko@pacujo.net> - 2015-06-11 07:08 +0300
            Re: enhancement request: make py3 read/write py2 pickle format Serhiy Storchaka <storchaka@gmail.com> - 2015-06-11 14:11 +0300
    Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 11:03 +1000

csiph-web