Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #92373 > unrolled thread

Re: enhancement request: make py3 read/write py2 pickle format

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 09:06 +1000
    Re: enhancement request: make py3 read/write py2 pickle format Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2015-06-10 02:17 +0200
      Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-09 17:47 -0700
        Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 13:52 +1000
          Re: enhancement request: make py3 read/write py2 pickle format random832@fastmail.us - 2015-06-09 23:57 -0400
            Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 15:43 +1000
          Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 14:00 +1000
          Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-09 21:48 -0700
            Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 19:36 +1000
              Re: enhancement request: make py3 read/write py2 pickle format Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2015-06-10 19:34 +0200
              Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 15:10 -0700
                Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve@pearwood.info> - 2015-06-11 13:21 +1000
                  Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 22:39 -0700
                    Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-11 19:20 +1000
              Re: enhancement request: make py3 read/write py2 pickle format Terry Reedy <tjreedy@udel.edu> - 2015-06-10 19:25 -0400
              Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 16:39 -0700
              Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 16:48 -0700
              Re: enhancement request: make py3 read/write py2 pickle format Terry Reedy <tjreedy@udel.edu> - 2015-06-10 19:46 -0400
              Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-11 09:58 +1000
              Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 17:02 -0700
                Re: enhancement request: make py3 read/write py2 pickle format Marko Rauhamaa <marko@pacujo.net> - 2015-06-11 07:08 +0300
              Re: enhancement request: make py3 read/write py2 pickle format Serhiy Storchaka <storchaka@gmail.com> - 2015-06-11 14:11 +0300
      Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 11:03 +1000

Page 2 of 2 — ← Prev page 1 [2]

#92453

Devin Jeanpierre <jeanpierreda@gmail.com>:

> For example, one can also imagine testing that a serialized structure
> is identical across version changes, so that it's guaranteed to be
> forwards/backwards compatible. It is not enough to test that the
> deserialized form is, because it might differ substantially, as long
> as the communicated serialized structure is the same.

There are merits to canonical serialization formats, but that approach
to testing is far too simplistic. A test case should accept all observed
behavior that is correct.


Marko

[toc] | [prev] | [next] | [standalone]

#92463

On 11.06.15 02:58, Chris Angelico wrote:
> On Thu, Jun 11, 2015 at 8:10 AM, Devin Jeanpierre
> <jeanpierreda@gmail.com> wrote:
>> The problem is that there are two different ways repr might write out
>> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle -- e.g.
>> it's why doctest fails badly at examples involving dictionaries. Text
>> format protocol buffers output everything sorted, so that you can do
>> textual diffs for compatibility tests and such.
>
> With Python's JSON module [1], you can pass sort_keys=True to
> stipulate that the keys be lexically ordered, which should make the
> output "canonical". Pike's Standards.JSON.encode() [2] can take a flag
> value to canonicalize the output, which currently has the same effect
> (sort mappings by their indices). I did a quick check for Ruby and
> didn't find anything in its standard library JSON module, but knowing
> Ruby, it'll be available somewhere in a gem.

AFAIK Ruby's dicts are ordered. So the output is pretty stable.

[toc] | [prev] | [next] | [standalone]

#92382

On Wed, Jun 10, 2015 at 10:47 AM, Devin Jeanpierre
<jeanpierreda@gmail.com> wrote:
> Passing around data that can be put into ast.literal_eval is
> synonymous with passing around data taht can be put into eval. It
> sounds like a trap.

Except that it's hugely restricted. There are JSON parsing libraries
all over the place, and one of the points in favour of JSON is that it
can be dumped into JavaScript as-is and it evaluates correctly.
Doesn't mean you have to use eval() to parse it, and usually you
won't.

> Other points against JSON / etc.: the lack of schema makes it easier
> to stuff anything in there (not as easily as pickle, mind), and by
> returning a plain dict, it becomes easier to require a field than to
> allow a field to be missing, which is bad for robustness and bad for
> data format migrations. (Protobuf (v3) has schemas and gives every
> field a default value.)

This is true. But there are plenty of cases where you can manage on a
simple dictionary that maps keywords to values that are either
strings, numbers, or lists/dicts of same - and without any schema to
tell you what keywords are valid. It's simple, extensible, and scales
reasonably well to mid-sized use cases. Sure, the biggest cases (and
some of the smaller ones) benefit nicely from schema definitions, but
there's room to manage without.

ChrisA

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web