Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #92381

Re: enhancement request: make py3 read/write py2 pickle format

Show key headers only | View raw

Passing around data that can be put into ast.literal_eval is
synonymous with passing around data taht can be put into eval. It
sounds like a trap.

Other points against JSON / etc.: the lack of schema makes it easier
to stuff anything in there (not as easily as pickle, mind), and by
returning a plain dict, it becomes easier to require a field than to
allow a field to be missing, which is bad for robustness and bad for
data format migrations. (Protobuf (v3) has schemas and gives every
field a default value.)

For human readable serialized data, text format protocol buffers are
seriously underrated. (Relatedly: underdocumented, too.)

/me lifts head out of kool-aid and gasps for air

-- Devin

On Tue, Jun 9, 2015 at 5:17 PM, Irmen de Jong <irmen.NOSPAM@xs4all.nl> wrote:
> On 10-6-2015 1:06, Chris Angelico wrote:
>> On Wed, Jun 10, 2015 at 6:07 AM, Devin Jeanpierre
>> <jeanpierreda@gmail.com> wrote:
>>> There's a lot of subtle issues with pickle compatibility. e.g.
>>> old-style vs new-style classes. It's kinda hard and it's better to
>>> give up. I definitely agree it's better to use something else instead.
>>> For example, we switched to using protocol buffers, which have much
>>> better compatibility properties and are a bit more testable to boot
>>> (since text format protobufs are always output in a canonical (sorted)
>>> form.)
>>
>> Or use JSON, if your data fits within that structure. It's easy to
>> read and write, it's human-readable, and it's safe (no chance of
>> arbitrary code execution). Forcing yourself to use a format that can
>> basically be processed by ast.literal_eval() is a good discipline -
>> means you don't accidentally save/load too much.
>>
>> ChrisA
>>
>
> I made a specialized serializer for this, which is more expressive than JSON. It outputs
> python literal expressions that can be directly parsed by ast.literal_eval(). You can
> find it on pypi (https://pypi.python.org/pypi/serpent).  It's the default serializer of
> Pyro, and it includes a Java and .NET version as well as an added bonus.
>
>
> Irmen
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 09:06 +1000
  Re: enhancement request: make py3 read/write py2 pickle format Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2015-06-10 02:17 +0200
    Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-09 17:47 -0700
      Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 13:52 +1000
        Re: enhancement request: make py3 read/write py2 pickle format random832@fastmail.us - 2015-06-09 23:57 -0400
          Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 15:43 +1000
        Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 14:00 +1000
        Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-09 21:48 -0700
          Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-10 19:36 +1000
            Re: enhancement request: make py3 read/write py2 pickle format Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2015-06-10 19:34 +0200
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 15:10 -0700
              Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve@pearwood.info> - 2015-06-11 13:21 +1000
                Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 22:39 -0700
                Re: enhancement request: make py3 read/write py2 pickle format Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-11 19:20 +1000
            Re: enhancement request: make py3 read/write py2 pickle format Terry Reedy <tjreedy@udel.edu> - 2015-06-10 19:25 -0400
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 16:39 -0700
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 16:48 -0700
            Re: enhancement request: make py3 read/write py2 pickle format Terry Reedy <tjreedy@udel.edu> - 2015-06-10 19:46 -0400
            Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-11 09:58 +1000
            Re: enhancement request: make py3 read/write py2 pickle format Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-10 17:02 -0700
              Re: enhancement request: make py3 read/write py2 pickle format Marko Rauhamaa <marko@pacujo.net> - 2015-06-11 07:08 +0300
            Re: enhancement request: make py3 read/write py2 pickle format Serhiy Storchaka <storchaka@gmail.com> - 2015-06-11 14:11 +0300
    Re: enhancement request: make py3 read/write py2 pickle format Chris Angelico <rosuav@gmail.com> - 2015-06-10 11:03 +1000

csiph-web