Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #92394

Re: enhancement request: make py3 read/write py2 pickle format

Show all headers | View raw

On 2015-06-10 12:04, Neal Becker wrote:
> Chris Warrick wrote:
>
>> On Tue, Jun 9, 2015 at 8:08 PM, Neal Becker <ndbecker2@gmail.com> wrote:
>>> One of the most annoying problems with py2/3 interoperability is that the
>>> pickle formats are not compatible.  There must be many who, like myself,
>>> often use pickle format for data storage.
>>>
>>> It certainly would be a big help if py3 could read/write py2 pickle
>>> format. You know, backward compatibility?
>>
>> Don’t use pickle. It’s unsafe — it executes arbitrary code, which
>> means someone can give you a pickle file that will delete all your
>> files or eat your cat.
>>
>> Instead, use a safe format that has no ability to execute code, like
>> JSON. It will also work with other programming languages and
>> environments if you ever need to talk to anyone else.
>>
>> But, FYI: there is backwards compatibility if you ask for it, in the
>> form of protocol versions. That’s all you should know — again, don’t
>> use pickle.
>
> I believe a good native serialization system is essential for any modern
> programming language.  If pickle isn't it, we need something else that can
> serialize all language objects.  Or, are you saying, it's impossible to do
> this safely?

By the very nature of the stated problem: serializing all language objects. 
Being able to construct any object, including instances of arbitrary classes, 
means that arbitrary code can be executed. All I have to do is make a pickle 
file for an object that claims that its constructor is shutil.rmtree().

This is fine in some use cases (e.g. wire format for otherwise-secured 
communication between two endpoints under your complete control), but it is 
worrying in others, like your use case of data storage (and presumably sharing).

Python 2/3 is also the least of your compatibility worries there. Refactor a 
class to a different module, or did one of your third-party dependencies do 
this? Poof! Your pickle files no longer work.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: enhancement request: make py3 read/write py2 pickle format Robert Kern <robert.kern@gmail.com> - 2015-06-10 12:22 +0100
  Re: enhancement request: make py3 read/write py2 pickle format Marko Rauhamaa <marko@pacujo.net> - 2015-06-10 15:08 +0300
    Re: enhancement request: make py3 read/write py2 pickle format random832@fastmail.us - 2015-06-10 09:38 -0400
    Re: enhancement request: make py3 read/write py2 pickle format Robert Kern <robert.kern@gmail.com> - 2015-06-10 14:52 +0100
      Re: enhancement request: make py3 read/write py2 pickle format Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-06-11 11:30 +1200
        Re: enhancement request: make py3 read/write py2 pickle format random832@fastmail.us - 2015-06-10 20:47 -0400

csiph-web