Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #87342

Re: Python3 "pickle" vs. stdin/stdout - unable to get clean byte streams in Python 3

Show all headers | View raw

On 3/12/2015 2:56 PM, Cameron Simpson wrote:
> On 12Mar2015 12:55, John Nagle <nagle@animats.com> wrote:
>> I have working code from Python 2 which uses "pickle" to talk to a
>> subprocess via stdin/stdio.  I'm trying to make that work in Python
>> 3. First, the subprocess Python is invoked with the "-d' option, so
>> stdin and stdio are supposed to be unbuffered binary streams.
> 
> You shouldn't need to use unbuffered streams specifically. It should
> be enough to .flush() the output stream (at whichever end) after you
> have written the pickle data.

    Doing that.

    It's a repeat-transaction thing.  Main process sends pickeled
item to subprocess, subprocess reads item, subprocess does work,
subprocess writes picked item to parent.  This repeats.

    I call writer.clear_memo() and set reader.memo = {} at the
end of each cycle, to clear Pickle's cache.  That all worked
fine in Python 2.  Are there any known problems with reusing
Python 3 "pickle"s streams?

    The identical code works with Python 2.7.9; it's converted to Python
3 using "six" so I can run on both Python versions and look for
differences.  I'm using Pickle format 2, for compatibility.
(Tried 0, the ASCII format; it didn't help.)

> I'm skipping some of your discussion; I can see nothing wrong. I
> don't use pickle itself so aside from saying that your use seems to
> conform to the python 3 docs I can't comment more deeply. That said,
> I do use subprocess a fair bit.

     I'll have to put in more logging and see exactly what's going
over the pipes.

				John Nagle

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Python3 "pickle" vs. stdin/stdout - unable to get clean byte streams in Python 3 John Nagle <nagle@animats.com> - 2015-03-12 12:55 -0700
  Re: Python3 "pickle" vs. stdin/stdout - unable to get clean byte streams in Python 3 Cameron Simpson <cs@zip.com.au> - 2015-03-13 08:56 +1100
    Re: Python3 "pickle" vs. stdin/stdout - unable to get clean byte streams in Python 3 John Nagle <nagle@animats.com> - 2015-03-12 17:18 -0700
      Re: Python3 "pickle" vs. stdin/stdout - unable to get clean byte streams in Python 3 John Nagle <nagle@animats.com> - 2015-03-12 23:05 -0700
        Re: Python3 "pickle" vs. stdin/stdout - unable to get clean byte streams in Python 3 Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-13 19:43 +1100
  Re: Python3 "pickle" vs. stdin/stdout - unable to get clean byte streams in Python 3 Peter Otten <__peter__@web.de> - 2015-03-12 22:57 +0100

csiph-web