Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #108146 > unrolled thread

Interacting with Subprocesses

Back to article view | Back to comp.lang.python

Contents

  Interacting with Subprocesses Dick Holmes <encore1@cox.net> - 2016-05-04 11:41 -0700
    Re: Interacting with Subprocesses Akira Li <4kir4.1i@gmail.com> - 2016-05-05 00:04 +0300
    Re: Interacting with Subprocesses Terry Reedy <tjreedy@udel.edu> - 2016-05-04 20:33 -0400
    Re: Interacting with Subprocesses Akira Li <4kir4.1i@gmail.com> - 2016-05-05 04:05 +0300
    Re: Interacting with Subprocesses eryk sun <eryksun@gmail.com> - 2016-05-05 04:38 -0500

#108146 — Interacting with Subprocesses

I am attempting to write a Python program that will interact with
a (non-Python) process. The programs will run under MinGW. The
process can use stdin/stdout commands and responses and can work 
with pipes. The problem I'm having is that I can't find any
way in Python to have a continuing dialog with the process. I
have tried Popen communicate, but that protocol seems to be
limited to a single message/response pair, and the response
is not returned to the message originator until the process
terminates. Unfortunately I don't have access to the process'
source code so I can't change the communication medium.

Is there some feature that will allow me to initiate the process
and execute multiple message/response pairs between the Python
program and the process during a single execution of the process?

Thanks!

Dick

[toc] | [next] | [standalone]

#108150

Dick Holmes <encore1@cox.net> writes:

> I am attempting to write a Python program that will interact with
> a (non-Python) process. The programs will run under MinGW. The
> process can use stdin/stdout commands and responses and can work 
> with pipes. The problem I'm having is that I can't find any
> way in Python to have a continuing dialog with the process. I
> have tried Popen communicate, but that protocol seems to be
> limited to a single message/response pair, and the response
> is not returned to the message originator until the process
> terminates. Unfortunately I don't have access to the process'
> source code so I can't change the communication medium.
>
> Is there some feature that will allow me to initiate the process
> and execute multiple message/response pairs between the Python
> program and the process during a single execution of the process?
>

Pass stdin=PIPE, stdout=PIPE and use p.stdin, p.stdout file objects to
write input, read output from the child process.

Beware, there could be buffering issues or the child process may change
its behavior some other way when the standard input/output streams are
redirected. See
http://pexpect.readthedocs.io/en/stable/FAQ.html#whynotpipe

btw, If pexpect module works in MingGW environment (if pty is
available); you could try it to communicate with the process
interactively.

You might also find the list of Stackoverflow question related to the
subprocess module useful http://stackoverflow.com/tags/subprocess/info


Akira

[toc] | [prev] | [next] | [standalone]

#108154

On 5/4/2016 2:41 PM, Dick Holmes wrote:
> I am attempting to write a Python program that will interact with
> a (non-Python) process. The programs will run under MinGW. The
> process can use stdin/stdout commands and responses and can work
> with pipes. The problem I'm having is that I can't find any
> way in Python to have a continuing dialog with the process. I
> have tried Popen communicate, but that protocol seems to be
> limited to a single message/response pair, and the response
> is not returned to the message originator until the process
> terminates. Unfortunately I don't have access to the process'
> source code so I can't change the communication medium.
>
> Is there some feature that will allow me to initiate the process
> and execute multiple message/response pairs between the Python
> program and the process during a single execution of the process?

I have been told that multiprocessing works better for this.  Not sure 
is true.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#108155

Terry Reedy <tjreedy@udel.edu> writes:

> On 5/4/2016 2:41 PM, Dick Holmes wrote:
>> I am attempting to write a Python program that will interact with
>> a (non-Python) process. The programs will run under MinGW. The
>> process can use stdin/stdout commands and responses and can work
>> with pipes. The problem I'm having is that I can't find any
>> way in Python to have a continuing dialog with the process. I
>> have tried Popen communicate, but that protocol seems to be
>> limited to a single message/response pair, and the response
>> is not returned to the message originator until the process
>> terminates. Unfortunately I don't have access to the process'
>> source code so I can't change the communication medium.
>>
>> Is there some feature that will allow me to initiate the process
>> and execute multiple message/response pairs between the Python
>> program and the process during a single execution of the process?
>
> I have been told that multiprocessing works better for this.  Not sure
> is true.

OP wants to interact with a *non-Python* process—*multiprocessing*
module won't help here.

[toc] | [prev] | [next] | [standalone]

#108171

On Wed, May 4, 2016 at 4:04 PM, Akira Li <4kir4.1i@gmail.com> wrote:
>
> Pass stdin=PIPE, stdout=PIPE and use p.stdin, p.stdout file objects to
> write input, read output from the child process.
>
> Beware, there could be buffering issues or the child process may change
> its behavior some other way when the standard input/output streams are
> redirected. See
> http://pexpect.readthedocs.io/en/stable/FAQ.html#whynotpipe

On Linux, you may be able to use stdbuf [1] to modify standard I/O
buffering. stdbuf sets the LD_PRELOAD [2] environment variable to load
libstdbuf.so [3]. For example, the following shows the environment
variables created by "stdbuf -oL":

    $ stdbuf -oL python -c 'import os;print os.environ["LD_PRELOAD"]'
    /usr/lib/coreutils/libstdbuf.so
    $ stdbuf -oL python -c 'import os;print os.environ["_STDBUF_O"]'
    L

[1]: http://www.gnu.org/software/coreutils/manual/html_node/stdbuf-invocation.html
[2]: http://www.linuxjournal.com/article/7795
[3]: http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/libstdbuf.c?id=v8.21

On Windows, if you can modify the program, then you can check for a
command-line option or an environment variable, like Python's -u and
PYTHONUNBUFFERED.

If you can't modify the source, I think you might be able to hack
something similar to the Linux LD_PRELOAD environment variable by
creating a stdbuf.exe launcher that debugs the process and injects a
DLL after the loader's first-chance breakpoint. The injected
stdbuff.dll would need to be able to get the standard streams for
common CRTs, such as by calling __acrt_iob_func for ucrtbase.dll.
Also, unlike the Linux command, stdbuf.exe would have to wait on the
child, since the Windows API doesn't have fork/exec.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web