Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail From: eryk sun Newsgroups: comp.lang.python Subject: Re: Interacting with Subprocesses Date: Thu, 5 May 2016 04:38:47 -0500 Lines: 36 Message-ID: References: <871t5h4jst.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Trace: news.uni-berlin.de KBjcUUS3QJM7IJlQGHs+twwDrRMlarmXk4d6hZQ2PYBA== Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.001 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'modify': 0.04; 'source,': 0.04; 'variable,': 0.07; 'api': 0.09; '[1]:': 0.09; '[2]:': 0.09; '[3]:': 0.09; 'input,': 0.09; 'python': 0.10; 'output': 0.13; 'wed,': 0.15; 'variables': 0.15; '2016': 0.16; 'buffering': 0.16; 'command,': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'url:faq': 0.16; 'url:gnu': 0.16; 'wrote:': 0.16; 'linux,': 0.18; 'hack': 0.18; 'variable': 0.18; 'load': 0.20; 'windows': 0.20; 'pass': 0.22; "python's": 0.23; 'sets': 0.23; 'unlike': 0.23; 'header:In-Reply-To:1': 0.24; "doesn't": 0.26; 'linux': 0.26; 'message-id:@mail.gmail.com': 0.27; '[2]': 0.27; 'dll': 0.27; 'url:article': 0.27; 'command-line': 0.29; 'i/o': 0.29; 'url:software': 0.29; 'windows,': 0.29; 'environment': 0.29; 'objects': 0.29; 'program,': 0.29; 'creating': 0.30; 'option': 0.31; "can't": 0.32; 'skip:_ 10': 0.32; '[1]': 0.32; 'common': 0.33; 'similar': 0.33; 'file': 0.34; 'received:google.com': 0.35; 'could': 0.35; 'something': 0.35; 'there': 0.36; 'url:org': 0.36; 'created': 0.36; 'received:209.85': 0.36; 'child': 0.36; 'to:addr :python-list': 0.36; 'pm,': 0.36; 'subject:: ': 0.37; 'received:209.85.213': 0.37; 'received:209': 0.38; 'skip:o 20': 0.38; 'url:en': 0.39; 'to:addr:python.org': 0.40; 'subject:with': 0.40; 'some': 0.40; 'behavior': 0.61; 'skip:u 10': 0.61; 'url:c': 0.76; 'child,': 0.84; 'skip:/ 30': 0.84; 'streams': 0.84; 'url:readthedocs': 0.84 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=O+d+EpdnIYApzmmWehepJyNYB9TgknbeD64Aoy09n4I=; b=TLqAyvcaaw2Yqajc+m6BpW/CUsgA19UuVZfpKy/+e5+lO1HDq8WbtthV2X7se9seWh aDDuN9wOqPbAoa/r+mWmoHx0oh1xthsNOAHMyPwooMxp1OfENlSrRdjzl7soiSpbDr+9 a0NNo/oP+GyuyTY4PAbC0yHgGCHv3BqQhhDUuICiiggvVj7XZzjtOEU/g2GnuLVw/p40 KT11gEFBGF0gLceR1dKbjcSuUbQCkINJoCyzQqir1ATA2kTRgIWwjn3DvQcDrcAr8qCL EcIWzoORSeTK4sagxdZ5X4OqrhZNDVYyWPsPH6tQu4glvBXyRjEMZcU+i19WqVRiCukq xi2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=O+d+EpdnIYApzmmWehepJyNYB9TgknbeD64Aoy09n4I=; b=bwR8Y2RMx8IL8nhqbkg+UX3pieKW08BbgFWPprd9rKayPa84DPGCmIKOTtBLgRtUMj rcPMOksNei2BitMo/QefylEQsq2meqPsvZN6bltRGjPZPTeEGaHDPKu0whRhCw/dWehh UNZBM6mbHL2SvTLlKjYJ5r8wAV32JR2PrMm1sEK5WD3Sqch0f3fFKxb2i2YH5Cx8AE6Y RjSFdg/WpzEjZQwePvaeaRbOwl7lhyuJW44ldWOVYRdEnf6gz/HfJMqrG0ln1YUdPXn3 PuA2LRBFOhDDInLEo0kTNxqHFaVl+XzSOvKYoQ76NM9mnrWGSdS2I2atUV3FqIjyKVih 1h9A== X-Gm-Message-State: AOPr4FXAxXP9veW1QyPzmlVDPX76PTbtEKSthO5PfACexPLQ+HPNmEeXaGwMGDnHBgIfmXB+A+su8THYvT8+EA== X-Received: by 10.50.29.73 with SMTP id i9mr2635715igh.32.1462441167068; Thu, 05 May 2016 02:39:27 -0700 (PDT) In-Reply-To: <871t5h4jst.fsf@gmail.com> X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Mailman-Original-Message-ID: X-Mailman-Original-References: <871t5h4jst.fsf@gmail.com> Xref: csiph.com comp.lang.python:108171 On Wed, May 4, 2016 at 4:04 PM, Akira Li <4kir4.1i@gmail.com> wrote: > > Pass stdin=PIPE, stdout=PIPE and use p.stdin, p.stdout file objects to > write input, read output from the child process. > > Beware, there could be buffering issues or the child process may change > its behavior some other way when the standard input/output streams are > redirected. See > http://pexpect.readthedocs.io/en/stable/FAQ.html#whynotpipe On Linux, you may be able to use stdbuf [1] to modify standard I/O buffering. stdbuf sets the LD_PRELOAD [2] environment variable to load libstdbuf.so [3]. For example, the following shows the environment variables created by "stdbuf -oL": $ stdbuf -oL python -c 'import os;print os.environ["LD_PRELOAD"]' /usr/lib/coreutils/libstdbuf.so $ stdbuf -oL python -c 'import os;print os.environ["_STDBUF_O"]' L [1]: http://www.gnu.org/software/coreutils/manual/html_node/stdbuf-invocation.html [2]: http://www.linuxjournal.com/article/7795 [3]: http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/libstdbuf.c?id=v8.21 On Windows, if you can modify the program, then you can check for a command-line option or an environment variable, like Python's -u and PYTHONUNBUFFERED. If you can't modify the source, I think you might be able to hack something similar to the Linux LD_PRELOAD environment variable by creating a stdbuf.exe launcher that debugs the process and injects a DLL after the loader's first-chance breakpoint. The injected stdbuff.dll would need to be able to get the standard streams for common CRTs, such as by calling __acrt_iob_func for ucrtbase.dll. Also, unlike the Linux command, stdbuf.exe would have to wait on the child, since the Windows API doesn't have fork/exec.