Re: Pass data to a subprocess

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.glorb.com!solaris.cc.vt.edu!news.vt.edu!newsfeed-00.mathworks.com!panix!roy
From	Roy Smith <roy@panix.com>
Newsgroups	comp.lang.python
Subject	Re: Pass data to a subprocess
Date	Wed, 01 Aug 2012 06:59:37 -0400
Organization	PANIX Public Access Internet and UNIX, NYC
Lines	25
Message-ID	<roy-DBE17E.06593701082012@news.panix.com> (permalink)
References	<CAF_E5JY5FgEJub7tNNzZEj0tri8Z7oPZUSLOLtD+5f0t7Yv+Yg@mail.gmail.com> <CAF_E5JY2sEsauXM-teW4hTHePqFCYK6z+9oLMZcjVWFyYs+gKg@mail.gmail.com> <5017EFB0.6080608@shopzeus.com> <CAF_E5JaEHxsjKoHRhZLzh+YdPU8UqKHae9efCsP7zEvm_37+2w@mail.gmail.com> <mailman.2809.1343809166.4697.python-list@python.org>
NNTP-Posting-Host	localhost
X-Trace	reader1.panix.com 1343818780 3658 127.0.0.1 (1 Aug 2012 10:59:40 GMT)
X-Complaints-To	abuse@panix.com
NNTP-Posting-Date	Wed, 1 Aug 2012 10:59:40 +0000 (UTC)
User-Agent	MT-NewsWatcher/3.5.3b3 (Intel Mac OS X)
Xref	csiph.com comp.lang.python:26348

Show key headers only | View raw

In article <mailman.2809.1343809166.4697.python-list@python.org>,
 Laszlo Nagy <gandalf@shopzeus.com> wrote:

> Yes, I think that is correct. Instead of detaching a child process, you 
> can create independent processes and use other frameworks for IPC. For 
> example, Pyro.  It is not as effective as multiprocessing.Queue, but in 
> return, you will have the option to run your service across multiple 
> servers.

You might want to look at beanstalk (http://kr.github.com/beanstalkd/).  
We've been using it in production for the better part of two years.  At 
a 30,000 foot level, it's an implementation of queues over named pipes 
over TCP, but it takes care of a zillion little details for you.

Setup is trivial, and there's clients for all sorts of languages.  For a 
Python client, go with beanstalkc (pybeanstalk appears to be 
abandonware).
> 
> The most effective IPC is usually through shared memory. But there is no 
> OS independent standard Python module that can communicate over shared 
> memory.

It's true that shared memory is faster than serializing objects over a 
TCP connection.  On the other hand, it's hard to imagine anything 
written in Python where you would notice the difference.

Thread

Re: Pass data to a subprocess Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-01 10:19 +0200
  Re: Pass data to a subprocess Roy Smith <roy@panix.com> - 2012-08-01 06:59 -0400
    Re: Pass data to a subprocess Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-01 13:07 +0200
    Re: Pass data to a subprocess Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-01 13:26 +0200
    Re: Pass data to a subprocess andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-01 14:25 +0100
    Re: Pass data to a subprocess Roy Smith <roy@panix.com> - 2012-08-01 15:07 -0400
  Re: Pass data to a subprocess Grant Edwards <invalid@invalid.invalid> - 2012-08-01 14:16 +0000
    Re: Pass data to a subprocess Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-01 16:32 +0200
      Re: Pass data to a subprocess Grant Edwards <invalid@invalid.invalid> - 2012-08-01 19:48 +0000
        Re: Pass data to a subprocess Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-02 08:10 +0200
          Re: Pass data to a subprocess Grant Edwards <invalid@invalid.invalid> - 2012-08-02 14:29 +0000
        Re: Pass data to a subprocess Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-08-02 13:21 -0400
    Re: Pass data to a subprocess Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-01 16:42 +0200
    Re: Pass data to a subprocess andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-01 16:24 +0100

csiph-web