Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #16450 > unrolled thread

Need some IPC pointers

Back to article view | Back to comp.lang.python

Contents

  Need some IPC pointers Andrew Berg <bahamutzero8825@gmail.com> - 2011-11-30 15:03 -0600
    Re: Need some IPC pointers Miki Tebeka <miki.tebeka@gmail.com> - 2011-11-30 13:22 -0800
    Re: Need some IPC pointers Miki Tebeka <miki.tebeka@gmail.com> - 2011-11-30 13:22 -0800
      Re: Need some IPC pointers Andrew Berg <bahamutzero8825@gmail.com> - 2011-12-06 08:05 -0600
    Re: Need some IPC pointers Irmen de Jong <irmen@-NOSPAM-xs4all.nl> - 2011-11-30 23:28 +0100
      Re: Need some IPC pointers Irmen de Jong <irmen@-NOSPAM-xs4all.nl> - 2011-11-30 23:31 +0100
    Re: Need some IPC pointers bobicanprogram <icanbob@gmail.com> - 2011-12-02 11:06 -0800
    Re: Need some IPC pointers Floris Bruynooghe <floris.bruynooghe@gmail.com> - 2011-12-06 05:37 -0800
    Re: Need some IPC pointers Floris Bruynooghe <floris.bruynooghe@gmail.com> - 2011-12-06 05:37 -0800

#16450 — Need some IPC pointers

I've done some research, but I'm not sure what's most appropriate for my
situation. What I want to do is have a long running process that spawns
processes (that aren't necessarily written in Python) and communicates
with them. The children can be spawned at any time and communicate at
any time. Being able to communicate with non-local processes would be
nice, but is not necessary. The implementation needs to be
cross-platform, but child processes will use the same OS as the parent
during runtime.
I don't think I'll ever need to transfer anything complicated or large -
just strings or possibly tuples/lists. I'd rather not go outside the
standard library (but I'd consider it). I don't need to worry about
compatibility with older Python versions; if it only works with Python
3.2, that's not a problem.
I'm thinking sockets, but perhaps there's something simpler/easier.

-- 
CPython 3.2.2 | Windows NT 6.1.7601.17640 | Thunderbird 7.0

[toc] | [next] | [standalone]

#16451

There are two different problems. One is the medium to pass messages, sockets are good but there are other options and depends on your requirement you need to pick the best one.
The other is serialization format, here you have marshal, pickle, JSON, XML and more.

For me JSON over sockets works well in the past. It has the nice property that you can read messages without need for a special decoder (like in the case of marshal/pickle).

[toc] | [prev] | [next] | [standalone]

#16452

There are two different problems. One is the medium to pass messages, sockets are good but there are other options and depends on your requirement you need to pick the best one.
The other is serialization format, here you have marshal, pickle, JSON, XML and more.

For me JSON over sockets works well in the past. It has the nice property that you can read messages without need for a special decoder (like in the case of marshal/pickle).

[toc] | [prev] | [next] | [standalone]

#16733

What about named pipes? I don't mind a bit of "if Windows do this, else,
do that" as long I'm not coding two or more completely different
approaches. I'm not too familiar with named pipes, though; perhaps
someone with some experience could chime in.


Apparently this didn't go through to Google Groups the first time; it
may show up twice for some.

-- 
CPython 3.2.2 | Windows NT 6.1.7601.17640 | Thunderbird 7.0

[toc] | [prev] | [next] | [standalone]

#16457

On 30-11-11 22:03, Andrew Berg wrote:
> I've done some research, but I'm not sure what's most appropriate for my
> situation. What I want to do is have a long running process that spawns
> processes (that aren't necessarily written in Python) and communicates
> with them. The children can be spawned at any time and communicate at
> any time. Being able to communicate with non-local processes would be
> nice, but is not necessary. The implementation needs to be
> cross-platform, but child processes will use the same OS as the parent
> during runtime.
> I don't think I'll ever need to transfer anything complicated or large -
> just strings or possibly tuples/lists. I'd rather not go outside the
> standard library (but I'd consider it). I don't need to worry about
> compatibility with older Python versions; if it only works with Python
> 3.2, that's not a problem.
> I'm thinking sockets, but perhaps there's something simpler/easier.
>

Standard library, local processes: multiprocessing module.

If that doesn't suit your needs, maybe check out Pyro:
http://packages.python.org/Pyro4/

Pyro allows objects to talk to each other over the network, with minimal 
programming effort. You can just use normal Python method calls to call 
objects on other machines (or locally, ofcourse). It's written in 100% 
pure Python and works on Python 2.6 and upwards (including 3.x).

Regards,
Irmen de Jong

[toc] | [prev] | [next] | [standalone]

#16459

On 30-11-11 23:28, Irmen de Jong wrote:
> On 30-11-11 22:03, Andrew Berg wrote:

>> processes (that aren't necessarily written in Python) and communicates

Oops, missed this on my first read. This rules out my suggestion of Pyro 
because that requires Python on both ends (or Java/.net on the client 
side). Sorry for the noise.

Irmen

[toc] | [prev] | [next] | [standalone]

#16568

On Nov 30, 4:03 pm, Andrew Berg <bahamutzero8...@gmail.com> wrote:
> I've done some research, but I'm not sure what's most appropriate for my
> situation. What I want to do is have a long running process that spawns
> processes (that aren't necessarily written in Python) and communicates
> with them. The children can be spawned at any time and communicate at
> any time. Being able to communicate with non-local processes would be
> nice, but is not necessary. The implementation needs to be
> cross-platform, but child processes will use the same OS as the parent
> during runtime.
> I don't think I'll ever need to transfer anything complicated or large -
> just strings or possibly tuples/lists. I'd rather not go outside the
> standard library (but I'd consider it). I don't need to worry about
> compatibility with older Python versions; if it only works with Python
> 3.2, that's not a problem.
> I'm thinking sockets, but perhaps there's something simpler/easier.
>
> --
> CPython 3.2.2 | Windows NT 6.1.7601.17640 | Thunderbird 7.0


You might want to take a look at the SIMPL toolkit.
http://www.icanprogram.com/06py/lesson1/lesson1.html

SIMPL modules can be written (and mixed) in any number of supported
languages including Python, C, C++, JAVA, Tcl/Tk or PHP.   Non local
communication is handled by generic surrogates.  Because of this
surrogate architecture SIMPL modules can often be tested locally and
then deployed into the cloud without any changes.

bob

[toc] | [prev] | [next] | [standalone]

#16730

I'm surprised no one has mentioned zeromq as transport yet.  It provides scaling from in proc (between threads) to inter-process and remote machines in a fairly transparent way.  It's obviously not the python stdlib and as any system there are downsides too.

Regards,
Floris

[toc] | [prev] | [next] | [standalone]

#16731

I'm surprised no one has mentioned zeromq as transport yet.  It provides scaling from in proc (between threads) to inter-process and remote machines in a fairly transparent way.  It's obviously not the python stdlib and as any system there are downsides too.

Regards,
Floris

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web