Re: Looking for an IPC solution

Return-Path	<gandalf@shopzeus.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.006
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'python,': 0.02; 'arguments': 0.07; 'space.': 0.07; 'python': 0.09; '(it': 0.09; 'buffer.': 0.09; 'deletion': 0.09; 'stl': 0.09; 'windows,': 0.09; '"free"': 0.16; '4-byte': 0.16; 'arrays.': 0.16; 'blocks': 0.16; 'did,': 0.16; 'processes.': 0.16; 'segment': 0.16; 'bytes': 0.17; 'integer': 0.17; 'memory': 0.18; 'requests': 0.18; 'module': 0.19; 'amounts': 0.22; 'libraries': 0.22; 'names.': 0.22; 'to:2**1': 0.23; 'header:In-Reply-To:1': 0.25; 'header:User-Agent:1': 0.26; 'creating': 0.26; 'values': 0.26; 'guess': 0.27; 'disk': 0.27; 'structures': 0.27; "doesn't": 0.28; 'arrays': 0.29; "i'm": 0.29; 'file': 0.32; 'operate': 0.32; 'space,': 0.32; 'running': 0.32; 'could': 0.32; "aren't": 0.33; 'to:addr:python-list': 0.33; 'another': 0.33; 'minimum': 0.34; 'server': 0.35; 'list': 0.35; 'faster': 0.35; 'locations': 0.35; 'doing': 0.35; 'there': 0.35; 'but': 0.36; 'operating': 0.36; 'uses': 0.37; 'subject:: ': 0.38; 'files': 0.38; 'object': 0.38; 'some': 0.38; 'sure': 0.38; 'system.': 0.39; 'to:addr:python.org': 0.39; 'received:192': 0.39; 'space': 0.39; 'received:192.168': 0.40; 'your': 0.60; 'address': 0.60; 'free': 0.61; 'more.': 0.62; 'provide': 0.62; 'different': 0.63; 'times': 0.63; 'within': 0.64; 'become': 0.65; 'details,': 0.65; 'received:204': 0.72; 'yourself': 0.77; 'conducive': 0.84; 'routines': 0.84; 'prone': 0.91
Date	Thu, 06 Sep 2012 12:00:10 +0200
From	Laszlo Nagy <gandalf@shopzeus.com>
User-Agent	Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0
MIME-Version	1.0
To	Aaron Brady <castironpi@gmail.com>, python-list@python.org
Subject	Re: Looking for an IPC solution
References	<mailman.9.1346440878.27098.python-list@python.org> <40ab9c6f-737e-47e3-950b-558472ad4a09@googlegroups.com>
In-Reply-To	<40ab9c6f-737e-47e3-950b-558472ad4a09@googlegroups.com>
Content-Type	text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding	7bit
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.285.1346925623.27098.python-list@python.org> (permalink)
Lines	11
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1346925623 news.xs4all.nl 6880 [2001:888:2000:d::a6]:42871
X-Complaints-To	abuse@xs4all.nl
Path	csiph.com!usenet.pasdenom.info!news.stben.net!border3.nntp.ams.giganews.com!border1.nntp.ams.giganews.com!nntp.giganews.com!usenetcore.com!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Xref	csiph.com comp.lang.python:28567

Show key headers only | View raw

> Hi Laszlo,
>
> There aren't a lot of ways to create a Python object in an "mmap" buffer.  "mmap" is conducive to arrays of arrays.  For variable-length structures like strings and lists, you need "dynamic allocation".  The C functions "malloc" and "free" allocate memory space, and file creation and deletion routines operate on disk space.  However "malloc" doesn't allow you to allocate memory space within memory that's already allocated.  Operating systems don't provide that capability, and doing it yourself amounts to creating your own file system.  If you did, you still might not be able to use existing libraries like the STL or Python, because one address might refer to different locations in different processes.
>
> One solution is to keep a linked list of free blocks within your "mmap" buffer.  It is prone to slow access times and segment fragmentation.  Another solution is to create many small files with fixed-length names.  The minimum file size on your system might become prohibitive depending on your constraints, since a 4-byte integer could occupy 4096 bytes on disk or more.  Or you can serialize the arguments and return values of your functions, and make requests to a central process.
I'm not sure about the technical details, but I was said that 
multiprocessing module uses mmap() under windows. And it is faster than 
TCP/IP. So I guess the same thing could be used from zmq, under Windows. 
(It is not a big concern, I plan to operate server on Unix. Some clients 
might be running on Windows, but they will use TCP/IP.)

Thread

Looking for an IPC solution Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-31 21:04 +0200
  Re: Looking for an IPC solution Marco Nawijn <nawijn@gmail.com> - 2012-08-31 13:25 -0700
  Re: Looking for an IPC solution Marco Nawijn <nawijn@gmail.com> - 2012-08-31 13:25 -0700
  Re: Looking for an IPC solution Paul Rubin <no.email@nospam.invalid> - 2012-08-31 13:36 -0700
    Re: Looking for an IPC solution Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-31 23:10 +0200
  Re: Looking for an IPC solution Wolfgang Keller <feliphil@gmx.net> - 2012-09-01 14:55 +0200
    Re: Looking for an IPC solution vasudevram <vasudevram@gmail.com> - 2012-09-03 14:43 -0700
    Re: Looking for an IPC solution Laszlo Nagy <gandalf@shopzeus.com> - 2012-09-06 12:13 +0200
  Re: Looking for an IPC solution Aaron Brady <castironpi@gmail.com> - 2012-09-01 19:10 -0700
    Re: Looking for an IPC solution Laszlo Nagy <gandalf@shopzeus.com> - 2012-09-06 12:00 +0200
  Re: Looking for an IPC solution Aaron Brady <castironpi@gmail.com> - 2012-09-01 19:10 -0700

csiph-web