Path: csiph.com!usenet.pasdenom.info!news.albasani.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
Sender: chris@rebertia.com
In-Reply-To: <516B37CB.1020607@earthlink.net>
References: <516B37CB.1020607@earthlink.net>
Date: Sun, 14 Apr 2013 19:32:54 -0700
Subject: Re: Threadpool item mailboxes design problem
From: Chris Rebert <clp2@rebertia.com>
To: Charles Hixson <charleshixsn@earthlink.net>
Content-Type: multipart/alternative; boundary=f46d04339ca801deb304da5d12c8
Cc: Python <python-list@python.org>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.615.1365993185.3114.python-list@python.org>
Lines: 93
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:43596

--f46d04339ca801deb304da5d12c8
Content-Type: text/plain; charset=UTF-8

On Apr 14, 2013 4:27 PM, "Charles Hixson" <charleshixsn@earthlink.net>
wrote:
>
> What is the best approach to implementing actors that accept and post
messages (and have no other external contacts).

You might look at how some of the existing Python actor libraries are
implemented (perhaps one of these might even save you from reinventing the
wheel):

http://www.pykka.org/en/latest/
http://www.kamaelia.org/Docs/Axon/Axon.html
https://pypi.python.org/pypi/pulsar

Kinda old:
http://candygram.sourceforge.net/contents.html
http://osl.cs.uiuc.edu/parley/

> So far what I've come up with is something like:
> actors     = {}
> mailboxs = {}
>
> Stuff actors with actor instances, mailboxes with multiprocessing.queue
instances.   (Actors and mailboxes will have identical keys, which are id#,
but it's got to be a dict rather than a list, because too many are rolled
out to disk.)  And I'm planning of having the actors running simultaneously
and continually in a threadpool that just loops through the actors that are
assigned to each thread of the pool.
<snip>
> It would, however, be better if the mailbox could be specific to the
threadpool instance, so less space would be wasted.  Or if the queues could
dynamically resize.  Or if there was a threadsafe dict.  Or...  But I don't
know that any of these are feasible.  (I mean, yes, I could write all the
mail to a database, but is that a better answer, or even a good one?)

My recollection is that the built-in collection types are threadsafe at
least to the limited extent that the operations exposed by their APIs (e.g.
dict.setdefault) are atomic.
Perhaps someone will be able to chime in with more details.

--f46d04339ca801deb304da5d12c8
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<p dir=3D"ltr">On Apr 14, 2013 4:27 PM, &quot;Charles Hixson&quot; &lt;<a h=
ref=3D"mailto:charleshixsn@earthlink.net">charleshixsn@earthlink.net</a>&gt=
; wrote:<br>
&gt;<br>
&gt; What is the best approach to implementing actors that accept and post =
messages (and have no other external contacts).</p>
<p dir=3D"ltr">You might look at how some of the existing Python actor libr=
aries are implemented (perhaps one of these might even save you from reinve=
nting the wheel):</p>
<p dir=3D"ltr"><a href=3D"http://www.pykka.org/en/latest/">http://www.pykka=
.org/en/latest/</a><br>
<a href=3D"http://www.kamaelia.org/Docs/Axon/Axon.html">http://www.kamaelia=
.org/Docs/Axon/Axon.html</a><br>
<a href=3D"https://pypi.python.org/pypi/pulsar">https://pypi.python.org/pyp=
i/pulsar</a></p>
<p dir=3D"ltr">Kinda old:<br>
<a href=3D"http://candygram.sourceforge.net/contents.html">http://candygram=
.sourceforge.net/contents.html</a><br>
<a href=3D"http://osl.cs.uiuc.edu/parley/">http://osl.cs.uiuc.edu/parley/</=
a></p>
<p dir=3D"ltr">&gt; So far what I&#39;ve come up with is something like:<br=
>
&gt; actors =C2=A0 =C2=A0 =3D {}<br>
&gt; mailboxs =3D {}<br>
&gt;<br>
&gt; Stuff actors with actor instances, mailboxes with multiprocessing.queu=
e instances. =C2=A0 (Actors and mailboxes will have identical keys, which a=
re id#, but it&#39;s got to be a dict rather than a list, because too many =
are rolled out to disk.) =C2=A0And I&#39;m planning of having the actors ru=
nning simultaneously and continually in a threadpool that just loops throug=
h the actors that are assigned to each thread of the pool.<br>

&lt;snip&gt;<br>
&gt; It would, however, be better if the mailbox could be specific to the t=
hreadpool instance, so less space would be wasted. =C2=A0Or if the queues c=
ould dynamically resize. =C2=A0Or if there was a threadsafe dict. =C2=A0Or.=
.. =C2=A0But I don&#39;t know that any of these are feasible. =C2=A0(I mean=
, yes, I could write all the mail to a database, but is that a better answe=
r, or even a good one?)</p>

<p dir=3D"ltr">My recollection is that the built-in collection types are th=
readsafe at least to the limited extent that the operations exposed by thei=
r APIs (e.g. dict.setdefault) are atomic.<br>
Perhaps someone will be able to chime in with more details.</p>

--f46d04339ca801deb304da5d12c8--