Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
Date: Mon, 23 Feb 2015 10:26:06 -0800
Subject: multiprocessing module best practices?
From: Christian Calderon <calderon.christian760@gmail.com>
To: python-list@python.org
Content-Type: multipart/alternative; boundary=001a11c20276154966050fc589b0
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.19093.1424716831.18130.python-list@python.org>
Lines: 48
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:86252

--001a11c20276154966050fc589b0
Content-Type: text/plain; charset=UTF-8

Is there a good blog or tutorial on when it is most appropriate to use the
Process vs Pool classes, and when you should subclass either of them? I
have written a few programs using these tools, and each time I have to sort
of make up my own design patterns as I go. I want to know what the
community thinks is the most pythonic use of the multiprocessing module (I
could just subclass Process every single time, why shouldn't I?)

I'm also curious about when I should be importing modules in the main
module to use in a Process, and when I should import the module inside the
run method/target function. Certain modules just won't work unless I import
it in the Process; I hate having a ton of import statements inside my
functions, but I also hate having to guess-and-check just to figure out
which modules I can import outside the function. And it is just kinda ugly
to have import statements at multiple parts of the same piece of code, IMO.

Here is an example of something I wrote that feels kinda funky (my excuse
is that the code-base I was working with was incredibly funky to begin
with, and I was trying to make a small corner of it nicer/actually work.):
    https://github.com/zack-bitcoin/augur-core/blob/master/database.py

--001a11c20276154966050fc589b0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Is there a good blog or tutorial on when it is most approp=
riate to use the Process vs Pool classes, and when you should subclass eith=
er of them? I have written a few programs using these tools, and each time =
I have to sort of make up my own design patterns as I go. I want to know wh=
at the community thinks is the most pythonic use of the multiprocessing mod=
ule (I could just subclass Process every single time, why shouldn&#39;t I?)=
<div><br></div><div>I&#39;m also curious about when I should be importing m=
odules in the main module to use in a Process, and when I should import the=
 module inside the run method/target function. Certain modules just won&#39=
;t work unless I import it in the Process; I hate having a ton of import st=
atements inside my functions, but I also hate having to guess-and-check jus=
t to figure out which modules I can import outside the function. And it is =
just kinda ugly to have import statements at multiple parts of the same pie=
ce of code, IMO.<br><br>Here is an example of something I wrote that feels =
kinda funky (my excuse is that the code-base I was working with was incredi=
bly funky to begin with, and I was trying to make a small corner of it nice=
r/actually work.):</div><div>=C2=A0 =C2=A0 <a href=3D"https://github.com/za=
ck-bitcoin/augur-core/blob/master/database.py">https://github.com/zack-bitc=
oin/augur-core/blob/master/database.py</a></div></div>

--001a11c20276154966050fc589b0--