Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #86252

multiprocessing module best practices?

Show key headers only | View raw

[Multipart message — attachments visible in raw view] - view raw

Is there a good blog or tutorial on when it is most appropriate to use the
Process vs Pool classes, and when you should subclass either of them? I
have written a few programs using these tools, and each time I have to sort
of make up my own design patterns as I go. I want to know what the
community thinks is the most pythonic use of the multiprocessing module (I
could just subclass Process every single time, why shouldn't I?)

I'm also curious about when I should be importing modules in the main
module to use in a Process, and when I should import the module inside the
run method/target function. Certain modules just won't work unless I import
it in the Process; I hate having a ton of import statements inside my
functions, but I also hate having to guess-and-check just to figure out
which modules I can import outside the function. And it is just kinda ugly
to have import statements at multiple parts of the same piece of code, IMO.

Here is an example of something I wrote that feels kinda funky (my excuse
is that the code-base I was working with was incredibly funky to begin
with, and I was trying to make a small corner of it nicer/actually work.):
    https://github.com/zack-bitcoin/augur-core/blob/master/database.py

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

multiprocessing module best practices? Christian Calderon <calderon.christian760@gmail.com> - 2015-02-23 10:26 -0800

csiph-web