Re: Finding the source of an exception in a python multiprocessing program

Date	2013-04-24 20:11 -0400
From	Dave Angel <davea@davea.name>
Subject	Re: Finding the source of an exception in a python multiprocessing program
References	<mailman.1032.1366835159.3114.python-list@python.org> <atqtpaFa3niU2@mid.individual.net> <5BC4D9D9-D757-40F3-81CA-CF4FA5C73A96@mac.com> <51786A2C.4020306@davea.name> <CAHVvXxShZJpHydbBO_X3v+WKoaPU871eOHiALbQoO97xryst6A@mail.gmail.com>
Newsgroups	comp.lang.python
Message-ID	<mailman.1041.1366848732.3114.python-list@python.org> (permalink)

Show all headers | View raw

On 04/24/2013 08:00 PM, Oscar Benjamin wrote:
> On 25 April 2013 00:26, Dave Angel <davea@davea.name> wrote:
>> On 04/24/2013 05:09 PM, William Ray Wing wrote:
>>>
>>>
>>>    <SNIP>
>>>
>>
>> My question is why bother with multithreading?  Why not just do these as
>> separate processes?  You said "they in no way interact with each other" and
>> that's a clear clue that separate processes would be cleaner.
>
> It's using multiprocessing rather than threads: they are separate processes.
>

You're right;  I was completely off base.  brain-freeze.

>>
>>     <SNIP>
>
> It's state that is passed to it by the subprocess and should only be
> accessed by the top-level process after the subprocess completes (I
> think!).
>
>>
>> Separate processes will find it much more difficult to interact, which is a
>> good thing most of the time.  Further, they seem to be scheduled more
>> efficiently because of the GIL, though that may not make that much
>> difference when you're time-limited by network data.
>
> They are separate processes and do not share the GIL (unless I'm very
> much mistaken).

No, you're not mistaken.  Somehow I interpreted the original as saying 
multi-thread, and everything else was wrong as a result.  Now it sounds 
like a bug in, or misuse of, the Pool class.



-- 
DaveA

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Finding the source of an exception in a python multiprocessing program William Ray Wing <wrw@mac.com> - 2013-04-24 15:25 -0400
  Re: Finding the source of an exception in a python multiprocessing program Neil Cerutti <neilc@norwich.edu> - 2013-04-24 20:31 +0000
    Re: Finding the source of an exception in a python multiprocessing program William Ray Wing <wrw@mac.com> - 2013-04-24 17:09 -0400
      Re: Finding the source of an exception in a python multiprocessing program Neil Cerutti <neilc@norwich.edu> - 2013-04-25 12:37 +0000
    Re: Finding the source of an exception in a python multiprocessing program Dave Angel <davea@davea.name> - 2013-04-24 19:26 -0400
    Re: Finding the source of an exception in a python multiprocessing program Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2013-04-25 01:00 +0100
    Re: Finding the source of an exception in a python multiprocessing program Dave Angel <davea@davea.name> - 2013-04-24 20:11 -0400

csiph-web