Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #53743

Re: Multiprocessing / threading confusion

Show key headers only | View raw

On Thu, Sep 5, 2013, at 03:27 PM, Paul Pittlerson wrote:
> I'm trying to understand data handling using multiprocessing and
> threading, haven't gotten very far without running into problems. This is
> my code:

[snip (not sure why you are using multiprocessing and threading at the
same time]


> What I expect to happen is the Debugger object will receive one string at
> a time, and read it from the queue. But that's not what I see the the
> output, the "started worker" stuff seems to print for every process, but
> "ticked" and "exited" will show up in unpredictable ways, I'm guessing
> they overwrite each other and therefore will not always appear in the
> output.
> 
> So I'm looking for help in trying to make sense of this kind of stuff, I
> thought this was the basic functionality that Queue() would take care of
> unless there is some other problem in my code.

My output is probably totally different than your output.  I only get
the processes starting.  Here's why:  This stuff all runs
asynchronously.  When you start the "Debugger" thread.. I see you put a
sleep() in it, but that guarantees nothing.  At least on my machine
which is somewhat loaded ATM, by the time the Processes are started, the
Debugger thread has already finished (because of the check to see if the
queue was empty).  Apparently it is took longer than 1 second from the
time the Debugger was started and the first Process was started.
Likewise, what you are getting is probably a case where the queue is
momentarily empty by the time the debugger loop gets ahold of the queue
lock and checks to see if it's empty.  Therefore the Debugger quits. 
Also because of the asynchronicity of processes, threads, you can not
guarantee the order that the processes will get the opportunity to put()
into the queue. 

Also you can't (and shouldn't) depend on the time that __del__ gets
called.  It can get called at any time, in any order and sometimes not
at all.*

Hope this helps.

*
http://docs.python.org/3/reference/datamodel.html?highlight=__del__#object.__del__

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Multiprocessing / threading confusion Paul Pittlerson <menkomigen6@gmail.com> - 2013-09-05 12:27 -0700
  Re: Multiprocessing / threading confusion "marduk@python.net" <marduk@python.net> - 2013-09-05 18:28 -0400
    Re: Multiprocessing / threading confusion Paul Pittlerson <menkomigen6@gmail.com> - 2013-09-05 16:34 -0700
      Re: Multiprocessing / threading confusion Chris Angelico <rosuav@gmail.com> - 2013-09-06 13:00 +1000
  Re: Multiprocessing / threading confusion Chris Angelico <rosuav@gmail.com> - 2013-09-06 08:46 +1000
    Re: Multiprocessing / threading confusion Paul Pittlerson <menkomigen6@gmail.com> - 2013-09-05 17:03 -0700
      Re: Multiprocessing / threading confusion Piet van Oostrum <piet@vanoostrum.org> - 2013-09-05 23:54 -0400
        Re: Multiprocessing / threading confusion Piet van Oostrum <piet@vanoostrum.org> - 2013-09-06 00:28 -0400
  Re: Multiprocessing / threading confusion Paul Pittlerson <menkomigen6@gmail.com> - 2013-09-06 11:27 -0700
    Re: Multiprocessing / threading confusion Skip Montanaro <skip@pobox.com> - 2013-09-06 13:53 -0500
    Re: Multiprocessing / threading confusion Dave Angel <davea@davea.name> - 2013-09-06 20:34 +0000
    Re: Multiprocessing / threading confusion Piet van Oostrum <piet@vanoostrum.org> - 2013-09-06 17:15 -0400

csiph-web