Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #11176

Re: multiprocessing timing issue

Show all headers | View raw

On Aug 9, 2011, at 1:07 PM, Tim Arnold wrote:

> Hi, I'm having problems with an empty Queue using multiprocessing.
> 
> The task:
> I have a bunch of chapters that I want to gather data on individually and then update a report database with the results.
> I'm using multiprocessing to do the data-gathering simultaneously.
> 
> Each chapter report gets put on a Queue in their separate processes. Then each report gets picked off the queue and the report database is updated with the results.
> 
> My problem is that sometimes the Queue is empty and I guess it's
> because the get_data() method takes a lot of time.
> 
> I've used multiprocessing before, but never with a Queue like this.
> Any notes or suggestions are very welcome.


Hi Tim,
THis might be a dumb question, but...why is it a problem if the queue is empty? It sounds like you figured out already that get_data() sometimes takes longer than your timeout. So either increase your timeout or learn to live with the fact that the queue is sometimes empty. I don't mean to be rude, I just don't understand the problem. 

Cheers
Philip

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

multiprocessing timing issue Tim Arnold <Tim.Arnold@sas.com> - 2011-08-09 13:07 -0400
  Re: multiprocessing timing issue Philip Semanchuk <philip@semanchuk.com> - 2011-08-10 23:36 -0400
    Re: multiprocessing timing issue Tim Arnold <Tim.Arnold@sas.com> - 2011-08-11 12:58 -0400
  Re: multiprocessing timing issue Tim Roberts <timr@probo.com> - 2011-08-10 22:26 -0700

csiph-web