Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.albasani.net!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <philip@semanchuk.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.029
X-Spam-Evidence: '*H*': 0.94; '*S*': 0.00; 'subject:issue': 0.07; 'tim,': 0.09; 'received:mindspring.com': 0.16; 'subject:timing': 0.16; 'task:': 0.16; 'timeout': 0.16; 'timeout.': 0.16; 'x-mailer:apple mail (2.1084)': 0.16; 'wrote:': 0.16; 'header:In- Reply-To:1': 0.22; 'cheers': 0.23; 'welcome.': 0.23; 'pm,': 0.24; 'aug': 0.24; 'guess': 0.26; "i'm": 0.27; 'fact': 0.27; 'separate': 0.28; 'problem': 0.28; 'gather': 0.29; 'updated': 0.29; 'processes.': 0.30; 'queue': 0.30; 'received:24': 0.32; 'this.': 0.32; 'hi,': 0.32; 'to:addr:python-list': 0.33; "i've": 0.34; 'notes': 0.35; 'charset:us-ascii': 0.36; 'problem.': 0.36; 'problems': 0.36; 'subject:skip:m 10': 0.37; 'using': 0.37; 'put': 0.37; 'but': 0.37; 'question,': 0.38; 'subject:: ': 0.39; 'suggestions': 0.39; 'header:Mime-Version:1': 0.39; 'either': 0.39; 'empty': 0.39; 'data': 0.39; 'to:addr:python.org': 0.39; 'might': 0.40; "it's": 0.40; 'your': 0.61; 'header:Message-Id:1': 0.61; 'increase': 0.63; 'received:69.73': 0.84
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: multiprocessing timing issue
From: Philip Semanchuk <philip@semanchuk.com>
In-Reply-To: <j1rpfm$v6l$1@foggy.unx.sas.com>
Date: Wed, 10 Aug 2011 23:36:52 -0400
Content-Transfer-Encoding: quoted-printable
References: <j1rpfm$v6l$1@foggy.unx.sas.com>
To: Lista-Comp-Lang-Python list <python-list@python.org>
X-Mailer: Apple Mail (2.1084)
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - deimos.nocdirect.com
X-AntiAbuse: Original Domain - python.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - semanchuk.com
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.2144.1313033824.1164.python-list@python.org>
Lines: 31
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1313033824 news.xs4all.nl 23926 [2001:888:2000:d::a6]:38751
X-Complaints-To: abuse@xs4all.nl
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:11176


On Aug 9, 2011, at 1:07 PM, Tim Arnold wrote:

> Hi, I'm having problems with an empty Queue using multiprocessing.
>=20
> The task:
> I have a bunch of chapters that I want to gather data on individually =
and then update a report database with the results.
> I'm using multiprocessing to do the data-gathering simultaneously.
>=20
> Each chapter report gets put on a Queue in their separate processes. =
Then each report gets picked off the queue and the report database is =
updated with the results.
>=20
> My problem is that sometimes the Queue is empty and I guess it's
> because the get_data() method takes a lot of time.
>=20
> I've used multiprocessing before, but never with a Queue like this.
> Any notes or suggestions are very welcome.


Hi Tim,
THis might be a dumb question, but...why is it a problem if the queue is =
empty? It sounds like you figured out already that get_data() sometimes =
takes longer than your timeout. So either increase your timeout or learn =
to live with the fact that the queue is sometimes empty. I don't mean to =
be rude, I just don't understand the problem.=20

Cheers
Philip