Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #11176
| Path | csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.albasani.net!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail |
|---|---|
| Return-Path | <philip@semanchuk.com> |
| X-Original-To | python-list@python.org |
| Delivered-To | python-list@mail.python.org |
| X-Spam-Status | OK 0.029 |
| X-Spam-Evidence | '*H*': 0.94; '*S*': 0.00; 'subject:issue': 0.07; 'tim,': 0.09; 'received:mindspring.com': 0.16; 'subject:timing': 0.16; 'task:': 0.16; 'timeout': 0.16; 'timeout.': 0.16; 'x-mailer:apple mail (2.1084)': 0.16; 'wrote:': 0.16; 'header:In- Reply-To:1': 0.22; 'cheers': 0.23; 'welcome.': 0.23; 'pm,': 0.24; 'aug': 0.24; 'guess': 0.26; "i'm": 0.27; 'fact': 0.27; 'separate': 0.28; 'problem': 0.28; 'gather': 0.29; 'updated': 0.29; 'processes.': 0.30; 'queue': 0.30; 'received:24': 0.32; 'this.': 0.32; 'hi,': 0.32; 'to:addr:python-list': 0.33; "i've": 0.34; 'notes': 0.35; 'charset:us-ascii': 0.36; 'problem.': 0.36; 'problems': 0.36; 'subject:skip:m 10': 0.37; 'using': 0.37; 'put': 0.37; 'but': 0.37; 'question,': 0.38; 'subject:: ': 0.39; 'suggestions': 0.39; 'header:Mime-Version:1': 0.39; 'either': 0.39; 'empty': 0.39; 'data': 0.39; 'to:addr:python.org': 0.39; 'might': 0.40; "it's": 0.40; 'your': 0.61; 'header:Message-Id:1': 0.61; 'increase': 0.63; 'received:69.73': 0.84 |
| Content-Type | text/plain; charset=us-ascii |
| Mime-Version | 1.0 (Apple Message framework v1084) |
| Subject | Re: multiprocessing timing issue |
| From | Philip Semanchuk <philip@semanchuk.com> |
| In-Reply-To | <j1rpfm$v6l$1@foggy.unx.sas.com> |
| Date | Wed, 10 Aug 2011 23:36:52 -0400 |
| Content-Transfer-Encoding | quoted-printable |
| References | <j1rpfm$v6l$1@foggy.unx.sas.com> |
| To | Lista-Comp-Lang-Python list <python-list@python.org> |
| X-Mailer | Apple Mail (2.1084) |
| X-AntiAbuse | This header was added to track abuse, please include it with any abuse report |
| X-AntiAbuse | Primary Hostname - deimos.nocdirect.com |
| X-AntiAbuse | Original Domain - python.org |
| X-AntiAbuse | Originator/Caller UID/GID - [47 12] / [47 12] |
| X-AntiAbuse | Sender Address Domain - semanchuk.com |
| X-BeenThere | python-list@python.org |
| X-Mailman-Version | 2.1.12 |
| Precedence | list |
| List-Id | General discussion list for the Python programming language <python-list.python.org> |
| List-Unsubscribe | <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe> |
| List-Archive | <http://mail.python.org/pipermail/python-list> |
| List-Post | <mailto:python-list@python.org> |
| List-Help | <mailto:python-list-request@python.org?subject=help> |
| List-Subscribe | <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe> |
| Newsgroups | comp.lang.python |
| Message-ID | <mailman.2144.1313033824.1164.python-list@python.org> (permalink) |
| Lines | 31 |
| NNTP-Posting-Host | 2001:888:2000:d::a6 |
| X-Trace | 1313033824 news.xs4all.nl 23926 [2001:888:2000:d::a6]:38751 |
| X-Complaints-To | abuse@xs4all.nl |
| Xref | x330-a1.tempe.blueboxinc.net comp.lang.python:11176 |
Show key headers only | View raw
On Aug 9, 2011, at 1:07 PM, Tim Arnold wrote: > Hi, I'm having problems with an empty Queue using multiprocessing. > > The task: > I have a bunch of chapters that I want to gather data on individually and then update a report database with the results. > I'm using multiprocessing to do the data-gathering simultaneously. > > Each chapter report gets put on a Queue in their separate processes. Then each report gets picked off the queue and the report database is updated with the results. > > My problem is that sometimes the Queue is empty and I guess it's > because the get_data() method takes a lot of time. > > I've used multiprocessing before, but never with a Queue like this. > Any notes or suggestions are very welcome. Hi Tim, THis might be a dumb question, but...why is it a problem if the queue is empty? It sounds like you figured out already that get_data() sometimes takes longer than your timeout. So either increase your timeout or learn to live with the fact that the queue is sometimes empty. I don't mean to be rude, I just don't understand the problem. Cheers Philip
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
multiprocessing timing issue Tim Arnold <Tim.Arnold@sas.com> - 2011-08-09 13:07 -0400
Re: multiprocessing timing issue Philip Semanchuk <philip@semanchuk.com> - 2011-08-10 23:36 -0400
Re: multiprocessing timing issue Tim Arnold <Tim.Arnold@sas.com> - 2011-08-11 12:58 -0400
Re: multiprocessing timing issue Tim Roberts <timr@probo.com> - 2011-08-10 22:26 -0700
csiph-web