Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #42519

Re: Get multiprocessing.Queue to do priorities

Path csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!newsfeed.eweka.nl!eweka.nl!feeder3.eweka.nl!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <madeleine.udell@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.008
X-Spam-Evidence '*H*': 0.98; '*S*': 0.00; 'jesse': 0.05; '*not*': 0.07; 'noller': 0.07; 'to:addr:comp.lang.python': 0.09; 'cc:addr :python-list': 0.11; 'thread': 0.14; '(which,': 0.16; 'cc:name:python list': 0.16; 'inaccessible': 0.16; 'logical,': 0.16; 'resolved?': 0.16; 'tracker,': 0.16; 'sat,': 0.16; 'wrote:': 0.18; 'all,': 0.19; 'bit': 0.19; 'module': 0.19; 'appears': 0.22; 'cc:addr:python.org': 0.22; 'cc:2**1': 0.23; 'creating': 0.23; 'header:User-Agent:1': 0.23; 'adds': 0.24; 'module,': 0.24; 'holds': 0.26; 'header:In-Reply-To:1': 0.27; 'correct': 0.29; '2009': 0.29; 'patch': 0.29; 'submitting': 0.31; 'ordinary': 0.31; 'file': 0.32; 'this.': 0.32; 'option': 0.32; 'quite': 0.32; 'received:209.85': 0.35; 'received:209.85.220': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'received:209': 0.37; 'manager': 0.38; 'issue': 0.38; 'pm,': 0.38; 'moving': 0.39; 'from:no real name:2**0': 0.61; 'different': 0.65; 'between': 0.67; 'subject:Get': 0.68; '6:11': 0.84; 'safe.': 0.84; 'received:209.85.220.184': 0.91; 'wishing': 0.93; 'enhancement': 0.95
X-Received by 10.50.153.101 with SMTP id vf5mr984628igb.3.1364855744392; Mon, 01 Apr 2013 15:35:44 -0700 (PDT)
Newsgroups comp.lang.python
Date Mon, 1 Apr 2013 15:35:44 -0700 (PDT)
In-Reply-To <mailman.5376.1241962506.11746.python-list@python.org>
Complaints-To groups-abuse@google.com
Injection-Info glegroupsg2000goo.googlegroups.com; posting-host=171.64.92.59; posting-account=H9EcpQoAAACKwG86xOMuaLgcg87Je64O
References <4a05b27a_2@news.arcor-ip.de> <4a05b6c6_2@news.arcor-ip.de> <28KdnTocUpw2X5jXnZ2dnUVZ_sudnZ2d@pdx.net> <529683651263584828.031438address_is-invalid.invalid@news.arcor.de> <-vKdnex07oQzQpjXnZ2dnUVZ_hZi4p2d@pdx.net> <4a05ff89_1@news.arcor-ip.de> <mailman.5376.1241962506.11746.python-list@python.org>
User-Agent G2/1.0
X-Google-Web-Client true
X-Google-IP 171.64.92.59
MIME-Version 1.0
Subject Re: Get multiprocessing.Queue to do priorities
From madeleine.udell@gmail.com
To comp.lang.python@googlegroups.com
Content-Type text/plain; charset=ISO-8859-1
Cc Python List <python-list@python.org>, uuid <M8R-gqmgb4@mailinator.com>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Message-ID <mailman.24.1364856317.17481.python-list@python.org> (permalink)
Lines 20
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1364856317 news.xs4all.nl 6945 [2001:888:2000:d::a6]:36584
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:42519

Show key headers only | View raw


Was this issue ever resolved? What is the current best practice for those wishing to use a priority queue with multiprocessing?

On Sunday, May 10, 2009 6:35:03 AM UTC-7, Jesse Noller wrote:
> On Sat, May 9, 2009 at 6:11 PM, uuid <M8R-gqmgb4@mailinator.com> wrote:
> > The Queue module, apparently, is thread safe, but *not* process safe. If you
> > try to use an ordinary Queue, it appears inaccessible to the worker process.
> > (Which, after all, is quite logical, since methods for moving items between
> > the threads of the same process are quite different from inter-process
> > communication.) It appears that creating a manager that holds a shared queue
> > might be an option
> > (http://stackoverflow.com/questions/342556/python-2-6-multiprocessing-queue-compatible-with-threads).
> 
> Using a manager, or submitting a patch which adds priority queue to
> the multiprocessing.queue module is the correct solution for this.
> 
> You can file an enhancement in the tracker, and assign/add me to it,
> but without a patch it may take me a bit (wicked busy right now).
> 
> jesse

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

Re: Get multiprocessing.Queue to do priorities madeleine.udell@gmail.com - 2013-04-01 15:35 -0700

csiph-web