Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Thu, 5 Dec 2013 13:29:58 +1100
From: Cameron Simpson <cs@zip.com.au>
To: python-list@python.org
Subject: Re: extracting a heapq in a for loop - there must be  more elegant solution
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <bg8bshFn3pfU6@mid.dfncis.de>
User-Agent: Mutt/1.5.21 (2010-09-15)
References: <bg8bshFn3pfU6@mid.dfncis.de>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.3597.1386210611.18130.python-list@python.org>
Lines: 72
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:61071

On 04Dec2013 09:44, Helmut Jarausch <jarausch@igpm.rwth-aachen.de> wrote:
> On Wed, 04 Dec 2013 08:13:03 +1100, Cameron Simpson wrote:
> > I iterate over Queues so often that I have a personal class called
> > a QueueIterator which is a wrapper for a Queue or PriorityQueue
> > which is iterable, and an IterablePriorityQueue factory function.
> > Example:
> > 
> >   from cs.queues import IterablePriorityQueue
> > 
> >   IPQ = IterablePriorityQueue()
> >   for item in [1,2,3,7,6,5,9,4]:
> >     IPQ.put(item)
> > 
> >   for item in IPQ:
> >     ... do stuff with item ...
> 
> Many thanks!
> I think you QueueIterator would be a nice addition to Python's library.

I'm not convinced. I'm still sorting out the edge cases, and it's
my own code and use cases!

The basic idea is simple enough:

  class QueueIterator:
    def __init__(self, Q, sentinel=None):
      self.q = Q
      self.sentinel = sentinel
    def __next__(self):
      item = self.q.get()
      if item is self.sentinel:
        # queue sentinel again for other users
        self.q.put(item)
        raise StopIteration
      return item
    def put(self, item):
      if item is self.sentinel:
        raise ValueError('.put of sentinel object')
      return self.q.put(item)
    def close(self):
      self.q.put(self.sentinel)

  QI = QueueIterator(Queue())

Note that it does not work well for PriorityQueues unless you can
ensure the sentinel sorted later than any other item.

But you really need to be sure nobody does a .get() in the internal
queue, or the close protocol doesn't work. This is why the above
does not expose a .get() method.

You want to prevent .put() on the queue after close().
Of course, I have use cases where I _want_ to allow .put() after
close:-(

You may want to issue a warning if more than one call to
.close happens.

And so on.

And I have a some weird cases where the close situation has trouble,
which probably points to .close() not being a great mapping to
my termination scenarios.

It is easy to code for the simple data-in, data-out case though.

Cheers,
-- 
Cameron Simpson <cs@zip.com.au>

Computer manufacturers have been failing to deliver working systems on
time and to budget since Babbage.       - Jack Schofield, in The Guardian