Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #61071

Re: extracting a heapq in a for loop - there must be more elegant solution

Show all headers | View raw

On 04Dec2013 09:44, Helmut Jarausch <jarausch@igpm.rwth-aachen.de> wrote:
> On Wed, 04 Dec 2013 08:13:03 +1100, Cameron Simpson wrote:
> > I iterate over Queues so often that I have a personal class called
> > a QueueIterator which is a wrapper for a Queue or PriorityQueue
> > which is iterable, and an IterablePriorityQueue factory function.
> > Example:
> > 
> >   from cs.queues import IterablePriorityQueue
> > 
> >   IPQ = IterablePriorityQueue()
> >   for item in [1,2,3,7,6,5,9,4]:
> >     IPQ.put(item)
> > 
> >   for item in IPQ:
> >     ... do stuff with item ...
> 
> Many thanks!
> I think you QueueIterator would be a nice addition to Python's library.

I'm not convinced. I'm still sorting out the edge cases, and it's
my own code and use cases!

The basic idea is simple enough:

  class QueueIterator:
    def __init__(self, Q, sentinel=None):
      self.q = Q
      self.sentinel = sentinel
    def __next__(self):
      item = self.q.get()
      if item is self.sentinel:
        # queue sentinel again for other users
        self.q.put(item)
        raise StopIteration
      return item
    def put(self, item):
      if item is self.sentinel:
        raise ValueError('.put of sentinel object')
      return self.q.put(item)
    def close(self):
      self.q.put(self.sentinel)

  QI = QueueIterator(Queue())

Note that it does not work well for PriorityQueues unless you can
ensure the sentinel sorted later than any other item.

But you really need to be sure nobody does a .get() in the internal
queue, or the close protocol doesn't work. This is why the above
does not expose a .get() method.

You want to prevent .put() on the queue after close().
Of course, I have use cases where I _want_ to allow .put() after
close:-(

You may want to issue a warning if more than one call to
.close happens.

And so on.

And I have a some weird cases where the close situation has trouble,
which probably points to .close() not being a great mapping to
my termination scenarios.

It is easy to code for the simple data-in, data-out case though.

Cheers,
-- 
Cameron Simpson <cs@zip.com.au>

Computer manufacturers have been failing to deliver working systems on
time and to budget since Babbage.       - Jack Schofield, in The Guardian

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

extracting a heapq in a for loop - there must be  more elegant solution Helmut Jarausch <jarausch@igpm.rwth-aachen.de> - 2013-12-03 12:18 +0000
  Re: extracting a heapq in a for loop - there must be more elegant solution Peter Otten <__peter__@web.de> - 2013-12-03 13:38 +0100
    Re: extracting a heapq in a for loop - there must be more elegant solution Helmut Jarausch <jarausch@igpm.rwth-aachen.de> - 2013-12-04 09:36 +0000
      Re: extracting a heapq in a for loop - there must be more elegant solution Peter Otten <__peter__@web.de> - 2013-12-04 10:50 +0100
  Re: extracting a heapq in a for loop - there must be  more elegant solution rusi <rustompmody@gmail.com> - 2013-12-03 04:40 -0800
    Re: extracting a heapq in a for loop - there must be  more elegant solution Helmut Jarausch <jarausch@igpm.rwth-aachen.de> - 2013-12-04 09:37 +0000
  Re: extracting a heapq in a for loop - there must be  more elegant solution Duncan Booth <duncan.booth@invalid.invalid> - 2013-12-03 13:06 +0000
    Re: extracting a heapq in a for loop - there must be  more elegant solution Helmut Jarausch <jarausch@igpm.rwth-aachen.de> - 2013-12-04 09:39 +0000
  Re: extracting a heapq in a for loop - there must be  more elegant solution Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-12-03 15:56 +0200
    Re: extracting a heapq in a for loop - there must be  more elegant solution Helmut Jarausch <jarausch@igpm.rwth-aachen.de> - 2013-12-04 09:41 +0000
  Re: extracting a heapq in a for loop - there must be  more elegant solution Cameron Simpson <cs@zip.com.au> - 2013-12-04 08:13 +1100
    Re: extracting a heapq in a for loop - there must be  more elegant solution Helmut Jarausch <jarausch@igpm.rwth-aachen.de> - 2013-12-04 09:44 +0000
      Re: extracting a heapq in a for loop - there must be  more elegant solution Cameron Simpson <cs@zip.com.au> - 2013-12-05 13:29 +1100
  Re: extracting a heapq in a for loop - there must be more elegant solution Ian Kelly <ian.g.kelly@gmail.com> - 2013-12-03 14:43 -0700
  Re: extracting a heapq in a for loop - there must be more elegant solution Ned Batchelder <ned@nedbatchelder.com> - 2013-12-03 17:32 -0500

csiph-web