Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
Date: Mon, 23 Feb 2015 11:23:44 -0600
Subject: Well, that was a head scratcher...
From: Skip Montanaro <skip.montanaro@gmail.com>
To: Python <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.19086.1424712227.18130.python-list@python.org>
Lines: 50
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:86242

I've been reworking some of the code in the platform I use at work.
I'm the sole developer/maintainer/hard-core user left, so I can pretty
much do what I want with it (convert modules to Cython, delete no
longer used modules, etc).  The platform uses PyGtk, so we use signals
and other features of that system heavily. In one place, I register a
generator function with gobject.idle_add:

    self.chunk_id = gobject.idle_add(self.read_chunk().next,
                                     priority=gobject.PRIORITY_LOW)

read_chunk is (as you might expect) a generator function.
Structurally, it looks like this:

    def read_chunk(self):
        count = 0
        while True:
            try:
                x = other_stuff()
            except StopIteration:
                yield False
            else:
                if some other condition holds:
                    yield False
                self.distribute(x)
            count += 1
            if count % 1000 == 0:
                # Let other folks have the CPU...
                yield True

I decided I needed to clean up that idle_add stuff when complete, so
instead of duplicating that bit before both "yield False" statements,
I wrapped the whole mess in a try/finally statement and did my cleanup
in the finally clause.  Thinking the "yield False" might circumvent
the finally clause (silly me), I changed those statements to break
statements.

Everything seemed to work okay for awhile, as long as I was running in
one (historical) mode, where the system only runs from historical data
(read_chunk above is responsible for much of that), exiting when the
historical data were exhausted.  When I switched to live mode
(initialize from historical data, then switch to listen for live
data), the system kept crashing with a StopIteration before switching
to live mode.

I thought I had handled all the StopIteration exceptions, but the
switch from "yield False" to "break" meant another one was raised
which I wasn't catching. Switching back to the yield statement solved
the problem. Took awhile to figure out what was going on.

Skip