Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #95096 > unrolled thread

except block isn't catching exception

Back to article view | Back to comp.lang.python

Contents

  except block isn't catching exception sohcahtoa82@gmail.com - 2015-08-06 17:34 -0700
    Re: except block isn't catching exception Chris Angelico <rosuav@gmail.com> - 2015-08-07 10:46 +1000
      Re: except block isn't catching exception sohcahtoa82@gmail.com - 2015-08-07 10:16 -0700
        Re: except block isn't catching exception Chris Angelico <rosuav@gmail.com> - 2015-08-08 12:44 +1000
        Re: except block isn't catching exception Ian Kelly <ian.g.kelly@gmail.com> - 2015-08-08 00:56 -0600
        Re: except block isn't catching exception Chris Angelico <rosuav@gmail.com> - 2015-08-08 17:08 +1000

#95096 — except block isn't catching exception

I've run into strange behavior involving a blocking call to a socket accept() on the main thread and thread.interrupt_main() being called on a worker thread.  Here's my code:

# BEGIN exception_test.py
import socket
import thread
import threading
import time


def worker():
    time.sleep(2)
    print 'Interrupting main'
    thread.interrupt_main()
    print 'main interrupted!'

sock = socket.socket()
sock.bind(('127.0.0.1', 8080))
sock.settimeout(5)
sock.listen(0)

t = threading.Thread(target=worker)
t.start()

try:
    connection, _ = sock.accept()
except KeyboardInterrupt:
    print 'KeyboardInterrupt!'
except socket.timeout:
    print 'Socket timeout!'

print 'exiting'

# END exception_test.py

I would expect this output:

Interrupting main
main interrupted!
KeyboardInterrupt caught!
exiting

But instead, I'm seeing this:

Interrupting main
main interrupted!
Traceback (most recent call last):
  File "exception_test.py", line 23, in <module>
    connection, _ = sock.accept()
KeyboardInterrupt

Despite my "except KeyboardInterrupt", the KeyboardInterrupt forced by the thread.interrupt_main() in the worker thread isn't being caught.

Other things worth noting is that the exception takes about 3 seconds after the call to thread.interrupt_main().  It appears to not actually happen until the sock.accept() times out.  If the call to sock.settimeout(5) is removed, the KeyboardInterrupt never appears and flow is still blocked on the sock.accept().

My Python version is 2.7.3.  I know its out of date, but I don't really have a choice.

[toc] | [next] | [standalone]

#95097

On Fri, Aug 7, 2015 at 10:34 AM,  <sohcahtoa82@gmail.com> wrote:
> Despite my "except KeyboardInterrupt", the KeyboardInterrupt forced by the thread.interrupt_main() in the worker thread isn't being caught.
>
> Other things worth noting is that the exception takes about 3 seconds after the call to thread.interrupt_main().  It appears to not actually happen until the sock.accept() times out.  If the call to sock.settimeout(5) is removed, the KeyboardInterrupt never appears and flow is still blocked on the sock.accept().
>
> My Python version is 2.7.3.  I know its out of date, but I don't really have a choice.

As far as I know, the only effect of thread.interrupt_main() is to set
a flag saying "Hey main thread, next time you go checking for these
things, there's a KeyboardInterrupt waiting for you". It doesn't
actually send an OS-level signal, which means it cannot interrupt a
blocking call. So you have a few choices:

1) Shorten the timeout on sock.accept() to an (if you'll excuse the
pun) acceptable delay, and then simply check a flag. Something like
this:

interrupt = False
while not interrupt:
    try:
        connection, _ = sock.accept()
    except socket.timeout:
        # On timeout, check flag and keep sleeping
        pass

If you wish to also have an actual timeout, you could monitor that separately.

2) Instead of blocking on sock.accept directly, have another event
which the other thread can raise, which will wake the main thread. You
could use select.select() for this.

3) Bury the details of select.select() away behind a nice framework
like asyncio, merge your two threads, and run everything through a
back-end event loop.

4) Use separate processes, and signal the interrupt using an OS-level
notification (on Unix systems, SIGINT; on Windows, there's an
equivalent called BreakSignal or something). This will break out of
the underlying system call that handles accept().

thread.interrupt_main() is similar to just setting a global that you
periodically check, except that the periodic check is handled for you
by the system. That's really all.

ChrisA

[toc] | [prev] | [next] | [standalone]

#95133

On Thursday, August 6, 2015 at 5:46:19 PM UTC-7, Chris Angelico wrote:
> On Fri, Aug 7, 2015 at 10:34 AM,  <sohcahtoa82@gmail.com> wrote:
> > Despite my "except KeyboardInterrupt", the KeyboardInterrupt forced by the thread.interrupt_main() in the worker thread isn't being caught.
> >
> > Other things worth noting is that the exception takes about 3 seconds after the call to thread.interrupt_main().  It appears to not actually happen until the sock.accept() times out.  If the call to sock.settimeout(5) is removed, the KeyboardInterrupt never appears and flow is still blocked on the sock.accept().
> >
> > My Python version is 2.7.3.  I know its out of date, but I don't really have a choice.
> 
> As far as I know, the only effect of thread.interrupt_main() is to set
> a flag saying "Hey main thread, next time you go checking for these
> things, there's a KeyboardInterrupt waiting for you". It doesn't
> actually send an OS-level signal, which means it cannot interrupt a
> blocking call. So you have a few choices:
> 
> 1) Shorten the timeout on sock.accept() to an (if you'll excuse the
> pun) acceptable delay, and then simply check a flag. Something like
> this:
> 
> interrupt = False
> while not interrupt:
>     try:
>         connection, _ = sock.accept()
>     except socket.timeout:
>         # On timeout, check flag and keep sleeping
>         pass
> 
> If you wish to also have an actual timeout, you could monitor that separately.
> 
> 2) Instead of blocking on sock.accept directly, have another event
> which the other thread can raise, which will wake the main thread. You
> could use select.select() for this.
> 
> 3) Bury the details of select.select() away behind a nice framework
> like asyncio, merge your two threads, and run everything through a
> back-end event loop.
> 
> 4) Use separate processes, and signal the interrupt using an OS-level
> notification (on Unix systems, SIGINT; on Windows, there's an
> equivalent called BreakSignal or something). This will break out of
> the underlying system call that handles accept().
> 
> thread.interrupt_main() is similar to just setting a global that you
> periodically check, except that the periodic check is handled for you
> by the system. That's really all.
> 
> ChrisA

I'll probably go for the first solution.  It's the simplest, and simple is usually good.  In my application, there aren't going to be any consequences for the main thread not unblocking *immediately* when a worker thread requests it.

Though I still doesn't understand why the exception isn't caught when I'm explicitly trying to catch it.  I even tried changing the try/except block to this:

try:
    connection, _ = sock.accept()
except KeyboardInterrupt:
    print 'KeyboardInterrupt caught!'
except socket.timeout:
    print 'Socket timeout caught!'
except:
    print 'other exception caught!'
finally:
    print 'finally!'

The result prints:

Interrupting main
main interrupted!
finally!
Traceback (most recent call last): 
  File "exception_test.py", line 23, in <module> 
    connection, _ = sock.accept() 
KeyboardInterrupt

[toc] | [prev] | [next] | [standalone]

#95154

On Sat, Aug 8, 2015 at 3:16 AM,  <sohcahtoa82@gmail.com> wrote:
>
> Though I still doesn't understand why the exception isn't caught when I'm explicitly trying to catch it.  I even tried changing the try/except block to this:
>
> try:
>     connection, _ = sock.accept()
> except KeyboardInterrupt:
>     print 'KeyboardInterrupt caught!'
> except socket.timeout:
>     print 'Socket timeout caught!'
> except:
>     print 'other exception caught!'
> finally:
>     print 'finally!'
>
> The result prints:
>
> Interrupting main
> main interrupted!
> finally!

The exception isn't happening inside sock.accept(), as I explained. So
you can't catch it there.

ChrisA

[toc] | [prev] | [next] | [standalone]

#95162

On Fri, Aug 7, 2015 at 8:44 PM, Chris Angelico <rosuav@gmail.com> wrote:
> The exception isn't happening inside sock.accept(), as I explained. So
> you can't catch it there.

Where does the exception happen then? Your explanation only covered
why the blocking call cannot be interrupted by it, not why the
exception isn't simply raised when the blocking call finishes.

I played around with this and found that if the try...except chain is
wrapped in another outer try statement, then the KeyboardInterrupt
exception does get caught by the outer exception handler as one might
expect. For the inner try statement though, neither any except block
nor the else block is executed, just the finally block. I didn't know
until now that was even possible.

The language reference mentions this in regard to the try statement:

"""
If the evaluation of an expression in the header of an except clause
raises an exception, the original search for a handler is canceled and
a search starts for the new exception in the surrounding code and on
the call stack (it is treated as if the entire try statement raised
the exception).
"""

So my theory as to what's going on here is that sock.accept raises a
socket.timeout exception, but then the KeyboardInterrupt is raised
before the first except block even begins to check the exception type,
and so it's treated as if the "entire try statement" raised
KeyboardInterrupt. Hence it can't be caught there, but only in an
outer try statement.

Whatever the reason, it definitely seems to be an interesting corner case.

[toc] | [prev] | [next] | [standalone]

#95163

On Sat, Aug 8, 2015 at 4:56 PM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Fri, Aug 7, 2015 at 8:44 PM, Chris Angelico <rosuav@gmail.com> wrote:
>> The exception isn't happening inside sock.accept(), as I explained. So
>> you can't catch it there.
>
> Where does the exception happen then? Your explanation only covered
> why the blocking call cannot be interrupted by it, not why the
> exception isn't simply raised when the blocking call finishes.
>

I'm not sure there's anything in the language spec about it; at least,
I can't find it. But the last time I was digging in the Python/C API,
there was a caveat that KeyboardInterrupt was raised *at some point
after* Ctrl-C was hit - a flag gets set, and after every N bytecode
instructions, the flag is checked, and then the signal gets raised. It
might happen on only some platforms, and I can't even find back the
page I was looking at when I read that. Maybe someone else knows?

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web