Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #11865 > unrolled thread

try... except with unknown error types

Back to article view | Back to comp.lang.python

Contents

  try... except with unknown error types Yingjie Lin <Yingjie.Lin@mssm.edu> - 2011-08-19 15:09 -0400
    Re: try... except with unknown error types John Gordon <gordon@panix.com> - 2011-08-19 19:14 +0000
      Re: try... except with unknown error types Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-20 06:13 +1000
        Re: try... except with unknown error types John Gordon <gordon@panix.com> - 2011-08-19 20:24 +0000
          Re: try... except with unknown error types Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-20 06:45 +1000
            Re: try... except with unknown error types Seebs <usenet-nospam@seebs.net> - 2011-08-20 05:46 +0000
          Re: try... except with unknown error types John Nagle <nagle@animats.com> - 2011-08-20 13:15 -0700
        Re: try... except with unknown error types Paul Rubin <no.email@nospam.invalid> - 2011-08-20 12:18 -0700
          Re: try... except with unknown error types Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-21 06:14 +1000
            Re: try... except with unknown error types Paul Rubin <no.email@nospam.invalid> - 2011-08-21 11:26 -0700
              Re: try... except with unknown error types Chris Angelico <rosuav@gmail.com> - 2011-08-21 20:17 +0100
                Re: try... except with unknown error types Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-22 10:30 +1000
                  Re: try... except with unknown error types Chris Angelico <rosuav@gmail.com> - 2011-08-22 01:41 +0100
                    Re: try... except with unknown error types Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-22 14:01 +1000
                    Re: try... except with unknown error types Paul Rubin <no.email@nospam.invalid> - 2011-08-23 00:21 -0700
                      Re: try... except with unknown error types Chris Angelico <rosuav@gmail.com> - 2011-08-23 09:26 +0100
                      Re: try... except with unknown error types gene heskett <gheskett@wdtv.com> - 2011-08-23 04:43 -0400
                        Re: try... except with unknown error types Paul Rubin <no.email@nospam.invalid> - 2011-08-23 10:43 -0700
                      Re: try... except with unknown error types Chris Angelico <rosuav@gmail.com> - 2011-08-23 10:07 +0100
                  Re: try... except with unknown error types John Nagle <nagle@animats.com> - 2011-08-31 21:22 -0700
              Re: try... except with unknown error types Ethan Furman <ethan@stoneleaf.us> - 2011-08-21 12:22 -0700
              Re: try... except with unknown error types Terry Reedy <tjreedy@udel.edu> - 2011-08-21 15:52 -0400
                Re: try... except with unknown error types Chris Torek <nospam@torek.net> - 2011-08-31 21:01 +0000
                  Re: try... except with unknown error types Stefan Krah <stefan-usenet@bytereef.org> - 2011-09-09 23:41 +0200
                  Re: try... except with unknown error types Nobody <nobody@nowhere.com> - 2011-09-10 10:33 +0100
                  Re: try... except with unknown error types Peter Otten <__peter__@web.de> - 2011-09-10 12:17 +0200
              Re: try... except with unknown error types Roy Smith <roy@panix.com> - 2011-08-21 17:14 -0400
              Re: try... except with unknown error types Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-22 10:25 +1000
                Re: try... except with unknown error types Roy Smith <roy@panix.com> - 2011-08-21 22:09 -0400
              Re: try... except with unknown error types Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-23 18:33 +1000

Page 2 of 2 — ← Prev page 1 [2]

#11956

Paul Rubin wrote:
> Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
>>> But there's no way to know what that minimum is.  Python libraries throw
>>> all sorts of exceptions that their documentation doesn't mention.
>> Yes, you're absolutely correct. But it's also irrelevant. Most of those
>> exceptions should not be caught, even if you know what they are, because
>> they represent either bugs that should be fixed, or bad data which should
>> raise an exception. A bare except, or except Exception, is hardly ever the
>> right approach.
> 
> I'm not sure what to do instead.  The exceptions I'm currently dealing
> with happen when certain network operations go wrong (e.g. network or
> remote host is down, connection fails, etc.)  The remedy in each case is
> to catch the exception, log the error, and try the operation again
> later.  But there's no guaranteed-to-be-complete list in the Python docs
> of all the exceptions that can be thrown.  A new and surprising mode of
> network failure can lead to an unhandled exception, unless you catch
> everything.

In a case like this I can see catching everything so long as (which you 
say you are doing) you log the error somehow -- what's really 
frustrating is when the error is simply tossed with no record 
whatsoever... what a pain to debug!

~Ethan~

[toc] | [prev] | [next] | [standalone]

#11959

On 8/21/2011 2:26 PM, Paul Rubin wrote:
> Steven D'Aprano<steve+comp.lang.python@pearwood.info>  writes:
>>> But there's no way to know what that minimum is.  Python libraries throw
>>> all sorts of exceptions that their documentation doesn't mention.
>>
>> Yes, you're absolutely correct. But it's also irrelevant. Most of those
>> exceptions should not be caught, even if you know what they are, because
>> they represent either bugs that should be fixed, or bad data which should
>> raise an exception. A bare except, or except Exception, is hardly ever the
>> right approach.
>
> I'm not sure what to do instead.  The exceptions I'm currently dealing
> with happen when certain network operations go wrong (e.g. network or
> remote host is down, connection fails, etc.)  The remedy in each case is
> to catch the exception, log the error, and try the operation again
> later.  But there's no guaranteed-to-be-complete list in the Python docs
> of all the exceptions that can be thrown.  A new and surprising mode of
> network failure can lead to an unhandled exception, unless you catch
> everything.

I would expect that catching socket.error (or even IOError) should catch 
all of those.

"exception socket.error
A subclass of IOError, this exception is raised for socket-related 
errors. It is recommended that you inspect its errno attribute to 
discriminate between different kinds of errors."

> It's a retail application that would cause some business disruption and
> a pissed off customer if the program went down.  Also it's in an
> embedded box on a customer site.  It's not in Antarctica or anything
> like that, but it's a few towns over, and someone would have to drive
> there (probably through heavy traffic) if something went wrong that
> power cycling the box couldn't fix.


-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#12525

In article <mailman.286.1313956388.27778.python-list@python.org>,
Terry Reedy  <tjreedy@udel.edu> wrote:
>I would expect that catching socket.error (or even IOError) should catch 
>all of those.
>
>"exception socket.error
>A subclass of IOError ...

Except that, as Steven D'Aprano almost noted elsethread, it isn't
(a subclass of IOError -- the note was that it is not a subclass
of EnvironmentError).  In 2.x anyway:

    >>> import socket
    >>> isinstance(socket.error, IOError)
    False
    >>> isinstance(socket.error, EnvironmentError)
    False
    >>> 

(I just catch socket.error directly for this case.)

(I have also never been sure whether something is going to raise
an IOError or an OSError for various OS-related read or write
operation failures -- such as exceeding a resource limit, for
instance -- so most places that do I/O operations on OS files, I
catch both.  Still, it sure would be nice to have a static analysis
tool that could answer questions about potential exceptions. :-) )
-- 
In-Real-Life: Chris Torek, Wind River Systems
Intel require I note that my opinions are not those of WRS or Intel
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W)  +1 801 277 2603
email: gmail (figure it out)      http://web.torek.net/torek/index.html

[toc] | [prev] | [next] | [standalone]

#13036

Chris Torek <nospam@torek.net> wrote:
> (I have also never been sure whether something is going to raise
> an IOError or an OSError for various OS-related read or write
> operation failures -- such as exceeding a resource limit, for
> instance -- so most places that do I/O operations on OS files, I
> catch both.  Still, it sure would be nice to have a static analysis
> tool that could answer questions about potential exceptions. :-) )

There is an effort to fix this:

http://www.python.org/dev/peps/pep-3151/


And an implementation ...

http://hg.python.org/features/pep-3151/


... together with a feature request:

http://bugs.python.org/issue12555


I think the whole concept makes a lot of sense and is really worth
taking a look at.


Stefan Krah

[toc] | [prev] | [next] | [standalone]

#13057

On Wed, 31 Aug 2011 21:01:34 +0000, Chris Torek wrote:

> Still, it sure would be nice to have a static analysis
> tool that could answer questions about potential exceptions. :-) )

That's an impossibility in a dynamic language.

If you call f.read() where f was passed in as a parameter, the exceptions
which f.read() may throw depend upon exactly what f is; there's no
guarantee that it will actually be a built-in file object, only that it
will have a .read() method (otherwise you will get an AttributeError).

Even if f was returned from open(), there's no guarantee that
__builtins__.open() won't have been replaced by the time of the call.

[toc] | [prev] | [next] | [standalone]

#13060

Chris Torek wrote:

> >>> import socket
> >>> isinstance(socket.error, IOError)
> False

Here you test if the socket.error *class* is an instance of IOError; this 
would print True if IOError were socket.error's metaclass. However:

>>> isinstance(socket.error(), IOError)
True

or more directly:

>>> issubclass(socket.error, IOError)
True
>>> issubclass(socket.error, EnvironmentError)
True

This is a relatively recent change:

$ python2.5 -c'from socket import error; print issubclass(error, IOError), 
issubclass(error, EnvironmentError)'
False False
$ python2.6 -c'from socket import error; print issubclass(error, IOError), 
issubclass(error, EnvironmentError)'
True True

[toc] | [prev] | [next] | [standalone]

#11965

In article <7xty9ahb84.fsf@ruckus.brouhaha.com>,
 Paul Rubin <no.email@nospam.invalid> wrote:

> It's a retail application that would cause some business disruption and
> a pissed off customer if the program went down.  Also it's in an
> embedded box on a customer site.  It's not in Antarctica or anything
> like that, but it's a few towns over, and someone would have to drive
> there (probably through heavy traffic) if something went wrong that
> power cycling the box couldn't fix.

I would do something like this:

try:
   do_some_network_stuff()
except IOError:
   do_normal_recovery()
except Exception:
   call_home()
   do_some_other_recovery()

You are right, in an embedded/unattended application, you don't want to 
ever crash.  If you ever get an error that you don't understand, you 
have no choice but to do the best you can to recover.  I would also 
generate some kind of alert back to home base to tell somebody to take a 
look at it and make sure things are fine.

I would expect that the whole application is wrapped in some kind of 
watchdog timer which will do a hard reset of the entire system.

[toc] | [prev] | [next] | [standalone]

#11975

Paul Rubin wrote:

> Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
>>> But there's no way to know what that minimum is.  Python libraries throw
>>> all sorts of exceptions that their documentation doesn't mention.
>>
>> Yes, you're absolutely correct. But it's also irrelevant. Most of those
>> exceptions should not be caught, even if you know what they are, because
>> they represent either bugs that should be fixed, or bad data which should
>> raise an exception. A bare except, or except Exception, is hardly ever
>> the right approach.
> 
> I'm not sure what to do instead.  The exceptions I'm currently dealing
> with happen when certain network operations go wrong (e.g. network or
> remote host is down, connection fails, etc.)  The remedy in each case is
> to catch the exception, log the error, and try the operation again
> later.  But there's no guaranteed-to-be-complete list in the Python docs
> of all the exceptions that can be thrown.  A new and surprising mode of
> network failure can lead to an unhandled exception, unless you catch
> everything.

I was waiting for you to raise network errors :)

Network errors are a particularly annoying case, because although rare and
undocumented, they are legitimate errors that should be caught. I feel your
pain.


> The Erlang approach is tempting.  Don't catch the exception at all--just
> let the process crash, and restart it.  But that's a more heavyweight
> operation in Python.

The Erland approach sounds good, but as I've never used it, I don't know how
well it works in practice.


>> After all, unless you're writing
>> software for a nuclear reactor, or an aeroplane's autopilot, chances are
>> that *bugs don't really matter*. That is to say, if you release software
>> with a hidden bug, the consequences generally aren't very important.
> 
> It's a retail application that would cause some business disruption and
> a pissed off customer if the program went down.  Also it's in an
> embedded box on a customer site.  It's not in Antarctica or anything
> like that, but it's a few towns over, and someone would have to drive
> there (probably through heavy traffic) if something went wrong that
> power cycling the box couldn't fix.

Customers are always pissed off when something goes wrong, but ask them to
pay an extra $300 for a battery backup unit to the hardware RAID controller
*they* insisted on against your advice, and they say no. But I'm not
bitter...

Customer gets pissed off. What's the consequences? Do they sue you? Leave?
Are they locked into a five year contract and have to pay you even if they
do leave? Do you have to fix the incident at no charge, or give them two
hours free support? Are you competing with armour-plated mature software
that never goes down, or is the field yours alone? How price sensitive are
your customers? Will they pay an extra $100,000 for an extra 9 in the
expected uptime?

Without knowing the consequences to *you* of failure, I can't tell you where
you should be spending your time: trying to predict errors ahead of time,
or fixing them once they've been seen. This is a business problem, not a
software problem.

Don't misunderstand me, I'm not suggesting that you shouldn't try to avoid
things going wrong. But it's a hard problem. Solving it is why they pay you
the big bucks *cough*:

http://www.joelonsoftware.com/items/2007/12/06.html

and is the difference between version 0.4 of an application, and version
3.4. If you're trying to go straight to version 3.4 without putting the
software through real-world testing, then your testing better be mean and
tough. *Really* tough.

http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#11985

In article <4e51a205$0$29974$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html

I *love* being the Chaos Monkey!

A few jobs ago, I had already turned in my resignation and was a 
short-timer, counting down the days to when I was out of there.  A new 
server had just come in and I was helping the guy who was going to 
replace me set it up.

This was the first server we had gotten with RAID and redundant power 
supplies and I wanted to test them out.  My accomplice was horrified, 
but technically I was still his boss, so I got to do what I wanted.  I 
opened up the drive shelve and yanked out a drive.  The console printed 
a nice neat warning about loss of media and kept chugging along, just 
like it should.

Then I pulled out a second drive.  Sure enough, the whole array failed, 
just like expected.

Then, I decided to have a go at the redundant power supplies.  We paid a 
lot of money for that, and I damn well was going to test them now, when 
there was no real risk.  I grabbed the big handle on the front of one of 
the (hot swappable) power supplies and pulled.  The whole thing went 
dark.  Turns out there had been an configuration error and the second 
power supply had never been installed (just a faceplate that looked like 
a power supply).  My buddy was pissed, but I figure I'd just done him a 
big favor.  Better to find out about it now than when the supply failed 
at some critical juncture.

[toc] | [prev] | [next] | [standalone]

#12069

On Mon, 22 Aug 2011 04:26 am Paul Rubin wrote:

> The Erlang approach is tempting.  Don't catch the exception at all--just
> let the process crash, and restart it.  But that's a more heavyweight
> operation in Python.

You might be interested in this paper:

http://usenix.org/events/hotos03/tech/full_papers/candea/candea.pdf


-- 
Steven

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web