Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #52926 > unrolled thread

Exception Handling Practices / Patterns

Back to article view | Back to comp.lang.python

Contents

  Exception Handling Practices / Patterns snarf <frank.ruiz@gmail.com> - 2013-08-23 22:25 -0700
    Re: Exception Handling Practices / Patterns Dave Angel <davea@davea.name> - 2013-08-24 07:18 +0000
      Re: Exception Handling Practices / Patterns frank.ruiz@gmail.com - 2013-08-24 16:30 -0700
    Re: Exception Handling Practices / Patterns Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-08-24 10:27 +0000
      Re: Exception Handling Practices / Patterns MRAB <python@mrabarnett.plus.com> - 2013-08-24 17:32 +0100
      Re: Exception Handling Practices / Patterns Terry Reedy <tjreedy@udel.edu> - 2013-08-24 15:57 -0400
        Re: Exception Handling Practices / Patterns Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-08-24 23:11 +0000
      Re: Exception Handling Practices / Patterns frank.ruiz@gmail.com - 2013-08-24 16:32 -0700

#52926 — Exception Handling Practices / Patterns

Greetings,

As I tread through my journey of OO I am trying to determine if there is a good approach for exception handling within classes.

From my readings and gatherings - it seems I have found a common theme, but I am trying to solicit from the experts.

Here is what I have found (I may be restating the obvious so please forgive me in advance):

* Seems like exception handing within Classes is largely avoided and is typically only used when calling external libraries.
* Try/Except type statements seem to be used more within modules, main functions, wrapper scripts.
* Classes should be coded in a way that exceptions 
* Better to never write your own exceptions (unless you absolutely have to).
* Using Exception is typically a bad. More specific the better.
* Exceptions should never fail silently. (Should exceptions always be logged?)

Best site I have found for exceptions (hopefully this helps someone):
* http://c2.com/cgi/wiki?ExceptionPatterns


I'd be interested in hearing others thoughts on this topic with regards to best practices for when to use exceptions, and when to avoid using exceptions.

Thank you in advance!

[toc] | [next] | [standalone]

#52929

snarf wrote:

> Greetings,
>
> As I tread through my journey of OO I am trying to determine if there is a good approach for exception handling within classes.
>
> From my readings and gatherings - it seems I have found a common theme, but I am trying to solicit from the experts.
>
> Here is what I have found (I may be restating the obvious so please forgive me in advance):
>
> * Seems like exception handing within Classes is largely avoided and is typically only used when calling external libraries.
> * Try/Except type statements seem to be used more within modules, main functions, wrapper scripts.

Exceptions are used when useful.  I don't see any bias towards any one
location.

> * Classes should be coded in a way that exceptions 

You seem to be missing the last part of this sentence.

> * Better to never write your own exceptions (unless you absolutely have to).

If you mean to avoid writing exception classes, then I say nonsense. 
Just derive them from the closest meaningful exception class, so that a
user can combine handlers when reasonable.

> * Using Exception is typically a bad. More specific the better.

If you mean in an except statement, then I'd agree.

> * Exceptions should never fail silently. (Should exceptions always be logged?)

Exceptions should be caught if you can handle them, or if you need to
convert them to a different exception that someone further up the stack
can handle.  Sometimes handling means do nothing.

>
> Best site I have found for exceptions (hopefully this helps someone):
> * http://c2.com/cgi/wiki?ExceptionPatterns

But that's for Java.  java is not C++, and neither is it Python.  For
one thing, Python exception overhead is deliberately much less, and they
are used more freely.

Notice that exceptions are used to terminate for loops, and that's a
*normal* exit to the loop.  They also appear in other places under the
covers.  Don't be afraid of them.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#52955

Appreciate the feedback. I was hoping to get as much perspective as possible. 

On Saturday, August 24, 2013 12:18:59 AM UTC-7, Dave Angel wrote:
> snarf wrote:
> 
> 
> 
> > Greetings,
> 
> >
> 
> > As I tread through my journey of OO I am trying to determine if there is a good approach for exception handling within classes.
> 
> >
> 
> > From my readings and gatherings - it seems I have found a common theme, but I am trying to solicit from the experts.
> 
> >
> 
> > Here is what I have found (I may be restating the obvious so please forgive me in advance):
> 
> >
> 
> > * Seems like exception handing within Classes is largely avoided and is typically only used when calling external libraries.
> 
> > * Try/Except type statements seem to be used more within modules, main functions, wrapper scripts.
> 
> 
> 
> Exceptions are used when useful.  I don't see any bias towards any one
> 
> location.
> 
> 
> 
> > * Classes should be coded in a way that exceptions 
> 
> 
> 
> You seem to be missing the last part of this sentence.
> 
> 
> 
> > * Better to never write your own exceptions (unless you absolutely have to).
> 
> 
> 
> If you mean to avoid writing exception classes, then I say nonsense. 
> 
> Just derive them from the closest meaningful exception class, so that a
> 
> user can combine handlers when reasonable.
> 
> 
> 
> > * Using Exception is typically a bad. More specific the better.
> 
> 
> 
> If you mean in an except statement, then I'd agree.
> 
> 
> 
> > * Exceptions should never fail silently. (Should exceptions always be logged?)
> 
> 
> 
> Exceptions should be caught if you can handle them, or if you need to
> 
> convert them to a different exception that someone further up the stack
> 
> can handle.  Sometimes handling means do nothing.
> 
> 
> 
> >
> 
> > Best site I have found for exceptions (hopefully this helps someone):
> 
> > * http://c2.com/cgi/wiki?ExceptionPatterns
> 
> 
> 
> But that's for Java.  java is not C++, and neither is it Python.  For
> 
> one thing, Python exception overhead is deliberately much less, and they
> 
> are used more freely.
> 
> 
> 
> Notice that exceptions are used to terminate for loops, and that's a
> 
> *normal* exit to the loop.  They also appear in other places under the
> 
> covers.  Don't be afraid of them.
> 
> 
> 
> -- 
> 
> DaveA

[toc] | [prev] | [next] | [standalone]

#52933

On Fri, 23 Aug 2013 22:25:55 -0700, snarf wrote:

[...]
> * Seems like exception handing within Classes is largely avoided and is
> typically only used when calling external libraries. 

There is certainly no rule "avoid exceptions inside classes". Methods 
often raise exceptions to signal an error, e.g.:

"My string".index("spam")


Less common, methods can raise exceptions as part of their flow control. 
The most obvious example is StopIteration, used by iterators and often by 
__iter__ or next methods.


> * Try/Except type
> statements seem to be used more within modules, main functions, wrapper
> scripts. 

It depends on whose code you are reading. I don't write a lot of classes, 
but when I do, I often use try...except inside them.

If try...except gets used more frequently in module's top level, it is 
because the sort of things that you do at the top level often needs 
exception handling. For example, you might have a fallback module:

try:
    import this_module
except ImportError:
    import that_module as this_module


You will very rarely see that inside a class, since you very rarely 
import modules inside a class.


> * Classes should be coded in a way that exceptions 

I think you forgot to finish the sentence.


> * Better to
> never write your own exceptions (unless you absolutely have to). 

That depends.

On the one hand, nobody wants a million different exception types. On the 
other hand, nobody wants just *one* exception type, and no way to 
distinguish between different kinds of errors. Somewhere between one and 
one million is an appropriate number of exception types.

The right answer is to be conservative about creating new exceptions, but 
don't be scared to create one when you need one.

But when you do, it is often better to subclass from an appropriate built-
in exception like ValueError or TypeError or similar, than to subclass 
from Exception itself.


> * Using
> Exception is typically a bad. More specific the better.

Yes, you should always try to catch the specific exceptions you care 
about:


# Best
except ValueError, OverflowError, ZeroDivisionError:


# Not so good
except Exception:


# Worst
except:


Don't use the last one, except maybe in the interactive interpreter, 
since it will catch *everything*, even exceptions that probably shouldn't 
be caught like KeyboardInterrupt.


> * Exceptions
> should never fail silently. (Should exceptions always be logged?)

Certainly not. Exceptions should fail silently if you don't care about 
them. For example, when connecting to a website, there are many temporary 
errors that can occur. The best way to handle them is to catch the 
exception, sleep for a little while, then try again. You need only care 
if repeated attempts to connect, with larger and larger sleeps, continue 
to fail.

Of course, you might have a debug mode that logs all of these, but if 
your web browser logged every single time a webpage was slow to respond, 
you would soon run out of disk space :-)

*Errors* should never fail silently, unless explicitly silenced. But an 
error isn't an error if you don't care about it, and an exception is not 
necessarily an error.

This is an error, because converting a number to uppercase cannot 
possibly mean anything:

mystring = 42
mystring.upper()


This is not necessarily an error, since "the list is empty" could be a 
legitimate situation:

mylist = []
first = mylist[0]

In this case, it may be appropriate to catch the exception, and either 
silently swallow it, or do something else.


> Best site I have found for exceptions (hopefully this helps someone): *
> http://c2.com/cgi/wiki?ExceptionPatterns

I haven't read that page for a long time, but as I recall the c2.com 
website, a lot of the ideas there are better suited to Java and C/C++ 
(and occasionally Lisp) rather than Python. But still, a valuable (if 
often confusing) resource.



> I'd be interested in hearing others thoughts on this topic with regards
> to best practices for when to use exceptions, and when to avoid using
> exceptions.

The try part of a try...except is *very* fast to set up. It's about as 
fast as a "pass" (do nothing), so it has little overhead.

On the other hand, actually *catching* an exception is quite heavy. So 
code that catches lots and lots of exceptions may be slow. In that case, 
it may be faster to "look before you leap" and test ahead of time:

# this is faster if most lists are not empty
try:
    process(mylist[0])
except IndexError:
    handle_empty_list()

# this is faster if many lists are empty
if mylist:
    process(mylist[0])
else:
    handle_empty_list()


Only profiling your data can tell you which you should use.


On the other hand, here you should *always* use try...except:

try:
    myfile = open("name")
except IOError:
    handle_error()


Because this code is wrong:

if os.path.exists("name"):
    myfile = open("name")
else:
    handle_error()


It's wrong for a couple of reasons:

- just because the file exists, doesn't mean you can open it;

- even if the file exists when you call the os.path.exists function, 
doesn't mean it will still exist a millisecond later when you try to open 
it.



Hope this is helpful,




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#52942

On 24/08/2013 11:27, Steven D'Aprano wrote:
> On Fri, 23 Aug 2013 22:25:55 -0700, snarf wrote:
[snip]
>> * Using
>> Exception is typically a bad. More specific the better.
>
> Yes, you should always try to catch the specific exceptions you care
> about:
>
>
> # Best
> except ValueError, OverflowError, ZeroDivisionError:
>
That should be:

except (ValueError, OverflowError, ZeroDivisionError):

>
> # Not so good
> except Exception:
>
>
> # Worst
> except:
>
[snip]

[toc] | [prev] | [next] | [standalone]

#52948

On 8/24/2013 6:27 AM, Steven D'Aprano wrote:
> On Fri, 23 Aug 2013 22:25:55 -0700, snarf wrote:
>
> [...]
>> * Seems like exception handing within Classes is largely avoided and is
>> typically only used when calling external libraries.
>
> There is certainly no rule "avoid exceptions inside classes". Methods
> often raise exceptions to signal an error, e.g.:
>
> "My string".index("spam")

The rule only makes sense if it referring to try: except: in top-level 
class code, outside of def statements. Even then, it is wrong.

...
> # Worst
> except:
>
> Don't use the last one, except maybe in the interactive interpreter,

Stick with never. "except:" means the same thing as "except 
BaseException:", except that the latter indicates a deliberate choice 
rather than an indication of carelessness or laziness.

A bare except: is a disservice to the next maintainer of the code.

> since it will catch *everything*, even exceptions that probably shouldn't
> be caught like KeyboardInterrupt.

In Idle, when you type 'expression(' and hesitate, Idle tries to 
evaluate 'expression' to a function, behind the scenes, in order to 
provide a calltip with the function signature. Any error in 
'eval(expression)' should be caught and ignored so the user can continue 
typing.

This is one place where the original authors were too specific. They 
only caught NameError and AttributeError, when those were not the only 
possible eval errors in practice. The result was that people would 
occasionally type '(' and see idle quit.

idlelib .py files have about 20 bare 'except:'s, which I will try to 
fill out when I have reviewed the try part and understand what should be 
caught. It would be easier for me to read the code if the original 
authors had added their best guess as to what should be expected and caught.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#52954

On Sat, 24 Aug 2013 15:57:55 -0400, Terry Reedy wrote:

>> # Worst
>> except:
>>
>> Don't use the last one, except maybe in the interactive interpreter,
> 
> Stick with never. "except:" means the same thing as "except
> BaseException:", except that the latter indicates a deliberate choice
> rather than an indication of carelessness or laziness.
> 
> A bare except: is a disservice to the next maintainer of the code.

Do you know anyone who maintains code typed in the interactive 
interpreter?

:-)



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#52956

Hi Steven,

Yea this is great. Thanks for the feedback.

On Saturday, August 24, 2013 3:27:45 AM UTC-7, Steven D'Aprano wrote:
> On Fri, 23 Aug 2013 22:25:55 -0700, snarf wrote:
> 
> 
> 
> [...]
> 
> > * Seems like exception handing within Classes is largely avoided and is
> 
> > typically only used when calling external libraries. 
> 
> 
> 
> There is certainly no rule "avoid exceptions inside classes". Methods 
> 
> often raise exceptions to signal an error, e.g.:
> 
> 
> 
> "My string".index("spam")
> 
> 
> 
> 
> 
> Less common, methods can raise exceptions as part of their flow control. 
> 
> The most obvious example is StopIteration, used by iterators and often by 
> 
> __iter__ or next methods.
> 
> 
> 
> 
> 
> > * Try/Except type
> 
> > statements seem to be used more within modules, main functions, wrapper
> 
> > scripts. 
> 
> 
> 
> It depends on whose code you are reading. I don't write a lot of classes, 
> 
> but when I do, I often use try...except inside them.
> 
> 
> 
> If try...except gets used more frequently in module's top level, it is 
> 
> because the sort of things that you do at the top level often needs 
> 
> exception handling. For example, you might have a fallback module:
> 
> 
> 
> try:
> 
>     import this_module
> 
> except ImportError:
> 
>     import that_module as this_module
> 
> 
> 
> 
> 
> You will very rarely see that inside a class, since you very rarely 
> 
> import modules inside a class.
> 
> 
> 
> 
> 
> > * Classes should be coded in a way that exceptions 
> 
> 
> 
> I think you forgot to finish the sentence.
> 
> 
> 
> 
> 
> > * Better to
> 
> > never write your own exceptions (unless you absolutely have to). 
> 
> 
> 
> That depends.
> 
> 
> 
> On the one hand, nobody wants a million different exception types. On the 
> 
> other hand, nobody wants just *one* exception type, and no way to 
> 
> distinguish between different kinds of errors. Somewhere between one and 
> 
> one million is an appropriate number of exception types.
> 
> 
> 
> The right answer is to be conservative about creating new exceptions, but 
> 
> don't be scared to create one when you need one.
> 
> 
> 
> But when you do, it is often better to subclass from an appropriate built-
> 
> in exception like ValueError or TypeError or similar, than to subclass 
> 
> from Exception itself.
> 
> 
> 
> 
> 
> > * Using
> 
> > Exception is typically a bad. More specific the better.
> 
> 
> 
> Yes, you should always try to catch the specific exceptions you care 
> 
> about:
> 
> 
> 
> 
> 
> # Best
> 
> except ValueError, OverflowError, ZeroDivisionError:
> 
> 
> 
> 
> 
> # Not so good
> 
> except Exception:
> 
> 
> 
> 
> 
> # Worst
> 
> except:
> 
> 
> 
> 
> 
> Don't use the last one, except maybe in the interactive interpreter, 
> 
> since it will catch *everything*, even exceptions that probably shouldn't 
> 
> be caught like KeyboardInterrupt.
> 
> 
> 
> 
> 
> > * Exceptions
> 
> > should never fail silently. (Should exceptions always be logged?)
> 
> 
> 
> Certainly not. Exceptions should fail silently if you don't care about 
> 
> them. For example, when connecting to a website, there are many temporary 
> 
> errors that can occur. The best way to handle them is to catch the 
> 
> exception, sleep for a little while, then try again. You need only care 
> 
> if repeated attempts to connect, with larger and larger sleeps, continue 
> 
> to fail.
> 
> 
> 
> Of course, you might have a debug mode that logs all of these, but if 
> 
> your web browser logged every single time a webpage was slow to respond, 
> 
> you would soon run out of disk space :-)
> 
> 
> 
> *Errors* should never fail silently, unless explicitly silenced. But an 
> 
> error isn't an error if you don't care about it, and an exception is not 
> 
> necessarily an error.
> 
> 
> 
> This is an error, because converting a number to uppercase cannot 
> 
> possibly mean anything:
> 
> 
> 
> mystring = 42
> 
> mystring.upper()
> 
> 
> 
> 
> 
> This is not necessarily an error, since "the list is empty" could be a 
> 
> legitimate situation:
> 
> 
> 
> mylist = []
> 
> first = mylist[0]
> 
> 
> 
> In this case, it may be appropriate to catch the exception, and either 
> 
> silently swallow it, or do something else.
> 
> 
> 
> 
> 
> > Best site I have found for exceptions (hopefully this helps someone): *
> 
> > http://c2.com/cgi/wiki?ExceptionPatterns
> 
> 
> 
> I haven't read that page for a long time, but as I recall the c2.com 
> 
> website, a lot of the ideas there are better suited to Java and C/C++ 
> 
> (and occasionally Lisp) rather than Python. But still, a valuable (if 
> 
> often confusing) resource.
> 
> 
> 
> 
> 
> 
> 
> > I'd be interested in hearing others thoughts on this topic with regards
> 
> > to best practices for when to use exceptions, and when to avoid using
> 
> > exceptions.
> 
> 
> 
> The try part of a try...except is *very* fast to set up. It's about as 
> 
> fast as a "pass" (do nothing), so it has little overhead.
> 
> 
> 
> On the other hand, actually *catching* an exception is quite heavy. So 
> 
> code that catches lots and lots of exceptions may be slow. In that case, 
> 
> it may be faster to "look before you leap" and test ahead of time:
> 
> 
> 
> # this is faster if most lists are not empty
> 
> try:
> 
>     process(mylist[0])
> 
> except IndexError:
> 
>     handle_empty_list()
> 
> 
> 
> # this is faster if many lists are empty
> 
> if mylist:
> 
>     process(mylist[0])
> 
> else:
> 
>     handle_empty_list()
> 
> 
> 
> 
> 
> Only profiling your data can tell you which you should use.
> 
> 
> 
> 
> 
> On the other hand, here you should *always* use try...except:
> 
> 
> 
> try:
> 
>     myfile = open("name")
> 
> except IOError:
> 
>     handle_error()
> 
> 
> 
> 
> 
> Because this code is wrong:
> 
> 
> 
> if os.path.exists("name"):
> 
>     myfile = open("name")
> 
> else:
> 
>     handle_error()
> 
> 
> 
> 
> 
> It's wrong for a couple of reasons:
> 
> 
> 
> - just because the file exists, doesn't mean you can open it;
> 
> 
> 
> - even if the file exists when you call the os.path.exists function, 
> 
> doesn't mean it will still exist a millisecond later when you try to open 
> 
> it.
> 
> 
> 
> 
> 
> 
> 
> Hope this is helpful,
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web