Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #40344 > unrolled thread

Dealing with exceptions

Back to article view | Back to comp.lang.python

Contents

  Dealing with exceptions bvdp <bob@mellowood.ca> - 2013-03-02 09:40 -0800
    Re: Dealing with exceptions Kwpolska <kwpolska@gmail.com> - 2013-03-02 18:52 +0100
      Re: Dealing with exceptions bvdp <bob@mellowood.ca> - 2013-03-02 11:35 -0800
        Re: Dealing with exceptions Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-03-03 01:41 +0000
      Re: Dealing with exceptions bvdp <bob@mellowood.ca> - 2013-03-02 11:35 -0800
      Re: Dealing with exceptions Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-03-03 00:41 +0000
        Re: Dealing with exceptions Chris Angelico <rosuav@gmail.com> - 2013-03-03 12:04 +1100
      Re: Dealing with exceptions Nobody <nobody@nowhere.com> - 2013-03-03 23:01 +0000
    Re: Dealing with exceptions Ian Kelly <ian.g.kelly@gmail.com> - 2013-03-02 10:58 -0700
    Re: Dealing with exceptions Ian Kelly <ian.g.kelly@gmail.com> - 2013-03-02 11:00 -0700
    Re: Dealing with exceptions Chris Angelico <rosuav@gmail.com> - 2013-03-03 05:21 +1100
      Re: Dealing with exceptions bvdp <bob@mellowood.ca> - 2013-03-02 11:39 -0800
      Re: Dealing with exceptions bvdp <bob@mellowood.ca> - 2013-03-02 11:39 -0800
    Re: Dealing with exceptions Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-03-02 19:18 +0000
    Re: Dealing with exceptions Devin Jeanpierre <jeanpierreda@gmail.com> - 2013-03-02 14:27 -0500
    Re: Dealing with exceptions Rick Johnson <rantingrickjohnson@gmail.com> - 2013-03-02 11:43 -0800
    Re: Dealing with exceptions Terry Reedy <tjreedy@udel.edu> - 2013-03-02 16:23 -0500
    Re: Dealing with exceptions Chris Angelico <rosuav@gmail.com> - 2013-03-03 09:16 +1100
    Re: Dealing with exceptions Terry Reedy <tjreedy@udel.edu> - 2013-03-02 18:08 -0500
    Re: Dealing with exceptions Chris Angelico <rosuav@gmail.com> - 2013-03-03 10:17 +1100

#40344 — Dealing with exceptions

Every time I write a program with exception handling (and I suppose that includes just about every program I write!) I need to scratch my brain when I create try blocks.

For example, I'm writing a little program do copy specific files to a USB stick. To do the actual copy I'm using:

    try:
       shutil.copy(s, os.path.join(usbpath, songname))
     except ...

now, I need to figure out just what exceptions to handle. Let's see: 

  IOError  that means that the disk is full or otherwise buggered. Better dump out of the loop.

But, I know there can be other errors as well. Doing some tests, I know that certain filenames are invalid (I think a "?" or unicode char is invalid when writing to a FAT32 filesystem). And, so what exception is that? Without actually creating the error, I can't figure it out.

In this case, I can run the program an number of times and parse out the errors and write code to catch various things. But, I think I'm missing something completely. Guess what I'm looking for is a list of possible (probable?) errors for the shutil.copy() command. And, in a much bigger manual, for most other commands.

Maybe I'm just venting about FAT32 filesystems :)

[toc] | [next] | [standalone]

#40348

On Sat, Mar 2, 2013 at 6:40 PM, bvdp <bob@mellowood.ca> wrote:
> Every time I write a program with exception handling (and I suppose that includes just about every program I write!) I need to scratch my brain when I create try blocks.
>
> For example, I'm writing a little program do copy specific files to a USB stick. To do the actual copy I'm using:
>
>     try:
>        shutil.copy(s, os.path.join(usbpath, songname))
>      except ...
>
> now, I need to figure out just what exceptions to handle. Let's see:
>
>   IOError  that means that the disk is full or otherwise buggered. Better dump out of the loop.
>
> But, I know there can be other errors as well. Doing some tests, I know that certain filenames are invalid (I think a "?" or unicode char is invalid when writing to a FAT32 filesystem). And, so what exception is that? Without actually creating the error, I can't figure it out.
>
> In this case, I can run the program an number of times and parse out the errors and write code to catch various things. But, I think I'm missing something completely. Guess what I'm looking for is a list of possible (probable?) errors for the shutil.copy() command. And, in a much bigger manual, for most other commands.
>
> Maybe I'm just venting about FAT32 filesystems :)
>
> --
> http://mail.python.org/mailman/listinfo/python-list

IOError and OSError should cover all copy problems, I think.

Also, you can do `except:` for a catch-all, but it is discouraged
unless you have REALLY good reasons to do this.  And, most of the
time, you don’t.

-- 
Kwpolska <http://kwpolska.tk> | GPG KEY: 5EAAEA16
stop html mail                | always bottom-post
http://asciiribbon.org        | http://caliburn.nl/topposting.html

[toc] | [prev] | [next] | [standalone]

#40356

> 
> IOError and OSError should cover all copy problems, I think.

How do you know that? I can figure it out as well by running the program, but I'd like to make the determination of what to catch when I'm writing the code.

[toc] | [prev] | [next] | [standalone]

#40381

On Sat, 02 Mar 2013 11:35:13 -0800, bvdp wrote:


>> IOError and OSError should cover all copy problems, I think.
> 
> How do you know that? I can figure it out as well by running the
> program, but I'd like to make the determination of what to catch when
> I'm writing the code.


In my experience, I would say:

20% by reading the documentation, 20% by experience, and 60% by 
experimentation at the interactive interpreter.

Generally if you read the docs, you will get some idea of the exceptions 
you can expect to get. E.g. the docs for open claim:

"Raise IOError upon failure."

(call "help(open)" in the interactive interpreter).

Experience and experimentation come into it in the (unfortunately very 
common case) where the docs don't describe the exceptions you can expect.

For the specific case of IOError and OSError, another very useful skill 
is googling for the specific errno you can test for:

try:
    something()
except IOError as e:
    if e.errno == whatever:
        do_this()
    else:
        raise


In Python 3.3, this becomes much nicer with individual exceptions for the 
most common errnos.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#40357

> 
> IOError and OSError should cover all copy problems, I think.

How do you know that? I can figure it out as well by running the program, but I'd like to make the determination of what to catch when I'm writing the code.

[toc] | [prev] | [next] | [standalone]

#40377

On Sat, 02 Mar 2013 18:52:19 +0100, Kwpolska wrote:

> Also, you can do `except:` for a catch-all, but it is discouraged unless
> you have REALLY good reasons to do this.  And, most of the time, you
> don’t.


`except Exception` is to be much preferred over a bare except. It 
excludes KeyboardInterruptError (the user hits Ctrl-C) and SystemExit, 
which you normally either want to let through, or handle separately.


But yes, in general, only catch the minimum you *know* you need to catch 
and can deal with. Anything else is a bug in your code that needs to be 
fixed, and you can't fix it if you never see the exception.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#40379

On Sun, Mar 3, 2013 at 11:41 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> But yes, in general, only catch the minimum you *know* you need to catch
> and can deal with. Anything else is a bug in your code that needs to be
> fixed, and you can't fix it if you never see the exception.

With the exception (if you'll excuse the expression) of "framework"
systems, where there's a distinct separation between "inside" and
"outside". Often then, the "outside" will catch any exception thrown
by the "inside" and deal with it in some generic way (for instance, a
web server might log the details and return HTTP 500 to the client,
then go back and handle the next request). Effectively, this is doing
the job of the top-level exception handler: log the exception (to the
console) and terminate.

ChrisA

[toc] | [prev] | [next] | [standalone]

#40427

On Sat, 02 Mar 2013 18:52:19 +0100, Kwpolska wrote:

> Also, you can do `except:` for a catch-all, but it is discouraged unless
> you have REALLY good reasons to do this.  And, most of the time, you
> don’t.

Most of the time you probably want to catch either Exception (which
excludes GeneratorExit, KeyboardInterrupt and SystemExit) or StandardError
(which excludes the above pluse warnings and StopIteration).

Only in specific circumstances can you reasonably go finer than that,
partly due to the set of exceptions a method may throw seldom being
documented, and partly due to duck typing; a parameter which is intended
to be a file object could realistically be any object which supports the
appropriate methods (e.g. .read()), and there's no guarantee that those
methods will have the same set of possible exceptions as a real file
object.

[toc] | [prev] | [next] | [standalone]

#40349

On Sat, Mar 2, 2013 at 10:40 AM, bvdp <bob@mellowood.ca> wrote:
> Every time I write a program with exception handling (and I suppose that includes just about every program I write!) I need to scratch my brain when I create try blocks.
>
> For example, I'm writing a little program do copy specific files to a USB stick. To do the actual copy I'm using:
>
>     try:
>        shutil.copy(s, os.path.join(usbpath, songname))
>      except ...
>
> now, I need to figure out just what exceptions to handle. Let's see:
>
>   IOError  that means that the disk is full or otherwise buggered. Better dump out of the loop.
>
> But, I know there can be other errors as well. Doing some tests, I know that certain filenames are invalid (I think a "?" or unicode char is invalid when writing to a FAT32 filesystem). And, so what exception is that? Without actually creating the error, I can't figure it out.

OSError.  In Python 3, I expect it would more specifically be a
FileNotFoundError, which is a subclass of OSError.

> In this case, I can run the program an number of times and parse out the errors and write code to catch various things. But, I think I'm missing something completely. Guess what I'm looking for is a list of possible (probable?) errors for the shutil.copy() command. And, in a much bigger manual, for most other commands.

OSError will cover a wide swath of possible exceptions here.

[toc] | [prev] | [next] | [standalone]

#40352

On Sat, Mar 2, 2013 at 10:52 AM, Kwpolska <kwpolska@gmail.com> wrote:
> IOError and OSError should cover all copy problems, I think.

And it may be worth pointing out here that as of Python 3.3, IOError
is just a synonym for OSError.

[toc] | [prev] | [next] | [standalone]

#40354

On Sun, Mar 3, 2013 at 4:40 AM, bvdp <bob@mellowood.ca> wrote:
> For example, I'm writing a little program do copy specific files to a USB stick. To do the actual copy I'm using:
>
>     try:
>        shutil.copy(s, os.path.join(usbpath, songname))
>      except ...
>
> now, I need to figure out just what exceptions to handle.

Here's a bit of a left-field thought: Maybe none of them.

What are you actually doing when you get an exception? Can you
plausibly recover? If not - that is, if you're going to abort the
whole operation anyway - then save yourself the trouble of writing the
try/catch, and just let the exception propagate up (to the console, if
nowhere else).

On the other hand, if you want to simply report the error and continue
on (meaning you get as many songs copied as possible), then do what
others have recommended and catch OSError.

ChrisA

[toc] | [prev] | [next] | [standalone]

#40359

> 
> Here's a bit of a left-field thought: Maybe none of them.
> 

Not far left at all :)
> 
> What are you actually doing when you get an exception? Can you
> 
> plausibly recover? If not - that is, if you're going to abort the
> 
> whole operation anyway - then save yourself the trouble of writing the
> 
> try/catch, and just let the exception propagate up (to the console, if
> 
> nowhere else).
> 

My first cut of the program did exactly that. Just abort the whole thing, figuring that the disk was full or buggered.

What I ran into was that half way through the process I ended up with a filename which the FAT32 didn't like. So, my brain-dead idea was to catch those (and ignore them) and continue on. But then I have to distinguish between a bad filename and "real" errors.

But, you are probably correct if you say I should check the filename first :) Is there a module for that?

[toc] | [prev] | [next] | [standalone]

#40361

> 
> Here's a bit of a left-field thought: Maybe none of them.
> 

Not far left at all :)
> 
> What are you actually doing when you get an exception? Can you
> 
> plausibly recover? If not - that is, if you're going to abort the
> 
> whole operation anyway - then save yourself the trouble of writing the
> 
> try/catch, and just let the exception propagate up (to the console, if
> 
> nowhere else).
> 

My first cut of the program did exactly that. Just abort the whole thing, figuring that the disk was full or buggered.

What I ran into was that half way through the process I ended up with a filename which the FAT32 didn't like. So, my brain-dead idea was to catch those (and ignore them) and continue on. But then I have to distinguish between a bad filename and "real" errors.

But, you are probably correct if you say I should check the filename first :) Is there a module for that?

[toc] | [prev] | [next] | [standalone]

#40355

On 02/03/2013 17:58, Ian Kelly wrote:
> On Sat, Mar 2, 2013 at 10:40 AM, bvdp <bob@mellowood.ca> wrote:
>> Every time I write a program with exception handling (and I suppose that includes just about every program I write!) I need to scratch my brain when I create try blocks.
>>
>> For example, I'm writing a little program do copy specific files to a USB stick. To do the actual copy I'm using:
>>
>>      try:
>>         shutil.copy(s, os.path.join(usbpath, songname))
>>       except ...
>>
>> now, I need to figure out just what exceptions to handle. Let's see:
>>
>>    IOError  that means that the disk is full or otherwise buggered. Better dump out of the loop.
>>
>> But, I know there can be other errors as well. Doing some tests, I know that certain filenames are invalid (I think a "?" or unicode char is invalid when writing to a FAT32 filesystem). And, so what exception is that? Without actually creating the error, I can't figure it out.
>
> OSError.  In Python 3, I expect it would more specifically be a
> FileNotFoundError, which is a subclass of OSError.

This wasn't introduced until Python 3.3 see 
http://docs.python.org/3/whatsnew/3.3.html#pep-3151-reworking-the-os-and-io-exception-hierarchy

>
>> In this case, I can run the program an number of times and parse out the errors and write code to catch various things. But, I think I'm missing something completely. Guess what I'm looking for is a list of possible (probable?) errors for the shutil.copy() command. And, in a much bigger manual, for most other commands.
>
> OSError will cover a wide swath of possible exceptions here.
>


-- 
Cheers.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#40358

On Sat, Mar 2, 2013 at 1:21 PM, Chris Angelico <rosuav@gmail.com> wrote:
>> now, I need to figure out just what exceptions to handle.
>
> Here's a bit of a left-field thought: Maybe none of them.
>
> What are you actually doing when you get an exception? Can you
> plausibly recover? If not - that is, if you're going to abort the
> whole operation anyway - then save yourself the trouble of writing the
> try/catch, and just let the exception propagate up (to the console, if
> nowhere else).

He can't know if he should handle the errors if he doesn't know what
those errors are. Thus the question.

-- Devin

[toc] | [prev] | [next] | [standalone]

#40360

On Saturday, March 2, 2013 11:40:11 AM UTC-6, bvdp wrote:
> Every time I write a program with exception handling (and
> I suppose that includes just about every program I write!)
> I need to scratch my brain when I create try blocks.
> 
> For example, I'm writing a little program do copy specific
> files to a USB stick. To do the actual copy I'm using:
> 
>     try:
>        shutil.copy(s, os.path.join(usbpath, songname))
>      except ...
> 
> now, I need to figure out just what exceptions to handle.
> Let's see:
> 
> IOError  that means that the disk is full or otherwise
> buggered. Better dump out of the loop.
> 
> But, I know there can be other errors as well. Doing some
> tests, I know that certain filenames are invalid (I think
> a "?" or unicode char is invalid when writing to a FAT32
> filesystem). And, so what exception is that? Without
> actually creating the error, I can't figure it out.

Well how can you expect an error to be thrown without creating the scenario that will throw one? *wink*

> In this case, I can run the program an number of times and
> parse out the errors and write code to catch various
> things. But, I think I'm missing something completely.
> Guess what I'm looking for is a list of possible
> (probable?) errors for the shutil.copy() command. And, in
> a much bigger manual, for most other commands.

No. What you are doing is *misunderstanding* that well written methods must follow the values of:

 "doing one thing and doing it well"
 
In the case of "shutil.copy", that "one thing" is "coping files"; but that "one thing" does NOT include: 

 * resolving file paths
 * validating file path chars on an OS by OS basis
 * ensuring disc space is adequate
 * etc...

Methods that take inputs, like in this case: "src" and "dst" file paths, should not be responsible for the validity of the inputs. It is the responsibility of the caller to validate these inputs PRIOR to injecting them into the method. 

But even *IF* a convincing argument could be made for methods to validate inputs, then at what point do we draw the line? How many foolish samples of the possible permutation set should we consider? Should we inform the user of such foolish usage as:
    
 shutil.copy(src=1, dst=2, follow_symlinks=isinstance)
 shutil.copy(src='cat', dst='hat', follow_symlinks=list)

Now whilst these example are quite absurd, they are in-fact possibilities that must be considered *IF* we intend to follow your "wish" until it's logical conclusion.

[toc] | [prev] | [next] | [standalone]

#40366

On 3/2/2013 12:40 PM, bvdp wrote:

> But, I know there can be other errors as well. Doing some tests, I
> know that certain filenames are invalid (I think a "?" or unicode
> char is invalid when writing to a FAT32 filesystem). And, so what
> exception is that? Without actually creating the error, I can't
> figure it out.

So use the interactive interpreter (or idle, or ipython) and create the 
error. You should always have it open when editing. Using less time that 
it took you to write the above. 3.3, win7, (idle)

 >>> open('sdjhfjshdfkjsh')
Traceback (most recent call last):
   File "<pyshell#2>", line 1, in <module>
     open('sdjhfjshdfkjsh')
FileNotFoundError: [Errno 2] No such file or directory: 'sdjhfjshdfkjsh'

Now, does shutil pass on FileNotFoundError? I will let you experiment.

There are error conditions that are hard to generate, but a bad file 
name is not one of them.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#40368

On Sun, Mar 3, 2013 at 8:23 AM, Terry Reedy <tjreedy@udel.edu> wrote:
>>>> open('sdjhfjshdfkjsh')
> Traceback (most recent call last):
>   File "<pyshell#2>", line 1, in <module>
>     open('sdjhfjshdfkjsh')
> FileNotFoundError: [Errno 2] No such file or directory: 'sdjhfjshdfkjsh'
>
> Now, does shutil pass on FileNotFoundError? I will let you experiment.
>
> There are error conditions that are hard to generate, but a bad file name is
> not one of them.

That's actually a perfectly valid file name, but one that doesn't
happen to have a corresponding file. However, the same technique will
work with the OP's description of "a filename which the FAT32 didn't
like" too.

ChrisA

[toc] | [prev] | [next] | [standalone]

#40372

On 3/2/2013 5:16 PM, Chris Angelico wrote:
> On Sun, Mar 3, 2013 at 8:23 AM, Terry Reedy <tjreedy@udel.edu> wrote:
>>>>> open('sdjhfjshdfkjsh')
>> Traceback (most recent call last):
>>    File "<pyshell#2>", line 1, in <module>
>>      open('sdjhfjshdfkjsh')
>> FileNotFoundError: [Errno 2] No such file or directory: 'sdjhfjshdfkjsh'
>>
>> Now, does shutil pass on FileNotFoundError? I will let you experiment.
>>
>> There are error conditions that are hard to generate, but a bad file name is
>> not one of them.
>
> That's actually a perfectly valid file name, but one that doesn't
> happen to have a corresponding file. However, the same technique will
> work with the OP's description of "a filename which the FAT32 didn't
> like" too.

Yeah, that gives a different error and message.
 >>> open('a~`!@#$%^&*()_-+={[}]|\<,>.?/')
Traceback (most recent call last):
   File "<pyshell#3>", line 1, in <module>
     open('a~`!@#$%^&*()_-+={[}]|\<,>.?/')
OSError: [Errno 22] Invalid argument: 'a~`!@#$%^&*()_-+={[}]|\\<,>.?/'


-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#40375

On Sun, Mar 3, 2013 at 10:08 AM, Terry Reedy <tjreedy@udel.edu> wrote:
> On 3/2/2013 5:16 PM, Chris Angelico wrote:
>>
>> On Sun, Mar 3, 2013 at 8:23 AM, Terry Reedy <tjreedy@udel.edu> wrote:
>>>>>>
>>>>>> open('sdjhfjshdfkjsh')
>>>
>>> Traceback (most recent call last):
>>>    File "<pyshell#2>", line 1, in <module>
>>>      open('sdjhfjshdfkjsh')
>>> FileNotFoundError: [Errno 2] No such file or directory: 'sdjhfjshdfkjsh'
>>>
>>> Now, does shutil pass on FileNotFoundError? I will let you experiment.
>>>
>>> There are error conditions that are hard to generate, but a bad file name
>>> is
>>> not one of them.
>>
>>
>> That's actually a perfectly valid file name, but one that doesn't
>> happen to have a corresponding file. However, the same technique will
>> work with the OP's description of "a filename which the FAT32 didn't
>> like" too.
>
>
> Yeah, that gives a different error and message.
>>>> open('a~`!@#$%^&*()_-+={[}]|\<,>.?/')
>
> Traceback (most recent call last):
>   File "<pyshell#3>", line 1, in <module>
>     open('a~`!@#$%^&*()_-+={[}]|\<,>.?/')
> OSError: [Errno 22] Invalid argument: 'a~`!@#$%^&*()_-+={[}]|\\<,>.?/'

That should be:

OSError: Profanity not permitted on respectable file systems

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web