Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #74148 > unrolled thread

error handling when opening files

Back to article view | Back to comp.lang.python

Contents

  error handling when opening files Alex Burke <alexjeffburke@gmail.com> - 2014-07-08 01:49 +0200
    Re: error handling when opening files Marko Rauhamaa <marko@pacujo.net> - 2014-07-08 11:30 +0300
    Re: error handling when opening files Steven D'Aprano <steve@pearwood.info> - 2014-07-08 09:00 +0000
      Re: error handling when opening files Chris Angelico <rosuav@gmail.com> - 2014-07-08 20:35 +1000
      Re: error handling when opening files Alex Burke <alexjeffburke@gmail.com> - 2014-07-09 21:59 +0200
    Re: error handling when opening files wxjmfauth@gmail.com - 2014-07-08 02:38 -0700

#74148 — error handling when opening files

Hi there,

While reading up on a previous thread 'open() and EOFError' I saw the
following (with minor changes to help make my question clearer) block
suggested as a canonical way to open files and do something:

try:
    f = open(path)
except IOError:
    handle_error()
else:
    with f:
        do_stuff()

This somewhat surprised me because I'd always written such a block
something more as follows:

try:
    with open(path) as f:
        do_stuff()
except IOError:
    handle_error('file never opened')
except ApplicationError:
    handle_error('doing something with the content went wrong')

Aside from the pythonic but less widely known else clause of the
try/except in the first form, what else is different? What nuances am
I missing?

The reason I preferred the second was in addition to catching the
IOError when attempting the open() if the file does not exist I
thought I was accounting for the possibility en error occurs while
reading data out of the file. That and to handle if do_stuff() was
actually calling readline() or such on the file which I thought was
possible source of issues.

I am wondering if this is also related to some misunderstanding around
the file context manager - on exit of the with() block a close is
arranged, but if that close() is called in an error situation I was
under the impression that the error would be raised again outside it.
Otherwise the only way I can see of getting errors is have a variable
set to None and if the with block succeeds set it to some other value
thus being able to do an if not None check afterward.

That's probably enough conflated questioning for now.. who'd have
thought opening a file could be such a poser! Yep, it's always the
error handling :)

Thanks in advance, Alex J Burke.

[toc] | [next] | [standalone]

#74151

Alex Burke <alexjeffburke@gmail.com>:

> While reading up on a previous thread 'open() and EOFError' I saw the
> following (with minor changes to help make my question clearer) block
> suggested as a canonical way to open files and do something:
>
> try:
>     f = open(path)
> except IOError:
>     handle_error()
> else:
>     with f:
>         do_stuff()
>
> This somewhat surprised me because I'd always written such a block
> something more as follows:
>
> try:
>     with open(path) as f:
>         do_stuff()
> except IOError:
>     handle_error('file never opened')
> except ApplicationError:
>     handle_error('doing something with the content went wrong')
>
> Aside from the pythonic but less widely known else clause of the
> try/except in the first form, what else is different? What nuances am
> I missing?

Your version catches IOError for do_stuff() as well as open(). You
usually want to guard each statement with try-except so you can log and
pinpoint the error better.


Marko

[toc] | [prev] | [next] | [standalone]

#74154

On Tue, 08 Jul 2014 01:49:58 +0200, Alex Burke wrote:

> Hi there,
> 
> While reading up on a previous thread 'open() and EOFError' I saw the
> following (with minor changes to help make my question clearer) block
> suggested as a canonical way to open files and do something:

Emphasis on "a" canonical way, out of many canonical ways ;-)

> try:
>     f = open(path)
> except IOError:
>     handle_error()
> else:
>     with f:
>         do_stuff()

I wrote it that way to show that there is no need to fall back on the old-
fashioned "try...except...finally" idiom that was used before the with 
statement and context managers. In practice, I would usually do what you 
do:

> This somewhat surprised me because I'd always written such a block
> something more as follows:
> 
> try:
>     with open(path) as f:
>         do_stuff()
> except IOError:
>     handle_error('file never opened')
> except ApplicationError:
>     handle_error('doing something with the content went wrong')

assuming that open() and do_stuff() don't raise the same exceptions. If 
they do raise the same exceptions, and it is important to distinguish 
IOError raised by open() from IOError raised by do_stuff, then you cannot 
use this idiom.


> Aside from the pythonic but less widely known else clause of the
> try/except in the first form, what else is different? What nuances am I
> missing?
> 
> The reason I preferred the second was in addition to catching the
> IOError when attempting the open() if the file does not exist I thought
> I was accounting for the possibility en error occurs while reading data
> out of the file. That and to handle if do_stuff() was actually calling
> readline() or such on the file which I thought was possible source of
> issues.

Correct. But now imagine you're happily reading through the file, 
processing line after line, and halfway through the file read() fails 
because it's on a network share and somebody just tripped over the cable. 
Your script now *lies to you* and says the file was never opened.

If you care about accurate error messages, you should be more careful 
about conflating two different errors (that is, catching too much in a 
single try...except block).


> I am wondering if this is also related to some misunderstanding around
> the file context manager - on exit of the with() block a close is
> arranged, but if that close() is called in an error situation I was
> under the impression that the error would be raised again outside it.

In general, context managers may choose whether or not to re-raise 
exceptions. Some exceptions are not errors and may be swallowed silently. 
However, file objects will re-raise the error.

Interestingly, did you know that even *closing* a file can fail?


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#74162

On Tue, Jul 8, 2014 at 7:00 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> Interestingly, did you know that even *closing* a file can fail?

I know that can happen with SSL sockets (which can require writing and
reading). Can't think of any situations on normal file systems where
that's true, unless the actual failure is in the flushing of buffers;
technically, that's not a failure of closing, but it could be a
failure that's detected on close(). Is that what you're thinking of?

ChrisA

[toc] | [prev] | [next] | [standalone]

#74278

> Interestingly, did you know that even *closing* a file can fail?

No I didn't, interesting piece on information for sure! I thought close() is
usually made to always succeed regardless if it actually hosed up. Any
idea what the context manager will do in that case? (I ask as that
else-with form looks vulnerable in that case but may not be).

Thanks!

[toc] | [prev] | [next] | [standalone]

#74159

Alex is right (at least not wrong) and I'm also working
in a similar way.

- If there is a problem in opening, the file is never
opened and only an IOError can happen.
- If do_stuff succeeds: the file is open and will be closed
because of the with statement.
- If do_stuff does not succeed (errors), the file will be
closed (with statement) and only later the error is raised.
Tricky, but it works like this.

In short: simple and elegant solution.

>>> def z():
...     try:
...         with open('eta.cfg', 'r') as f:
...             a = 1 / 0
...     except IOError:
...         print('error in opening file')
...     except ZeroDivisionError:
...         print('ZeroDivisionError', f.closed)
...         
>>> z()
ZeroDivisionError True
>>> def z2():
...     try:
...         with open('xxx.cfg', 'r') as f:
...             a = 1 / 0
...     except IOError:
...         print('error in opening file')
...     except ZeroDivisionError:
...         print('ZeroDivisionError', f.closed)
...         
>>> z2()
error in opening file
>>> def z3():
...     try:
...         with open('xxx.cfg', 'r') as f:
...             a = 1 / 0
...     except IOError:
...         print('error in opening file')
...         print(f.closed)
...     except ZeroDivisionError:
...         print('ZeroDivisionError', f.closed)
...         
>>> z3()
error in opening file
Traceback (most recent call last):
  File "<eta last command>", line 1, in <module>
  File "<eta last command>", line 7, in z3
UnboundLocalError: local variable 'f' referenced before assignment

----

One could consider a case where do_stuff is opening a file...

----

Quote: "Interestingly, did you know that even *closing* "
a file can fail? 

Answer: Does "with" fail? I never explicitely closed the file.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web