Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #104551 > unrolled thread

Re: context managers inline?

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: context managers inline? Chris Angelico <rosuav@gmail.com> - 2016-03-11 07:17 +1100
    Re: context managers inline? Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-03-11 08:09 +0200
      Re: context managers inline? Paul Rubin <no.email@nospam.invalid> - 2016-03-10 23:30 -0800
        Re: context managers inline? Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-03-11 11:16 +0200

#104551 — Re: context managers inline?

On Fri, Mar 11, 2016 at 5:33 AM, Neal Becker <ndbecker2@gmail.com> wrote:
> Is there a way to ensure resource cleanup with a construct such as:
>
> x = load (open ('my file', 'rb))
>
> Is there a way to ensure this file gets closed?

Yep!

def read_file(fn, *a, **kw):
    with open(fn, *a, **kw) as f:
        return f.read()

Now you can ensure resource cleanup, because the entire file has been
read in before the function returns. As long as your load() function
is okay with reading from a string, this is effective.

Alternatively, push the closing the other way: pass a file name to
your load() function, and have the context manager in there.

If you don't do it one of those ways, the question is: WHEN should the
file be closed? How does Python know when it should go and clean that
up? There's no "end of current expression" rule as there is in C++, so
it's safer to use the statement form, which has a definite end (at the
unindent).

ChrisA

[toc] | [next] | [standalone]

#104585

Chris Angelico writes:

> On Fri, Mar 11, 2016 at 5:33 AM, Neal Becker wrote:

>> Is there a way to ensure resource cleanup with a construct such as:
>>
>> x = load (open ('my file', 'rb))
>>
>> Is there a way to ensure this file gets closed?
>
> Yep!
>
> def read_file(fn, *a, **kw):
>     with open(fn, *a, **kw) as f:
>         return f.read()
>
> Now you can ensure resource cleanup, because the entire file has been
> read in before the function returns. As long as your load() function
> is okay with reading from a string, this is effective.

Make the string look like a file object to the load function:

import io

def clopen(fn, mode = 'r', encoding = None):
    with open(fn, mode = mode, encoding = encoding) as f:
        ModeIO = io.BytesIO if 'b' in mode else io.StringIO
        return ModeIO(f.read())

x = load(clopen('my-file', 'rb'))

[toc] | [prev] | [next] | [standalone]

#104587

Jussi Piitulainen <jussi.piitulainen@helsinki.fi> writes:
>         return ModeIO(f.read())

These suggestions involving f.read() assume the file contents are small
enough to reasonably slurp into memory.  That's unlike the original
where "load" receives a stream and might process it piecewise.

[toc] | [prev] | [next] | [standalone]

#104598

Paul Rubin writes:

> Jussi Piitulainen writes:
>>         return ModeIO(f.read())
>
> These suggestions involving f.read() assume the file contents are small
> enough to reasonably slurp into memory.  That's unlike the original
> where "load" receives a stream and might process it piecewise.

If you strike that word "reasonably", then yes, of course they do. A
_reasonable_ answer to the original question may well be "no", even
though it's not strictly true (because load can be written to close the
file, because open can be replaced with something that closes the file,
or registers the file with some mechanism that closes the file at a
convenient time, assuming the programmer ensures that that time
eventually comes, because one can trust the memory management system to
close the file, assuming the unreachable file object is eventually
collected).

I think the following was also effectively given by someone already. It
would work, _assuming_ that the user is happy to use higher order
functions and especially _lambda_ (or restrict themself to always use
unary functions in place of their load, which to me is the most bizarre
suggestion so far). My impression of the Python community is that there
is some resistance to higher order functions and _especially_ to lambda.

def load(o, length = 1): return [ line[:length] for line in o ]

def closing(p, o):
    with o as f:
        return p(f)

x = closing(load, open('Chipsta/Demo.org', mode = 'rb'))

y = closing(load, open('Chipsta/Demo.org', encoding = 'UTF-8'))

print(x)
print(y)

x = closing(lambda o: load(o, 2), open('Chipsta/Demo.org', mode = 'rb'))

y = closing(lambda o: load(o, 2), open('Chipsta/Demo.org', encoding = 'UTF-8'))

print(x)
print(y)

A problem with _every_ solution is that they cannot be idiomatic Python.
The idiomatic Python thing is to rearrange the code as a with-statement.

I think. (I hedge.)

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web