Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #17582 > unrolled thread

nesting context managers

Back to article view | Back to comp.lang.python

Contents

  nesting context managers Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2011-12-20 15:15 +0100
    Re: nesting context managers Rami Chowdhury <rami.chowdhury@gmail.com> - 2011-12-20 16:25 +0000
    Re: nesting context managers Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2011-12-20 17:56 +0100
      Re: nesting context managers Rami Chowdhury <rami.chowdhury@gmail.com> - 2011-12-20 17:46 +0000
      Re: nesting context managers Ian Kelly <ian.g.kelly@gmail.com> - 2011-12-20 11:46 -0700
      Re: nesting context managers Ethan Furman <ethan@stoneleaf.us> - 2011-12-20 10:02 -0800
      Re: nesting context managers Ian Kelly <ian.g.kelly@gmail.com> - 2011-12-20 11:55 -0700

#17582 — nesting context managers

Hi!

Let us assume I had a class HTTPClient that has a socket for HTTP and a 
logfile for filing logs. I want to make this HTTPClient a context 
manager, so that I can write

   with HTTPClient(url) as client:
       pass

and reliably have both the socket and the logfile closed. The easy way 
is wrong

   def __enter__(self):
       with self._mysock, self._myfile:
           return self

because it closes the file and socket as part of the return statement. 
Of course I can go the hard way, code everything myself, but I wonder if 
there isn't some useful tool that would help me achieve this (common?) task.

Thanks!

Uli

[toc] | [next] | [standalone]

#17588

On Tue, Dec 20, 2011 at 14:15, Ulrich Eckhardt
<ulrich.eckhardt@dominolaser.com> wrote:
> Hi!
>
> Let us assume I had a class HTTPClient that has a socket for HTTP and a
> logfile for filing logs. I want to make this HTTPClient a context manager,
> so that I can write
>
>  with HTTPClient(url) as client:
>      pass
>
> and reliably have both the socket and the logfile closed. The easy way is
> wrong
>
>  def __enter__(self):
>      with self._mysock, self._myfile:
>          return self
>

It seems like some of the functions in the contextlib module might
help? You could try and reorganize your code so that you can use the
@contextmanager decorator, for instance?

That having been said, it doesn't seem that difficult to me to code
your own simple __exit__ method if you're already coding up __enter__
?

HTH,
Rami

-- 
Rami Chowdhury
"Never assume malice when stupidity will suffice." -- Hanlon's Razor
+44-7581-430-517 / +1-408-597-7068 / +88-0189-245544

[toc] | [prev] | [next] | [standalone]

#17590

Am 20.12.2011 15:15, schrieb Ulrich Eckhardt:
> Let us assume I had a class HTTPClient that has a socket for HTTP and a
> logfile for filing logs. I want to make this HTTPClient a context
> manager, so that I can write
>
> with HTTPClient(url) as client:
>     pass

Actually, I overestimated the task:

   class HTTPClient(url):
       def __enter__(self):
           return self
       def __exit__(self, ctx_type, ctx_val, ctx_tb):
           self.close()
       def close(self):
           self._myfile.close()
           self._mysocket.close()

I'll simply ignore the fact that closing a file can fail if you can't 
flush buffers. Now, my error was that I have to somehow invoke the 
file's and socket's enter function in my client's enter function. The 
default for all files and sockets is that they are already open, they 
are not opened in the enter function! That way, the client remains 
usable outside a with context but gives me the desired guarantees inside 
one.

For the case of on-demand allocated stuff, I could write the enter 
function like this:

   def __enter__(self):
       # allocate resources
       f = ... # open file
       s = ... # connect socket
       # attach after completion
       self._myfile = f
       self._mysocket = s

To be extra safe or in more complicated scenarios, I could wrap this in 
a try-except and explicitly close those that were already created, but 
normally I'd expect the garbage collector to do that for me ... or am I 
then implicitly assuming a specific implementation?


Uli

[toc] | [prev] | [next] | [standalone]

#17593

On Tue, Dec 20, 2011 at 16:56, Ulrich Eckhardt
<ulrich.eckhardt@dominolaser.com> wrote:
> To be extra safe or in more complicated scenarios, I could wrap this in a
> try-except and explicitly close those that were already created, but
> normally I'd expect the garbage collector to do that for me ... or am I then
> implicitly assuming a specific implementation?

I'm no expert but I believe the basic garbage collection behavior is
part of the language, and it's only details of breaking cycles that
are specific to CPython -- can anyone correct me if I'm wrong?


-- 
Rami Chowdhury
"Never assume malice when stupidity will suffice." -- Hanlon's Razor
+44-7581-430-517 / +1-408-597-7068 / +88-0189-245544

[toc] | [prev] | [next] | [standalone]

#17599

On Tue, Dec 20, 2011 at 9:56 AM, Ulrich Eckhardt
<ulrich.eckhardt@dominolaser.com> wrote:
> Am 20.12.2011 15:15, schrieb Ulrich Eckhardt:
>
>> Let us assume I had a class HTTPClient that has a socket for HTTP and a
>> logfile for filing logs. I want to make this HTTPClient a context
>> manager, so that I can write
>>
>> with HTTPClient(url) as client:
>>    pass
>
>
> Actually, I overestimated the task:
>
>  class HTTPClient(url):
>      def __enter__(self):
>          return self
>      def __exit__(self, ctx_type, ctx_val, ctx_tb):
>          self.close()
>      def close(self):
>          self._myfile.close()
>          self._mysocket.close()
>
> I'll simply ignore the fact that closing a file can fail if you can't flush
> buffers. Now, my error was that I have to somehow invoke the file's and
> socket's enter function in my client's enter function. The default for all
> files and sockets is that they are already open, they are not opened in the
> enter function! That way, the client remains usable outside a with context
> but gives me the desired guarantees inside one.
>
> For the case of on-demand allocated stuff, I could write the enter function
> like this:
>
>  def __enter__(self):
>      # allocate resources
>      f = ... # open file
>      s = ... # connect socket
>      # attach after completion
>      self._myfile = f
>      self._mysocket = s
>
> To be extra safe or in more complicated scenarios, I could wrap this in a
> try-except and explicitly close those that were already created, but
> normally I'd expect the garbage collector to do that for me ... or am I then
> implicitly assuming a specific implementation?

For full generality, copy the code for the contextlib.nested function
in Python 2.7 or 3.1 and use that.  Don't use contextlib.nested
itself, though, because it's deprecated and does not exist in Python
3.2.

Cheers,
Ian

[toc] | [prev] | [next] | [standalone]

#17600

Rami Chowdhury wrote:
> On Tue, Dec 20, 2011 at 16:56, Ulrich Eckhardt
> <ulrich.eckhardt@dominolaser.com> wrote:
>> To be extra safe or in more complicated scenarios, I could wrap this in a
>> try-except and explicitly close those that were already created, but
>> normally I'd expect the garbage collector to do that for me ... or am I then
>> implicitly assuming a specific implementation?
> 
> I'm no expert but I believe the basic garbage collection behavior is
> part of the language, and it's only details of breaking cycles that
> are specific to CPython -- can anyone correct me if I'm wrong?

Garbage collection is part of the language, but how, and when, is 
implementation specific.

Resource management (beyond basic memory allocation) should be handled 
directly.  Python 3 encourages this by complaining if there were still 
open files when it shuts down.

~Ethan~

[toc] | [prev] | [next] | [standalone]

#17601

On Tue, Dec 20, 2011 at 11:46 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Tue, Dec 20, 2011 at 9:56 AM, Ulrich Eckhardt
> <ulrich.eckhardt@dominolaser.com> wrote:
>> Am 20.12.2011 15:15, schrieb Ulrich Eckhardt:
>>
>>> Let us assume I had a class HTTPClient that has a socket for HTTP and a
>>> logfile for filing logs. I want to make this HTTPClient a context
>>> manager, so that I can write
>>>
>>> with HTTPClient(url) as client:
>>>    pass
>>
>>
>> Actually, I overestimated the task:
>>
>>  class HTTPClient(url):
>>      def __enter__(self):
>>          return self
>>      def __exit__(self, ctx_type, ctx_val, ctx_tb):
>>          self.close()
>>      def close(self):
>>          self._myfile.close()
>>          self._mysocket.close()
>>
>> I'll simply ignore the fact that closing a file can fail if you can't flush
>> buffers. Now, my error was that I have to somehow invoke the file's and
>> socket's enter function in my client's enter function. The default for all
>> files and sockets is that they are already open, they are not opened in the
>> enter function! That way, the client remains usable outside a with context
>> but gives me the desired guarantees inside one.
>>
>> For the case of on-demand allocated stuff, I could write the enter function
>> like this:
>>
>>  def __enter__(self):
>>      # allocate resources
>>      f = ... # open file
>>      s = ... # connect socket
>>      # attach after completion
>>      self._myfile = f
>>      self._mysocket = s
>>
>> To be extra safe or in more complicated scenarios, I could wrap this in a
>> try-except and explicitly close those that were already created, but
>> normally I'd expect the garbage collector to do that for me ... or am I then
>> implicitly assuming a specific implementation?
>
> For full generality, copy the code for the contextlib.nested function
> in Python 2.7 or 3.1 and use that.  Don't use contextlib.nested
> itself, though, because it's deprecated and does not exist in Python
> 3.2.

Also note that in versions that do support the nested with syntax, you
can do something like:

class HTTPClient(url):
    @contextlib.contextmanager
    def _contextmanager(file, socket):
        with file, socket:
            yield
    def __enter__(self):
        self._cm = self._contextmanager(self._myfile, self._mysocket)
        return self._cm.__enter__()
    def __exit__(self, exc_type, exc_value, traceback):
        try:
            return self._cm.__exit__(exc_type, exc_value, traceback)
        finally:
            del self._cm

Cheers,
Ian

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web