Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #20659 > unrolled thread

Should I acquire lock for logging.Handler.flush()?

Back to article view | Back to comp.lang.python

Contents

  Should I acquire lock for logging.Handler.flush()? Fayaz Yusuf Khan <fayaz.yusuf.khan@gmail.com> - 2012-02-21 12:53 +0530
    Re: Should I acquire lock for logging.Handler.flush()? Vinay Sajip <vinay_sajip@yahoo.co.uk> - 2012-02-21 12:52 -0800
      Re: Should I acquire lock for logging.Handler.flush()? Fayaz Yusuf Khan <fayaz.yusuf.khan@gmail.com> - 2012-02-22 10:14 +0530
        Re: Should I acquire lock for logging.Handler.flush()? Vinay Sajip <vinay_sajip@yahoo.co.uk> - 2012-02-23 08:23 -0800
          Re: Should I acquire lock for logging.Handler.flush()? Fayaz Yusuf Khan <fayaz.yusuf.khan@gmail.com> - 2012-02-23 23:25 +0530
            Re: Should I acquire lock for logging.Handler.flush()? Vinay Sajip <vinay_sajip@yahoo.co.uk> - 2012-02-23 11:08 -0800
            Re: Should I acquire lock for logging.Handler.flush()? Vinay Sajip <vinay_sajip@yahoo.co.uk> - 2012-02-23 12:06 -0800

#20659 — Should I acquire lock for logging.Handler.flush()?

[Multipart message — attachments visible in raw view] — view raw

I'm writing a custom logging Handler that sends emails through AWS Simple 
Email Service using the boto library.
As there's a quota cap on how many (200) emails I can send within 24hrs, I 
think I need to buffer my log messages from the emit() calls (Or is that a bad 
idea?).
And I was reading the Handler documentation and was confused if I should call 
the acquire() and release() methods from within a flush() call.
-- 
Fayaz Yusuf Khan
Cloud developer and architect
Dexetra SS, Bangalore, India
fayaz.yusuf.khan_AT_gmail_DOT_com
fayaz_AT_dexetra_DOT_com
+91-9746-830-823

[toc] | [next] | [standalone]

#20667

On Feb 21, 7:23 am, Fayaz Yusuf Khan <fayaz.yusuf.k...@gmail.com>
wrote:
> I'm writing a custom logging Handler that sends emails through AWS Simple
> Email Service using the boto library.
> As there's a quota cap on how many (200) emails I can send within 24hrs, I
> think I need to buffer my log messages from the emit() calls (Or is that a bad
> idea?).
> And I was reading the Handler documentation and was confused if I should call
> the acquire() and release() methods from within a flush() call.
> --
> Fayaz Yusuf Khan
> Cloud developer and architect
> Dexetra SS, Bangalore, India
> fayaz.yusuf.khan_AT_gmail_DOT_com
> fayaz_AT_dexetra_DOT_com
> +91-9746-830-823
>
>  signature.asc
> < 1KViewDownload

If you are using SMTPHandler, calling flush() won't do anything.
You'll probably need to subclass the handler to implement rate
limiting.

In the stdlib, only StreamHandler and its subclasses actually
implement flush(), which flushes I/O buffers to disk.

Regards,

Vinay Sajip

[toc] | [prev] | [next] | [standalone]

#20673

[Multipart message — attachments visible in raw view] — view raw

On Tuesday 21 Feb 2012 12:52:14 PM Vinay Sajip wrote:
> If you are using SMTPHandler, calling flush() won't do anything.
> You'll probably need to subclass the handler to implement rate
> limiting.
Oh, no! I'm writing my own handler.
https://github.com/fayazkhan/ses_handler/blob/master/ses_handler.py
Now I understand from here 
http://docs.python.org/library/logging.html#handler-objects
that emit() calls are wrapped acquire() and release() in the handle() method.

Anyway, I read the source and found many interesting things that ought to be 
mentioned in the docs.
Such as flush() should be called from close() whenever it's implemented. 
(FileHandler.close() is doing it)
And how come close()/flush() isn't being called from inside a lock?
(Handler.close() calls the module level _acquireLock() and _releaseLock()s but 
nothing about the instance level acquire() or release())
Or is it being locked from somewhere else?
-- 
Fayaz Yusuf Khan
Cloud developer and architect
Dexetra SS, Bangalore, India
fayaz.yusuf.khan_AT_gmail_DOT_com
fayaz_AT_dexetra_DOT_com
+91-9746-830-823

[toc] | [prev] | [next] | [standalone]

#20734

On Feb 22, 4:44 am, Fayaz Yusuf Khan <fayaz.yusuf.k...@gmail.com>
wrote:

> Anyway, I read the source and found many interesting things that ought to be
> mentioned in the docs.
> Such as flush() should be called from close() whenever it's implemented.
> (FileHandler.close() is doing it)

This is entirely handler-dependent - there's no absolute rule that you
*have* to call flush() before close(). Some underlying will do
flushing when you close.

> And how come close()/flush() isn't being called from inside a lock?

Why does it need to be? Are you aware of any race conditions or other
concurrency problems which will occur with the code as it is now?

> (Handler.close() calls the module level _acquireLock() and _releaseLock()s but
> nothing about the instance level acquire() or release())
> Or is it being locked from somewhere else?

The module level acquisitions are because module-level handler lists
are changed in Handler.close().

If locking is required in a particular handler class for close or
flush, that can be implemented by the developer of that handler class.
AFAIK there is no such need for the handler classes in the stdlib - if
you have reason to believe otherwise, please give some examples of
potential problems and with example code if possible.

Regards,

Vinay Sajip

[toc] | [prev] | [next] | [standalone]

#20738

[Multipart message — attachments visible in raw view] — view raw

On Thursday 23 Feb 2012 8:23:42 AM Vinay Sajip wrote:
> If locking is required in a particular handler class for close or
> flush, that can be implemented by the developer of that handler class.
> AFAIK there is no such need for the handler classes in the stdlib - if
> you have reason to believe otherwise, please give some examples of
> potential problems and with example code if possible.
Well, I'm not currently facing any race-around conditions. As I said, I was 
mostly familiarizing myself with the API.
Well, as emit() is always being called from within a lock, I assumed that 
flush() should/would also be handled similarly. Afterall, they are handling the 
same underlying output stream or in case of the BufferingHandler share the same 
buffer. Shouldn't the access be synchronized?
-- 
Fayaz Yusuf Khan
Cloud developer and architect
Dexetra SS, Bangalore, India
fayaz.yusuf.khan_AT_gmail_DOT_com
fayaz_AT_dexetra_DOT_com
+91-9746-830-823

[toc] | [prev] | [next] | [standalone]

#20739

On Feb 23, 5:55 pm, Fayaz Yusuf Khan <fayaz.yusuf.k...@gmail.com>
wrote:

> Well, as emit() is always being called from within a lock, I assumed that
> flush() should/would also be handled similarly. Afterall, they are handling the
> same underlying output stream or in case of the BufferingHandler share the same
> buffer. Shouldn't the access be synchronized?

Yes, you might well be right - though no problems have been reported,
it's probably best to be safe.

Regards,

Vinay Sajip

[toc] | [prev] | [next] | [standalone]

#20745

On Feb 23, 5:55 pm, Fayaz Yusuf Khan <fayaz.yusuf.k...@gmail.com>
wrote:

> buffer. Shouldn't the access be synchronized?

I've now updated the repos for 2.7, 3.2 and default to add locking for
flush/close operations. Thanks for the suggestion.

Regards,


Vinay Sajip

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web