Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #18858 > unrolled thread

Two questions about logging

Back to article view | Back to comp.lang.python

Contents

  Two questions about logging Matthew Pounsett <matt.pounsett@gmail.com> - 2012-01-11 18:19 -0800
    Re: Two questions about logging Roy Smith <roy@panix.com> - 2012-01-11 21:34 -0500
      Re: Two questions about logging Matthew Pounsett <matt.pounsett@gmail.com> - 2012-01-11 19:31 -0800
    Re: Two questions about logging Jean-Michel Pichavant <jeanmichel@sequans.com> - 2012-01-12 11:28 +0100
    Re: Two questions about logging K Richard Pixley <rich@noir.com> - 2012-01-12 17:03 -0800
      Re: Two questions about logging Matthew Pounsett <matt.pounsett@gmail.com> - 2012-01-14 18:03 -0800
    Re: Two questions about logging Vinay Sajip <vinay_sajip@yahoo.co.uk> - 2012-01-15 17:03 -0800

#18858 — Two questions about logging

I'm trying to figure out a couple of things with the logging module,
and I'm hoping someone can provide some pointers.  I've read through
the module docs on python.org, the basic and advanced tutorials, and
the cookbook post, but a couple of things still elude me.

First, I'd like to be able to permit users to do more typical log
rotation, based on their OS's log rotation handler, rather than
rotating logs from inside an application.  This is usually handled by
signalling an application with a HUP, whereupon it closes and then re-
opens all of its logs, getting new file handles (and new inodes).  I
don't see anything in the Handler methods (or anywhere else) that
would let me tell a logger object to refresh the file handles on a log
file.  Is there some standard way to deal with this?

Second, I'm trying to get a handle on how libraries are meant to
integrate with the applications that use them.  The naming advice in
the advanced tutorial is to use __name__ to name loggers, and to allow
log messages to pass back up to the using application's logger for
processing, but these two pieces of advice seem contradictory.. since
log messages only pass back up to the root if the loggers are named
hierarchically.

> cat foo.py
#!/usr/bin/env python

import logging
import bar

logger = logging.getLogger(__name__)
fh = logging.FileHandler('foo.log')
ch = logging.StreamHandler()
logger.addHandler(fh)
logger.addHandler(ch)
logger.warning('from the first')
bar.funct()

> cat bar.py
#!/usr/bin/env python

import logging

def funct():
	logger = logging.getLogger(__name__)
	logger.warning('from the second')

> ./foo.py
from the first
No handlers could be found for logger "bar"

So, given this state of affairs, how is a library author to use
loggers, given that he or she can't know how authors who use the
library will name their logger objects?  In the above example, what
would the author of bar.py do to hook up bar's logger with foo's,
without knowing in advance what foo's logger will be named?


Thanks very much for any suggestions, or pointers to documentation
that I've missed.

[toc] | [next] | [standalone]

#18860

In article 
<7dabf43f-3814-47b6-966a-1439f56548b5@i6g2000vbk.googlegroups.com>,
 Matthew Pounsett <matt.pounsett@gmail.com> wrote:

> First, I'd like to be able to permit users to do more typical log
> rotation, based on their OS's log rotation handler, rather than
> rotating logs from inside an application.  This is usually handled by
> signalling an application with a HUP, whereupon it closes and then re-
> opens all of its logs, getting new file handles (and new inodes).  I
> don't see anything in the Handler methods (or anywhere else) that
> would let me tell a logger object to refresh the file handles on a log
> file.  Is there some standard way to deal with this?

What I would do is log to syslog (logging.handlers.SysLogHandler) and 
let syslog worry about rotating log files.  Why reinvent the wheel?

[toc] | [prev] | [next] | [standalone]

#18863

On Jan 11, 9:34 pm, Roy Smith <r...@panix.com> wrote:
> What I would do is log to syslog (logging.handlers.SysLogHandler) and
> let syslog worry about rotating log files.  Why reinvent the wheel?

Syslog is fine for an application run by an administrator, but isn't
an option for a user.

[toc] | [prev] | [next] | [standalone]

#18871

Matthew Pounsett wrote:
> [snip]
>
> Second, I'm trying to get a handle on how libraries are meant to
> integrate with the applications that use them.  The naming advice in
> the advanced tutorial is to use __name__ to name loggers, and to allow
> log messages to pass back up to the using application's logger for
> processing, but these two pieces of advice seem contradictory.. since
> log messages only pass back up to the root if the loggers are named
> hierarchically.
>
> [snip]
>
> So, given this state of affairs, how is a library author to use
> loggers, given that he or she can't know how authors who use the
> library will name their logger objects?  In the above example, what
> would the author of bar.py do to hook up bar's logger with foo's,
> without knowing in advance what foo's logger will be named?
>
>
> Thanks very much for any suggestions, or pointers to documentation
> that I've missed.
>   

All loggers share at least one parent : the root logger. In the standard 
way of using the loggers, the root logger is *the* logger responsible of 
processing logs.


Considering that extlib is an external library from anApp, and intlib an 
internal library to the anApp application:

People may expect logs that way:

anApp.extlib : INFO : a log message
anApp.intlib : INFO : another message

However, what will happen is :

extlib : INFO : message from the external lib
anApp.intlib : INFO : message from the internal lib

As you mentioned in your post, there is no way to 'attach' an external 
lib to your application since the external lib knows nothing about the 
application, but there is no need to attach it actually.


JM

[toc] | [prev] | [next] | [standalone]

#18887

On 1/11/12 18:19 , Matthew Pounsett wrote:
> Second, I'm trying to get a handle on how libraries are meant to
> integrate with the applications that use them.  The naming advice in
> the advanced tutorial is to use __name__ to name loggers, and to allow
> log messages to pass back up to the using application's logger for
> processing, but these two pieces of advice seem contradictory.. since
> log messages only pass back up to the root if the loggers are named
> hierarchically.

Here's the confusion.  Each log named __name__ is under the root logger. 
  If you want them all, then catch them all with the root logger.

In foo.py, change getLogger(__name__) to getLogger('').  Only the 
included modules need __name__.  Or use two - one logger for setting 
handlers based on the root logger, and another based on __name__ for 
logging from the top level.

--rich

[toc] | [prev] | [next] | [standalone]

#18989

On Jan 12, 8:03 pm, K Richard Pixley <r...@noir.com> wrote:
> Here's the confusion.  Each log named __name__ is under the root logger.
>   If you want them all, then catch them all with the root logger.

Thanks!  I knew I was missing something obvious.  Between you and Jean-
Michael Pichavant I've figured out what I need to do here.


On Jan 11, 9:34 pm, Roy Smith <r...@panix.com> wrote:
> What I would do is log to syslog (logging.handlers.SysLogHandler) and
> let syslog worry about rotating log files.  Why reinvent the wheel?

I've also worked out what I need to reset file handles, although it
took a lot of reading in the various logging.Handler subclasses.  What
I needed isn't explicitly documented anywhere, but it turns out that
calling the close() method on a FileHandler instance does what I
need.  There's no method to re-open the handler, but the next call to
emit() will automatically re-open the file if it isn't already open.

The upshot is that this does the expected thing if you rename its log
file and then send the running script a HUP signal.

#!/usr/bin/env python

import logging
import signal
import time

logger = logging.getLogger()
lh = logging.FileHandler('./foo.log')
lh.setFormatter(logging.Formatter('%(asctime)s %(name)s: %(message)s',
'%T'))
logger.addHandler(lh)

def sighup(signum, frame):
    lh.close()
    logger.error("handled {0}: {1}".format(signum, frame))

def main():
    signal.signal(signal.SIGHUP, sighup)
    while 1:
        time.sleep(1)
        logger.error('a message')

if __name__ == '__main__':
    main()

[toc] | [prev] | [next] | [standalone]

#19023

On Jan 12, 2:19 am, Matthew Pounsett <matt.pouns...@gmail.com> wrote:

> First, I'd like to be able to permit users to do more typical log
> rotation, based on their OS's log rotation handler, rather than
> rotating logs from inside an application.  This is usually handled by
> signalling an application with a HUP, whereupon it closes and then re-
> opens all of its logs, getting new file handles (and new inodes).  I
> don't see anything in the Handler methods (or anywhere else) that
> would let me tell a logger object to refresh the file handles on a log
> file.  Is there some standard way to deal with this?

There's the WatchedFileHandler, which checks to see if a file's device
or inode has changed (which happens when the external rotator does
rotation) and if so, closes and reopens the file. (This handler is for
Unix/Linux/OS X - it doesn't work on Windows).

See

http://docs.python.org/library/logging.handlers.html#watchedfilehandler

for more information.

Regards,

Vinay Sajip

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web