Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #15762 > unrolled thread

Got some problems when using logging Filter

Back to article view | Back to comp.lang.python

Contents

  Got some problems when using logging Filter sword <john.37@gmail.com> - 2011-11-15 22:09 -0800
    Re: Got some problems when using logging Filter Jean-Michel Pichavant <jeanmichel@sequans.com> - 2011-11-16 12:40 +0100
      Re: Got some problems when using logging Filter sword <john.37@gmail.com> - 2011-11-16 04:31 -0800
        Re: Got some problems when using logging Filter Peter Otten <__peter__@web.de> - 2011-11-16 15:50 +0100
          Re: Got some problems when using logging Filter sword <john.37@gmail.com> - 2011-11-17 01:06 -0800
            Re: Got some problems when using logging Filter Vinay Sajip <vinay_sajip@yahoo.co.uk> - 2011-11-19 16:42 -0800
              Re: Got some problems when using logging Filter sword <john.37@gmail.com> - 2011-11-20 23:15 -0800
                Re: Got some problems when using logging Filter Vinay Sajip <vinay_sajip@yahoo.co.uk> - 2011-11-21 02:11 -0800
        Re: Got some problems when using logging Filter Jean-Michel Pichavant <jeanmichel@sequans.com> - 2011-11-21 12:39 +0100

#15762 — Got some problems when using logging Filter

The logging cookbook gives an Filter example, explainning how to add
contextural info to log. I can't figure out how to filter log from it.

Suppose I have 3 file, a.py, b.py and main.py
#file: a.py
import logging

logger=logging.getLogger(__name__)
def print_log():
    logger.debug("I'm module a")

#file: b.py just like a.py
import logging
logger=logging.getLogger(__name__)
def print_log():
    logger.debug("I'm module b")

#file: main.py
import logging
from logging import Filter
logging.basicConfig(level=logging.DEBUG)
logger=logging.getLogger("main")
logger.debug("This is main process")
logger.addFilter(Filter("a"))

And I expected that the console output would contain main and b module
log only. But it turned out that all logs there.  Is it the problem of
root logger?

[toc] | [next] | [standalone]

#15767

sword wrote:
> The logging cookbook gives an Filter example, explainning how to add
> contextural info to log. I can't figure out how to filter log from it.
>
> Suppose I have 3 file, a.py, b.py and main.py
> #file: a.py
> import logging
>
> logger=logging.getLogger(__name__)
> def print_log():
>     logger.debug("I'm module a")
>
> #file: b.py just like a.py
> import logging
> logger=logging.getLogger(__name__)
> def print_log():
>     logger.debug("I'm module b")
>
> #file: main.py
> import logging
> from logging import Filter
> logging.basicConfig(level=logging.DEBUG)
> logger=logging.getLogger("main")
> logger.debug("This is main process")
> logger.addFilter(Filter("a"))
>
> And I expected that the console output would contain main and b module
> log only. But it turned out that all logs there.  Is it the problem of
> root logger?
>
>
>   
Hi,

First of all, in the code you provided we can't see where you import a & 
b, and when you call their respective print_log method.
Secondly,Filter("a") would allow only the "a" log events, not forbid 
them. quoting the docs: "if name is specified, it names a logger which, 
together with its children, will have its events allowed through the 
filter."

As for your problem it may come from the fact that you applied the 
filter to the 'main' logger, while you probably want to add the filter 
to the *root* logger. Your current hierarchy is

root
  - main
  - a
  - b

events fired from 'a' will be handled by the root logger, not the main.
root = logging.getLogger()
root.addFilter('main')
root.addFilter('a')
root.addFilter('b')

JM

[toc] | [prev] | [next] | [standalone]

#15768

On Nov 16, 7:40 pm, Jean-Michel Pichavant <jeanmic...@sequans.com>
wrote:
> sword wrote:
> > The logging cookbook gives an Filter example, explainning how to add
> > contextural info to log. I can't figure out how to filter log from it.
>
> > Suppose I have 3 file, a.py, b.py and main.py
> > #file: a.py
> > import logging
>
> > logger=logging.getLogger(__name__)
> > def print_log():
> >     logger.debug("I'm module a")
>
> > #file: b.py just like a.py
> > import logging
> > logger=logging.getLogger(__name__)
> > def print_log():
> >     logger.debug("I'm module b")
>
> > #file: main.py
> > import logging
> > from logging import Filter
> > logging.basicConfig(level=logging.DEBUG)
> > logger=logging.getLogger("main")
> > logger.debug("This is main process")
> > logger.addFilter(Filter("a"))
>
> > And I expected that the console output would contain main and b module
> > log only. But it turned out that all logs there.  Is it the problem of
> > root logger?
>
> Hi,
>
> First of all, in the code you provided we can't see where you import a &
> b, and when you call their respective print_log method.
> Secondly,Filter("a") would allow only the "a" log events, not forbid
> them. quoting the docs: "if name is specified, it names a logger which,
> together with its children, will have its events allowed through the
> filter."
>
> As for your problem it may come from the fact that you applied the
> filter to the 'main' logger, while you probably want to add the filter
> to the *root* logger. Your current hierarchy is
>
> root
>   - main
>   - a
>   - b
>
> events fired from 'a' will be handled by the root logger, not the main.
> root = logging.getLogger()
> root.addFilter('main')
> root.addFilter('a')
> root.addFilter('b')
>
> JM

Thanks for your reply. I tried to edit the source a bit, now the
main.py looks like this:
#main.py
import logging
from logging import Filter
import a
import b

logging.basicConfig(level=logging.DEBUG)
root = logging.getLogger()
root.addFilter(Filter("GoneWithTheWind")) #so I suppose no log msg
would pass this filter

logger = logging.getLogger("main")
logger.debug("main process")
a.print_log()
b.print_log()

####
And It still prints out all the log msg. :(

[toc] | [prev] | [next] | [standalone]

#15775

sword wrote:

> Thanks for your reply. I tried to edit the source a bit, now the
> main.py looks like this:
> #main.py
> import logging
> from logging import Filter
> import a
> import b
> 
> logging.basicConfig(level=logging.DEBUG)
> root = logging.getLogger()
> root.addFilter(Filter("GoneWithTheWind")) #so I suppose no log msg
> would pass this filter
> 
> logger = logging.getLogger("main")
> logger.debug("main process")
> a.print_log()
> b.print_log()
> 
> ####
> And It still prints out all the log msg. :(

Here's a little demo to explore how filtering works:

$ cat demo.py
import logging
class Filter(logging.Filter):
    def filter(self, record):
        print "applying filter", self.name
        return True

logging.basicConfig()

loggers = [logging.getLogger(path) for path in ["", "a", "a.b"]]
for logger in loggers:
    logger.addFilter(Filter("filter@" + logger.name))

[handler] = logging.getLogger().handlers
handler.addFilter(Filter("filter@handler"))

for logger in loggers:
    logger.critical("whatever")
$ python demo.py
applying filter filter@root
applying filter filter@handler
CRITICAL:root:whatever
applying filter filter@a
applying filter filter@handler
CRITICAL:a:whatever
applying filter filter@a.b
applying filter filter@handler
CRITICAL:a.b:whatever
$

As you can infer from the output only the filter(s) of the original logger 
and of the handler(s) are applied.

[toc] | [prev] | [next] | [standalone]

#15811

On Nov 16, 10:50 pm, Peter Otten <__pete...@web.de> wrote:
> sword wrote:
> > Thanks for your reply. I tried to edit the source a bit, now the
> > main.py looks like this:
> > #main.py
> > import logging
> > from logging import Filter
> > import a
> > import b
>
> > logging.basicConfig(level=logging.DEBUG)
> > root = logging.getLogger()
> > root.addFilter(Filter("GoneWithTheWind")) #so I suppose no log msg
> > would pass this filter
>
> > logger = logging.getLogger("main")
> > logger.debug("main process")
> > a.print_log()
> > b.print_log()
>
> > ####
> > And It still prints out all the log msg. :(
>
> Here's a little demo to explore how filtering works:
>
> $ cat demo.py
> import logging
> class Filter(logging.Filter):
>     def filter(self, record):
>         print "applying filter", self.name
>         return True
>
> logging.basicConfig()
>
> loggers = [logging.getLogger(path) for path in ["", "a", "a.b"]]
> for logger in loggers:
>     logger.addFilter(Filter("filter@" + logger.name))
>
> [handler] = logging.getLogger().handlers
> handler.addFilter(Filter("filter@handler"))
>
> for logger in loggers:
>     logger.critical("whatever")
> $ python demo.py
> applying filter filter@root
> applying filter filter@handler
> CRITICAL:root:whatever
> applying filter filter@a
> applying filter filter@handler
> CRITICAL:a:whatever
> applying filter fil...@a.b
> applying filter filter@handler
> CRITICAL:a.b:whatever
> $
>
> As you can infer from the output only the filter(s) of the original logger
> and of the handler(s) are applied.

Thanks, so if I want to see my own log out of all logs produced by
different module in the project, I should addFilter to each
corresponding logger. I thought I could add Filter in root and filter
out only the interested info from it before.

[toc] | [prev] | [next] | [standalone]

#15940

On Nov 17, 9:06 am, sword <john...@gmail.com> wrote:
> On Nov 16, 10:50 pm, Peter Otten <__pete...@web.de> wrote:
>
>
>
>
>
>
>
>
>
> > sword wrote:
> > > Thanks for your reply. I tried to edit the source a bit, now the
> > > main.py looks like this:
> > > #main.py
> > > importlogging
> > > fromloggingimport Filter
> > > import a
> > > import b
>
> > >logging.basicConfig(level=logging.DEBUG)
> > > root =logging.getLogger()
> > > root.addFilter(Filter("GoneWithTheWind")) #so I suppose no log msg
> > > would pass this filter
>
> > > logger =logging.getLogger("main")
> > > logger.debug("main process")
> > > a.print_log()
> > > b.print_log()
>
> > > ####
> > > And It still prints out all the log msg. :(
>
> > Here's a little demo to explore how filtering works:
>
> > $ cat demo.py
> > importlogging
> > class Filter(logging.Filter):
> >     def filter(self, record):
> >         print "applying filter", self.name
> >         return True
>
> >logging.basicConfig()
>
> > loggers = [logging.getLogger(path) for path in ["", "a", "a.b"]]
> > for logger in loggers:
> >     logger.addFilter(Filter("filter@" + logger.name))
>
> > [handler] =logging.getLogger().handlers
> > handler.addFilter(Filter("filter@handler"))
>
> > for logger in loggers:
> >     logger.critical("whatever")
> > $ python demo.py
> > applying filter filter@root
> > applying filter filter@handler
> > CRITICAL:root:whatever
> > applying filter filter@a
> > applying filter filter@handler
> > CRITICAL:a:whatever
> > applying filter fil...@a.b
> > applying filter filter@handler
> > CRITICAL:a.b:whatever
> > $
>
> > As you can infer from the output only the filter(s) of the original logger
> > and of the handler(s) are applied.
>
> Thanks, so if I want to see my own log out of all logs produced by
> different module in the project, I should addFilter to each
> corresponding logger. I thought I could add Filter in root and filter
> out only the interested info from it before.

Or you can add a filter to the handler (but then you can't use
basicConfig() to configure it - you need to do it explicitly).

Regards,

Vinay Sajip

[toc] | [prev] | [next] | [standalone]

#15987

On Nov 20, 8:42 am, Vinay Sajip <vinay_sa...@yahoo.co.uk> wrote:
> On Nov 17, 9:06 am, sword <john...@gmail.com> wrote:
>
>
>
>
>
>
>
>
>
> > On Nov 16, 10:50 pm, Peter Otten <__pete...@web.de> wrote:
>
> > > sword wrote:
> > > > Thanks for your reply. I tried to edit the source a bit, now the
> > > > main.py looks like this:
> > > > #main.py
> > > > importlogging
> > > > fromloggingimport Filter
> > > > import a
> > > > import b
>
> > > >logging.basicConfig(level=logging.DEBUG)
> > > > root =logging.getLogger()
> > > > root.addFilter(Filter("GoneWithTheWind")) #so I suppose no log msg
> > > > would pass this filter
>
> > > > logger =logging.getLogger("main")
> > > > logger.debug("main process")
> > > > a.print_log()
> > > > b.print_log()
>
> > > > ####
> > > > And It still prints out all the log msg. :(
>
> > > Here's a little demo to explore how filtering works:
>
> > > $ cat demo.py
> > > importlogging
> > > class Filter(logging.Filter):
> > >     def filter(self, record):
> > >         print "applying filter", self.name
> > >         return True
>
> > >logging.basicConfig()
>
> > > loggers = [logging.getLogger(path) for path in ["", "a", "a.b"]]
> > > for logger in loggers:
> > >     logger.addFilter(Filter("filter@" + logger.name))
>
> > > [handler] =logging.getLogger().handlers
> > > handler.addFilter(Filter("filter@handler"))
>
> > > for logger in loggers:
> > >     logger.critical("whatever")
> > > $ python demo.py
> > > applying filter filter@root
> > > applying filter filter@handler
> > > CRITICAL:root:whatever
> > > applying filter filter@a
> > > applying filter filter@handler
> > > CRITICAL:a:whatever
> > > applying filter fil...@a.b
> > > applying filter filter@handler
> > > CRITICAL:a.b:whatever
> > > $
>
> > > As you can infer from the output only the filter(s) of the original logger
> > > and of the handler(s) are applied.
>
> > Thanks, so if I want to see my own log out of all logs produced by
> > different module in the project, I should addFilter to each
> > corresponding logger. I thought I could add Filter in root and filter
> > out only the interested info from it before.
>
> Or you can add a filter to the handler (but then you can't use
> basicConfig() to configure it - you need to do it explicitly).
>
> Regards,
>
> Vinay Sajip


Thank you! Maybe I should find out another way to manipulate the log,
like wrap the getLogger function and add the filter at the first
time :)

[toc] | [prev] | [next] | [standalone]

#15993

On Nov 21, 7:15 am, sword <john...@gmail.com> wrote:
>
> Thank you! Maybe I should find out another way to manipulate the log,
> like wrap the getLogger function and add the filter at the first
> time :)

If you are using Python 2.7, 3.2 or later, you can use dictionary-
based configuration - it's fairly painless.

http://docs.python.org/library/logging.config.html#logging.config.dictConfig

If you are using an earlier version of Python, the logutils project
includes the same dictionary-based configuration logic.

http://code.google.com/p/logutils/

Regards,

Vinay Sajip

[toc] | [prev] | [next] | [standalone]

#15995

sword wrote:
> On Nov 16, 7:40 pm, Jean-Michel Pichavant <jeanmic...@sequans.com>
> wrote:
>   
>> sword wrote:
>>     
>>> The logging cookbook gives an Filter example, explainning how to add
>>> contextural info to log. I can't figure out how to filter log from it.
>>>       
>>> Suppose I have 3 file, a.py, b.py and main.py
>>> #file: a.py
>>> import logging
>>>       
>>> logger=logging.getLogger(__name__)
>>> def print_log():
>>>     logger.debug("I'm module a")
>>>       
>>> #file: b.py just like a.py
>>> import logging
>>> logger=logging.getLogger(__name__)
>>> def print_log():
>>>     logger.debug("I'm module b")
>>>       
>>> #file: main.py
>>> import logging
>>> from logging import Filter
>>> logging.basicConfig(level=logging.DEBUG)
>>> logger=logging.getLogger("main")
>>> logger.debug("This is main process")
>>> logger.addFilter(Filter("a"))
>>>       
>>> And I expected that the console output would contain main and b module
>>> log only. But it turned out that all logs there.  Is it the problem of
>>> root logger?
>>>       
>> Hi,
>>
>> First of all, in the code you provided we can't see where you import a &
>> b, and when you call their respective print_log method.
>> Secondly,Filter("a") would allow only the "a" log events, not forbid
>> them. quoting the docs: "if name is specified, it names a logger which,
>> together with its children, will have its events allowed through the
>> filter."
>>
>> As for your problem it may come from the fact that you applied the
>> filter to the 'main' logger, while you probably want to add the filter
>> to the *root* logger. Your current hierarchy is
>>
>> root
>>   - main
>>   - a
>>   - b
>>
>> events fired from 'a' will be handled by the root logger, not the main.
>> root = logging.getLogger()
>> root.addFilter('main')
>> root.addFilter('a')
>> root.addFilter('b')
>>
>> JM
>>     
>
> Thanks for your reply. I tried to edit the source a bit, now the
> main.py looks like this:
> #main.py
> import logging
> from logging import Filter
> import a
> import b
>
> logging.basicConfig(level=logging.DEBUG)
> root = logging.getLogger()
> root.addFilter(Filter("GoneWithTheWind")) #so I suppose no log msg
> would pass this filter
>
> logger = logging.getLogger("main")
> logger.debug("main process")
> a.print_log()
> b.print_log()
>
> ####
> And It still prints out all the log msg. :(
>   
You need to add you filter to the handler(s) of the root logger, not the 
root logger itself.

Filters to loggger object are applied only when the log event is raised, 
i.e. when one of the logging method error, info, warning, debug is called.
In your case, log events are raised by other loggers than the root 
logger, so its filter will not apply.

However any filter applied to the root *handlers* will be applied to any 
log processed by the handler, including log event raised by sub-logger.

root.handlers[-1].addFilter(Filter("GoneWithTheWind")) #so I suppose no log msg

JM

Quoting http://docs.python.org/library/logging.html#filter-objects

"Note that filters attached to handlers are consulted whenever an event is emitted by the handler, whereas filters attached to loggers are consulted whenever an event is logged to the handler (using debug(), info(), etc.) This means that events which have been generated by descendant loggers will not be filtered by a logger’s filter setting, unless the filter has also been applied to those descendant loggers."

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web