Why doesn't a filter attached to the root log apply to descendants?

Question

From paragraph 15.7.4 of the python protocol documentation :

Please note that filters attached to handlers are checked whenever an event is emitted by a handler, while filters attached to registrars are consulted whenever an event is registered by a handler (using debug (), info (), etc. ) This means that events that were created by registering descendant flows will not be filtered using the log filter parameter if the filter was not applied to these descendant registrars.

I do not understand this design decision. Wouldn't it be more reasonable for the root registrar filter to also be applied to the descendant registrars?

Answer 1

I agree: this is a controversial design decision, IMHO.

The simplest solution is to attach a filter to each available handler. For example, let's say you have a console handler, a mail handler and a database handler, you must attach your "root filter" to each of them.: - /

import logging import logging.config class MyRootFilter(logging.Filter): def filter(self, record): # filter out log messages that include "secret" if "secret" in record.msg: return False else: return True LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'filters': { 'my_root_filter': { '()': MyRootFilter, }, }, 'handlers': { 'stderr': { 'level': 'DEBUG', 'class': 'logging.StreamHandler', 'filters': ['my_root_filter'], }, 'mail_admins': { 'level': 'ERROR', 'class': 'some.kind.of.EmailHandler', 'filters': ['my_root_filter'], }, 'database': { 'level': 'ERROR', 'class': 'some.kind.of.DatabaseHandler', 'filters': ['my_root_filter'], }, }, 'loggers': { 'some.sub.project': { 'handlers': ['stderr'], 'level': 'ERROR', }, }, } logging.config.dictConfig(LOGGING) logging.getLogger("some.sub.project").error("hello") # logs 'hello' logging.getLogger("some.sub.project").error("hello secret") # filtered out! :-) 

If there are many handlers, you might want to attach your root filter to each handler programmatically, rather than manually. I recommend that you do this directly in your configuration dictionary (or file, depending on how you load the logging configuration), instead of doing it after loading the configuration, since there seems to be no documentary way to get a list of all the handlers, I found logger.handlers and logging._handlers, but since they are not documented, they may break in the future. In addition, there is no guarantee that they are thread safe.

The previous solution (attaching your root filter to each handler directly in the configuration before loading it) assumes that you control the logging configuration before loading it, and also that no handler will be added dynamically (using Logger # AddHandler ()). If this is not the case, you may want to decapitate the registration module (good luck with that!).

change

I made an attempt to intercept the Logger monkey # addHandler, just for fun. It really works great and simplifies setup, but I'm not sure I recommend it (I hate beheading monkeys, it’s very difficult to debug when something goes wrong). Use your risks ...

 import logging import logging.config class MyRootFilter(logging.Filter): [...] # same as above LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'handlers': { 'stderr': { 'level': 'DEBUG', 'class': 'logging.StreamHandler', # it shorter: there no explicit reference to the root filter }, [...] # other handlers go here }, 'loggers': { 'some.sub.project': { 'handlers': ['stderr'], 'level': 'ERROR', }, }, } def monkey_patched_addHandler(self, handler): result = self.old_addHandler(handler) self.addFilter(MyRootFilter()) return result logging.Logger.old_addHandler = logging.Logger.addHandler logging.Logger.addHandler = monkey_patched_addHandler logging.config.dictConfig(LOGGING) logging.getLogger("some.sub.project").error("hello") # logs 'hello' logging.getLogger("some.sub.project").error("hello secret") # filtered out! :-) 

Answer 2

Think of it this way. Lumberjacks look like drain pipes from your home. The filter on the recorder does not allow you to dump material into the sewer, it does not filter the entire sewer. If you are in a stream (upstream, downstream), this does not change this behavior.

Handlers are pipes. Pipes accumulate flow upstream. The default is skip, pass to parent. If you want to influence the flow upstream, you need to place the filter out of the tube (handler). If you look at the logging flowchart , you should add a NullHandler (without formatting or output), which then filters the message.

This is the behavior you desire.