How to add a custom log level to the Python logger

Question

I would like to have loglevel TRACE (5) for my application, since I don't think debug() enough. Also, log(5, msg) not what I want. How to add custom log level to Python logger?

I have mylogger.py with the following contents:

 import logging @property def log(obj): myLogger = logging.getLogger(obj.__class__.__name__) return myLogger 

In my code, I use it as follows:

 class ExampleClass(object): from mylogger import log def __init__(self): '''The constructor with the logger''' self.log.debug("Init runs") 

Now I would like to call self.log.trace("foo bar")

Thanks in advance for your help.

Edit (December 8, 2016): I changed the accepted answer to pfa , which is IMHO, a great solution based on the best offer from Eric C.

Answer 1

@ Eric S.

The answer to Eric S. is excellent, but I experimented with the experiment that it would always cause messages written at a new debugging level to be printed - no matter what the log level is for. Therefore, if you produce a new level 9, if you call setLevel (50), lower-level messages will be mistakenly printed. To prevent this from happening, you need another line inside the "debugv" function to check if the logging level is indeed enabled.

Fixed an example that checks if the logging level is enabled:

 import logging DEBUG_LEVELV_NUM = 9 logging.addLevelName(DEBUG_LEVELV_NUM, "DEBUGV") def debugv(self, message, *args, **kws): if self.isEnabledFor(DEBUG_LEVELV_NUM): # Yes, logger takes its '*args' as 'args'. self._log(DEBUG_LEVELV_NUM, message, args, **kws) logging.Logger.debugv = debugv 

If you look at the code for class Logger in logging.__init__.py for Python 2.7, this is what all the standard log functions (.critical, .debug, etc.) do.

I, apparently, cannot post answers to other people's answers due to a lack of reputation ... I hope Eric updates his post if he sees this. =

Answer 2

I accepted the answer "don't see lambda" and had to change where log_at_my_log_level is added. I also saw the problem Paul made: "I don’t think this works. Do you need a logger as the first argument to log_at_my_log_level?" It worked for me

 import logging DEBUG_LEVELV_NUM = 9 logging.addLevelName(DEBUG_LEVELV_NUM, "DEBUGV") def debugv(self, message, *args, **kws): # Yes, logger takes its '*args' as 'args'. self._log(DEBUG_LEVELV_NUM, message, args, **kws) logging.Logger.debugv = debugv 

Answer 3

This question is quite old, but I just touched on the same topic and found a way similar to the one already mentioned, which seems a little cleaner to me. This has been tested on 3.4, so I'm not sure if methods exist in older versions:

 from logging import getLoggerClass, addLevelName, setLoggerClass, NOTSET VERBOSE = 5 class MyLogger(getLoggerClass()): def __init__(self, name, level=NOTSET): super().__init__(name, level) addLevelName(VERBOSE, "VERBOSE") def verbose(self, msg, *args, **kwargs): if self.isEnabledFor(VERBOSE): self._log(VERBOSE, msg, args, **kwargs) setLoggerClass(MyLogger) 

Answer 4

Combining all the existing answers with a ton of usage experience, I think I have a list of all the things that need to be done to ensure full use of the new level. In the steps below, suppose you add a new TRACE level with the value logging.DEBUG - 5 == 5 :

• logging.addLevelName(logging.DEBUG - 5, 'TRACE') must be called to get the new level registered internally so that it can be referenced by name.
• A new level should be added as an attribute for logging for consistency: logging.TRACE = logging.DEBUG - 5 .
• You must add the TRACE method to the logging module. It should behave just like debug , info , etc.
• You must add a method named TRACE to the current log class. Since this is not 100% guaranteed by logging.Logger , use logging.getLoggerClass() instead.

All steps are illustrated as follows:

 def addLoggingLevel(levelName, levelNum, methodName=None): """ Comprehensively adds a new logging level to the `logging` module and the currently configured logging class. `levelName` becomes an attribute of the `logging` module with the value `levelNum`. `methodName` becomes a convenience method for both `logging` itself and the class returned by `logging.getLoggerClass()` (usually just `logging.Logger`). If `methodName` is not specified, `levelName.lower()` is used. To avoid accidental clobberings of existing attributes, this method will raise an `AttributeError` if the level name is already an attribute of the `logging` module or if the method name is already present Example ------- >>> addLoggingLevel('TRACE', logging.DEBUG - 5) >>> logging.getLogger(__name__).setLevel("TRACE") >>> logging.getLogger(__name__).trace('that worked') >>> logging.trace('so did this') >>> logging.TRACE 5 """ if not methodName: methodName = levelName.lower() if hasattr(logging, levelName): raise AttributeError('{} already defined in logging module'.format(levelName)) if hasattr(logging, methodName): raise AttributeError('{} already defined in logging module'.format(methodName)) if hasattr(logging.getLoggerClass(), methodName): raise AttributeError('{} already defined in logger class'.format(methodName)) # This method was inspired by the answers to Qaru post # http://stackoverflow.com/q/2183233/2988730, especially # http://stackoverflow.com/a/13638084/2988730 def logForLevel(self, message, *args, **kwargs): if self.isEnabledFor(levelNum): self._log(levelNum, message, args, **kwargs) def logToRoot(message, *args, **kwargs): logging.log(levelNum, message, *args, **kwargs) logging.addLevelName(levelNum, levelName) setattr(logging, levelName, levelNum) setattr(logging.getLoggerClass(), methodName, logForLevel) setattr(logging, methodName, logToRoot) 

Answer 5

Who started the bad practice of using internal methods ( self._log ) and why is each answer based on this? The python solution would be to use self.log instead, so that you don't have to interfere with any internal things:

 import logging SUBDEBUG = 5 logging.addLevelName(SUBDEBUG, 'SUBDEBUG') def subdebug(self, message, *args, **kws): self.log(SUBDEBUG, message, *args, **kws) logging.Logger.subdebug = subdebug logging.basicConfig() l = logging.getLogger() l.setLevel(SUBDEBUG) l.subdebug('test') l.setLevel(logging.DEBUG) l.subdebug('test') 

Answer 6

It’s easier for me to create a new attribute for the logger object that passes the log () function. I think the logger module provides addLevelName () and log () for exactly this reason. Thus, no subclasses or new method is required.

 import logging @property def log(obj): logging.addLevelName(5, 'TRACE') myLogger = logging.getLogger(obj.__class__.__name__) setattr(myLogger, 'trace', lambda *args: myLogger.log(5, *args)) return myLogger 

Now

 mylogger.trace('This is a trace message') 

should work as expected.

Answer 7

I think you will have to subclass the Logger class and add a method called trace , which basically calls Logger.log with a level below DEBUG . I have not tried this, but that is what indicate.

Answer 8

Tips for creating a custom registrar:

• Do not use _log , use log (you do not need to check isEnabledFor )
• the logging module should be the one that creates the user registrar instance, as it does some magic in getLogger , so you will need to set the class via setLoggerClass
• You do not need to define __init__ for the log, class, if you do not store anything

 # Lower than debug which is 10 TRACE = 5 class MyLogger(logging.Logger): def trace(self, msg, *args, **kwargs): self.log(TRACE, msg, *args, **kwargs) 

When calling this registrar, use setLoggerClass(MyLogger) to make this the default registrar from getLogger

 logging.setLoggerClass(MyLogger) log = logging.getLogger(__name__) # ... log.trace("something specific") 

You will need setFormatter , setHandler and setLevel(TRACE) on the handler and on the log itself in order to actually perform this low level trace

Answer 9

This worked for me:

 import logging logging.basicConfig( format=' %(levelname)-8.8s %(funcName)s: %(message)s', ) logging.NOTE = 32 # positive yet important logging.addLevelName(logging.NOTE, 'NOTE') # new level logging.addLevelName(logging.CRITICAL, 'FATAL') # rename existing log = logging.getLogger(__name__) log.note = lambda msg, *args: log._log(logging.NOTE, msg, args) log.note('school\ out for summer! %s', 'dude') log.fatal('file not found.') 

The lambda / funcName problem is fixed using logger._log, as pointed out by @marqueed. I think using lambda looks a little cleaner, but the disadvantage is that it cannot accept keyword arguments. I have never used this myself, so it’s not a biggie.

   NOTE setup: school out for summer!  dude
   FATAL setup: file not found.

Answer 10

In my experience, this is a complete solution to the op problem ... so as not to see lambda as the function in which the message is emitted, go deeper:

 MY_LEVEL_NUM = 25 logging.addLevelName(MY_LEVEL_NUM, "MY_LEVEL_NAME") def log_at_my_log_level(self, message, *args, **kws): # Yes, logger takes its '*args' as 'args'. self._log(MY_LEVEL_NUM, message, args, **kws) logger.log_at_my_log_level = log_at_my_log_level 

I have never tried working with a standalone logger class, but I think the main idea is the same (use _log).

Answer 11

Add the Mad Physicists example to get the correct file name and line number:

 def logToRoot(message, *args, **kwargs): if logging.root.isEnabledFor(levelNum): logging.root._log(levelNum, message, args, **kwargs) 

Answer 12

As an alternative to adding an additional method to the Logger class, I would recommend using the Logger.log(level, msg) method.

 import logging TRACE = 5 logging.addLevelName(TRACE, 'TRACE') FORMAT = '%(levelname)s:%(name)s:%(lineno)d:%(message)s' logging.basicConfig(format=FORMAT) l = logging.getLogger() l.setLevel(TRACE) l.log(TRACE, 'trace message') l.setLevel(logging.DEBUG) l.log(TRACE, 'disabled trace message') 

Answer 13

I do not quite understand; with python 3.5, at least it just works:

 import logging TRACE = 5 """more detail than debug""" logging.basicConfig() logging.addLevelName(TRACE,"TRACE") logger = logging.getLogger('') logger.debug("n") logger.setLevel(logging.DEBUG) logger.debug("y1") logger.log(TRACE,"n") logger.setLevel(TRACE) logger.log(TRACE,"y2") 

exit:

DEBUG: root: y1

TRACE: root: y2

Answer 14

based on the pinned answer, I wrote a small method that automatically creates new levels of logging

 def set_custom_logging_levels(config={}): """ Assign custom levels for logging config: is a dict, like { 'EVENT_NAME': EVENT_LEVEL_NUM, } EVENT_LEVEL_NUM can't be like already has logging module logging.DEBUG = 10 logging.INFO = 20 logging.WARNING = 30 logging.ERROR = 40 logging.CRITICAL = 50 """ assert isinstance(config, dict), "Configuration must be a dict" def get_level_func(level_name, level_num): def _blank(self, message, *args, **kws): if self.isEnabledFor(level_num): # Yes, logger takes its '*args' as 'args'. self._log(level_num, message, args, **kws) _blank.__name__ = level_name.lower() return _blank for level_name, level_num in config.items(): logging.addLevelName(level_num, level_name.upper()) setattr(logging.Logger, level_name.lower(), get_level_func(level_name, level_num)) 

the config may be something like this:

 new_log_levels = { # level_num is in logging.INFO section, that why it 21, 22, etc.. "FOO": 21, "BAR": 22, } 

Answer 15

If someone wants to automate the way of adding a new level of logging to the logging module (or its copy) dynamically, I created this function by extending @pfa's answer:

 def add_level(log_name,custom_log_module=None,log_num=None, log_call=None, lower_than=None, higher_than=None, same_as=None, verbose=True): ''' Function to dynamically add a new log level to a given custom logging module. <custom_log_module>: the logging module. If not provided, then a copy of <logging> module is used <log_name>: the logging level name <log_num>: the logging level num. If not provided, then function checks <lower_than>,<higher_than> and <same_as>, at the order mentioned. One of those three parameters must hold a string of an already existent logging level name. In case a level is overwritten and <verbose> is True, then a message in WARNING level of the custom logging module is established. ''' if custom_log_module is None: import imp custom_log_module = imp.load_module('custom_log_module', *imp.find_module('logging')) log_name = log_name.upper() def cust_log(par, message, *args, **kws): # Yes, logger takes its '*args' as 'args'. if par.isEnabledFor(log_num): par._log(log_num, message, args, **kws) available_level_nums = [key for key in custom_log_module._levelNames if isinstance(key,int)] available_levels = {key:custom_log_module._levelNames[key] for key in custom_log_module._levelNames if isinstance(key,str)} if log_num is None: try: if lower_than is not None: log_num = available_levels[lower_than]-1 elif higher_than is not None: log_num = available_levels[higher_than]+1 elif same_as is not None: log_num = available_levels[higher_than] else: raise Exception('Infomation about the '+ 'log_num should be provided') except KeyError: raise Exception('Non existent logging level name') if log_num in available_level_nums and verbose: custom_log_module.warn('Changing ' + custom_log_module._levelNames[log_num] + ' to '+log_name) custom_log_module.addLevelName(log_num, log_name) if log_call is None: log_call = log_name.lower() setattr(custom_log_module.Logger, log_call, cust_log) return custom_log_module