Geek Answers Handbook

What is the most pythonic logging method for multiple modules and multiple handlers with the specified encoding?

I am looking for specific advice on how to make multiple protocols and multiple handler handlers. I added my simplified code here, but I don't want to give in to the answers - tell me what is the best practice.

I would like to write everything to a file and warn and higher for the console.

This is my level0.py , which I want it to register in the specified file:

 import logging from flask import Flask from level1 import function1 app = Flask(__name__) logger = logging.getLogger('logger0') logger.setLevel(logging.DEBUG) file_handler = logging.FileHandler('../logs/logger0','w','utf-8') file_handler.setLevel(logging.DEBUG) file_format = logging.Formatter('%(asctime)s %(levelname)s: %(message)s [in %(pathname)s:%(lineno)d in %(funcName)s]') file_handler.setFormatter(file_format) logger.addHandler(file_handler) console_handler = logging.StreamHandler() console_handler.setLevel(logging.INFO) console_format = logging.Formatter('%(message)s') console_handler.setFormatter(console_format) logger.addHandler(console_handler) @app.route('/', methods=['GET', 'POST']) def function0(foo): bar = function1(foo) logger.debug('function0') ...

In addition, level1 can be a standalone module when calling a script. In this case, I want it to be registered in another file. Below is level1.py (has duplicate logging lines):

 import logging logger = logging.getLogger('level0.level1') from level2 import function2 def function1(foo): bar = function2(foo) logger.debug('function1') ... if __name__ == "__main__": logger = logging.getLogger('logger0') logger.setLevel(logging.DEBUG) file_handler = logging.FileHandler('../logs/logger1','w','utf-8') file_handler.setLevel(logging.DEBUG) file_format = logging.Formatter('%(asctime)s %(levelname)s: %(message)s [in %(pathname)s:%(lineno)d in %(funcName)s]') file_handler.setFormatter(file_format) logger.addHandler(file_handler) console_handler = logging.StreamHandler() console_handler.setLevel(logging.INFO) console_format = logging.Formatter('%(message)s') console_handler.setFormatter(console_format) logger.addHandler(console_handler) bar = function1('foo') logger.info('level1 main') ...

I copied this piece of the log from level0 because I needed the same log and it seemed intuitive to put it in the main one. level2 not autonomous, so it only has:

 import logging logger = logging.getLogger('level0.level1.level2') def function2(foo): logger.info('function2') ....

I started with logging.basicSetup , but could not set the encoding for the file and continued to receive UnicodeEncodeError when trying to write strings without ascii:

 logger.warn(u'foo bar {}'.format(NON_ASCII_STR))

(I still get an error message when the registrar sends a message to their parents)

So, what is the best log design for this case or in general - several modules (with the choice of encoding - I want utf-8)

+4
source share
2 answers

For modules consisting of many parts, I use the method recommended in the documentation , which has only one line per module, logger = logging.getLogger(__name__) . As you noticed, the module does not need to know or care about how and where its messages go, it simply passes it to the registrar, which should have been configured by the main program.

To reduce cut-n-paste, depending on your main program, make sure that your modules have a hierarchy that makes sense, and only has one function somewhere that sets up your journal, which can then be called up by whatever your main desire .

For example, enter logsetup.py:

 import logging def configure_log(level=None,name=None): logger = logging.getLogger(name) logger.setLevel(level) file_handler = logging.FileHandler('../logs/%s' % name,'w','utf-8') file_handler.setLevel(logging.DEBUG) file_format = logging.Formatter('%(asctime)s %(levelname)s: %(message)s [in %(pathname)s:%(lineno)d in %(funcName)s]') file_handler.setFormatter(file_format) logger.addHandler(file_handler) console_handler = logging.StreamHandler() console_handler.setLevel(logging.INFO) console_format = logging.Formatter('%(message)s') console_handler.setFormatter(console_format) logger.addHandler(console_handler)

To have your individual modules have a mode in which they behave as the main one, define a separate function, for example main .

In level0.py and / or level1.py:

 def main(): # do whatever

And in the top-level program itself, call this function:

 import logging from logsetup import configure_log configure_log(logging.DEBUG,'level0') # or 'level1' from level0 import main # or level1 if __name__ == "__main__" main()

You should still have the sentence __name__ == "__main__" , some modules (cough with multiple coughing) have different behavior depending on whether the sentence exists or not.

+8
source

To curiously wrap this, this is what I did; I put the following two lines in each module / file:

 import logging logger = logging.getLogger(__name__)

This sets up the registration, but does not add handlers. Then I add handlers to the root log in the main file, in which I run the imported modules, so I transfer their entries to the root log, and everything will be saved and displayed. I do this as captain Murphy suggested, but with logger = logging.getLogger('') to work with the root log

To access the encoding - I had problems saving strings without ascii for the file. So I just added FileHandler to the root log, where you can specify the encoding. I could not do this using logging.basicConfig .

Thanks again @CaptainMurphy - Sorry, I cannot support you with my low reputation, but I checked the answer.

+4
source

More articles:


All Articles