Why is my TimedRotatingFileHandler not turning at midnight?

Question

This is my configuration file:

[loggers] keys=root [handlers] keys=TimedRotatingFileHandler [formatters] keys=simpleFormatter [logger_root] level=DEBUG handlers=TimedRotatingFileHandler [handler_TimedRotatingFileHandler] class=handlers.TimedRotatingFileHandler level=DEBUG formatter=simpleFormatter args=('driver.log', 'midnight', 1, 30) [formatter_simpleFormatter] format=%(asctime)s - %(name)s - %(levelname)s - %(message)s datefmt= 

In my code, I install and use the logger as follows:

 import logging import logging.config logging.config.fileConfig('logging.conf') logging.info('Some message...') 

Messages are logged in the file I specify (driver.log), but rotations at midnight never happen.

Should the process run at midnight to turn? This is a batch process that I run every 15 minutes and it never runs at midnight.

Answer 1

The answer is that the process must work all the time for it to work correctly.

From http://bytes.com/topic/python/answers/595931-timedrotatingfilehandler-isnt-rotating-midnight :

The rotation should occur when the logging process creates a handler before midnight and logs a call for that handler after midnight.

Answer 2

I would suggest that this really only happens when the process starts at midnight. In your case (cronjob doesn't work very long), you have to go with a simple log file where the current date is added to logfilename. Thus, the "rollover" occurs automatically.

Answer 3

I also ran into this problem, for various reasons I could not use rotatelog, and cron to rotate the logs just adds an extra thing that could go wrong. I used the function below for daily file rotation.

 import os import datetime import glob def sort_number_ext(s): try: return int(os.path.splitext(s)[1][1:]) except: return s def rotate_file(file, keep=30): """ Rotate a file if needed. If the file wasn't modified today then we rotate it around and remove old files """ modified_date = datetime.datetime.fromtimestamp(os.path.getmtime(file)) if modified_date.date() == datetime.datetime.today().date(): return False old_files = glob.glob(file + ".*") old_files.sort(key=sort_number_ext, reverse=True) for f in old_files: try: number = int(os.path.splitext(f)[1][1:]) except ValueError: continue if number >= keep: # If at or above keep limit, remove. os.unlink(f) else: # Increment. new = "%s.%s" % (os.path.splitext(f)[0], number + 1) os.rename(f, new) # Finally rename our log. os.rename(file, "%s.1" % file) return True 

I call this to rotate my logs before logger initialization.