Python Progress Bar Protocol Control Module

Question

I have seen various progress bar solutions in Python, but simple stdout solutions do not work for my project. I have several classes and use the "logging" module to output information to STDOUT. I have a function from which I want to show the progress bar on one line, each time flushing the buffer.

An example of a simple move:

for i in range(100): time.sleep(1) sys.stdout.write("\r%d%%" %i) sys.stdout.flush() 

When I try to write via STDOUT and then clear the buffer, either the buffer is not flushed or progress is not going anywhere. I hope to avoid any thread or complex process to make this possible. Does anyone have a preferred way to do this?

Answer 1

I could not find a good solution for this, so I wrote an enlighten progress bar to handle this. This basically changes the scroll area of ​​the terminal, so registration is performed on the progress indicator (s) rather than redrawing the progress indicator (s) every time you want to write to STDOUT. This allows you to write to the terminal as much as you want, without the need to make changes to the logs , print , etc.

 import logging import time import enlighten # Setup logging logging.basicConfig(level=logging.INFO) logger = logging.getLogger() # Setup progress bar manager = enlighten.get_manager() pbar = manager.counter(total=100, desc='Ticks', unit='ticks') for i in range(1, 101): logger.info("Processing step %s" % i) time.sleep(.2) pbar.update() 

Answer 2

I solved it like this:

 import logging import time from tqdm import tqdm import io class TqdmToLogger(io.StringIO): """ Output stream for TQDM which will output to logger module instead of the StdOut. """ logger = None level = None buf = '' def __init__(self,logger,level=None): super(TqdmToLogger, self).__init__() self.logger = logger self.level = level or logging.INFO def write(self,buf): self.buf = buf.strip('\r\n\t ') def flush(self): self.logger.log(self.level, self.buf) if __name__ == "__main__": logging.basicConfig(format='%(asctime)s [%(levelname)-8s] %(message)s') logger = logging.getLogger() logger.setLevel(logging.DEBUG) tqdm_out = TqdmToLogger(logger,level=logging.INFO) for x in tqdm(range(100),file=tqdm_out,mininterval=30,): time.sleep(.5) 

Exit

 2016-12-19 15:35:06 [INFO ] 16%|#####9 | 768/4928 [07:04<40:50, 1.70it/s] 2016-12-19 15:36:07 [INFO ] 18%|######6 | 865/4928 [08:04<40:34, 1.67it/s] 

Answer 3

You can use tqdm progress bar with a custom handler through logging, as described here :

 import logging import time import colorlog from tqdm import tqdm class TqdmHandler(logging.StreamHandler): def __init__(self): logging.StreamHandler.__init__(self) def emit(self, record): msg = self.format(record) tqdm.write(msg) if __name__ == "__main__": for x in tqdm(range(100)): logger = colorlog.getLogger("MYAPP") logger.setLevel(logging.DEBUG) handler = TqdmHandler() handler.setFormatter(colorlog.ColoredFormatter( '%(log_color)s%(name)s | %(asctime)s | %(levelname)s | %(message)s', datefmt='%Y-%d-%d %H:%M:%S', log_colors={ 'DEBUG': 'cyan', 'INFO': 'white', 'SUCCESS:': 'green', 'WARNING': 'yellow', 'ERROR': 'red', 'CRITICAL': 'red,bg_white'},)) logger.addHandler(handler) logger.debug("Inside subtask: "+str(x)) time.sleep(.5) 

Answer 4

If you know the progress bar is always written to STDOUT, you should use print instead of the logger. See the documentation in the documentation for the Python protocol documentation