Check on `concurrent.futures.ThreadPoolExecutor`

Question

I have a live concurrent.futures.ThreadPoolExecutor . I want to check his status. I want to know how many threads exist, how many processing tasks and what tasks, how many free and which tasks are in the queue. How can I find out?

Answer 1

There is some visibility to the pool and queue of pending work items. To find out what is available, type poolx.__dict__ to see the structure. Read the threadpool code, it's pretty good: concurrent.futures.thread

The following creates a single-thread pool. Then he creates two works: one sleeps for 3 seconds, the other returns immediately. The pool number of pending work items is then printed.

After that, we print out the items from the work queue. In this case, the thread is already executing time.sleep(3) , so it is not in the queue. The sleep with args [0] and kwargs {} function prints because this is the next work item for the pool to run.

Kudos to @dano for a non-destructive understanding of the lineup, and @abarnert.

a source

 import concurrent.futures, time poolx = concurrent.futures.ThreadPoolExecutor(max_workers=1) poolx.submit(time.sleep, 3) poolx.submit(time.sleep, 0) # very fast print('pending:', poolx._work_queue.qsize(), 'jobs') print('threads:', len(poolx._threads)) print() # TODO: make thread safe; work on copy of queue? print('Estimated Pending Work Queue:') for num,item in enumerate(poolx._work_queue.queue): print('{}\t{}\t{}\t{}'.format( num+1, item.fn, item.args, item.kwargs, )) poolx.shutdown(wait=False) 

exit

 pending: 1 jobs threads: 1 Pending Work Queue: 1 <built-in function sleep> (0,) {} 

Answer 2

Not a very clean and reliable way to find pending futures, but I do it like this:

 if 'state=pending' in str(future): logger.debug('PENDING') elif future.running(): logger.debug('RUNNING') elif future.cancelled(): logger.debug('CANCELLED') elif future.exception(): logger.debug('EXCEPTION') elif future.done(): logger.debug('DONE')