In a Python thread, how can I track the end of a thread?

Question

I have a python program that generates multiple threads. These flows last from 2 seconds to 30 seconds. In the main thread, I want to track every time that each thread completes and prints a message. If I just sequentially .join () all the threads and the first thread lasts 30 seconds and the others end much earlier, I cannot print the message earlier - all messages will be printed in 30 seconds.

Basically I want to block until the any thread is completed. As soon as the thread completes, print a message about it and return to blocking if any other threads are still alive. If all threads are complete, then exit the program.

One way that I could think of is to have a queue that is passed to all threads and block queue.get (). Whenever a message is received from a queue, print it, check if any other threads are alive using threading.active_count (), and if so, return to blocking on queue.get (). This will work, but here all threads must follow the discipline of sending a message to the queue before terminating.

I am wondering if this is the usual way to achieve this behavior or are there any other / better ways?

Answer 1

Here's the @detly answer option, which allows you to specify messages from the main stream, rather than printing them from your target functions. This creates a wrapper function that calls your target, and then prints a message before terminating. You can change this to perform any standard cleanup after each thread completes.

#!/usr/bin/python import threading import time def target1(): time.sleep(0.1) print "target1 running" time.sleep(4) def target2(): time.sleep(0.1) print "target2 running" time.sleep(2) def launch_thread_with_message(target, message, args=[], kwargs={}): def target_with_msg(*args, **kwargs): target(*args, **kwargs) print message thread = threading.Thread(target=target_with_msg, args=args, kwargs=kwargs) thread.start() return thread if __name__ == '__main__': thread1 = launch_thread_with_message(target1, "finished target1") thread2 = launch_thread_with_message(target2, "finished target2") print "main: launched all threads" thread1.join() thread2.join() print "main: finished all threads" 

Answer 2

The thread must be checked by calling Thread.is_alive() .

Answer 3

Why aren't the threads themselves just printing a completion message or calling some other completion callback when this is done?

You can simply join these threads from your main program, so you will see a bunch of completion messages, and your program will end when everything is done, as needed.

Here is a quick and easy demo:

 #!/usr/bin/python import threading import time def really_simple_callback(message): """ This is a really simple callback. `sys.stdout` already has a lock built-in, so this is fine to do. """ print message def threaded_target(sleeptime, callback): """ Target for the threads: sleep and call back with completion message. """ time.sleep(sleeptime) callback("%s completed!" % threading.current_thread()) if __name__ == '__main__': # Keep track of the threads we create threads = [] # callback_when_done is effectively a function callback_when_done = really_simple_callback for idx in xrange(0, 10): threads.append( threading.Thread( target=threaded_target, name="Thread #%d" % idx, args=(10 - idx, callback_when_done) ) ) [t.start() for t in threads] [t.join() for t in threads] # Note that thread #0 runs for the longest, but we'll see its message first! 

Answer 4

What I would suggest is a loop like this

 while len(threadSet) > 0: time.sleep(1) for thread in theadSet: if not thread.isAlive() print "Thread "+thread.getName()+" terminated" threadSet.remove(thread) 

There is a 1 second sleep, so there will be a slight delay between the end of the stream and the printed message. If you can live with this delay, I think this is a simpler solution than the one you proposed in your question.

Answer 5

You can allow threads to contribute their results to threading.Queue . Ask another thread to wait in this queue and print a message as soon as a new item appears.

Answer 6

I'm not sure I see a problem using: threading.activeCount ()

to track the number of threads that are still active?

Even if you do not know how many threads you are going to start before starting, it is quite easy to track. I usually generate thread collections through list comprehension, then a simple comparison using activeCount with the size of the list can tell you how many of them ended.

See here: http://docs.python.org/library/threading.html

Alternatively, if you have stream objects, you can simply use the .isAlive method on stream objects to check.

I just checked by throwing this into the multi-threaded program that I have, and it looks fine:

 for thread in threadlist: print(thread.isAlive()) 

Gives me a True / False list when threads turn on and off. So you have to do this and check for something False to find out if any thread has completed.

Answer 7

I use a slightly different technique because of the nature of the threads that I used in my application. To illustrate this, this is a piece of the test belt program that I wrote to raise the barrier class for my thread class:

  while threads: finished = set(threads) - set(threading.enumerate()) while finished: ttt = finished.pop() threads.remove(ttt) time.sleep(0.5) 

Why am I doing this? In my production code, I have a time limit, so the first line actually reads "while threads and time.time () <cutoff_time". If I get to the cutoff, then I have code to disable threads.