Python 2.6.x theading / signals / atexit crash on some versions?

Question

I saw a lot of questions related to this ... but my code works on python 2.6.2 and does not work on python 2.6.5. I am mistaken in thinking that all atexit functions registered through this module do not call when a program is killed by a signal, "the thing should not be taken into account here, because I will catch the signal and then exit cleanly? What is happening here? What is the right way to do this ?

import atexit, sys, signal, time, threading terminate = False threads = [] def test_loop(): while True: if terminate: print('stopping thread') break else: print('looping') time.sleep(1) @atexit.register def shutdown(): global terminate print('shutdown detected') terminate = True for thread in threads: thread.join() def close_handler(signum, frame): print('caught signal') sys.exit(0) def run(): global threads thread = threading.Thread(target=test_loop) thread.start() threads.append(thread) while True: time.sleep(2) print('main') signal.signal(signal.SIGINT, close_handler) if __name__ == "__main__": run() 

python 2.6.2:

 $ python halp.py looping looping looping main looping main looping looping looping main looping ^Ccaught signal shutdown detected stopping thread 

python 2.6.5:

 $ python halp.py looping looping looping main looping looping main looping looping main ^Ccaught signal looping looping looping looping ... looping looping Killed <- kill -9 process at this point 

The main thread on 2.6.5 seems to never perform atexit functions.

Answer 1

The root difference here is not actually associated with both signals and atexit, but rather with a change in the behavior of sys.exit .

Prior to 2.6.5, sys.exit (more precisely, SystemExit caught at the top level) will force the interpreter to exit; if the threads are still running, they will be terminated, as in POSIX threads.

Around 2.6.5, the behavior has changed: the sys.exit effect now essentially coincides with the return from the main function of the program. When you do this - in both versions - the interpreter waits until all threads are connected before exiting.

The corresponding change is that Py_Finalize now calls wait_for_thread_shutdown() near the vertex where it was not there before.

This behavioral change seems to be wrong, primarily because it no longer functions as documented, but simply: "Quit Python". The practical effect is not exiting Python, but simply exiting the stream. (As a note, sys.exit never exited Python when called from another thread, but this implicit discrepancy with the documented behavior does not justify much more.)

I see the attractiveness of the new behavior: instead of two ways to exit the main thread ("exit and wait for threads" and "exit immediately"), there is only one, since sys.exit is essentially identical to the simple returning from the top function. However, this is a terrific change and diverges from documented behavior that far outweighs this.

Due to this change, after sys.exit from the signal handler above, the interpreter sits around waiting for the threads to exit and then starts the atexit handlers after that. Since it is the handler itself that informs the threads about the exit, the result is a dead end.

Answer 2

The output due to the signal does not match the output from the signal handler. Signal capture and output using sys.exit is a clean output, not an output due to a signal handler. So, yes, I agree that atexit handlers should work here - at least in principle.

However, something complicated is in signal handlers: they are completely asynchronous. They can interrupt the program flow at any time, between any VM operating code. Take this code for example. (Treat this as the same form as your code above, I skipped the code for short.)

 import threading lock = threading.Lock() def test_loop(): while not terminate: print('looping') with lock: print "Executing synchronized operation" time.sleep(1) print('stopping thread') def run(): while True: time.sleep(2) with lock: print "Executing another synchronized operation" print('main') 

There is a serious problem here: a signal (e.g. ^ C) can be received, and run () holds the lock . If this happens, your signal handler will be started while retaining the lock. Then it will wait for test_loop to exit, and if this thread is waiting for a lock, you will close.

This is a whole category of problems, and so many APIs say they don't call them from signal handlers. Instead, you should set a flag to inform the main thread at the appropriate time.

 do_shutdown = False def close_handler(signum, frame): global do_shutdown do_shutdown = True print('caught signal') def run(): while not do_shutdown: ... 

My preference is to completely exclude the program using sys.exit and explicitly clear it at the main exit point (e.g. end of run ()), but you can use atexit here if you want.

Answer 3

I'm not sure if this has been reversed, but this is how I made my atexit in 2.6.5

 atexit.register(goodbye) def goodbye(): print "\nStopping..."