Preventing threaded subprocess when closing my main script when a child is killed?

Question

Python 2.7.3 on Solaris 10

Questions

• When my subprocess has an internal separation problem (core), or a user externally kills it from the shell using SIGTERM or SIGKILL, my main program signal handler processes SIGTERM (-15), and my parent program terminates. It's real? or is it a poor python build?

Background and code

I have a python script that first spawns a worker control flow. Then, the control workflow spawns one or more workflows. I have other things that happen in my main thread that I cannot block. My management and workflows are robust. My services have been working for years without restarting, but then we have this subprocess.Popen script:

In the run method of a workflow, I use:

 class workerThread(threading.Thread): def __init__(self) : super(workerThread, self).__init__() ... def run(self) ... atempfile = tempfile.NamedTempFile(delete=False) myprocess = subprocess.Popen( ['third-party-cmd', 'with', 'arguments'], shell=False, stdin=subprocess.PIPE, stdout=atempfile, stderr=subprocess.STDOUT,close_fds=True) ... 

I need to use myprocess.poll() to check the completion of the process, because I need to scan atempfile until I find the relevant information (the file may be> 1 GiB), and I need to terminate the process due to a user request or because the process is running too long. As soon as I find what I'm looking for, I will stop checking the temp stdout file. I will clean it after the external process is dead, and before the workflow completes. I need stdin PIPE in case I need to insert a response to something interactive in the stdin stream for a child.

In my main program, I installed a SIGINT and SIGTERM handler to perform a cleanup if my main python program exits with SIGTERM or SIGINT (Ctrl-C) if executed from the shell.

Does anyone have a solid 2.x recipe for handling child signals in streams? ctypes sigprocmask etc.

Any help would be greatly appreciated. I'm just looking for an “official” recipe or a BEST hack , if it even exists.

Notes

I am using a limited build of Python. I have to use 2.7.3. Third-party-cmd is a program for which I have no source - modification is not possible.

Answer 1

There are many things in your description that look weird. First, you have several different threads and processes. Who is crashing, who gets SIGTERM and who gets SIGKILL and due to what operations?

Secondly: why does your parent get SIGTERM? It cannot be implicitly sent. Someone directly or indirectly kills your parent process (for example, killing the entire parent group).

Third point: how does your program end when you process SIGTERM? By definition, a program terminates if it is not processed. If it is processed, it does not stop. What is really going on?

Suggestions:

  $ cat crsh.c #include <stdio.h> int main(void) { int *f = 0x0; puts("Crashing"); *f = 0; puts("Crashed"); return 0; } $ cat a.py import subprocess, sys print('begin') p = subprocess.Popen('./crsh') a = raw_input() print(a) p.wait() print('end') $ python a.py begin Crashing abcd abcd end 

It works. No signal is transmitted to the parent device. Have you isolated a problem in your program?

If the problem is a signal sent to several processes: can you use setpgid to configure a separate process group for a child?

Is there a reason to create a temporary file? These are 1 GB files created in the temporary directory. Why not a conveyor belt?

If you are really sure that you need to process the signals in your parent program (why don’t you try, with the exception of KeyboardInterrupt, for example?): It may signal () unspecified behavior with multi-threaded programs causing these problems (for example, sending a signal to a stream that does not process signals)?

 NOTES The effects of signal() in a multithreaded process are unspecified. 

In any case, try to explain in more detail what topics and the process of your program are, what they do, how the signal handlers were configured and why, who sends the signals, who receives, etc. etc. etc. etc.