How to determine process id in Python

Question

I work with a cluster system via linux (www.mosix.org), which allows me to run tasks and run them on different computers. Tasks run as follows:

mosrun ls & 

This, of course, will create the process and start it in the background, returning the process identifier, for example:

 [1] 29199 

He will return later. I am writing a Python framework that will run tasks and control them. To do this, I want to run tasks using mosrun, as described above, and save the process ID of the generated process (29199 in this case). This, of course, cannot be done using os.system or commands.getoutput, since the printed identifier is not what the process prints for output ... Any hints?

Edit

Since the python script is only for the initial launch of the script, the scripts should run longer than the python shell. I assume this means that the mosrun process cannot be a child of the script process. Any suggestions?

thanks

Answer 1

It looks like you want the child process to be daemonized - PEP 3143, which I point to, documents and points to the reference implementation for this, and also points to others.

As soon as your process (still running Python code) is demonized, be it using the tools offered in PEP 3143 or others, you can os.execl (or another function os.exec... ) your target code - this runs the specified target code exactly in the same process that we just said is demonized, and thus it continues to be demonized at will.

The last step cannot use subprocess , because it must run in the same (demonized) process, overlaying its executable code - exactly that for os.execl and for friends.

The first step before demonizing it would be to do this through subprocess , but it is somewhat inconvenient (you need to put the daemonize-then-os.exec code in a separate .py ): most often you just want os.fork and immediately os.fork child process.

subprocess pretty convenient as it is basically a cross-platform way to start other processes, but it cannot really replace the good old Unix fork and exec approach for extended use (like demonization) - that’s why it’s good that the standard Python library also allows you to do the latter using these functions in the os module! -)

Answer 2

Use subprocess . Popen instances have the pid attribute.

Answer 3

Thank you all for your help. Here is what I did at the end, and it seems to be working fine. The code uses python-daemon . Maybe you need to do something smarter about passing the process identifier from child to father, but it's easier.

 import daemon def run_in_background(command, tmp_dir="/tmp"): # Decide on a temp file beforehand warnings.filterwarnings("ignore", "tempnam is a potential security") tmp_filename = os.tempnam(tmp_dir) # Duplicate the process pid = os.fork() # If we're child, daemonize and run if pid == 0: with daemon.DaemonContext(): child_id = os.getpid() file(tmp_filename,'w').write(str(child_id)) sp = command.split(' ') os.execl(*([sp[0]]+sp)) else: # If we're a parent, poll for the new file n_iter = 0 while True: if os.path.exists(tmp_filename): child_id = int(file(tmp_filename, 'r').read().strip()) break if n_iter == 100: raise Exception("Cannot read process id from temp file %s" % tmp_filename) n_iter += 1 time.sleep(0.1) return child_id