How to run a subprocess in python background

Question

I am writing a python application that initiates a JVM in java by invoking a shell script using the python subprocess. However, my problem is that the correct way I wrote it starts the JVM and blocks the rest of the processes that happen after it. I need the JVM to start while I call another function, and I need to stop the JVM after the process completes.

Python Code:

process = subprocess.Popen('runJVM.sh', shell=True, stderr=subprocess.STDOUT) process.wait() r = Recommender() r.load() assert len(sys.argv) > 1, '%d arguments supplied, one needed' %(len(sys.argv)-1) print "recommendations" + str(r.get_users_recommendation(sys.argv[1:])) .... def get_users_recommendation(self, user_list): j_id_list = ListConverter().convert(class_list, self.gateway._gateway_client) recs = self.gateway.entry_point.recommend(j_id_list) return recs 

Where:

  from py4j.java_gateway import JavaGateway self.gateway = JavaGateway() 

I cannot run get_user_recommendations because the JVM server is blocking the process. How can I not block the rest of the Python script and then kill it as soon as the python methods shut down and return a value? Many thanks.

Answer 1

process.wait() blocks your process. Delete this line and the rest of the code will run. You can then complete it by calling Popen.terminate()

Popen calls CreateProcess, which means the OS starts a new process and returns the PID that will be stored in your process variable.

If you use process.wait() , then your code will wait for the recently created process to complete (it never ends in this case if you don't terminate it from the outside).

If you do not call process.wait() , your code will continue to execute. But you still have control over another process, and you can do things like process.terminate() or even process.kill() if necessary.