How to check status or kill external process using python

Question

I have a python script that runs on my web server. The main function is called when it returns, it just sleeps for a few seconds and is called again. The goal is to pick up all the new uploaded videos uploaded by users and convert them to a website, pull out the middle frame as an image and a bunch of other funky stuff. I am using an external ffmpeg call. Below is a snippet of code, as I call it.

duration = output[durationIndex+10:durationIndex+18] durationBits = duration.split(":") lengthInSeconds = (int(durationBits[0])*60*60) + (int(durationBits[1])*60) + (int(durationBits[2])) child = subprocess.Popen(["ffmpeg","-y","-i",sourceVideo,"-f","mjpeg","-vframes","1","-ss",str(lengthInSeconds/2),destination], shell=True, stderr=subprocess.PIPE) output = "" while True: out = child.stderr.read(1) if out == '' and child.poll() != None: break if out != '': output += out updateSQL = "update `videos_graduatevideo` set thumbnail = '" + str(destination) + "' where `original_video` = '" + sourceVideo + "'" cursor.execute(updateSQL) 

This script runs on a computer running Windows, but I will most likely deploy it on a Unix system when it is complete.

Problem. I need this python script to continue working. If something goes wrong when ffmpeg and my script freeze, the videos uploaded by the user will just be in a wait state until I mix up the python script. I know that the specific mov file that I have, ffmpeg hangs indefinitely. Is there any way to check how long the process has been running and then kill it if it runs too long?

Answer 1

I agree with S. Lott that it seems to be useful for you to consider MQ for your architecture, but for this particular problem, I think your use of Popen is OK.

For each process you create, save the creation time (something like datetime.datetime.today() will be enough). Then, every minute or so, go through the list of open processes and time and retrieve those that shouldn't be there using Popen.send_signal (signal), terminate (), or kill ().

Example:

 import time from subprocess import Popen from datetime import datetime jobs = [] max_life = 600 # in seconds def reap_jobs(jobs): now = datetime.datetime.today() for job in jobs: if job[0] < now - datetime.timedelta(seconds=max_life) job[1].kill() # remove the job from the list if you want. # but remember not to do it while iterating over the list for video in list_of_videos: time = datetime.datetime.today() job = Popen(...) jobs.append((time,child)) while True: reap_jobs(jobs) time.sleep(60) 

Answer 2

Since the control script is the one that ran it, and since you want it to be killed depending on the time and not the use of system resources, it should be pretty simple. Below is an example of your code with some changes; find the comment lines.

 import time timeout = 60 #child is allowed to run for 1 minute. duration = output[durationIndex+10:durationIndex+18] durationBits = duration.split(":") lengthInSeconds = (int(durationBits[0])*60*60) + (int(durationBits[1])*60) + (int(durationBits[2])) child = subprocess.Popen(["ffmpeg","-y","-i",sourceVideo,"-f","mjpeg","-vframes","1","-ss",str(lengthInSeconds/2),destination], shell=True, stderr=subprocess.PIPE) killtime = time.time() + timeout #timestamp after which the child process should be killed output = "" while True: out = child.stderr.read(1) if out == '' and child.poll() != None: break if out != '': output += out if time.time() > killtime: #check if 60 seconds have passed child.kill() #tell the child to exit raise RuntimeError("Child process still going %i seconds after launch" %killtime) #raise an exception so that updateSQL doesn't get executed updateSQL = "update `videos_graduatevideo` set thumbnail = '" + str(destination) + "' where `original_video` = '" + sourceVideo + "'" cursor.execute(updateSQL) 

You can change the RuntimeError to something else or set a flag instead of throwing an exception, depending on what you need for this. The line child.kill () will lead to the death of the child process, but this may not be the most elegant way to terminate it. If you host it on a posix system, you can instead use os.system ('kill -s 15% i'% child.pid) to kill it more elegantly.

Answer 3

Look God - The Process Monitor , which monitors the process you specify and performs some actions in accordance with your monitoring status. For example, it can monitor CPU usage and restart the process if CPU usage exceeds 50%:

 # code in Ruby # copyied from the documentation w.restart_if do |restart| restart.condition(:cpu_usage) do |c| c.above = 50.percent c.times = 5 end end 

Answer 4

There is a python module that provides an interface for extracting information about all running processes and using the system (CPU, disk, memory) in a portable way, implementing many functions offered by command-line tools, such as: ps, top, df, kill, free, lsof , free, netstat, ifconfig, nice, ionice, iostato, iotop, uptime, tty: psutil . This should help.

Answer 5

Step 1. Do not use CGI scripts. Use the framework.

Step 2. Do not start the subprocess directly in the function that creates the response. Use celery .

this process just works on the server all the time. It is independent of any structure and reads from the same db that django fills

Step 2, again. Do not leave this subprocess all the time. Use celery so that it starts when a request arrives, processes this request (and only this request), and then stops.