Python background process

Question

I need to create a background process that will wait for incoming commands and execute them. Here is the code:

instance_tuple.popen = subprocess.Popen(['python',\ os.path.join(config['scripts_dir'],\ 'instance_script.py')],\ stdin = subprocess.PIPE,\ stdout = subprocess.PIPE) 

Process Function Code:

 if __name__ == '__main__': config = dict() is_config_valid = False print 'Hello from instance process' while True: cmd_str = raw_input() if (cmd_str.strip() != ''): print 'received %s' % cmd_str command = json.loads(cmd_str) print 'received command: %s' % str(command) sys.stdout.flush() if command['name'] == 'set_variable': name = command['args'][0] value = command['args'][1] config[name] = value is_config_valid = validate_instance_dict(config) elif is_config_valid: if (command['name'] == 'init_model'): config['instance'].init_model() elif (command['name'] == 'get_tree'): tree = config['instance'].get_fidesys_tree(command['args']) result = CommandResult(command.name, tree) print 'process exit' 

The way I send data to the process: 1st test run works fine:

 (input, errors) = instance_tuple.popen \ .communicate(json.dumps({'name': 'name', 'args': list()})) 

Later, for some reason, raw_input() gets an EOF, and the process ends. What is the correct way to configure interprocess communication?

Answer 1

I like to use zeromq . I am setting up a server using the zmq.PULL socket, which listens for clients sending messages using the zmq.PUSH socket. It is very easy to work with:

 import zmq def client(msg) context = zmq.Context() client = context.socket(zmq.PUSH) client.connect('tcp://127.0.0.1:9999') client.send(msg) def server(): context = zmq.Context() server = context.socket(zmq.PULL) server.bind('tcp://127.0.0.1:9999') while True: msg = server.recv() ..do something with each message if __name__ == '__main__': server() 

Answer 2

What happens if you use "sys.stdin.readline ()" instead of raw_input?

Answer 3

I believe that binding () closes stdin, which gives your EOF process.

Use popen.stdin.write (...) if you want to talk to him several times.

Answer 4

The subprocess module allows you to do this, but you cannot use the communication function.

I like to use the pexpect module. It is easier!
Below is an example in which an ftp connection is created and the python script interacts with the created process:

  import pexpect import sys child = pexpect.spawn('ftp ftp.openbsd.org') child.expect('name .*: ') child.sendline('anonymous') child.expect('(password') child.sendline(' [e-mail] ') child.expect('ftp> ') child.sendline('cd /pub/OpenBSD/3.7/packages/i386') ... if child.isalive(): child.sendline('bye') # Try to ask ftp child to exit. child.close()