Can you communicate only once using a subprocess?

Question

communicate in the documentation says:

Interaction with the process: sending data to stdin. Read data from stdout and stderr until the end of the file is reached. Wait for the process to complete.

What will you do if you need to send input to the process more than once? For example, I start a process, send it some data, does the process do something with this, return some output, and then I need to send the input again? How do I handle this?

Answer 1

Do not use subprocess for this, you will enter the world of pain associated with buffering.

I recommend pexpect for this purpose - it works very well. Unfortunately, it does not work under windows at the moment, although I heard about a port (which I can no longer find).

Answer 2

Then you cannot use .communicate() . You can poll threads, use select or in some other way that allows you to listen to FD changes (for example, gtk and Qt have tools for this).

Answer 3

Take a look at the Doug Hellman Python Module of the week about registration . Search until you see "repeatater.py".

Here you will find an example of how to send and receive input / output to a process.

Answer 4

Here is the module I wrote. Be sure to use this -u argument to avoid buffering problems:

 import os import pickle import subprocess from subprocess import PIPE import struct import builtins def call_thru_stream(stream,funcname,*args,**kwargs): """Used for calling a function through a stream and no return value is required. It is assumed the receiving program is in the state where it is expecting a function.""" transmit_object(stream,(funcname,args,kwargs)) def function_thru_stream(in_stream,out_stream,funcname,*args,**kwargs): """Used for calling a function through a stream where a return value is required. It is assumed the receiving program is in the state where it is expecting a function.""" transmit_object(in_stream,(funcname,args,kwargs)) return receive_object(out_stream) #--------------------- Object layer ------------------------------------------------------------ def transmit_object(stream,obj): """Transmits an object through a binary stream""" data=pickle.dumps(obj,2)#Uses pickle version 2 for compatibility with 2x transmit_atom(stream,data) def receive_object(stream): """Receive an object through a binary stream""" atom=receive_atom(stream) return pickle.loads(atom) #--------------------- Atom layer -------------------------------------------------------------- def transmit_atom(stream, atom_bytes): """Used for transmitting a some bytes which should be treated as an atom through a stream. An integer indicating the size of the atom is appended to the start.""" header=struct.pack("=L",len(atom_bytes)) stream.write(header) stream.write(atom_bytes) stream.flush() def receive_atom(stream): """Read an atom from a binary stream and return the bytes.""" input_len=stream.read(4) l=struct.unpack("=L",input_len)[0] return stream.read(l)