How to execute a process in Python where data is written to stdin?

Question

I have a flag in my Python script that indicates whether I install and use an external process or not. This process is a command called my_command , and it takes data from standard input. If I ran this on the command line, it would be something like:

 $ my_command < data > result 

I want to use a Python script to generate data strings by changing the standard input and feeding it to my_command .

I am doing something like this:

 import getopt, sys, os, stat, subprocess # for argument sake, let say this is set to True for now # in real life, I use getopt.getopt() to decide whether this is True or False useProcess = True if useProcess: process = subprocess.Popen(['my_command'], stdin=subprocess.PIPE, stdout=subprocess.PIPE) for line in sys.stdin: # parse line from standard input and modify it # we store the result in a variable called modified_line modified_line = line + "foo" # if we want to feed modified_line to my_command, do the following: if useProcess: process.stdin.write(modified_line) # otherwise, we just print the modified line else: print modified_line 

However, my_command behaves as if it does not receive any data and does not exit the error state. What am I doing wrong?

EDIT

Let's say my Python script is called my_Python_script . Let's say I usually passed my_command file called data over standard inputs:

 $ my_command < data > result 

But now I pass it my_Python_script :

 $ my_Python_script < data > some_other_result 

I want my_Python_script conditionally configure a subprocess that runs my_command in the data content (which changes to my_Python_script before passing to my_command ). It makes sense?

If I used bash as a scripting language, I would conditionally decide to run one of two functions. One of them will pass data strings to my_command . There will be no other. Can this be done with Python?

Answer 1

After writing to stdin you need to close it:

  process.stdin.write(modified_line) process.stdin.close() 

Update

I did not notice that process.stdin.write() was executed in a for loop. In this case, you should move process.stdin.close() outside the loop.

In addition, Raymond mentioned that we should also call process.wait() . Therefore, the updated code should be:

 for ... process.stdin.write(modified_line) process.stdin.close() process.wait() 

Answer 2

In addition to process.stdin.close() , as mentioned in @HaiVu, did you do process.wait() to wait for the command to complete before you get the result?

Answer 3

It seems that you can be misleading with arguments and stdin. Your team should be

 $ <data> | mycommand result 

when data is transferred after calling the command.

The input is executed using the built-in raw_input function. ( http://docs.python.org/2/library/functions.html )