Inter-program communication in python on Linux

Question

There are many examples of how to do this:

1) Communicate between different processes in the same program.

2) Communicate between the client / server through the network

However, this question has no good example anywhere:

• What is the canonical way to send a string from Python A to program B, which locks and processes this string and then waits for another in a loop ?

It seems to me that I have met the answer many times, but I could not create a working example.

Additional implied requirements:

• There are actually two different programs: the example should actually have two different programs (i.e. two progA.py, progB.py files that can be separately launched from the command line on two screens on the same computer) do not use any forking or multiprocess to create a client and server.
• Please suggest a way to do this, allowing you to send strings with delimiters of length to a reasonable length, rather than receiving the exact number of bytes of the correct data size. (The latter is more prone to implementation errors).
• Ideal to do this without using localhost internet connections.

For instance; when the reader uses:

pipein = open(pipe_name, 'r') while program.KeepRunning: action = pipein.readline()[:-1] program.processLine(line) time.sleep(1) 

and the author uses:

 command = "enable" pipeout = os.open(pipe_name, os.O_WRONLY) os.write(pipeout, command) os.write(pipeout, "\n") 

as suggested at http://www.python-course.eu/pipes.php the reader gets stuck in an infinite loop by reading an empty line.

Interestingly, adding if(action == 'enable'): longFunction() to the program.processLine function causes some of the code in the longFunction execute before getting stuck, reading out empty lines forever.

On the other hand, all examples using the more modern, lower-level subprocess module include only multi-threaded applications, not several applications. Other implementations include sockets and networks.

While I was trying to use sockets, this leads to an error like “something went wrong” with many possible reasons for Error 111: "connection refused" showing “some time”. As part of the python code that executes when certain commands are received, it actually changes the network configuration (for example, it calls commands such as ip , tc and iptables with various arguments) using a network connection to localhost - this is something that probably follows Avoid, causing serious debugging and generally unpleasant problems. In addition to the part where it is not needed, since the second program runs on the same computer, therefore, for any inter-program communication, you do not need to use network interfaces.

Answer 1

This is the intended behavior. See this answer to a similar problem and an overview of FIFO behavior. Part relevant to your question:

When there are no more writers (...), readers are notified of this via read() , returning EOF .

file.readline() docs says that '' (empty line) means that EOF being reached:

if f.readline() returns an empty line, the end of the file is reached, and the empty line is represented by ' '\n' , a line containing only one new line.

What is it. In an infinite loop, every time you try to read, you get an empty line that says that there are no more writers connected.

Nothing prevents you from using named pipes to solve your problem. The easiest way is to sleep for some time when there are no writers. Here is a working example:

 # server.py import os import time pipe_name = 'pipe_test' if not os.path.exists(pipe_name): os.mkfifo(pipe_name) with open(pipe_name, 'r') as pipe: print("Listening for actions...") while True: action = pipe.readline()[:-1] if action == '': print("No clients. Sleeping...") time.sleep(1) else: print("Action received:", repr(action)) 

 # client.py import os pipe_name = 'pipe_test' if not os.path.exists(pipe_name): os.mkfifo(pipe_name) print("Waiting for server to start...") with open(pipe_name, 'w') as pipe: action = input("Enter action to send: ") pipe.write(action + '\n') 

Notes:

• It makes no sense to use os.open() , which is a low-level function. You can use open() to interact with named pipes.
• Opening a named pipe for reading blocks until the first writer is connected. This way, you will not see Listening for actions... at the output, until the first client connects to the channel. The same thing happens with a writer who begins without readers.
• You requested "which blocks and processes this line, and then waits for another in a loop." If you do not process a line in a separate thread, it will not try to read the next line until the current one has been processed.