Java process with parallel I / O streams

Question

I am trying to create a console / terminal that allows the user to enter a string, which then turns into a process and the results are printed. Like a regular console. But I am having problems managing I / O streams. I looked at this thread , but this solution, unfortunately, does not apply to my problem.

Along with standard commands such as "ipconfig" and "cmd.exe", I need to be able to run the script and use the same input stream to pass some arguments if the script requests input.

For example, after running the script "python pyScript.py", I should be able to pass further input to the script if it requests it (example: raw_input), and also print the output from the script. The main behavior that you expect from the terminal.

What I have so far:

import java.awt.BorderLayout; import java.awt.Color; import java.awt.Dimension; import java.awt.event.KeyEvent; import java.awt.event.KeyListener; import java.io.BufferedReader; import java.io.BufferedWriter; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStream; import java.io.OutputStreamWriter; import javax.swing.JFrame; import javax.swing.JPanel; import javax.swing.JScrollPane; import javax.swing.JTextPane; import javax.swing.text.BadLocationException; import javax.swing.text.Document; public class Console extends JFrame{ JTextPane inPane, outPane; InputStream inStream, inErrStream; OutputStream outStream; public Console(){ super("Console"); setPreferredSize(new Dimension(500, 600)); setLocationByPlatform(true); setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); // GUI outPane = new JTextPane(); outPane.setEditable(false); outPane.setBackground(new Color(20, 20, 20)); outPane.setForeground(Color.white); inPane = new JTextPane(); inPane.setBackground(new Color(40, 40, 40)); inPane.setForeground(Color.white); inPane.setCaretColor(Color.white); JPanel panel = new JPanel(new BorderLayout()); panel.add(outPane, BorderLayout.CENTER); panel.add(inPane, BorderLayout.SOUTH); JScrollPane scrollPanel = new JScrollPane(panel); getContentPane().add(scrollPanel); // LISTENER inPane.addKeyListener(new KeyListener(){ @Override public void keyPressed(KeyEvent e){ if(e.getKeyCode() == KeyEvent.VK_ENTER){ e.consume(); read(inPane.getText()); } } @Override public void keyTyped(KeyEvent e) {} @Override public void keyReleased(KeyEvent e) {} }); pack(); setVisible(true); } private void read(String command){ println(command); // Write to Process if (outStream != null) { System.out.println("Outstream again"); BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(outStream)); try { writer.write(command); //writer.flush(); //writer.close(); } catch (IOException e1) { e1.printStackTrace(); } } // Execute Command try { exec(command); } catch (IOException e) {} inPane.setText(""); } private void exec(String command) throws IOException{ Process pro = Runtime.getRuntime().exec(command, null); inStream = pro.getInputStream(); inErrStream = pro.getErrorStream(); outStream = pro.getOutputStream(); Thread t1 = new Thread(new Runnable() { public void run() { try { String line = null; while(true){ BufferedReader in = new BufferedReader(new InputStreamReader(inStream)); while ((line = in.readLine()) != null) { println(line); } BufferedReader inErr = new BufferedReader(new InputStreamReader(inErrStream)); while ((line = inErr.readLine()) != null) { println(line); } Thread.sleep(1000); } } catch (Exception e) { e.printStackTrace(); } } }); t1.start(); } public void println(String line) { Document doc = outPane.getDocument(); try { doc.insertString(doc.getLength(), line + "\n", null); } catch (BadLocationException e) {} } public static void main(String[] args){ new Console(); } } 

I do not use the mentioned ProcessBuilder , since I like to distinguish between error and regular thread.

UPDATE 08/29/2016

With @ArcticLord, we achieved what was asked in the original question. Now it is just a matter of smoothing out any strange behavior, such as a process without interruption. The console has a stop button that simply calls pro.destroy (). But for some reason this does not work for endlessly running processes, these are spam outputs.

Console: http://pastebin.com/vyxfPEXC

InputStreamLineBuffer: http://pastebin.com/TzFamwZ1

Example code that stops not :

 public class Infinity{ public static void main(String[] args){ while(true){ System.out.println("."); } } } 

Example code that stops:

 import java.util.concurrent.TimeUnit; public class InfinitySlow{ public static void main(String[] args){ while(true){ try { TimeUnit.SECONDS.sleep(1); } catch (InterruptedException e) { e.printStackTrace(); } System.out.println("."); } } } 

Answer 1

You are on the right track with the code. There are only some minor things you missed.
Let's start with your read method:

 private void read(String command){ [...] // Write to Process if (outStream != null) { [...] try { writer.write(command + "\n"); // add newline so your input will get proceed writer.flush(); // flush your input to your process } catch (IOException e1) { e1.printStackTrace(); } } // ELSE!! - if no outputstream is available // Execute Command else { try { exec(command); } catch (IOException e) { // Handle the exception here. Mostly this means // that the command could not get executed // because command was not found. println("Command not found: " + command); } } inPane.setText(""); } 

Now fix your exec method. You should use separate threads to read the normal output process and output errors. In addition, I present a third thread that waits for the process to complete and closes the output stream, so the next user input is not intended for the process, but is a new command.

 private void exec(String command) throws IOException{ Process pro = Runtime.getRuntime().exec(command, null); inStream = pro.getInputStream(); inErrStream = pro.getErrorStream(); outStream = pro.getOutputStream(); // Thread that reads process output Thread outStreamReader = new Thread(new Runnable() { public void run() { try { String line = null; BufferedReader in = new BufferedReader(new InputStreamReader(inStream)); while ((line = in.readLine()) != null) { println(line); } } catch (Exception e) { e.printStackTrace(); } System.out.println("Exit reading process output"); } }); outStreamReader.start(); // Thread that reads process error output Thread errStreamReader = new Thread(new Runnable() { public void run() { try { String line = null; BufferedReader inErr = new BufferedReader(new InputStreamReader(inErrStream)); while ((line = inErr.readLine()) != null) { println(line); } } catch (Exception e) { e.printStackTrace(); } System.out.println("Exit reading error stream"); } }); errStreamReader.start(); // Thread that waits for process to end Thread exitWaiter = new Thread(new Runnable() { public void run() { try { int retValue = pro.waitFor(); println("Command exit with return value " + retValue); // close outStream outStream.close(); outStream = null; } catch (InterruptedException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } } }); exitWaiter.start(); } 

Now this should work.
If you enter ipconfig , it prints the output of the command, closes the output stream, and is ready for a new command.
If you enter cmd , it prints the output and allows you to enter additional cmd commands, such as dir or cd , and so on, until you type exit . Then it closes the output stream and is ready for a new team.

You may have problems running python scripts because there are problems reading Process InputStreams with Java if they are not dumped to the system pipeline.
See python script example

 print "Input something!" str = raw_input() print "Received input is : ", str 

You can run this with your Java program and also enter input, but you will not see the output of the script until the script completes.
The only fix I could find was to manually clear the output in the script.

 import sys print "Input something!" sys.stdout.flush() str = raw_input() print "Received input is : ", str sys.stdout.flush() 

Running this script will crash as you expect.
You can learn more about this issue at

• Java: is there a way to run a system command and print the output at run time?
• Why reading from "InputStream block altough" process data is available
• Java: it is not possible to get stdout data from a process unless manually clearing it

EDIT: I found another very simple solution to the problem with stdout.flush() using Python scripts. Run them using python -u script.py and you do not need to crash manually. This should solve your problem.

EDIT2: We discussed in the comments that Stream will mix up with this solution and error, as they work in different streams. The problem here is that we cannot distinguish whether the recording of the output ends when a stream of the error stream appears. Otherwise, classic lock-based thread planning can handle this situation. But we have a continuous stream until the process is complete regardless of whether the data streams or not. Therefore, we need a mechanism that records how much time has passed since the last line was read from each stream.

To do this, I will introduce a class that receives an InputStream and runs Thread to read incoming data. This thread stores each line in the queue and stops when the end of the thread arrives. In addition, it contains the time when the last line was read and added to the queue.

 public class InputStreamLineBuffer{ private InputStream inputStream; private ConcurrentLinkedQueue<String> lines; private long lastTimeModified; private Thread inputCatcher; private boolean isAlive; public InputStreamLineBuffer(InputStream is){ inputStream = is; lines = new ConcurrentLinkedQueue<String>(); lastTimeModified = System.currentTimeMillis(); isAlive = false; inputCatcher = new Thread(new Runnable(){ @Override public void run() { StringBuilder sb = new StringBuilder(100); int b; try{ while ((b = inputStream.read()) != -1){ // read one char if((char)b == '\n'){ // new Line -> add to queue lines.offer(sb.toString()); sb.setLength(0); // reset StringBuilder lastTimeModified = System.currentTimeMillis(); } else sb.append((char)b); // append char to stringbuilder } } catch (IOException e){ e.printStackTrace(); } finally { isAlive = false; } }}); } // is the input reader thread alive public boolean isAlive(){ return isAlive; } // start the input reader thread public void start(){ isAlive = true; inputCatcher.start(); } // has Queue some lines public boolean hasNext(){ return lines.size() > 0; } // get next line from Queue public String getNext(){ return lines.poll(); } // how much time has elapsed since last line was read public long timeElapsed(){ return (System.currentTimeMillis() - lastTimeModified); } } 

With this class, we could combine the output stream and the error stream into one. It lives while the data input buffer streams live and have no data. In each run, he checks to see if some time has passed since the last exit was read, and if he prints all the unsealed lines with a stroke. Same thing with error output. Then he sleeps for some milliseconds, so as not to waste CPU time.

 private void exec(String command) throws IOException{ Process pro = Runtime.getRuntime().exec(command, null); inStream = pro.getInputStream(); inErrStream = pro.getErrorStream(); outStream = pro.getOutputStream(); InputStreamLineBuffer outBuff = new InputStreamLineBuffer(inStream); InputStreamLineBuffer errBuff = new InputStreamLineBuffer(inErrStream); Thread streamReader = new Thread(new Runnable() { public void run() { // start the input reader buffer threads outBuff.start(); errBuff.start(); // while an input reader buffer thread is alive // or there are unconsumed data left while(outBuff.isAlive() || outBuff.hasNext() || errBuff.isAlive() || errBuff.hasNext()){ // get the normal output if at least 50 millis have passed if(outBuff.timeElapsed() > 50) while(outBuff.hasNext()) println(outBuff.getNext()); // get the error output if at least 50 millis have passed if(errBuff.timeElapsed() > 50) while(errBuff.hasNext()) println(errBuff.getNext()); // sleep a bit bofore next run try { Thread.sleep(100); } catch (InterruptedException e) { e.printStackTrace(); } } System.out.println("Finish reading error and output stream"); } }); streamReader.start(); // remove outStreamReader and errStreamReader Thread [...] } 

This may not be the perfect solution, but it should handle the situation here.

---

EDIT (8/31/2016)
We discussed in the comments that there is still a problem with the code when implementing the stop button, which kills the initial one using Process#destroy() . A process that produces a very large output, for example. in an infinite loop, destroy immediately by calling destroy() . But since it has already produced a lot of products that should be consumed by our streamReader we cannot return to the normal behavior of the program.
Therefore, we need small changes here:

We will introduce the destroy() method for InputStreamLineBuffer , which stops the output and clears the queue.
The changes will look like this:

 public class InputStreamLineBuffer{ private boolean emergencyBrake = false; [...] public InputStreamLineBuffer(InputStream is){ [...] while ((b = inputStream.read()) != -1 && !emergencyBrake){ [...] } } [...] // exits immediately and clears line buffer public void destroy(){ emergencyBrake = true; lines.clear(); } } 

And some small changes in the main program

 public class ExeConsole extends JFrame{ [...] // The line buffers must be declared outside the method InputStreamLineBuffer outBuff, errBuff; public ExeConsole{ [...] btnStop.addActionListener(new ActionListener() { public void actionPerformed(ActionEvent e) { if(pro != null){ pro.destroy(); outBuff.destroy(); errBuff.destroy(); } }}); } [...] private void exec(String command) throws IOException{ [...] //InputStreamLineBuffer outBuff = new InputStreamLineBuffer(inStream); //InputStreamLineBuffer errBuff = new InputStreamLineBuffer(inErrStream); outBuff = new InputStreamLineBuffer(inStream); errBuff = new InputStreamLineBuffer(inErrStream); [...] } } 

Now he should be able to destroy even some spamming processes.

Note. I found out that Process#destroy() cannot destroy child processes. So, if you run cmd in windows and run the Java program from there, you will end the destruction of the cmd process while the Java program is still running. You will see it in the task manager. This problem cannot be solved with java. it will require some os depend on external tools to get pids of these processes and kill them manually.

Answer 2

Although the @ArticLord solution is nice and neat, I recently ran into the same problem and came up with a solution that is conceptually equivalent, but slightly different in its implementation.

The concept is the same, namely, “volume reading”: when a reader stream receives its turn, it consumes the entire stream that it processes, and passes the hand only when it is done.
This ensures print order / print error. A.

But instead of using a timer-based rotation assignment, I use non-blocking read with lock:

 // main method for testability: replace with private void exec(String command) public static void main(String[] args) throws Exception { // create a lock that will be shared between reader threads // the lock is fair to minimize starvation possibilities ReentrantLock lock = new ReentrantLock(true); // exec the command: I use nslookup for testing on windows // because it is interactive and prints to stderr too Process p = Runtime.getRuntime().exec("nslookup"); // create a thread to handle output from process (uses a test consumer) Thread outThread = createThread(p.getInputStream(), lock, System.out::print); outThread.setName("outThread"); outThread.start(); // create a thread to handle error from process (test consumer, again) Thread errThread = createThread(p.getErrorStream(), lock, System.err::print); errThread.setName("errThread"); errThread.start(); // create a thread to handle input to process (read from stdin for testing purpose) PrintWriter writer = new PrintWriter(p.getOutputStream()); Thread inThread = createThread(System.in, null, str -> { writer.print(str); writer.flush(); }); inThread.setName("inThread"); inThread.start(); // create a thread to handle termination gracefully. Not really needed in this simple // scenario, but on a real application we don't want to block the UI until process dies Thread endThread = new Thread(() -> { try { // wait until process is done p.waitFor(); logger.debug("process exit"); // signal threads to exit outThread.interrupt(); errThread.interrupt(); inThread.interrupt(); // close process streams p.getOutputStream().close(); p.getInputStream().close(); p.getErrorStream().close(); // wait for threads to exit outThread.join(); errThread.join(); inThread.join(); logger.debug("exit"); } catch(Exception e) { throw new RuntimeException(e.getMessage(), e); } }); endThread.setName("endThread"); endThread.start(); // wait for full termination (process and related threads by cascade joins) endThread.join(); logger.debug("END"); } // convenience method to create a specific reader thread with exclusion by lock behavior private static Thread createThread(InputStream input, ReentrantLock lock, Consumer<String> consumer) { return new Thread(() -> { // wrap input to be buffered (enables ready()) and to read chars // using explicit encoding may be relevant in some case BufferedReader reader = new BufferedReader(new InputStreamReader(input)); // create a char buffer for reading char[] buffer = new char[8192]; try { // repeat until EOF or interruption while(true) { try { // wait for your turn to bulk read if(lock != null && !lock.isHeldByCurrentThread()) { lock.lockInterruptibly(); } // when there nothing to read, pass the hand (bulk read ended) if(!reader.ready()) { if(lock != null) { lock.unlock(); } // this enables a soft busy-waiting loop, that simultates non-blocking reads Thread.sleep(100); continue; } // perform the read, as we are sure it will not block (input is "ready") int len = reader.read(buffer); if(len == -1) { return; } // transform to string an let consumer consume it String str = new String(buffer, 0, len); consumer.accept(str); } catch(InterruptedException e) { // catch interruptions either when sleeping and waiting for lock // and restore interrupted flag (not necessary in this case, however it a best practice) Thread.currentThread().interrupt(); return; } catch(IOException e) { throw new RuntimeException(e.getMessage(), e); } } } finally { // protect the lock against unhandled exceptions if(lock != null && lock.isHeldByCurrentThread()) { lock.unlock(); } logger.debug("exit"); } }); } 

Please note that both solutions, @ArticLord and mine, are not completely protected from hunger, and the chances (really little) are inversely proportional to the speed of consumers.

Happy 2016 !;)