Shell script call from java freezes

Question

So, I am trying to execute a shell script that produces a lot of output (in 100's MB) from a Java file. It freezes the process and never ends.

However, in a shell script, if I redirect the script output to some log file or / dev / null, the Java file is executed and terminated in jiffy.

Is it because of the amount of data that a Java program never completes? If so, is there any documentation as such? or is there a limit on the amount of data (documented)?

Here is how you can simulate this scenario.

The Java file will look like this:

import java.io.InputStream; public class LotOfOutput { public static void main(String[] args) { String cmd = "sh a-script-which-outputs-huuggee-data.sh"; try { ProcessBuilder pb = new ProcessBuilder("bash", "-c", cmd); pb.redirectErrorStream(true); Process shell = pb.start(); InputStream shellIn = shell.getInputStream(); int shellExitStatus = shell.waitFor(); System.out.println(shellExitStatus); shellIn.close(); } catch (Exception ignoreMe) { } } } 

script 'a- script -which-outputs-huuggee-data.sh' might look like this:

 #!/bin/sh # Toggle the line below exec 3>&1 > /dev/null 2>&1 count=1 while [ $count -le 1000 ] do cat some-big-file ((count++)) done echo echo Yes I m done 

Free beer for the right answer. :)

Answer 1

This is because you are not reading the result of the Process .

According to class' Javadocs , if you do not, you may be at a dead end; the process fills its I / O buffer and expects the shell (or listening process) to read and lower it. Meanwhile, your process, which should do this, blocks waiting for the process to exit.

You want to call getInputStream () and read reliably (possibly from another thread) to stop the process blocking.

Also consider Five Java Process Errors and Runtime Execution. exec () will not - as informative articles about common problems with Process .

Answer 2

You never read the input stream, so it is probably blocked because the input buffer is full.

Answer 3

The I / O buffer is limited in size (depending on the operating system). If I remember correctly, it was not significant or Windows XP, at least. Try creating a stream that reads InputStream as quickly as possible.

Something like that:

 class StdInWorker implements Worker { private BufferedReader br; private boolean run = true; private int linesRead = 0; private StdInWorker (Process prcs) { this.br = new BufferedReader( new InputStreamReader(prcs.getInputStream())); } public synchronized void run () { String in; try { while (this.run) { while ((in = this.br.readLine()) != null) { this.buffer.add(in); linesRead++; } Thread.sleep(50); } } catch (IOException ioe) { ioe.printStackTrace(); } catch (InterruptedException ie) {} } } }