Getting exit status from parallel GNU

Question

The Perl wrapper below executes parallel commands, saving STDOUT and STDERR to / tmp files:

open(A,"|parallel"); for $i ("date", "ls", "pwd", "factor 17") { print A "$i 1> '/tmp/$i.out' 2> '/tmp/$i.err'\n"; } close(A); 

How to get exit status values ​​from individual commands?

Answer 1

To get the status of individual tasks, parallel will need to write information somewhere. I don’t know if it works or not. If this is not the case, you can do it yourself.

 my %jobs = ( "date" => "date", "ls" => "ls", "pwd" => "pwd", "factor" => "factor 17", ); open(my $parallel, "|parallel"); for my $id (keys(%jobs)) { print $parallel $jobs{$id} ." 1> '/tmp/$id.out'" ." 2> '/tmp/$id.err' ; " ."echo \$?" ." > '/tmp/$id.exit'\n"; } close($parallel); my $exit_status = $? >> 8; if ($exit_status >= 255) { print("Failed\n"); } else { printf("%d failed jobs\n", $exit_status); } for my $id (keys(%jobs)) { ...grab output and exit code from files... } 

---

Update : I went and installed parallel .

It has an option --joblog {file} , which creates a report with exit codes. It accepts - for the file name, if you want it to be displayed on STDOUT.

Please note that parallel does not recognize abnormal death by signal, so this is not included in the --joblog report. Using the solution posted above, the missing .exit file indicates an abnormal death. (You must make sure that this does not exist in the first place.)

Answer 2

GNU Parallel 20110722 has a val output and a signal in --joblog :

 parallel --joblog /tmp/log false ::: a cat /tmp/log Seq Host Starttime Runtime Send Receive Exitval Signal Command 1 : 1311332758 0 0 0 1 0 false a 

Answer 3

If you want to avoid the wrapper, you can consider:

 cat foo | parallel "{} >\$PARALLEL_SEQ.out 2>\$PARALLEL_SEQ.err; echo \$? >\$PARALLEL_SEQ.status" 

Version 20110422 or later makes it even shorter:

 cat foo | parallel "{} >{#}.out 2>{#}.err; echo \$? >{#}.status" 

If your lines do not contain, then this should work too:

 cat foo | parallel "{} >'{}'.out 2>'{}'.err; echo \$? >'{}'.status" 

Answer 4

Instead of wrapping parallel you can use any of the tons of modules available from CPAN, providing similar functionality.

For instance:

 use Proc::Queue size => 10, qw(run_back); my @pids; for $i ("date", "ls", "pwd", "factor 17") { push @pids, run_back { open STDOUT, '>', '/tmp/$i.out'; open STDERR, '>', '/tmp/$i.err'; exec $i; } } for (@pids) { 1 while waitfor($_, 0) <= 0; say "process $_ exit code: ", ($? >> 8); }