How to do non-blocking IPC reading on Windows?

Question

I have a Perl script that uses an external tool (cleartool) to collect information about a list of files. I want to use IPC to avoid a new process for each file:

use IPC::Open2; my ($cin, $cout); my $child = open2($cout, $cin, 'cleartool'); 

Commands that return single-line lines work well. eg.

 print $cin "describe -short $file\n"; my $description = <$cout>; 

Commands that return multiple lines make me stump for how to use the whole answer without focusing on the lock:

 print $cin "lshistory $file\n"; # read and process $cout... 

I tried to set the file descriptor for non-blocking reads via fcntl :

 use Fcntl; my $flags = ''; fcntl($cout, F_GETFL, $flags); $flags |= O_NONBLOCK; fcntl($cout, F_SETFL, $flags); 

but Fcntl dies with the message "Your provider has not defined the Fcntl macro F_GETFL."

I tried to use IO :: Handle to set $cout->blocking(0) , but that fails (it returns undef and sets $! "Unknown error").

I tried using select to determine if data is available before trying to read:

 my $rfd = ''; vec($rfd, fileno($cout), 1) = 1; while (select($rfd, undef, undef, 0) >= 0) { my $n = read($cout, $buffer, 1024); print "Read $n bytes\n"; # do something with $buffer... } 

but it hangs without even reading anything. Does anyone know how to make this work (on Windows)?

Answer 1

select only works on sockets on Windows. IPC :: OpenX seems to use regular file descriptors, so you cannot use select with the handles you create.

If you do not need a timeout / activity detection that you choose, you can configure the pens to be non-blocking and just read or write as normal.

If you need more nuanced control, IPC :: Run might work well for you.

You can also look at creating socketpair and use these descriptors with child processes. Newer perls (5.8 and higher) support socketpair emulation on Windows using TCP sockets.

If you try to clone STDOUT and STDERR for a program that runs without a console (i.e. it runs using wperl, not perl), you will not be able to receive data through STDIO.

In practice, this was a huge pain for me on several projects. I found work best to write a child process to connect to the parent server via TCP. If you do not control child processes, look at IPC::Run or socketpair .

Answer 2

Another kludge should use sysread with a large or unlikely buffer size.

  print $cin "$command_for_cleartool\n"; my ($description, $maxlen, $buffer) = ("", 65336); while (my $n = sysread $cout, $buffer, $maxlen) { $description .= $buffer; last if $n < $maxlen; } ... do something with $description ... 

sysread will sysread if there are only 0 input bytes waiting to be read. Thus, the code above will hang if cleartool produces exactly a few multiple 65336 bytes. If you know a good upper bound on the size of the output from the program, you can use this value for $maxlen above. Otherwise, you could choose a large and unlikely number and pray ...

Answer 3

Non-blocking IO is highly dependent on Windows. In this case, you can send cleartool output to a regular file and use seek to reset with the eof flag in the file every time after reading from the file:

 my($cin, $cout); my $somefile = "some/file"; open($cin, "| cleartool > $somefile"); open($cout, '<', $somefile); ... print $cin "$command_for_cleartool\n"; # if necessary, wait until cleartool finishes with new output seek $cout, 0, 1; # clear eof condition from previous read my @cleartool_output = <$cout>; # capture recent output ... process output ... 

Although this probably won't work as well if cleartool buffers its output.