Perl, Parallel :: ForkManager - how to implement fork timeout

Question

Is it possible to implement some kind of timeout (time limit) for fork using Parallel :: ForkManager?

The Base Parallel :: ForkManager script is as follows

use Parallel::ForkManager; my $pm = Parallel::ForkManager->new( 10 ); for ( 1 .. 1000 ) { $pm->start and next; # some job for fork $pm->finish; } $pm->wait_all_children(); 

I would like to limit the time for # # some fork work. For example, if he did not finish after 90 seconds. then it (fork) should be killed / completed. I was thinking about using this , but I have to say that I do not know how to use it with Parallel :: ForkManager.

EDIT

Thanks hobbs and ikegami. Both of your suggestions worked ..... but only in this basic example, not in my actual script :(. These forks will be there forever and, to be honest, I don't know why. I have been using this script for a couple of months. Did not change anything (although much depends on external variables). Each fork should download a page from a website, analyze it, and save the results to a file. This should not take more than 30 seconds per fork. The timeout is set to 180 seconds. These hanging forks are completely random, so it’s very difficult to trace the problem. That's why I came up with a temporary, simple solution - timeout and kill.

What could disable (interrupt) your timeout methods in my code? I don't have another alarm() in my code.

EDIT 2

One of the forks hung at 1h38m and returned the “PID timeout” - this is what I type die() for alarm() . So the timeout works ... but its end is about 1h36.5m;). Do you have any ideas?

Answer 1

Update

Sorry to update after closing, but I would have missed it if I hadn’t noticed that Parallel :: ForkManager also supports the run_on_start . This can be used to set the "baby registration" function, which takes care of the PID time() print for you.

eg.

 $pm->run_on_start(sub { my $pid = shift; $workers{$pid} = time(); }); 

The result is that in combination with run_on_wait , as described below, the main P :: FM loop should not do anything special. That is, it can remain simple $pm->start and next , and callbacks will take care of the rest.

Original answer

Parallel :: ForkManager run_on_wait handler and a little bookkeeping can force to suspend and restore ALRM files for children.

The callback registered by this function can be executed periodically, and $pm waits for the completion of the child process.

 use strict; use warnings; use Parallel::ForkManager; use constant PATIENCE => 90; # seconds our %workers; sub dismiss_hung_workers { while (my ($pid, $started_at) = each %workers) { next unless time() - $started_at > PATIENCE; kill TERM => $pid; delete $workers{$pid}; } } ... sub main { my $pm = Parallel::ForkManager->new(10); $pm->run_on_wait(\&dismiss_hung_workers, 1); # 1 second between callback invocations for (1 .. 1000) { if (my $pid = $pm->start) { $workers{$pid} = time(); next; } # Here we are child. Do some work. # (Maybe install a $SIG{TERM} handler for graceful shutdown!) ... $pm->finish; } $pm->wait_all_children; } 

(Others claim that it’s best for the children to adjust themselves through alarm() , but this seems irregular for you. You can also resort to wasteful, crude hacks, such as having each child fork() or exec('bash', '-c', 'sleep 90; kill -TERM $PPID') .)

Answer 2

All you need is one line:

 use Parallel::ForkManager; my $pm = Parallel::ForkManager->new( 10 ); for ( 1 .. 1000 ) { $pm->start and next; alarm 90; # <--- # some job for fork $pm->finish; } $pm->wait_all_children(); 

You do not need to set up signal handlers because you want the process to die.

It even works if you are exec in a child. It will not work on Windows, but using fork on Windows is in doubt first.

Answer 3

Just do what your answer tells you inside the child process (ie between $pm->start and next and the end of the loop. There is nothing special so that you can interact with Parallel :: ForkManager, except make sure that you don’t accidentally killed the parent :)