PHP Warning: exec () cannot work with fork

Question

So, here is a little info on my setup. Running Centos with apache and php 5.2.17. I have a website that lists products from many retail sites. I have scanner scripts that run to capture products from each website. Because each site is different, each script crawler must be configured to crawl a particular retail website. So basically I have 1 tracked move per seller. At this time, I have 21 scanners that are constantly working to collect and update products from these sites. Each crawler is a php file, and after the PHP script is executed, it checks to make sure that it is the only instance itself, and at the very end of the script, it uses exec to start all over again, while the original instance is closed. This helps protect against memory leaks as each crawler restarts before it closes. However, lately I will check the scanner scripts and notice that one of them no longer works, and in the error log I found the following.

PHP Warning: exec() [<a href='function.exec'>function.exec</a>]: Unable to fork [nice -n 20 php -q /home/blahblah/crawler_script.php &gt;/dev/null &amp;] 

This is what this particular crawler should start again, but since it was "incapable", it never restarted, and the original crawler instance ended as usual.

Obviously, this is not a resolution problem, because each of these 21 crawler scripts runs this exec command every 5 or 10 minutes at the end of its launch, and most of the time it works as it should. It seems to happen once or twice a day. It seems that its limit is of some degree, as I just recently began to see that this has been happening since I added my 21st seeker. And its not always the same crawler that gets this error, there will be any of them at a random time that will not be able to unlock the exec restart command.

Does anyone have an idea what could cause php to be unable to develop, or perhaps even a better way to handle these processes in order to get around the error all together? Is there a limit to the process I have to learn, or something like that? Thank you in advance!

Answer 1

Process limit

"Is there a limit to the process I need to study?"

He suspected that someone (a system administrator?) Had set a max user process restriction. Could you try this?

 $ ulimit -a .... .... max user processes (-u) 16384 .... 

Run the previous command in PHP. Something like:

 echo system("ulimit -a"); 

I searched for this limit in php.ini or httpd.conf but could not find it.

Error processing

"even the best way to handle these processes to get around the error all together?"

The third parameter, exec() returns the exit code $cmd . 0 for success, not zero for the error code. See http://php.net/function.exec .

 exec($cmd, &$output, &$ret_val); if ($ret_val != 0) { // do stuff here } else { echo "success\n"; } 

Answer 2

In my case (a large set of PHPUnit tests), he would unable to fork as soon as the process reaches 57% of the memory usage. So, one more thing that you can pay attention to, this may not be the limit of the process, but memory.

Answer 3

I ran into the same problem and I tried this and it worked for me;

 ulimit -n 4096 

Answer 4

For anyone who encounters this problem, this may be a few problems, as indicated in the answer to this question.

However, my problem was that my nginx user did not have the proper shell to execute the commands I wanted. Adding .bashrc to nginx user home directory has been fixed.

Answer 5

The problem is often caused by a system or process or by running out of available memory. Make sure you have enough by running free -m . You will get the following result:

total used free shared buffers cached Mem: 7985 7722 262 19 189 803 -/+ buffers/cache: 6729 1255 Swap: 0 0 0

The line of buffers / caches is what you want to see. Please note: free memory is 1255 MB on this machine. When you start your program, continue to try free -m and check the free memory to see if it falls into the lower hundreds. If so, you will need to find a way to run the program, while the consumer has less memory.