Best practice for finding source completion php script

Question

I have a PHP script that grabs a piece of data from a database, processes it, and then looks to see if there is more data. These processes run indefinitely, and I run several of them at a time on the same server.

It looks something like this:

<?php while($shouldStillRun) { // do stuff } logThatWeExitedLoop(); ?> 

The problem is that after some time something forces the process to stop working, and I could not debug it and determine the reason.

Here is what I use to get information:

• error_log - logging all errors, but errors are not displayed in the error log.
• register_shutdown_function - a user disconnect function is registered. This is caused so that I know that the process is not killed by the server, and it is allowed to terminate. (or at least I assume this is so when is it called?)
• debug_backtrace I wrote down the debug_backtrace () function in my custom shutdown function. This shows only one call, and this is my custom shutdown function.
• Log, if it reaches the end of the script - Outside of the loop, I have a function that registers that the script exited the loop (and therefore reaches the end of the source file normally). When a script dies randomly, it does not register it, so everything that kills it kills it while it is in the middle of processing.

What other debugging methods do you offer for finding the culprit?

Note. I should add that this is not a problem with max_execution_time , which is disabled for these scripts. The time to kill is controversial. He can work for 10 seconds or 12 hours before he dies.

---

Update / solution . Thank you all for your suggestions. By logging the output, I found that when the MySql request failed, the script was set to die (). D'o. Updated to log mysql errors and then complete. Now it works like a charm!

Answer 1

Remember that PHP has a variable in the ini file that indicates how long the script will run. max-execution-time

Make sure you don't jump to this, or use set_time_limit () to increase the execution time. Does this program work through a web server or through cli?

Addendum: My bad experience with PHP. Looking through some background scripts that I wrote earlier this year. Sorry, but PHP is a terrible scripting language for executing anything over long periods of time. I see that the new PHP (which we have not yet updated) adds functionality to make GC work. The problem I am facing is using too much memory, because the GC almost never starts to clear itself. If you use things that recursively reference them, they will also never be released.

Creating an array of 100,000 elements makes the memory, but then, installing the array in an empty array or splicing all this, DOES NOT release it immediately and does not mark it as unused (also creating a new array of 100,000 elements memory).

My personal decision was to write a perl script, which was maintained forever, and system ("php my_php.php"); when necessary, so that the translator is completely free. I currently support 5.1.6, this can be fixed in 5.3+, or at least now they have GC commands that you can use to force a clean GC.

Simple script

  #! / usr / bin / perl -w

 use strict;

 while (1) {
   if (system ("php /to/php/script.php")! = 0) {
     sleep (30);
   }
 }

then in your php script

  <? php

 // do a single processing block

 if ($ moreblockstodo) {
   exit (0);
 } else {
   // no?  then lets sleep for a bit until we get more
   exit (1);
 }

 ?>

Answer 2

I would write down the memory usage of your script. Maybe he gains too much memory, a memory limit and dies?

Answer 3

I would write the state of the function to a file in several different places in each loop.

You can get the contents of most variables as a string with var_export using the form var_export($varname,true) .

You can simply register this in a specific file and keep track of it. The last state of the function before the end of the log should contain some hints.

Answer 4

It sounds like this is not a standard php error. You should be able to create your own errors using the try ... catch statement, which must be logged. I have no other details besides this, because I am on my phone from the computer.

Answer 5

I came across this before in one of our projects at work. We have a similar setup - a PHP script checks the database if there are tasks that need to be performed (for example, sending email, updating records, processing some data). PHP script has a while loop inside that is set to

 while(true) { //do something } 

After a while, the script will also be killed. I already tried most of what was said here, like setting max_execution_time, using var_export to register all the output, place try_catch, create the output of the script (php ...> output.txt), etc., and we never were able to find out what the problem is.

I think that PHP is simply not designed to perform background tasks on its own. I know that this does not answer your question (how to debug this), but the way we worked it, we used cronjob to call a PHP file every 5 minutes. This is similar to Jeremy’s response to using a perl script - it ensures that the interpreter is free after execution.

Answer 6

If it is on Linux, try looking at the system logs - the process can be killed by the OOM killer (without memory) (it is unlikely that you will also see other problems if this happens), or segmentation fault (some versions of PHP do not like some versions of extensions that lead to to strange crashes).