Suggestions / Tricks for throttling PHP script

Question

I have a scheduled task that runs a script on a regular basis (every hour). This script does some intense interaction with the database and file system and takes several minutes regularly. The problem is that when using the script, the cpu-use processor runs and normal operation slows down. Is there a way to throttle this process so that it takes longer, but does not consume so many resources?

I looked at various configuration options for PHP, but it seems that not everything fits my needs.

Setting memory_limit in php.ini to something lower causes data objects to overflow quite easily.

I saw similar entries in which people suggested using sleep () at specific points in the script, but this does not prevent the script from tearing itself away from the server.

The best solution would be to somehow tell the Glass (in this case Wamp) glass to use only 10% of the maximum CPU usage. I'm not worried about runtime at all, and would prefer it to take longer if that means saving CPU cycles per second. My alternative solution would be to configure another server with database replication so that cron can go to the city without slowing down everything else.

Environment: Windows Server 2k3, Apache 2.2.11, PHP 5.2.9, MySQL 5.1

I appreciate any understanding of this situation.

EDIT: I appreciate all the answers, even those that are * nix-specific. It's still early enough in my situation to change the placement environment. Hope this question helps others regardless of OS.

Answer 1

This is a difficult problem. If you use a PHP script through the command line, you can set the priority of the process planning to low ( start /low php.exe myscript.php I think). If your PHP script itself actually does most of the processing that your processor consumes, this might work. However, you said that you are performing very complex interactions with the database and file system, which will not help this solution. It looks like there is a MySQL prompt "LOW_PRIORITY" for INSERT and UPDATE queries that may help you, but I have not tried them.

Answer 2

You can set processes on Windows as a lower priority. I'm not sure how the process starts, but if you set this process to low priority, no matter what CPU resources get, if you set the priority to really low.

Answer 3

On UNIX (LAMP), I was able to solve the problem by checking the server load before continuing with the loop

 function get_server_load($windows = 0) { $os = strtolower(PHP_OS); if(strpos($os, "win") === false) { if(file_exists("/proc/loadavg")) { $load = file_get_contents("/proc/loadavg"); $load = explode(' ', $load); return $load; } elseif(function_exists("shell_exec")) { $load = explode(' ', `uptime`); return $load; } else { return ""; } } } for(... ... ...){ $data = get_server_load(); if($data[0] < 0.2){ // continue }else{ sleep(1); } } 

This function should work on windows as well, but I cannot guarantee it. In linux, it returns an array with the load of the last minute, 5 minutes and 15 minutes.

Also, consider running your scripts (if using the CLI) with a lower priority (on Linux use "nice")

You can also use other values ​​before continuing the cycle, for example, the number of active Apache processes (you can analyze the page 127.0.0.1/server_status?auto if you included mod_status in httpd.conf), as well as the situation with MySQL (active connections?)

Answer 4

Can you change the cron entry to run the script using nice ?

Answer 5

It is not recommended to use the server for customer service and data analysis.

So, if you are looking for the final solution, make several redesigns of your application and upload the data analysis from the interfaces and the live database to another system designed for this task.

Even if you can successfully compress the analyzer, it will use valuable resources, otherwise they will be available to serve users.

Answer 6

It may be a complex change, but it may be worth refactoring your data structures in iterators. Also, if you have circular references in your code, specify the clearReferences () method, which disables these objects. This is a problem that is solved in PHP 5.3 by the way.

So, if you have:

 class Row { protected $_table; public function __construct($table) { $this->_table = $table; } } class Table { protected $_row; public function __construct() { $this->_row = new Row($this); } } 

Add the clearReferences () method to the Row class:

 class Row { public function clearReferences() { $this->_table = null; } } 

That’s all I can think of at the moment.

Answer 7

I have a bunch of scripts that I run from cron in a similar way using nice:

0 * * * * nice -n19 php myscript.php

This will not help the use of RAM (only this can change the script entry), but it uses only the processor, which otherwise would be inactive.

EDIT: did not notice that the question is related to the Windows environment, sorry ... leaving this for any * nix users having the same problem.

Answer 8

It is possible that your script is simply trying to do too much at once. Will it do less if it works three times an hour?

Another solution might be to install an additional server just to start this kind of backend processing. This would be especially effective if it did not put excessive load on the database, namely the web server.

Another approach, to look at whether it works, can be divided in another direction. In such scenarios, there are often several large SQL statements that generate the results used to generate many small SQL statements. If the latter can be delayed somewhere, they can be started with the database as a later step. This approach may also allow you to use an unbuffered query to retrieve preprocessing data that can significantly reduce memory consumption with PHP code.

Answer 9

If you have (Apache) running as a service, you can change the configuration priority in the Win Control Center / Services. In any case, your CPU usage will be spike, but other programs will be preferable by the scheduler. Also try putting the database / server on a different hd than your Application.