Diagnostics of memory leaks. Allowed memory size in # bytes.

Question

I came across a terrible error message, perhaps due to painstaking effort, PHP ran out of memory:

The allowed memory size #### bytes has been exhausted (tried to allocate #### bytes) in the .php file on line 123

Limit increase

If you know what you are doing and want to increase the limit, see memory_limit :

ini_set('memory_limit', '16M'); ini_set('memory_limit', -1); // no limit 

Caution! You can solve only a symptom, not a problem!

Leak diagnosis:

The error message points to a line with a loop that I believe is leaking or unnecessarily accumulating memory. I typed the expression memory_get_usage() at the end of each iteration and can see that the number slowly grows until it reaches the limit:

 foreach ($users as $user) { $task = new Task; $task->run($user); unset($task); // Free the variable in an attempt to recover memory print memory_get_usage(true); // increases over time } 

For the purposes of this question, suppose the worst spaghetti code imaginable is hiding in a global area somewhere in $user or Task .

What tools, PHP tricks, or voodoo debugging can help me find and fix the problem?

Answer 1

There is no garbage collector in PHP. It uses reference counting to manage memory. Therefore, circular references and global variables are the most common source of memory leaks. I'm afraid if you use the framework, you will have a lot of code to find it. The easiest tool is to selectively place calls on memory_get_usage and narrow it down to where the code leaks. You can also use xdebug to create a trace of code. Run the code with the execution trace and show_mem_delta .

Answer 2

There are several possible memory leak points in php:

• php
• php extension
• Php library you are using
• your php code

It is hard to find and fix the first 3 without deep reverse engineering or knowledge of php source code. For the latter, you can use a binary search for a memory leak code using memory_get_usage

Answer 3

I noticed once in an old script that PHP will support the "as" variable as in scope even after my foreach loop. For example,

 foreach($users as $user){ $user->doSomething(); } var_dump($user); // would output the data from the last $user 

I am not sure whether future versions of PHP are fixed or not, since I saw this. If so, you can unset($user) after the doSomething() remove it from memory. YMMV.

Answer 4

I recently ran into this problem in an application, under what I collect to be similar circumstances. A script that runs in PHP cli that iterates through many iterations. My script depends on several core libraries. I suspect that the reason is a particular library, and I spent several hours in vain trying to add suitable destruction methods to them. Faced with the long process of converting to another library (which may be in the same problems), I came up with rough work for this problem in my case.

In my situation, in linux cli, I went over a bunch of user records and for each of them I created a new instance of several classes that I created. I decided to try creating new instances of classes using the PHP exec method so that this process runs in a "new thread". Here is a really basic example of what I mean:

 foreach ($ids as $id) { $lines=array(); exec("php ./path/to/my/classes.php $id", $lines); foreach ($lines as $line) { echo $line."\n"; } //display some output } 

Obviously, this approach has limitations, and you need to know about the dangers of this, since it would be easy to create work on a rabbit, but in some rare cases it could help to overcome a difficult place, until it is better corrected, you can find, as in my case.

Answer 5

I ran into the same problem and my solution was to replace foreach with the usual one. I'm not sure about the specifics, but it seems that foreach is creating a copy of the object (or somehow a new link). Using a regular loop, you access the element directly.

Answer 6

Here's the trick we used to determine which scripts use the most memory on our server.

Save the following fragment of the file in a file, for example, /usr/local/lib/php/strangecode_log_memory_usage.inc.php :

 <?php function strangecode_log_memory_usage() { $site = '' == getenv('SERVER_NAME') ? getenv('SCRIPT_FILENAME') : getenv('SERVER_NAME'); $url = $_SERVER['PHP_SELF']; $current = memory_get_usage(); $peak = memory_get_peak_usage(); error_log("$site current: $current peak: $peak $url\n", 3, '/var/log/httpd/php_memory_log'); } register_shutdown_function('strangecode_log_memory_usage'); 

Use it by adding the following to httpd.conf:

 php_admin_value auto_prepend_file /usr/local/lib/php/strangecode_log_memory_usage.inc.php 

Then /var/log/httpd/php_memory_log file in /var/log/httpd/php_memory_log

You may need to touch /var/log/httpd/php_memory_log && chmod 666 /var/log/httpd/php_memory_log before your web user can write to the log file.

Answer 7

I recently noticed that PHP 5.3 lambda functions leave extra memory used to delete them.

 for ($i = 0; $i < 1000; $i++) { //$log = new Log; $log = function() { return new Log; }; //unset($log); } 

I'm not sure why, but it seems to take an extra 250 bytes each lambda even after the function is removed.

Answer 8

If what you are saying that PHP only executes GC after the function is true, you can wrap the contents of the loop inside the function as a workaround / experiment.

Answer 9

I would advise you to check the php manual or add the gc_enable() function to collect garbage ... This memory leak does not affect how your code works.

PS: php has a gc_enable() garbage collector that takes no arguments.

Answer 10

One huge problem I ran into was using create_function . As in lambda functions, it leaves the generated temporary name in memory.

Another cause of memory leak (in the case of Zend Framework) is Zend_Db_Profiler. Make sure this is disabled if you run scripts under the Zend Framework. For example, I used the following in my application.ini application:

 resources.db.profiler.enabled = true resources.db.profiler.class = Zend_Db_Profiler_Firebug 

By running about 25,000 requests + downloads before that, bringing the memory to a good 128 MB (maximum limit for maximum memory).

Just setting:

 resources.db.profiler.enabled = false 

this was enough to keep it below 20 MB

And this script was run in the CLI, but it created an instance of Zend_Application and launched Bootstrap, so it used the development configuration.

It really helped to run the script with the xDebug profile

Answer 11

I'm a little late for this conversation, but I will tell you something about the Zend Framework.

I had a memory leak problem after installing php 5.3.8 (using phpfarm) to work with a ZF application that was developed using php 5.2.9. I found that a memory leak was triggered in the Apache httpd.conf file in my virtual host definition, which says SetEnv APPLICATION_ENV "development" . After commenting on this line, memory leaks ceased. I am trying to create a built-in workaround in my PHP script (mainly by manually defining it in the main index.php file).

Answer 12

I have not seen this mentioned here, but one thing that may be useful is to use xdebug and xdebug_debug_zval ('variableName') to view them.

I can also give an example php extension: Zend Server Z-Ray. If data collection is enabled, memory usage will be thrown at each iteration just as if garbage collection was disabled.