Php-fpm child process exited signal 11

Question

Our application runs on an AWS docker container. Operating system: Ubuntu 14.04.2 LTS Nginx Version: nginx / 1.4.6 (Ubuntu) Memcached version: memcached 1.4.14 PHP version: PHP 5.5.9-1ubuntu4.11 (cli) (built: July 2, 2015 15:23 : 08) System memory: 7.5 GB.

We get blank pages and 404 error less frequently. When checking the logs, it was found that the php-child process was killed, and it seems that the memory is mainly used by the memcache and php-fpm process and very low free memory.

memcache is configured to use 2 GB of memory.

Here is php www.conf

pm = dynamic pm.max_children = 30 pm.start_servers = 9 pm.min_spare_servers = 4 pm.max_spare_servers = 14 rlimit_files = 131072 rlimit_core = unlimited 

Error logs

 /var/log/nginx/php5-fpm.log [29-Jul-2015 14:37:09] WARNING: [pool www] child 259 exited on signal 11 (SIGSEGV - core dumped) after 1339.412219 seconds from start /var/log/nginx/error.log 2015/07/29 14:37:09 [error] 141#0: *2810 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: xxxx, server: _, request: "GET /suggestions/business?q=Selectfrom HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "example.com", referrer: "http://example.com/" /var/log/nginx/php5-fpm.log [29-Jul-2015 14:37:09] NOTICE: [pool www] child 375 started /var/log/nginx/php5-fpm.log:[29-Jul-2015 14:37:56] WARNING: [pool www] child 290 exited on signal 11 (SIGSEGV - core dumped) after 1078.606356 seconds from start 

Coredump

 Core was generated by php-fpm: pool www.Program terminated with signal SIGSEGV, Segmentation fault.#0 0x00007f41ccaea13a in memcached_io_readline(memcached_server_st*, char*, unsigned long, unsigned long&) () from /usr/lib/x86_64-linux-gnu/libmemcached.so.10 

dmesg

 [Wed Jul 29 14:26:15 2015] php5-fpm[12193]: segfault at 7f41c9e8e2da ip 00007f41ccaea13a sp 00007ffcc5730ce0 error 4 in libmemcached.so.10.0.0[7f41ccad2000+32000] [Wed Jul 29 14:28:26 2015] php5-fpm[12211]: segfault at 7f41c966b2da ip 00007f41ccaea13a sp 00007ffcc5730ce0 error 4 in libmemcached.so.10.0.0[7f41ccad2000+32000] [Wed Jul 29 14:29:16 2015] php5-fpm[12371]: segfault at 7f41c9e972da ip 00007f41ccaea13a sp 00007ffcc5730b70 error 4 in libmemcached.so.10.0.0[7f41ccad2000+32000] [Wed Jul 29 14:35:36 2015] php5-fpm[12469]: segfault at 7f41c96961e9 ip 00007f41ccaea13a sp 00007ffcc5730ce0 error 4 in libmemcached.so.10.0.0[7f41ccad2000+32000] [Wed Jul 29 14:35:43 2015] php5-fpm[12142]: segfault at 7f41c9e6c2bd ip 00007f41ccaea13a sp 00007ffcc5730b70 error 4 in libmemcached.so.10.0.0[7f41ccad2000+32000] [Wed Jul 29 14:37:07 2015] php5-fpm[11917]: segfault at 7f41c9dd22bd ip 00007f41ccaea13a sp 00007ffcc5730ce0 error 4 in libmemcached.so.10.0.0[7f41ccad2000+32000] [Wed Jul 29 14:37:54 2015] php5-fpm[12083]: segfault at 7f41c9db72bd ip 00007f41ccaea13a sp 00007ffcc5730ce0 error 4 in libmemcached.so.10.0.0[7f41ccad2000+32000] 

Please let me know if you need more information.

Thank you in advance

Answer 1

While searching for the same problem and trying to find a solution not related to sessions (because I made it), as well as bad PHP code (because I have several sites running exactly in the same version of WordPress, and no one has problems ... except one), I came up with the answer, which said that a possible solution is to remove some buggy extension (usually memcache / d, but maybe something else).

Since I had the same site working flawlessly on one Ubuntu server, when switching to a newer server, I immediately suspected that it caused a problem with PHP 5.5 to 7. It was just weird because no other site Suffered. Then I remembered that there was something else on this new server: I also installed New Relic . This is both an extension and a small server that runs in the background and sends a lot of analytic data to New Relic for processing. This is supposedly an extension of PHP 5, but, surprisingly, it loads well on PHP 7 as well.

Now comes the tricky bit. At some point, I installed W3 Total Cache to install WordPress for this particular site. Subsequently, I saw that the performance of this server was so spectacular that W3TC was not required, and simply adhered to a much simpler configuration. Therefore, I was able to remove W3TC. This is all very nice, but ... I forgot that I also turned the new relic into W3TC (presumably, it adds some additional analytic data to send to New Relic). When uninstalling W3TC, there was probably something in the New Relic configuration on my server that was still trying to send data through the W3TC interface (assuming W3TC has an interface ... I really don't know how this works at that level) , and since this particular bit of code was missing, the php_fpm handler for this website fails ... sometimes. Not all the time, because I assume that in most cases nginx sent static pages back. Or maybe php_fpm, configured to "recycle" after 100 calls or so, will crash. No matter what exactly happened, it was definitely related to New Relic - as soon as I removed the New Relic extension from PHP, this website returned to normal operation.

Since this is such a specific scenario, I simply write this as an answer, in the distant case, that someone in the future will catch a problem for a specific problem.

Answer 2

This can happen if php cannot write session information to a file. By default it is /var/lib/php/session You can change it using the session_save_path configuration

https://serverfault.com/questions/427596/phpmyadmin-having-problems-on-nginx-and-php-fpm-on-rhel-6/429445

Answer 3

In my case, it was called by the New Relic PHP Agent. So for the specific function that causes the crash, I added this code to disable New Relic

 if (function_exists('newrelic_ignore_transaction')) { newrelic_ignore_transaction(); } 

: https://discuss.newrelic.com/t/how-to-disable-a-specific-transaction-in-php-agent/42384/2

Answer 4

I had this problem after installing xdebug, adding some properties to / etc / php / 7.1 / fpm / php.ini and restarting nginx. It works on the Homestead Laravel box.

Just restarting the php7.1-fpm service solved it for me.

Answer 5

In my case, this was due to zend debug / xdebug . It forwards some tcp packets to an IDE (phpstorm) that did not listen on this port (debugging was disabled). The solution is to either disable these extensions or enable debugging listening on the debug port.

Answer 6

In our case, it was caused by Guzzle + New Relic. The New Relic Agent change log mentioned that version 7.3 was fixed by Guzzle, but even using 8.0 did not work, so there is still something wrong. In our case, this happened only in our 2 scripts that used Guzzle. We found that there are two solutions:

1. Set newrelic.guzzle.enabled = false to newrelic.ini . This way you will lose data on the External Services tab, but you may not need it anyway.
2. Bring the new Relic Agent back to version 6.x, which also works
3. If you read this when they released something newer than version 8.0, you can also try updating the new relic agent to the latest version, and maybe they fixed it.

Answer 7

You can find more detailed information about these crashes by looking at your syslog ( /var/log/syslog on linux, /var/log/system.log on mac os).

I had Sep 14 11:16:26 bob ReportCrash[89504]: Saved crash report for php-fpm[13757] version 0 to /Users/bob/Library/Logs/DiagnosticReports/php-fpm_2017-09-14-111626_MacBob.crash on my syslog, and the generated file contained everything to find out which extension has problems.

Answer 8

in my case, I deactivated the buffering function ob_start("buffer"); in my code;)

Answer 9

Possible php7.3 + xdebug problem , please change Xdebug 2.7.0beta1 to Xdebug 2.7.0rc1 or latest xdebug