Geek Answers Handbook

Tomcat process will suddenly crush

Trying to diagnose some strange Tomcat 6 and / or JVM errors on a 64-bit Linux machine:

OS BASIS: RED ENTERPRISE WARNING LINUX SERVER RELEASE 6.3 (SANTIAGO)
SERVER MODEL: VIRTUAL PLATFORM VMWARE ()

I run some application tests on virtual machines and run into some strange problem. The problem occurs on some virtual machines where exactly 2 processes are running: the Tomcat application server process (Java process) and the monitoring process at one random point, after some random hours of execution, you can find the following message on the catalina.out (server log) VM, which fails (the problem does not always happen on the same machine):

org.apache.coyote.http11.Http11Protocol pause INFO: Pausing Coyote HTTP/1.1 on http-50100 INFO: Stopping service Catalina org.apache.coyote.http11.Http11Protocol destroy INFO: Stopping Coyote HTTP/1.1 on http-8080

After this point, the Tomcat process terminates and execution fails. No samples are visible in memory or CPU usage. In / var / log, system messages are not visible.

  • At the first stage, we suspected that an external signal was killing our Tomcat process, and we tried to use the -Xrs flag for the JVM arguments of the process, but the problem still occurred.

    The java flag ( -Xrs ) should prevent the Java runtime from handling exception signals such as SIGSEGV and SIGABRT do not help

    Before adding the xrs flag xrs we saw an INFO: Pausing Coyote HTTP/1.1 on http-50100 error INFO: Pausing Coyote HTTP/1.1 on http-50100 , which occurred because the result signal

    After adding the xrs flag, kettle / tomcat behaves the same and suppresses error-free log files.

  • Running the starce command in the corresponding process gives us the following result:

    Process 12917 is attached - interrupt for futex output (0x7f6b9e8ab9d0, FUTEX_WAIT, 12918, NULLPANIC: attached pid 12917 exited from 143

    We cannot run starce -f to turn on all threads because it makes our application get stuck. However, the process of killing a son with the kill command behaves similarly - we see that the parent JVM exit with error code 143, and we also see the signal sent to the sons process.

  • We also run java -XX:+HeapDumpOnOutOfMemoryError optind to create a heap dump for the jvm crash due to a memory error, but no core dumps or pile heaps due to failures.

Any ideas?

+4
source share
1 answer

Is it possible that some memory depletion occurs before the failure, so that the java process can be killed by OOM. In addition, I found a link http://bugs.centos.org/view.php?id=4059 that describes a similar problem.

0
source

More articles:


All Articles