Tomcat memory management

I am running Tomcat7, the server is quite powerful, 8 GB of RAM is 8-core.

tomcat taking a LOT of memory for nothing?

My problem is that the RES memory is getting higher and higher until the server just no longer responds without even raising an OnOutOfMemoryError.

Tomcat configuration:

-Xms1024M -Xmx2048M -XX:PermSize=256m -XX:MaxPermSize=512m -XX:+UseConcMarkSweepGC -XX:OnOutOfMemoryError='/var/tomcat/conf/restart_tomcat.sh' 

Memory Information:

 Memory: Non heap memory = 106 Mb (Perm Gen, Code Cache), Loaded classes = 14,055, Garbage collection time = 47,608 ms, Process cpu time = 4,296,860 ms, Committed virtual memory = 6,910 Mb, Free physical memory = 4,906 Mb, Total physical memory = 8,192 Mb, Free swap space = 26,079 Mb, Total swap space = 26,079 Mb Perm Gen memory: 88 Mb / 512 Mb ++++++++++++ Free disk space: 89,341 Mb 

The memory used by Tomcat does not look so high compared to the top command.

app memory graph

I also had java.net.SocketException: No buffer space available when trying to connect to an SMTP server or when trying to connect to facebook servers.

I am using Hibernate, with c3p0 connection pool with this configuration:

  <property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property> <property name="hibernate.connection.url">jdbc:mysql://urldb/schema?autoReconnect=true</property> <property name="hibernate.connection.username">username</property> <property name="hibernate.dialect">org.hibernate.dialect.MySQL5InnoDBDialect</property> <property name="hibernate.connection.password"></property> <property name="connection.characterEncoding">UTF-8</property> <property name="hibernate.c3p0.acquire_increment">1</property> <property name="hibernate.c3p0.idle_test_period">300</property> <property name="hibernate.c3p0.timeout">5000</property> <property name="hibernate.c3p0.max_size">50</property> <property name="hibernate.c3p0.min_size">1</property> <property name="hibernate.c3p0.max_statement">0</property> <property name="hibernate.c3p0.preferredTestQuery">select 1;</property> <property name="hibernate.connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property> 

I could not find anything ... someone tell me where I should look?

Thanks!

[UPDATE 1] HEAP DUMP:

 HEAP HISTOGRAM : class [C 269780 34210054 class [B 5600 33836661 class java.util.HashMap$Entry 221872 6212416 class [Ljava.util.HashMap$Entry; 23797 6032056 class java.lang.String 271170 5423400 class org.hibernate.hql.ast.tree.Node 103588 4972224 class net.bull.javamelody.CounterRequest 28809 2996136 class org.hibernate.hql.ast.tree.IdentNode 23461 2205334 class java.lang.Class 14677 2113488 class org.hibernate.hql.ast.tree.DotNode 13045 1852390 class [Ljava.lang.String; 48506 1335600 class [Ljava.lang.Object; 12997 1317016 Instance Counts for All Classes (excluding platform) : 103588 instances of class org.hibernate.hql.ast.tree.Node 33366 instances of class antlr.ANTLRHashString 28809 instances of class net.bull.javamelody.CounterRequest 24436 instances of class org.apache.tomcat.util.buf.ByteChunk 23461 instances of class org.hibernate.hql.ast.tree.IdentNode 22781 instances of class org.apache.tomcat.util.buf.CharChunk 22331 instances of class org.apache.tomcat.util.buf.MessageBytes 13045 instances of class org.hibernate.hql.ast.tree.DotNode 10024 instances of class net.bull.javamelody.JRobin 9084 instances of class org.apache.catalina.loader.ResourceEntry 7931 instances of class org.hibernate.hql.ast.tree.SqlNode 

[UPDATE 2] server.xml:

 <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" URIEncoding="UTF-8" maxThreads="150" minSpareThreads="25" maxSpareThreads="75" enableLookups="false" acceptCount="1024" server="unknown" address="public_ip" /> 

**** [UPDATE 3] Output from log files: ****

  2012-06-04 06:18:24,152 [http-bio-ip-8080-exec-3500] ERROR org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/api].[Jersey REST Service]- Servlet.ser vice() for servlet [Jersey REST Service] in context with path [/socialapi] threw exception java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:532) at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501) at org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:563) at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:118) at org.apache.coyote.http11.AbstractInputBuffer.doRead(AbstractInputBuffer.java:326) at org.apache.coyote.Request.doRead(Request.java:422) 

[UPDATE 4] ServletContext

I use ServletContextListener in my application to create controllers and save the link using event.getServletContext().setAttribute . These controllers download configurations and transfers (88Mb in Perm).

Then to use the database, I use:

 SessionFactory sf = dbManager.getSessionFactory(DatabaseManager.DB_KEY_DEFAULT); Session session = sf.openSession(); Transaction tx = null; try { tx = session.beginTransaction(); //Do stuuf tx.commit(); } catch (Exception e){ //Do something } finally { session.close(); } 
  • Could this be a source of leakage?
  • Why not use a manual transaction / session and how would you do it?
+7
source share
2 answers

Try with this option:

 +XX:+HeapDumpOnOutOfMemoryError -XX:+HeapDumpPath=dump.log 

Also try with lower initial memory -Xms .

then you can check the dump to see if the object allocation was a problem.

When run try

 jps 

This will output all java processes, say Tomcat - PID 4444:

 jmap -dump:format=b,file=heapdump 4444 

and

 jhat heapdump 

If you run out of memory while running jhat, just add more memory. From there you can check a bunch of your application.

Another way is to enable Hibernate statistics to verify that you are not receiving more objects. Although it looks like a complete garbage collection, every hour should not be a problem (a room for this is better there).

 -verbose:gc -Xloggc:/opt/tomcat/logs/gc.out -XX:+PrintGCDetails -XX:+PrintGCTimeStamps 

And with GCViewer, for example, look at each memory space (ternured, eden, survivors, perm).

Another handy tool:

 jstack 4444 > stack.txt 

This will allow you to get a full stack trace for each thread running inside the java process using pid 4444.

Keep in mind that you need privileges if you run Tomcat as root or another user.

 jps 

will not display a process that does not have rights, so you cannot connect to it.

Since I don’t know what your application is about (and therefore I don’t know its requirements), 3 million instances look like a lot.

With Hibernate statistics you can see which classes you create the most.

Then setting up your eden and ternured garbage eden and ternured can be more efficient.

New instances of objects are sent to eden. When it fills the meager gc triggers. What is not removed falls into the space of the survivor. When it fills, it goes into state. Full gc will occur when ternured is full.

In this picture (which is inaccurate), I left out the String , which became interned and memory-mapped files (which are not on the heap). See which classes you create the most. Intensive use of String can quickly fill perm.

I assume you are doing this, but use a managed factory session like Spring (if on your stack) and avoid manually managing transactions and sessions.

Keep in mind that objects are deleted in the GC if the object does not belong to it. As long as the object is available in your application, the object remains.

If your ServletContextListener instantiates the controllers and is stored in the getServletContext event. Make sure you completely delete the link after that, if you save the link, the objects will not be deleted, since they are still available.

If you manage your own transactions and session (this is normal if you cannot use the framework), you need to deal with code maintenance and errors that Spring-tx , for example, solved and improved.

I would personally use FOSS . But of course, sometimes you cannot expand the stack.

If you use Hibernate , I would look at Spring-orm and Spring-tx for transaction and session management. Also take a look at the Hibernate patter Open Session In View .

+14
source

I also recommend that you download Visual VM 1.3.3, install all the plugins, and attach it to the Tomcat PID controller so you can see what is happening in real time. Why wait for a dump stream? He will also tell you the processor, threads, all generations of the heap, whose objects consume most of the memory, etc.

+2
source

All Articles