Java Application Server Performance

Question

I have a somewhat outdated Java EE application running on Sun Application Server 8.1 (aka SJSAS, the predecessor of Glassfish). With over 500 concurrent users, the application is becoming unacceptably slow, and I'm trying to help determine where most of the runtime is spent and what can be done to speed it up. So far, we have experimented and measured using LoadRunner, application server logs, Oracle statpack, snoop, configured acceptor and session flows of the (working) application server, adjusted the Hibernate batch size and used the extraction option, etc., But after some initial win we are struggling to improve the situation.

Well, with this introduction to the problem, here is the real question: if you had a slow Java EE application running on a box whose processor and memory never exceeded 20%, and while working with more than 500 users you showed two things: 1) that requesting even static files in the same process of the server JVM process was extremely slow, and 2) requesting a static file outside the application JVM server process, but in the same field was fast, what would you investigate?

My thoughts initially jumped on the application server threads, both acceptor and session threads, assuming that even requests for static files were queued, waiting for the available thread, and if the CPU / memory was not taxed, then more threads were fine. But then we significantly increased both acceptor and session flows, and there was no improvement.

Editing Permissions:

1) Static files should be served by a web server, not an application server. I use the fact that in our case this (unfortunately) is not a configuration, so I can see the performance of the application server for files that it does not execute, therefore eliminating the cost of database performance, etc.

2) I don’t think there is a proxy server between the proxies and the application server, but even if it did not seem to be overloaded, because the static files requested from the same application server, but outside the JVM application instance coming back.

3) The JVM (Xmx) heap size is set to 1 GB.

Thanks for any help!

Answer 1

After using the Sun performance monitoring tool, we found that the garbage collector worked every couple of seconds and that only about 100 MB of the 1 GB heap was used. Therefore, we tried to add the following JVM parameters, and so far this new configuration has significantly improved performance.

-XX: + DisableExplicitGC -XX: + AggressiveHeap

See http://java.sun.com/docs/performance/appserver/AppServerPerfFaq.html

Our lesson: do not leave JVM settings and garbage collection settings to the end. If you have performance issues, review these options at the beginning of the troubleshooting process.

Answer 2

Sunone itself is a pain in the ass. I have the same problem and you know what? Just redeploying the same application to Weblogic reduced memory and CPU consumption by about 30%.

SunONE is the reference implementation server and should not be used for production (I don't know about Glassfish).

I know this answer really does not help, but I noticed significant pauses even in very simple operations, such as getting a bean instance from the pool.

Maybe trying to deploy JBoss or Weblogic on the same computer will give you a hint?

PS You should not serve static content from under the application server (although sometimes I do it when the processor is full).

PPS 500 concurrent users of a rather high load, I definitely set SunONE to be a caching proxy or Apache that serves static content.