The same program, the same JVM, but significantly different memory requirements and runtime on different machines - why?

Question

I am trying to run NetLogo (Java simulation simulation) in a cluster as part of a big experiment. I was surprised at the seemingly huge memory requirement for a relatively (relatively) simple simulation. On the cluster, it throws "java.lang.OutOfMemoryError: Java heap space" exceptions for anything less than "-Xmx2500M", heapsizes. It takes 5 hours to complete. I did the same experiment on both of my computers (iMac and MacBook Pro), and they ran for less than one hour, and "-Xmx1024" did not produce any errors. Cluster tasks require "-XX: MaxPermSize = 250M", whereas on my Mac computers no increase is required that exceeds the default value. I ran the same code, the same inputs, using all the same banks in all cases.

In each case, 64-bit JVMs are used (and as far as I know, they are very similar):

<on the cluster> $ java -version java version "1.6.0_26" Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) <on my macs> $ java -version java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04-415-10M3646) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01-415, mixed mode) 

And I start the Client JVM in all cases (I originally used the server on the cluster, the transition to the client did not matter). I tried to execute on a cluster with java 7, the same huge memory and runtime problems.

I am completely puzzled, no one I spoke to about can explain this. Has anyone there encountered this before? Any help is much appreciated!

Answer 1

I suspect you have a faster network or IO drive. If you use queues to write to disk or write to a network where one computer can keep up and the other cannot, the queue can slow down the machine and use an unlimited amount of memory.

If you have faster network I / O, it can either send data faster (keeping the queues small), or it may mean that you receive data too quickly (the value of the queue can grow faster than it is consumed).

Much depends on what your application actually does. When your program receives OOME, I suggest you get a bunch of heaps and parse it and look for collections (like a queue) that consume a lot of memory.

Answer 2

I suspect the problem is that you are using a server-side JVM. Client JVM is not available on 64-bit machines. Even if you ask the JVM client, it will provide you with a server server.