All memory cannot be used.

Question

I am trying to write a program that should consume memory of a certain size. I am wondering that I get an outOfMemory exception when there really is free space on the heap.

Here is the code:

 import java.util.Vector; import java.lang.*; public class MemoryEater1 { public static void main(String[] args) { try { long mb = Long.valueOf(args[0]); Vector v = new Vector(); Runtime rt = Runtime.getRuntime(); while (true) { if (v.size() > 0) { if (((long) v.size())*100 < mb) { System.out.println("total memory: " + rt.totalMemory()/1024/1024); System.out.println("max memory: " + rt.maxMemory()/1024/1024); System.out.println("free memory: " + rt.freeMemory()/1024/1024); System.out.println("Trying to add 100 mb"); //100mb byte b[] = new byte[104857600]; v.add(b); } } else { //100mb byte b[] = new byte[104857600]; v.add(b); System.out.println("Added 100 mb"); } } } catch (Exception e) { e.printStackTrace(); } } } 

Command to run:

  java -Xmx4096m MemoryEater1 3000 

And the conclusion:

 total memory: 2867 max memory: 3641 free memory: 59 Trying to add 100 mb Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at MemoryEater1.main(MemoryEater1.java:18) 

Well, the difference between maximum memory and shared memory is 774 MB, which should be enough to consume 100 MB more, but still there is an error, and even sufficient machine resources:

 [ user@machine ~]$ free -m total used free shared buffers cached Mem: 15950 3447 12502 0 210 2389 -/+ buffers/cache: 847 15102 Swap: 4031 1759218603 8941 

Why could this be?

Answer 1

I do not think that this is fragmentation, since you have only one thread that allocates memory and does not restore anything.

This is your specific garbage collector that is to blame, they manage memory in different ways, which leads to more or less inaccessible to your application. You can find out which one is used when parsing java -XX:+PrintCommandLineFlags output java -XX:+PrintCommandLineFlags ).

You can try using G1, which manages memory in different ways.

 java -Xmx4096m -XX:+UseG1GC MemoryEater1 3000 

Or play with the sizes of generations, for example. -XX:NewSize , etc.

For more information, read Virtual Machine Settings and anything on garbage collection algorithms, for example. [GC setup]

Here is a brief illustration of how sharing memory for different generations can make it inaccessible ( http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html ).

Answer 2

You can set JVM flags for more memory than physical memory on your computer. With the information provided so far, most likely you do not have enough physical memory to allocate this array. In other words, the JVM is requesting more memory from the OS, and the OS says it is not there.

Another possibility is the memory fragmentation issue noted in the comments on your question. I think it is less likely, however, in this case because of the structure of your program. I do not think this can be ruled out.