Debugging memory issues in java - understanding freeMemory

Question

I have an application that throws OutOfMemoryError, so I'm trying to debug it using Runtime.getRuntime (). freeMemory (). Here is what I get:

freeMemory=48792216 ## Reading real sentences file...map size=4709. freeMemory=57056656 ## Reading full sentences file...map size=28360. freeMemory=42028760 freeMemory=42028760 ## Reading suffix array files of main corpus ...array size=513762 freeMemory=90063112 ## Reading reverse suffix array files... array size=513762. freeMemory=64449240 

I am trying to understand the behavior of freeMemory. It starts with 48 MB, then - after I read a large file - it will move to 57 MB, then it will go down to 42 MB again, then - after I read a very large file (513762 elements), it jumps to 90 MB, then down to 64 MB.

What's going on here? How can I understand these numbers?

Answer 1

java memory is a bit complicated. Your program runs inside jvm, jvm runs inside os, os uses the resources of your computer. When your program needs memory, jvm will see if it already requested some memory that is not currently in use, if there is not enough memory, jvm will ask for os and, if possible, get some memory.

From time to time, jvm will look for memory that is no longer in use and free it. Depending on the (huge) number of factors, jvm can also put that memory back into os so other programs can use it.

This means that at any given time, you have a certain amount of memory received by jvm from os, and a certain amount that jvm uses.

At any given point, jvm may refuse to receive more memory because it was patched up for this, or os may deprive jvm of access to more memory, either because it is instructed to do it again, or simply because there is no more free ram .

When you run your program on your computer, you probably don't give any restrictions to jvm, so you can use a lot of rams. When launched in Google applications, there may be restrictions imposed by jvm by Google operators, so the available meory may be less.

Runtime.freeMemory will tell you which part of the bar received by jvm from os is currently free.

When you allocate a large object, say one megabyte, jvm may require more bar for os, say 5 MB, resulting in freeMemory will be 4 MB more than before, which contradicts intuition. Allocating another MB is likely to reduce free memory as expected, but later jvm may decide to free some memory to thenos, and freeMemory is compressed again for no apparent reason.

Using totalMemory and maxMemory in combination with freeMemory, you can better understand your current limits and consumption.

To understand WHY you are consuming more bars than you expected, you should use a memory profiler. A simple but effective package is packaged with visualvm, a tool that is already installed using jdk. There you can see what ram uses in your program and why this memory cannot be restored by jvm.

(note that the jvm memory system is much more complicated than that, but I hope this simplification helps you understand more than the complete and complicated picture).

Answer 2

This is not terribly clear or convenient. If you look at the api runtime, you will see 3 different calls in memory:

freeMemory Returns the amount of free memory in a virtual Java machine. Calling the gc method may increase the value returned by freeMemory.

totalMemory Returns the total memory in a virtual Java machine. The value returned by this method may change over time, depending on the host environment.

maxMemory Returns the maximum amount of memory that the Java virtual machine will try to use.

When starting jvm, you can set the initial heap size (-Xms) as well as the maximum heap size (-Xmx). for example, java -Xms100m -Xmx 200m starts with a heap of 100 m, the heap grows, since more space is required up to 200, and OutOfMemory will fail if it needs to grow beyond this. So there is a ceiling that gives you maxMemory ().

The memory currently available in the JVM is somewhere between your start and max. Somwhere. This is your shared memory (). freeMemory () is how much free of this amount.

To add to the confusion, see what they say about gc - "Calling the gc method may increase the value returned by the freeMemory function." This means that irrevocable garbage is not included in free memory.

Answer 3

OK, based on your comments, I wrote this function, which prints a summary of memory measurements:

 static String memory() { final int unit = 1000000; // MB long usedMemory = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory(); long availableMemory = Runtime.getRuntime().maxMemory() - usedMemory; return "Memory: free="+(Runtime.getRuntime().freeMemory()/unit)+" total="+(Runtime.getRuntime().totalMemory()/unit)+" max="+(Runtime.getRuntime().maxMemory()/unit+" used="+usedMemory/unit+" available="+availableMemory/unit); } 

It seems that the best measures for using my program are using Memory and the additional available memory. They increase / decrease monotonously when I use more memory:

 Memory: free=61 total=62 max=922 used=0 available=921 Memory: free=46 total=62 max=922 used=15 available=906 Memory: free=46 total=62 max=922 used=15 available=876 Memory: free=44 total=118 max=922 used=73 available=877 Memory: free=97 total=189 max=922 used=92 available=825 

Answer 4

Try running the application against http://download.oracle.com/javase/1.5.0/docs/guide/management/jconsole.html . It comes with a JVM (or, of course, is used), and is invaluable in terms of monitoring what happens inside the JVM at runtime.

This will provide more useful insight into what is happening to your memory than your debug statements.

Also, if you are really interested, you can learn a little about setting up garbage collectors through something like: http://www.oracle.com/technetwork/java/gc-tuning-5-138395.html

This is pretty detailed, but it's good to get an idea of ​​the different generations of memory in the JVM and how objects are stored in these generations. If you see that objects are preserved in the old generations, and the old gene is constantly growing, this may be an indicator of leakage.

For debugging, why data is saved and not collected, you can not go past the profilers. Check out JProfiler or Yourkit.

Good luck.