Using Java RAM does not match what Task Manager says

Question

I played with the Java JVM by creating an array of bytes with a length of 1024^3 (mostly 1 GB). I measured the use of RAM before, after creating the array and after the array was destroyed by the garbage collector, using the task manager (looking at the process) and this small fragment:

 public static void showMemory() { System.out.println("Memory used: " + (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) / (1024.D * 1024.D) + "mB."); } 

The above code shows 2Mb, 1029Mb and 2Mb, respectively. → Everything seems normal. However, looking at the TaskManager, the use of Java RAM is initially 2 MB, then it goes to 1052 MB and remains there, even if the fragment shows 2 MB.

How I would like Java to use the smallest resource, how can I solve this problem?

Edit:

I did some research and found out the conditions of use. Actually, native memory is not like the value of heap memory, and often more than heap memory. Is there a way to reduce the internal memory used so that it is close to heap memory?

Answer 1

Output:

Use garbage first (G1) GC (default GC in Java 9), this garbage collector also reduces heap size (which, in conclusion, will also reduce the total "native memory" used) on garabel collections compared to ParallelOldGC (default GC in Java 7 and Java 8), which rarely rarely reduces heap size!

---

Generally:

Your basic assumption is incorrect.

You assume that the code snippet shows the heap size. This is not true. It shows heap usage. This means: "How much space of my heap is used?". Runtime.getRuntime().totalMemory() shows the size of the heap, Runtime.getRuntime().freeMemory() shows the size of the free heap, their difference shows the use of the heap (used size).

Your heap starts with the initial size using 0 bytes, because the object has not yet been created and the maximum heap size. The maximum heap size describes the size by which the garbage collector can resize the heap (for example, if there is not enough space for a very large object)

As a next step, after creating an empty heap, some objects (class objects, etc.) are automatically loaded, they usually should easily fit in the size of the initial heap.

Then your code starts working and selects objects. As soon as there is no more space in your Eden space (the heap is divided into the young generation (Eden, surviving and surviving into space) and the old generation, look for additional resources if you are interested in these details), garbage collection starts.

During garbage collection, the garbage collector may decide to resize the heap (as mentioned above when it comes to maximum heap size). This happens in your case because your initial heap size is too small to fit your 1GB object. Therefore, the heap size increases, somewhere between the initial heap size and the maximum heap size.

Then, after your large object has died, the next GC could make the heap smaller again, but shouldn't . What for? It is below the maximum heap size that all GCs need. Some garbage collection algorithms compress the heap again, and some do not.

In espacially ParallelOldGC , the default GC in Java 7 and Java 8 rarely rarely compresses a bunch.

If you want a GC that also tried to reduce the heap size by decreasing it during garbage collection, try the garabage first (G1) GC by setting the -XX:+UseG1GC Java flag.

Example:

This will print all the values ​​in bytes.

You will get an overview of how both GCs work and how much space is used when using either of them.

 System.out.println(String.format("Init:\t%,d",ManagementFactory.getMemoryMXBean().getHeapMemoryUsage().getInit())); System.out.println(String.format("Max:\t%,d%n", ManagementFactory.getMemoryMXBean().getHeapMemoryUsage().getMax())); Thread outputThread = new Thread(() -> { try { int i = 0; for(;;) { System.out.println(String.format("%dms\t->\tUsed:\t\t%,d", i, ManagementFactory.getMemoryMXBean().getHeapMemoryUsage().getUsed())); System.out.println(String.format("%dms\t->\tCommited:\t%,d", i, ManagementFactory.getMemoryMXBean().getHeapMemoryUsage().getCommitted())); Thread.sleep(100); i += 100; } } catch (Exception e) { } }); Thread allocThread = new Thread(() -> { try { int val = 0; Thread.sleep(500); // Wait 1/2 second createArray(); Thread.sleep(500); // Wait another 1/2 seconds System.gc(); // Force a GC, array should be cleaned return; } catch (Exception e) { } }); outputThread.start(); allocThread.start(); 

createArray() is just a small method:

 private static void createArray() { byte[] arr = new byte[1024 * 1024 * 1024]; } 

- Result of ParallelOldGC :

 Init: 262,144,000 Max: 3,715,629,056 0ms -> Used: 6,606,272 0ms -> Commited: 251,658,240 100ms -> Used: 6,606,272 100ms -> Commited: 251,658,240 200ms -> Used: 6,606,272 200ms -> Commited: 251,658,240 300ms -> Used: 6,606,272 300ms -> Commited: 251,658,240 400ms -> Used: 6,606,272 400ms -> Commited: 251,658,240 500ms -> Used: 1,080,348,112 500ms -> Commited: 1,325,924,352 600ms -> Used: 1,080,348,112 600ms -> Commited: 1,325,924,352 700ms -> Used: 1,080,348,112 700ms -> Commited: 1,325,924,352 800ms -> Used: 1,080,348,112 800ms -> Commited: 1,325,924,352 900ms -> Used: 1,080,348,112 900ms -> Commited: 1,325,924,352 1000ms -> Used: 1,080,348,112 1000ms -> Commited: 1,325,924,352 1100ms -> Used: 1,080,348,112 1100ms -> Commited: 1,325,924,352 1200ms -> Used: 2,261,768 1200ms -> Commited: 1,325,924,352 1300ms -> Used: 2,261,768 1300ms -> Commited: 1,325,924,352 

You can see my heap starts with an initial size of about 260 MB, with a permitted maximum size (the size of which the GC may decide to resize your heap) of about 3.7 GB.

Before creating the array, about 6 MB of my heap is used. Then a large array is created, and my heap size (size specified) is increased to 1.3 GB, with about 1 GB used (array). I then force the garbage collection, which collects the array. However, my heap size remains at 1.3 GB because the GC does not care about reducing it, just using it is reduced by 2 MB.

- Result G1 :

 Init: 262,144,000 Max: 4,179,623,936 0ms -> Used: 2,097,152 0ms -> Commited: 262,144,000 100ms -> Used: 2,097,152 100ms -> Commited: 262,144,000 200ms -> Used: 2,097,152 200ms -> Commited: 262,144,000 300ms -> Used: 2,097,152 300ms -> Commited: 262,144,000 400ms -> Used: 2,097,152 400ms -> Commited: 262,144,000 500ms -> Used: 1,074,364,464 500ms -> Commited: 1,336,934,400 600ms -> Used: 1,074,364,464 600ms -> Commited: 1,336,934,400 700ms -> Used: 1,074,364,464 700ms -> Commited: 1,336,934,400 800ms -> Used: 1,074,364,464 800ms -> Commited: 1,336,934,400 900ms -> Used: 1,074,364,464 900ms -> Commited: 1,336,934,400 1000ms -> Used: 492,520 1000ms -> Commited: 8,388,608 1100ms -> Used: 492,520 1100ms -> Commited: 8,388,608 1200ms -> Used: 492,520 1200ms -> Commited: 8,388,608 

And here we go! G1 GC takes care of small heaps! After cleaning the object, not only the usage is reduced to 0.5 MB, but the heap size also shrinks to 8 MB (compared to 1.3 GB in ParallelOldGC).

Additional Information:

But keep in mind that the heap size will still be different from what is shown in the task manager. The following image from Wikipedia - The Java Virtual Machine shows that the heap is only part of the full JVM memory:

Answer 2

A heap is just one area of ​​JVM memory. For the JVM, there aren't many extra 200-400 MB beyond the maximum heap size for things like shared libraries, code, stack threads, direct memory, and GUI components.

Thus, while a 2 MB (MB = megabyte, Mb = Mega-bit) object can be used at this moment, the application can reserve a lot more.

Is there a way to reduce the internal memory used so that it is close to heap memory?

You can use an older version of Java, which typically uses less memory, a smaller maximum heap, and perm gen uses less additional resources.

Answer 3

You can reduce the heap size if you use the G1 collector for the GC, but the size of your own memory will not decrease. In some applications, it is possible that the allocated internal memory is larger than the actual heap. A native pile suffers from Thrashing.