Configuring MaxDirectMemory and MaxHeapMemory for Java Applications

Question

For my Java application, I tried to limit heap memory and use direct memory using command line options.

I came across the next VMware article when I tried to learn more about the memory layout of Java applications.

In this article, I suggested that the -Xmx parameter can be used to limit the use of the heap, and the MaxDirectMemory parameter can be used to limit the internal memory outside the heap (guest OS memory in the diagram). But the results are different when I ran a simple program. I used ByteBuffer.allocateDirect to allocate my own memory, and ByteBuffer.allocate to allocate heapmemory.

This is a 64-bit processor (OSX) and a 64-bit JVM.

First experiment

import java.nio.ByteBuffer; import java.nio.channels.FileChannel; import java.nio.file.Paths; import java.nio.file.Path; import java.util.*; public class javalimits { public static void main (String [] args) throws Exception { ArrayList al = new ArrayList(); for(int i = 0; i< 100;i++) { ByteBuffer bb = ByteBuffer.allocateDirect(1024 * 1024* 1024); al.add(bb); System.out.println(" Buffer loop "+ i); Thread.sleep(500); } Thread.sleep(10000); } } 

When I ran the above program without any parameters, it crashed after allocating 3.6 GB of memory. When I used the option "-XX: MaxDirectMemorySize = 100g" or "-Xms100g -Xmx100g", it crashed after 65 cycles or about 65 GB of memory.

I do not understand

• Since my physical drum is just 16G, why didn't it fall after 16G of memory allocation? What is special about 64G memory allocation?
• How do the restrictions on the allocation of main memory when using "-Xms100g -Xmx100g" change? I assumed that the limitations of its own memory are controlled only by the option “-XX: MaxDirectMemorySize = 100g” in accordance with the above diagram in the link I. But the results are different. The settings in the memory cells also changed the limits on the direct memory buffer.
• What is especially important in allocating 3.6G memory if there are no command line options?

Second experiment

I changed ByteBuffer.allocateDirect to ByteBuffer.allocate to allocate on the memory heap instead of my own memory.

 import java.nio.ByteBuffer; import java.nio.channels.FileChannel; import java.nio.file.Paths; import java.nio.file.Path; import java.util.*; public class javalimits { public static void main (String [] args) throws Exception { ArrayList al = new ArrayList(); for(int i = 0; i< 100;i++) { ByteBuffer bb = ByteBuffer.allocate(1024 * 1024* 1024); al.add(bb); System.out.println(" Buffer loop "+ i); Thread.sleep(500); } Thread.sleep(10000); } } 

When I ran the above program without any parameters, it crashed after allocating 2.7G memory. When I used the "-XX: MaxDirectMemorySize = 100g" option, it had no effect. It crashed after allocating 2.7G memory. I felt it made sense. But when I added the parameter “-Xms100g -Xmx100g”, it crashed after 48 cycles or about 48 GB of memory.

I do not understand why,

• Since my physical drum is just 16G, why didn't it fall after 16G of memory allocation? What is special about 48G heap memory allocation?
• What is special about 2.7G memory allocation if command line options are not specified?

Third experiment

I turned on allocateDirect and allocated functions inside the loop. when I added the parameter "-Xms100g -Xmx100g", it crashed after 24 cycles or effectively 48 GB of memory, combining both. (24G internal memory + 24G heap memory)

Can someone help me figure out where I am wrong in understanding the Java memory layout? (Referring to the chart in the link)

Answer 1

A pretty good explanation of memory management can be found here: https://smarttechie.org/2016/08/15/understanding-the-java-memory-model-and-the-garbage-collection/

To answer your questions:

I do not understand why,

• Since my physical drum is just 16G, why didn't it fall after 16G of memory allocation? What is special about 48G heap memory allocation?

Physical memory is not the limit for a system; it can use the swap technique. This allows the system to remove rarely used pages from a modified page from physical memory, so that the system uses physical memory more efficiently for more frequently visited pages.

What is special about 48G is that your system can only handle this amount of memory. You can try playing with swamping and let the system allocate all 100G.

2. What is special about 2.7G memory allocation if command line options are not specified?

Before starting java from the command line, check the following:

Window

 java -XX:+PrintFlagsFinal -version | findstr /i "HeapSize PermSize ThreadStackSize" 

Linux

 java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize' 

You can also track the distribution of visual memory over jconsol.

Why different memory sizes were located, you should read this:

ByteBuffer.allocate () and ByteBuffer.allocateDirect ()