For a simple "Hello World" do you need 10G virtual memory on a 64-bit machine versus 1G on a 32-bit one?

Question

Running a simple Java program on our production machine, I noticed that this program consumes more than 10 GB. I know that virtual memory is not so important, but at least I would like to understand why this is necessary.

public class Main { public static void main(String[] args) { System.out.println("Hello World!"); try { Thread.sleep(10000); } catch(InterruptedException e) { /* ignored */ } } } 

This is what top says when I run this little program:

  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18764 myuser 20 0 10.2g 20m 8128 S 1.7 0.1 0:00.05 java 

Does anyone know why this is happening?

uname -a says:

 Linux m4fxhpsrm1dg 2.6.32-358.18.1.el6.x86_64 #1 SMP Fri Aug 2 17:04:38 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux 

On an older machine with a 32-bit line, the same program consumes only about 1 GB. The old machine has 4 GB of RAM, the new 32 GB.

Answer 1

default sizes for the initial heap and maximum heap are defined as a percentage of the physical memory of the machine, of which today the production server tends to have a lot.

You can select both options using the -Xms and -Xmx command-line options .

Answer 2

Virtual memory really doesn't matter to you.

The main difference between 32-bit and 64-bit is that the address space in 64-bit mode is incredibly large. If 10 GiB looks a lot to you, note that .NET on a 64-bit version can use TiBs of memory like this. However, on the 32-bit version, .NET is much more conservative (and therefore the JVM) - the address space is 4 gigabytes - this is not so much.

But it does not matter - it does not matter. This simply simplifies programming and does not adversely affect the host OS. It creates a continuous address space for the virtual machine, which means you don’t need to fragment the heap (or, worse, a stack where it is more or less impossible), but they are usually only MiB or so), since you more "real" memory will be required. When you finally make virtual memory, it becomes a little more real - at this moment it needs more or less to be supported by some data storage - whether it is a page file (swap) or physical RAM.

The fact is that the physical location of the memory is not necessarily continuous, but this is done out of your reach, and the mapping is usually very fast. On the other hand, you need to, say, index an array that is actually fragmented into 10 different memory blocks of virtual addresses that (completely unnecessary) work.

So, you have it - virtual memory is almost free on 64-bit. The basic approach is "if it is there, use it." You do not limit other applications, and it saves you a lot of work if you really use it. But up to this point you have only a reservation. This does not mean any physical memory. You don’t pay for friends who could come tonight and sit at your table, but you still have a place for them to sit if they really come - and only when they finally arrive do you actually get “charged”.

See this question for more information on how Java behaves on different machines and in different versions: What is the default maximum heap size for the Sun JVM from Java SE 6? The maximum heap size also determines the amount of reserved virtual memory, since the heap must be a contiguous address space. If it had not been previously reserved, it might happen that the heap could not expand to this maximum value, because someone else had reserved the address space in the place where the heap should expand.

Answer 3

This is not your program that uses this memory; it is the Java VM that reserves this memory, no matter what program is loaded.

Answer 4

It turns out that on a modern computer architecture that uses virtual memory addressing (where the "memory space" that the application sees is actually not associated with memory that is actually physically allocated), it really does not matter how much of this virtual "Memory" is provided to the application when start up. This does not mean that this large memory was allocated by the system.

If the application sees a 10 GB virtual address space, all it signals to the application is that it can use memory addresses up to 10 GB if it wants. However, memory is not actually allocated in physical RAM until it is actually written, and this is done on every page, where the page is a 4 KB memory section. A virtual address space is just virtual until actually used.

Let's say an application is provided with 10 GB of address space, and it starts using some of them. First, a "fresh" - previously untouched page of this virtual memory is recorded, the system at a low level "matches" this virtual page with a section of physical memory, and then writes it. But this application itself does not need to worry about such details, it just acts as if it has full access to the virtual memory area.

In the case of Java applications, this is not the application itself, but Java, which is allocated this address space, and Java simply requests a huge address space by default - the requested amount is calculated relative to the size of the physical memory, but not because it has some kind of need to be conservative, but just for practicality - the application probably won’t want to get enough heap to completely bring the server to its knees, so it works if you assume that this will not happen. As I said above, this does not mean that it is "allocated" or that the system had to spend a lot of resources on this.

Answer 5

Imagine that you are working in a document repository. You have a small facility in the city center where boxes of papers are stored and a much larger warehouse outside the city, 1000 times the space. Each box has a label indicating its contents.

The internal object is the main memory. A warehouse is disk space.

Distributing 10 GB of virtual memory for a new process does not mean finding a place for 10 billion mailboxes for a new client. This means printing 10 billion labels for boxes with adjacent ID numbers on them.

Answer 6

This is not the amount of physical memory that the application actually uses. The virtual memory used by all processes can be several orders of magnitude larger than the physical memory on the machine without any obvious problems.

Answer 7

Your program does NOT use that much memory. JVM / OS reserves this memory, i.e. The UPTO limit that your program can use. In addition, as one answer clearly states. 32 bits and 64 bits have nothing to do with this. 32 bits means that you can access physical memory cells up to 2 ^ 32. and 64 bits means up to 2 ^ 64.