Why does memory fragmentation occur on a 64-bit machine?

Question

In a 32-bit machine, each process receives 4 GB of virtual space. In this case, you may worry that we might run into problems due to fragmentation. But in the case of a 64-bit machine, theoretically we have a huge addressable virtual memory, so why is memory fragmentation still a problem (if any) on a 64-bit machine?

Answer 1

Each virtual address that you are trying to obtain is mapped by the operating system to physical memory. Physical memory is allocated on pages (for example, 4 KB). If you manage to select a byte at an offset of 1,000,000 * n and do it for n from 1 to 1,000,000 (I think you could do it with mmap), then the OS will have to return this with a million pages of physical memory, which is something like 4G . This physical memory will not be available for anything else. If you allocated bytes contiguously, you only need about 1M of physical memory (256 pages) for your million bytes.

You may encounter similarly bad situations if you assigned 4G for legitimate reasons, and then free some of it, saving a little of each page. The OS will not be able to really reuse the freed memory for anything else, because there are no completely free physical pages. So the problem is fragmentation.

In theory, you could imagine that the virtual addresses 1,000,000 and 2,000,000 would be mapped onto the same page of physical memory, avoiding fragmentation. But in practice and for good reasons, virtual memory mapping is done page by page. You can read more about this here: http://en.wikipedia.org/wiki/Page_table .

Answer 2

Because all this memory is "wasted" considering an application in which you have a lot of internal fragmentation. This process requires more pages in memory, because the working set is now scattered across memory, which means that its memory capacity is much higher. If this application competes for physical slots in RAM (machines still actually have about 4 - 8 GB of RAM for normal home setup), then this leads to a larger rearrangement of pages. Typically, you want to reduce the application memory in order to avoid excessive pressure and competition with other applications.

There are times when it doesn’t really matter, it won’t kill you to use extra megabytes here or there, but all this is added to larger applications. It depends on the situation regarding whether it is important to have as little fragmentation as possible, depending on what you are coding or what is the purpose of your project.