If / When does the recovery of freed heap memory occur?

Question

I ran memory tests overnight on the embedded Linux system. Using vmstat, I noticed that free memory is steadily decreasing over time. According to some smaps analysis in procfs , a bunch of one process grows at about the same rate. I suspected a memory leak and found a few spots in the code where new and delete regularly used. However, I have not seen new calls without matching delete calls.

I ran the memory test again and cleared the memory caches with the next call this morning

 echo 3 > /proc/sys/vm/drop_caches 

The free memory specified in vmstat is reduced to a value close to the beginning of the test.

Does the kernel regularly restore unused heap pages? If so, are there other points besides what has been done above? Probably when free memory falls below a certain threshold?

Answer 1

As others have said, it is the responsibility of a process to return memory to the kernel.

Usually there are 2 ways to allocate memory: if you malloc() / new memory block is above a certain size, the memory receives the allocation from the OS via mmap() and is called as soon as it is free. Smaller blocks are allocated by increasing the process data area, shifting the sbrk boundary up. This memory is freed only if a block of a certain size is free at the end of this segment.

For example: (pseudo code, I don't know C ++ very well)

 a = new char[1000]; b = new char[1000]; 

Memory card:

 ---------------+---+---+ end of program | a | b | ---------------+---+---+ 

If you free a now, you have a hole in the middle. He is not released because he cannot be released. If you free b , the process memory may or may not be reduced; unused balance is returned to the system.

A test with such a simple program as

 #include <stdlib.h> int main() { char * a = malloc(100000); char * b = malloc(100000); char * c = malloc(100000); free(c); free(b); free(a); } 

causes strace to exit, sort of

 brk(0) = 0x804b000 brk(0x8084000) = 0x8084000 brk(0x80b5000) = 0x80b5000 brk(0x809c000) = 0x809c000 brk(0x8084000) = 0x8084000 brk(0x806c000) = 0x806c000 

shows that the brk value first increases (for malloc() ) and then decreases again (for free() ).

Answer 2

The kernel will restore the cached pages of memory when they need them, that is, when the system would otherwise end the memory. whether the memory pages from the process heap (free storage) are ever returned to the OS, at the discretion of the process memory manager, in this case, the new / delete implementation in the C ++ library. This is a completely voluntary operation with which the kernel has nothing to do.

Because drop_caches did the trick, you can conclude that this is the kernel cache, not a bunch of the process that populated the memory. Use the free command to find out how much memory is actually available to use the application, especially. the line -/+ buffers/cache , which it reports.

Answer 3

Calling delete in your program causes memory to return to the memory manager, which is part of your program runtime. In principle, this could be written to return the freed memory of the OS, but I would be surprised if this happened. Rather, the secondary memory is kept aside for subsequent calls to new .

Please note that this is the virtual memory of your process; how many of them are actually in physical memory at any time during program execution depends on the total system load and is processed by the operating system.

Answer 4

Custom calls to malloc and free (or new and deleted), as far as I know, never return unnecessary pages to O / S. Instead, they simply remember which memory was freed, so if you make a malloc / new size that can be satisfied with the previously freed memory, then it will use this instead of going to O / S and using sbrk to get more memory.

So this code:

 for (;;) { struct { char data[200 * 1024 * 1024] } HugeBuffer; HugeBuffer *buff = new HugeBuffer; delete buff; } 

Will allocate 200 MB once, and then just constantly use this memory forever. It will go to O / S once upon initial distribution, and then scroll around in user space.