C ++ program heap size reduction after big calculations

Question

Consider an MPI application based on two steps, which we will call load and globalReduce. Just for simplicity, the software is described as such, but much more is happening, so this is not just a Map / Reduce problem.

During the boot phase, all ranks in each given node are queued, so that one and only one rank has full access to all node memory. The reason for this design is due to the fact that during the boot phase there is a set of large I / O blocks that are read, and all of them must be loaded into memory before local reduction occurs. The result of this local reduction will be called the named variable myRankVector. After receiving the variable myRankVector, the IO blocks are freed. The myRankVector variable itself uses a small amount of memory, therefore, during its creation, the node can use all the memory, after the rank is completed, it takes only 2-3 GB to store myRankVector.

During the globalReduce stage in node, it is expected that all ranks in node have loaded their respective globalReduce.

So, here is my problem, while I guaranteed that there are no memory leaks (I program using shared pointers, I double-checked Valgrind, etc.), I am sure that the heap remains extended even after all destructors have issued blocks input-output When the next rank in the queue comes to work, it starts to request more memory, as the previous rank did, and, of course, the program kills Linux, yielding "Out of memory: Kill process xxx (xxxxxxxx) score xxxx or sacrifice a child". It is clear why this is so, the second rank in the queue wants to use all the memory, but the first rank remains with a large pile.

So, after setting up the context of this question: is there a way to manually reduce the heap size in C ++ in order to really free memory that is not in use?

Thanks.

Answer 1

So, after setting up the context of this question: is there a way to manually reduce the heap size in C ++ in order to really free memory that is not in use?

This operating system is dependent, but most likely not possible.

Most operating systems leave you allocated resources from one process until this process is completely completed and destroyed.

Answer 2

Heaps are implemented using mmap on linux, and you will need to use your own heap, which you can completely utilize and munmap.

munmap free up the required space.

Look at the code in boost: pool for an implementation that allows you to independently manage the underlying heaps.

In my experience, it is very difficult to manage std containers with custom allocators, since they are derived from the class and not derived from the instance.

Answer 3

Could shared memory solve your problem (even if you do not want to share this memory)? You can allocate a shared memory block in the “boot” phase and disable it after computing “myRankVector”.

(see shmget, shmat, shmdt, shmctl (..., IPC_RMID ,.))