Memory allocation failed even if there is still enough memory

I am working on Linux (ubuntu 13.04 for sure), and currently I have a question: why the allocation of memory will fail, even if there is still enough memory?

Today I wrote a simple test application, and I ran into this problem when running this test application. Below is the code snippet that I used for testing:

#include <stdio.h> #include <unistd.h> #include <list> #include <vector> #include <strings.h> using namespace std; unsigned short calcrc(unsigned char *ptr, int count) { unsigned short crc; unsigned char i; //high cpu-consumption code //implements CRC algorithm: Cylic //Redundancy code } void* CreateChild(void* param){ vector<unsigned char*> MemoryVector; pid_t PID = fork(); if (PID == 0){ const int MEMORY_TO_ALLOC = 1024 * 1024; unsigned char* buffer = NULL; while(1){ buffer = NULL; try{ buffer = new unsigned char [MEMORY_TO_ALLOC](); calcrc(buffer, MEMORY_TO_ALLOC ); MemoryVector.push_back(buffer); } catch(...){ printf("an exception was thrown!\n"); continue; } //try ... catch } //while } // if pid == 0 return NULL; } int main(){ int children = 4; while(--children >= 0){ CreateChild(NULL); }; while(1) sleep(3600); return 0; } 

During my test, the above code starts throwing an exception when 220M RAM is available . And from now on, it looks like the application can no longer get more memory because the free memory displayed by the TOP command remains above 210M. So why does this happen?

UPDATE
1. Software and Hardware Information
The RAM is 4G and the swap is about 9G . Running "uname -a" gives: Linux steve-ThinkPad-T410 3.8.0-30-generi # 44-Ubuntu SMP Thu Aug 22 20:54:42 UTC 2013 i686 i686 i686 GNU / Linux <w> 2. Statistics in testing time

  Right after Test App Starts Throwing Exception steve@steve-ThinkPad-T410 :~$ free total used free shared buffers cached Mem: 3989340 3763292 226048 0 2548 79728 -/+ buffers/cache: 3681016 308324 Swap: 9760764 9432896 327868 10 minutes after Test App Starts Throwing Exception steve@steve-ThinkPad-T410 :~$ free total used free shared buffers cached Mem: 3989340 3770808 218532 0 3420 80632 -/+ buffers/cache: 3686756 302584 Swap: 9760764 9436168 324596 20 minutes after Test App Starts Throwing Exception steve@steve-ThinkPad-T410 :~$ free total used free shared buffers cached Mem: 3989340 3770960 218380 0 4376 104716 -/+ buffers/cache: 3661868 327472 Swap: 9760764 9535700 225064 40 minutes after Test App Starts Throwing Exception steve@steve-ThinkPad-T410 :~$ free total used free shared buffers cached Mem: 3989340 3739168 250172 0 2272 139108 -/+ buffers/cache: 3597788 391552 Swap: 9760764 9556292 204472 
+4
source share
3 answers

You are running x86-32, so your processes are 32-bit. Even with a large amount of memory + swap, they will be limited by their address space, run:

 grep "HIGHMEM" /boot/config-`uname -r` grep "VMSPLIT" /boot/config-`uname -r` 

to find out how your kernel is configured.

Perhaps your 4 child processes are limited to 3G and use 12G + other processes ~ 700M to achieve the number you see.

EDIT:

So, your kernel is configured to provide each 3G namespace with an address space, some of which will be processed by the program, libraries, and the source run-time memory (which will be shared because of the plug).

Therefore, you have 4 children who use ~ 3G each - ~ 12G + ~ 780M of other programs. This leaves around 220M for free as soon as the kids start reporting bugs.

You can simply start another child process or reinstall using the Ubuntu version with the AND64 / x86-64 version, where each process can allocate much more memory.

0
source

You may have no more than 1MB consecutive memory pages in your address space. you have fragmentation of free space.

+2
source

During my test, the above code starts throwing an exception when there is about 220 MB of memory. And from now on, it looks like the application can no longer get more memory, because the free memory displayed by the TOP command remains above 210M. So why does this happen?

The top output is updated every N seconds (configured) and does not actually show the current status.

On the other hand, memory allocation is very fast.

What happens, your program eats memory, and at a certain point (when the top shows 200 MB for free), it starts to crash.

+2
source

All Articles