How does NSMutableData allocate memory?

Question

When I run the following code, it slowly swallows up my memory and even starts using swap:

long long length = 1024ull * 1024ull * 1024ull * 2ull; // 2 GB db = [NSMutableData dataWithLength:length]; char *array = [db mutableBytes]; for(long long i = 0; i < length - 1; i++) { array[i] = i % 256; } 

If I run it without a for loop, the memory is not used at all:

  long long length = 1024ull * 1024ull * 1024ull * 2ull; db = [NSMutableData dataWithLength:length]; char *array = [db mutableBytes]; /* for(long long i = 0; i < length - 1; i++) { array[i] = i % 256; } */ 

I can only conclude that NSMutableData is only a "backup" memory, and when it is accessed, it really "allocates" it. How is this done exactly?

Is this done through hardware (CPU)?

Is there a way for NSMutableData to catch records in memory in "reserved" memory and only then do the "allocation"?

Does this also mean that the call to [NSMutableData dataWithLength:length] cannot fail? Can he allocate any amount of memory using swap to get it if necessary?

If it can fail, will my db variable be null?

In the apple "NSMutableData Class Reference", I saw only vague sentences on these topics.

Answer 1

This is not so much a NSMutableData problem, but a kernel / OS problem. If a process requests a (large) chunk of memory, the kernel will usually just say "it's ok, here you go." But only in fact, using it, is it really ("physically") allocated. This is normal, because if your program starts with 2 GB of malloc (as you do here), it otherwise otherwise immediately crowds out other exchange programs, whereas in practice you often will not use 2 GB at once.

When accessing a memory page that is not actually present in physical memory, the kernel will receive a signal from the CPU. If the page should be there (because it is in your 2 GB block), it will be inserted into place (possibly from a swap), and you won’t even notice. If there should not be a page (because the address is not allocated in your virtual memory), you will receive a segmentation error (error type SIGSEGV or EXC_BAD_ACCESS).

One related topic is "overcommit (ment)", where the kernel promises more memory than is actually available. This can cause serious problems if all processes start using the promised memory. It depends on the OS.

There are many pages on the Internet explaining this better and in more detail; I just wanted to give a short introduction, so you have conditions for posting on Google.

change has just been tested, linux easily promises me 4 TB of memory, and, I assure you, this device does not even have 1 TB of total disk storage. You can imagine that this, if not taken care of, can cause some headaches when creating mission-critical systems.