Dynamic memory allocation in RAM

Question

On a 32-bit and 64-bit Windows machine, I have to allocate memory for storing large amounts of data that are broadcast in real time, for a total of about 1 GB. If I use malloc (), I am going to get the virtual memory address, and this address can actually cause some paging to the hard disk depending on the amount of memory that I have. Unfortunately, I'm afraid that HD will affect performance and result in a lack of data.

Is there a way to make the memory allocate only in RAM, even if it means that I get an error when there is not enough memory (so the user needs to close other things or use another machine)? I want to guarantee that all operations will be performed in memory. If this fails, forcing the removal of the application is acceptable.

I know that another process may come and take some memory by itself, but I'm not worried because this does not happen on this computer (this will be the only application on the machine that will make this big allocation).

[Edit:] My attempt so far has been to try using VirtualLock as follows:

if(!SetProcessWorkingSetSize(this, 300000, 300008)) printf("Error Changing Working Set Size\n"); // Allocate 1GB space unsigned long sz = sizeof(unsigned char)*1000000000; unsigned char * m_buffer = (unsigned char *) malloc(sz); if(m_buffer == NULL) { printf("Memory Allocation failed\n"); } else { // Protect memory from being swapped if(!VirtualLock(m_buffer , sz)) { printf("Memory swap protection failed\n"); } } 

But a failure in the working set fails, as well as VirtualLock. Malloc returns nonzero.

[Edit2] I also tried:

  unsigned long sz = sizeof(unsigned char)*1000000000; LPVOID lpvResult; lpvResult = VirtualAlloc(NULL,sz, MEM_PHYSICAL|MEM_RESERVE, PAGE_NOCACHE); 

But lpvResult is 0, so you're out of luck too.

Answer 1

You can use the functions mlock, mlockall, munlock, munlockall to prevent the exchange of data with pages (part of POSIX, also available in MinGW). Unfortunately, I have no experience with Windows, but it seems like VirtualLock is doing the same.

Hope this helps. Good luck

Answer 2

I think VirtualAlloc can help you with what you want.

This problem really comes down to simply writing your own memory manager instead of using the CRT function.

Answer 3

You need to use the undocumented NtLockVirtualMemory lock function 2 ( LOCK_VM_IN_RAM ); make sure that you first request and receive the SE_LOCK_MEMORY_NAME privilege, and remember that it may not be granted (I am sure that Group Policy uses the privilege by default, but it can be very granted to no one).

I suggest using VirtualLock as a backup, and if that fails, use SetProcessWorkingSetSize . If this fails, then just let it fail, I think ...

See this link for a pleasant discussion on this. One person says:

When you specify the LOCK_VM_IN_WSL flag, you simply tell the balance settings manager that you do not want any particular page to be replaced with a disk, and ask her to leave this page alone when cropping the target’s working set to process. This is just an indicator, so the landing page can still be replaced if there is not enough RAM in the system. However, when you specify the LOCK_VM_IN_RAM flag, you issue a directive to the memory manager to treat this page as unavoidable (i.e. do something that the driver does when it calls MmProbeAndLockPages () to block the pages described in MDL), so the question on the page is guaranteed to be loaded into RAM all the time.

---

Edit:

Read this .

Answer 4

One option is to create a RAM disk from your host memory. Although there is no longer built-in support in common Windows code, you can still find drivers that are needed for free or available through commercial products. For example, DRDataRam provides a free driver for personal use and a commercially licensed product for business use at: http://memory.dataram.com/products-and-services/software/ramdisk

There is also a virtual ImDisk driver, available at: http://www.ltr-data.se/opencode.html/#ImDisk. It is open and available for commercial use. It is digitally signed with a trusted certificate from Microsoft.

For more information about RAM disks in Windows, visit ServerFault.com.

Answer 5

You should take a look at Address Window Extensions (AWE) . It looks like it matches the memory limitations that you have (my attention):

AWE uses physical non-paged memory and window views of various parts of this physical memory in a 32-bit virtual address space.