Memory management strategy

Question

We are developing an enterprise application that caches a lot of data from the back side. Users are allowed to open an arbitrary number of application windows, and each downloads its own data and caches it. In order to somehow manage the memory consumption and reduce the overall OS performance, we decided to write a cache manager that will automatically monitor the application’s memory size and delete data from the cache if necessary.

So the problem is that it’s hard for us to determine whether it’s time to free up memory. We are currently using a very simple approach - we are just starting to drop things from the cache when application memory usage exceeds 80% of physical memory.

Are there any (alternative?) Established methods to solve this problem?

Answer 1

This is basically normal. There is no really good strategy. If there are several competing applications, this can lead to cash contests and false evictions.

If you select a threshold that is too low, you are losing cache space. If it is too high, nothing can fit into memory, including file cache, DLLs, ...

What do you mean by "accessible physical memory"? Do you mean installed memory or memory, which is free? How can an application use 80% of free memory? I do not understand what you are using.

SQL Server uses memory until the OS reports that it is inactive in memory (I believe this happens when 95% of "something" is used).

You certainly do not want to use the GC to free memory. It will usually kill your entire cache.

Maybe you can completely move the contents of the cache to disk? Or you could share the cache between .NET processes, having a hidden cache server process, which could be a request by application processes.

---

I want to emphasize that if your application consumes 99% of the installed RAM (as an example), the performance will be very bad, because the cache file is almost empty. This means that even DLLs and .NET NGEN'ed code will be downloaded frequently.

Perhaps the best strategy is to assume that 1GB is required for the proper caching of OS and application files. Thus, you can use the memory until there is only 10% of the installed RAM minus 1 GB.