The larger cache size always leads to better performance?

Question

Since the cache inside the processor increases the speed of command execution. I am wondering if we increase the cache size to many MB, for example, 1 GB. Is it possible? If this will increase the size of the cache will always lead to an increase in performance?

Answer 1

This is a simplification, but one of the main reasons that the cache increases its “speed” is that it provides fast memory very close to the processor - it is much faster than accessing the main memory. So, theoretically, increasing the cache size should allow you to store more information in this "fast" memory and thereby improve performance. In the real world, things are obviously much more complex than that, and of course there will be added complexity and cost associated with such a large cache, as well as problems such as cache consistency, cache algorithms, etc.

Answer 2

There is a difference between cache size and click speed on one side and read latency with power consumption on the other. Thus, the answer to your first question: technically (possible) is possible, but it hardly makes sense, since L3 cache memory in modern processors with a size of only a few MB has a latency of reading of about dozens of cycles.

Performance depends more on the memory access pattern than on the cache size. More precisely, if the program is mostly sequential, the size of the cache does not matter much. If there is quite a bit of random access (for example, when associative containers are heavily used), the cache size really matters.

The above is true for single computing tasks. In a multiprocessor environment with several active processes, a larger cache size is always better, due to reduced competition between processes.