Redis: How does performance_rragmentation_ratio affect performance?

Question

Here is my question for you: how does "mem_fragmentation_rateo" affect performance in Redis?

I did some investigation on stackoverflow and redis.io, but I could not find a clear explanation about my question.

As also explained in this book https://www.datadoghq.com/wp-content/uploads/2013/09/Understanding-the-Top-5-Redis-Performance-Metrics.pdf on page # 16, “The mem_fragmentation_ratio tag gives the ratio of used memory, as seen (used_memory_rss) to the memory allocated by Redis (used_memory). "

So, as explained at http://redis.io/commands/INFO , "ideally, the value of used_memory_rss should be slightly higher than used_memory. When rss is used, a big difference means memory fragmentation (internal or external) that can be evaluated by checking mem_fragmentation_ratio. When using → rss, it means that some of the Redis memory has been disabled by the operating system: expect some significant delays. "

So, when we have a higher "used_memory" than "used_memory_rss", the negative effect on the Redis node is clear, because it means that Redis started using Swap, and this is clearly not suitable for performance.

But what about a situation where we have a higher "used_memory_rss" than "used_memory"? This should probably mean that our memory allocator did not “take” part of the memory assigned to Redis in the past, and that Redis is no longer using it (possibly due to a temporary outburst). However, I cannot understand how this could adversely affect Redis performance. Why am I reading a lot of posts on the Internet where people are so worried about the high "mem_fragmentation_ratio"? Also in the book I mentioned earlier, they suggest having a "mem_fragmentation_rateo" between 1 and 1.5.

In this situation, when we have a higher "used_memory_rss", are Redis really affected? Should we expect a performance degradation of Redis with a high "mem_fragmentation_ratio"?

I decided to ask you this question because I honestly believe that with a high "mem_fragmentation_ratio" we may have a problem "only" in the OS, which "may" suffer from a lack of memory: the OS can’t allocate new memory to new processes. in any case, it should not directly affect Redis (unless the OS starts to slow down, etc.).

Could you clarify my doubts? Did you have experience when you experienced the high "mem_fragmentation_ratio" and you started to notice the performance degradation of Redis?

thanks

Answer 1

I'm not a Redis guru, but from my experience and the documentation I read, fragmentation_ratio does not directly affect performance. But it increases memory consumption. When your application adds new data and the data does not fill in free fragments in memory (when new data bundles are larger than these fragments), this memory remains free. And so you can run out of memory faster than you expect.