Geek Answers Handbook

PHP based evaluation cache

I am trying to create a PHP memory cache based on points, but I have a performance problem (how to read the score and delete entries with a low score).

Goals

I have about 10 million records. And I want to cache only 0.1% of the most frequent entries in memory, not on disk.

I would like to install 10,000 slots for cache (0.1%), and I would like to leave only the most visited slots in these slots.

Attempts / Problems

I tried file-based cache and it is very slow.

I tried MySQL and PostgreSQL, but it has too high a cost of working with account counting and deleting records with a low score.

I tried the timer cache, for example. xcache, but due to too much data in my project, it writes too much. There is also a problem with deleting entries with the smallest count and listing all cache slots, since this is a "key-> value".

I found Redis, but it seems that there is not a single point or something like that.

My question is:

What cache method should be used for evaluation-based cache?

Please note that all of these posts are similar, but do not contain any useful answers:

Fastest PHP memory cache / hash table

LRU Expired Memory Cache

Architecture / memory caching technology?

Need php caching recommendation

+4
source share
2 answers

It seems that the LRU cache should give you what you need. You can configure Redis as an LRU cache. This will probably do a good job of your situation. Here is a link to redis docs: http://redis.io/topics/lru-cache

To give a brief summary, you can use the eviction policy "allkeys-lru" and set "maxmemory" to something that you would like. Once the memory limit has been removed, redis will release the least recently used items and save the memory usage in "maxmemory".

Another option is to use "memcached", this is the inmemory key value store and is configured as LRU cache by default.

If you want to really track the results yourself, and you already have some kind of scoring mechanism for your items, you can use Redis, you can save the SortedSet along with the hash to rank your cache items.

The hash will save your data in the cache, and SortedSet will save your positions.

You need these SortedSet commands:

  • You can add and change items using "ZADD"
  • You can also use "ZINCRBY" to change positions.
  • You can use the "ZCARD" command to get the total number of sorted sets.
  • You can use ZRANGE to get the least points.
  • And you can use ZREM to remove items.

After each insertion, you need to manually check the SortedSet account and limit the number of items in the cache. In general, the algorithm will be as follows:

Cache Insert:

HSET "cacheKey" "itemName" "itemValue" ZADD "rankingKey" "itemScore" "itemName" count = ZCARD "rankingKey" if (count > limit) lowestRankedItem = ZRANGE "rankingKey" 0 0 ZREM "rankingKey" lowestRankedItem HDEL "cacheKey" lowestRankedItem

And the search will look like this:

 itemValue = HGET "cacheKey" "itemName"
+1
source

Pure LRU is not very good, as it caches all new keys, and I need to cache only 0.1% of the data (with high rating), so there will not be much unnecessary writing.

Of course, I will implement the Redis method, as suggested in the next version, as this should be much faster. Therefore, use this only if you do not have Redis, if you cannot decide to score any other than just counting hits for some time or simply if you want it to be simple.

But my best attempt is to use key => value memcached, just test now and look fast and stable. Before I thought wrong ...

When a new element needs to be cached, we check if there is an account in the cache (the cache for the key contains int), if yes and less than our limit, we increase the score, if yes and above the limit, we get the content and save the cache. If there is anything else in the cache besides the number, this is our content (high score). If the cache for the key does not exist at all, we save int 1 as an account in the cache with a very short TTL, and we just get the content without caching (low score).

 <?php $minhits = 10; // Min 10 hits for one key $minhitstime = 60 // Max 60 seconds between hits $cachettl = 3600; // Cache for 3600 seconds $key = "article1"; $mem = new Memcached(); $mem->addServer('localhost', 11211); $content = $mem->get($key); if(!$content OR (is_int($content) AND $content<$minhits)){ $content = getArticleContent(); // Your own function } if(is_int($c)){ if($c>=$minhits){ $mem->set($key, $content, $cachettl); } else { $mem->set($key, ($c+1), $minhitstime); } } elseif($c) { $content = $c; } else { $mem->set($key, 1, $minhitstime); } echo $content; ?>

Also do not try to cache int-only values;) If so, you need to edit the code.

0
source

More articles:


All Articles