Compress Python Objects Before Caching

Question

What is a quick way to compress Python objects (list, dictionary, string, etc.) before storing them in the cache and unpacking them after reading from the cache?

I am using Django, and I hope to add support for compression / decompression directly to the backend of the Django cache, which makes it accessible to all my Django applications.

I looked at django / core / cache / backends / memcached.py

import cmemcache as memcache

class CacheClass(BaseCache):

    def __init__(self, server, params):
        BaseCache.__init__(self, params)
        self._cache = memcache.Client(server.split(';'))

    def get(self, key, default=None):
        val = self._cache.get(smart_str(key))
        if val is None:
            return default
        return val

    def set(self, key, value, timeout=0):
        self._cache.set(smart_str(key), value, self._get_memcache_timeout(timeout))

It seems that the brine / unickle is executed by the cmemcache library. I don’t know where to put the compression / decompression code.

Answer 1

I looked further into the python-memcache source code.

zlib memcached.

lv = len(val)
# We should try to compress if min_compress_len > 0 and we could
# import zlib and this string is longer than our min threshold.
if min_compress_len and _supports_compress and lv > min_compress_len:
    comp_val = compress(val)
    # Only retain the result if the compression result is smaller
    # than the original.
    if len(comp_val) < lv:
        flags |= Client._FLAG_COMPRESSED
        val = comp_val

def _set(self, cmd, key, val, time, min_compress_len = 0):

Django "set" memcache:

def set(self, key, value, timeout=0):
    self._cache.set(smart_str(key), value, self._get_memcache_timeout(timeout))

-, "min_compress_len".

Answer 2

-, , ? , ? / , - , .

, , , zlib.

zlib, , compress, :

zlib.compress(string[, level])
, , . level - 1 9, ; 1 , 9 . - 6. error.