Efficient way to store a 32-bit signed integer in Redis

Question

Since Redis is trying to parse strings up to 64-bit integers, is it useful to store a binary representation of a 32-bit signed integer instead of radix integers?

In our system, we have lists of 32-bit signed integer identifiers.

I can store them like lpush mykey 102450 --> redis cast 102450 to 8 bytes long or store it like lpush mykey \x00\x01\x19\x32 ---> this is just 4 bytes 

Answer 1

Internally, Redis stores strings in the most efficient way. Forcing integers into radix 10 strings will actually use more memory.

This is how Redis stores strings -

• Integers less than 10,000 are stored in a shared memory pool and do not have memory overhead. If you want, you can increase this limit by changing the REDIS_SHARED_INTEGERS constant in redis.h and recompiling Redis.
• Integers greater than 10,000 and within the long consumption range of 8 bytes.
• Regular strings take len (string) + 4 bytes for length + 4 bytes for marking free space + 1 byte for null terminator + 8 bytes for malloc service messages.

In the example you pointed out, the question is about 8 bytes for a long v / s 21 bytes for a string.

EDIT:

So, if I have a set of numbers just under 10,000, how does Redis save my set?

It depends on how many items you have.

If your set contains less than 512 elements (see set-max-intset-entries ), the set will be saved as an IntSet. IntSet is the distinguished name for a sorted integer array. Since your numbers are less than 10000, it will use 16 bits per element. This is (almost) how memory is efficient as an array of C.

If you have more than 512 items, the set becomes a HashTable. Each element in the set is enclosed in a structure called robj , which has overhead of 16 bytes. The robj structure has a pointer to a common pool of integers, so you do not pay anything extra for the whole integer. Finally, robj instances are stored in a hash table, and the hash table has overhead proportional to the size of the set.

If you are interested in how much memory an element consumes, run redis-rdb-tools in your dataset. Or you can read the source code for the MemoryCallback class, comments explain how memory is laid out.

Answer 2

Strings are stored with a length, so it will not be only 4 bytes in the database - it is probably stored as 4 bytes of data + 4 bytes of length + padding, so you get nothing.