Does couchbase really support datasets larger than memory?

Question

The Couchbase documentation says that “Persistence of the drive allows you to perform backup and restore operations and allows you to grow your datasets beyond the built-in cache level,” but I can't get it working.

I am testing Couchbase 2.5.1 on a three node cluster, with a total of 56.4 GB memory configured for the bucket. After ~ 124,000,000 100-byte objects — about 12 GB of raw data — it stops accepting extra buttons . 1 configured.

Is there a “go ahead and splash out to disk” magic that I miss? There are no suspicious entries in the error log.

Answer 1

It supports data with more memory - see Emission and Work Set Management in the manual.

In your case, what errors do you get from your application? When you begin to reach the low memory watermark, items must be ejected from memory to make room for new items.

Depending on the disk speed / TEMP_OOM element speed, this can lead to TEMP_OOM errors being sent back to the client, indicating that dialing should be temporarily back off, but in most cases they should be rare. Details on how to handle them can be found in the Manual the developer .

Answer 2

I assume that this is not raw data that fills your memory, but the metadata associated with it. Couchbase 2.5 requires 56 bytes per key, so in your case it will be approximately 7 GB of metadata, which is much less than your memory quota.

But ... metadata can be fragmented in memory. If you turned on all 124M objects in batch in a short time, I would suggest that you get at least 90% fragmentation. This means that using only 7 GB of useful metadata, the space needed to store it filled your RAM with lots of unused parts in each allocated block.

The solution to your problem is to defragment the data. It can either be done manually, or called if necessary:

• manually:
• automatically:

If you need to understand more about why compaction is needed, you can read this blog article by Couchbase .

Answer 3

Even if none of your documents is stored in RAM, CouchBase stores all document identifiers and metadata in memory (this will change in version 3), and it will also require some available memory to work efficiently. Relevant section in the docs: http://docs.couchbase.com/couchbase-manual-2.5/cb-admin/#memory-quota

Note that when using a replica, you need twice as much RAM. The formula is approximately equal to:

(56 + avg_size_of_your_doc_ID) * nb_docs * 2 (replica) * (1 + height) / (high_water_mark)

Therefore, depending on your configuration, it is possible that 124,000,000 documents require 56 GB of memory.