Does LevelDB support hot backups (or their equivalent)?

Question

We are currently evaluating several data warehouses with key + values ​​to replace the old ion currently used by the owr main application (over 20 years!) ...

The problem is that our current isam does not support crash recovery.

So LevelDB seemed good to us (also checking out BerkleyDB, etc.)

But we were faced with the issue of hot backups, and given the fact that LevelDB is a library and not a server, it’s weird to request a “hot backup” because it intuitively implies an external backup process.

Perhaps someone would like to suggest options (or well-known solutions)?

For example: - Hot backup through the internal flow of the main application? - Hot backup by simply copying a LevelDB data directory?

Thanks in advance

Answer 1

You can iterate the snapshot through LevelDB, which is probably the best way to make a hot copy (remember to close the iterator).

To back up LevelDB via the file system, I previously used a script that creates hard links to all .sst files (which are immutable after recording) and regular copies of the log files (and MANIFEST, CURRENT files, etc.) to the backup directory in the same section. This is fast because the log files are small compared to .sst files.

The database must be closed (by the application) during the backup, but the time will obviously be much less than the time taken to copy the entire database to another partition or load on S3, etc. This may be after the DB application opens again.

Answer 2

LMDB is a repository of built-in keys, but unlike LevelDB, it supports the multiprocessor concurrency process, so you can use an external backup process. The mdb_copy utility will make an atomic hot backup of the database, your application does not need to stop or do anything special during the backup. http://symas.com/mdb/

Answer 3

I'm a little late to this question, but there are LevelDB forks that offer good backup features like HyperLevelDB and RocksDB. Both are available as npm modules, i.e. level-hyper and level-rocksdb. For a more detailed discussion, see How to back up RocksDB? and HyperDex Question .