Geek Answers Handbook

Riak data migration when resizing a ring

Is this trivial? I will use Bitcask and file backups (from files on each node).

Say my initial ring size is 256 with 16 nodes. Now, if I need to expand to 1024, can I configure 16 new instances with a call size of 1024, copy the backup files for the old cluster to these 16 new instances, and start Riak? Can Riak take this old data?

I suppose not, since the section identifiers and their mapping to individual nodes may also change after the ring is resized. But what other way? Will riak-backup work in this case (when resizing the ring)?


I just want to know that the choice I made is the future enough . Obviously, at some point when requirements change dramatically or database balloons arise, you may need to change the whole architecture. But I hope that I can make such changes (to the size of the ring) at some point - naturally, with SOME effort, but - without this impossible.

+4
source share
2 answers

Transferring clusters to a different ring size is difficult with backing up files based on a node (which means that if you simply back up the data directories on each node, as recommended in Riak Backup ). Because, as you suspected, the backend data files depend on matching nodes and partitions with a given ring size.

What should you do instead?

You must use “logical” backups of the entire cluster using one of these two tools:

Using one of them basically brings the contents of the entire cluster into one place (so be careful not to leave the disk space, obviously). Then you can transfer and restore a new cluster with a different ring size.

Beware:

  • Back up inanimate clusters. Meaning: either reset the cluster, or at least make sure that new records do not occur with the old cluster during backup. Otherwise, if you start the backup, but the new records are still arriving, there is no guarantee that they will fall into the backup data set.

  • Before performing backup / restore, be sure to transfer the app.config and user bucket options to the new cluster.

Hope this helps. Thus, this is not trivial (this means that it will take some time and require a lot of disk space, but it is true when you transfer large amounts of data), but it is also not too difficult.

+4
source

I know this is an old question, but with Riak 2.x you can now dynamically resize the ring without closing the cluster:

riak-admin cluster resize-ring <new_size> riak-admin cluster plan riak-admin cluster commit

Note. The size of the Riak ring should always be an integer 2n, for example. 16, 32, 64, etc.

http://docs.basho.com/riak/latest/ops/advanced/ring-resizing/

+1
source

More articles:


All Articles