Alternatives to ZooKeeper? (cluster coordination service)

Question

ZooKeeper is a highly accessible data center coordination service. It originated in the Hadoop project. You can implement blocking, fail, election of leaders, group membership and other coordination problems. Are there any alternatives to ZooKeeper? (of course, free software)

Answer 1

I looked broadly at Zookeeper / Curator , Eureka , etcd , and the consul. Zookeeper / curator and Eureka are in many ways the most polished and easiest to integrate if you are in the Java world. etcd is pretty cool and very flexible, but it's really just an HA key store, so you will need to write a lot of code to turn it into a stubborn service for the discovery system.

Consul (for me) is the best of both worlds. This is a service self-assertion system written on top of serf , using a raft for cluster consensus and gossip for communication. It provides discovery / registration endpoints with a well-documented REST api, and also allows you to discover services with DNS SRV records and register services with configuration (i.e. you can register a database or application with which you cannot integrate the client, or if you just want your service discovery to be disconnected from your application)

I wrote a blog post about the consul where you can find out more and go through my "try" demo

I also discussed service discovery with etcd and docker if you want to know more about what this user code looks like.

The last thing! etcd and consul are written in go, so maintaining them is much easier than Java solutions like zookeeper. All you need is the consul / etcd binary. no dependencies, no related libraries, no jvm.

Answer 2

There's a very promising alternative to ZooKeeper called etcd (github.com/coreos/etcd) , written by the CoreOS team. Unlike Doozerd, etc. Actively developing.

Answer 3

Just discovered Accord (C) and OpenReplica / ConCoord (Python), which might be interesting solutions

[EDIT] The Hashicorp team, famed for Vagrant and Packer, is preparing a “decentralized service discovery and orchestration solution” called Serf .

[EDIT2] Hashikorp strikes again! They just released Consul , built on top of Surf. Step: "A service discovery and configuration solution, fully distributed, highly available, scalable to thousands of nodes and services in multiple data centers."

Answer 4

Yes, there is also Doozerd (https://github.com/ha/doozerd) . Take a good look at this; this is a one-time distributed coordination service developed by Heroku. With bindings / libraries for java / python / ruby ​​/ node. It is very easy to start and play.

Answer 5

Take a look at Serf . There is a comparison with Zookeeper here .

Answer 6

OpenReplica from my research team is the highly available FOSS data center coordination service. It can be used to implement blocking, malfunctioning, election of leaders, group memberships and other coordination services. It differs from ZooKeeper in two critical ways:

• It uses an object oriented API. This greatly simplifies the work with coordination services. The synchronization code for OpenReplica looks exactly like its copy of the tutorial; there is no need to manage the file and the upcall-based API, as in ZooKeeper and Chubby.

• It allows dynamic replica membership membership updates. Static configuration files are not required. The system is integrated into DNS (authoritative, subordinate to OpenReplica or Amazon Route 53).

We actively support the system, do not hesitate to inform us if you have additional questions.

Answer 7

There's a project called Noah on github that looks interesting, it says it's "free based on Apache ZooKeeper" https://github.com/lusis/Noah with REST support (ZK has this as a contribution / option, not built-in).

Answer 8

There are various tools that are optimized for various technical compromises.

• ZooKeeper Scales for reading; writes with many observers can be slow. It is proven and has a significant community.
• Accord It seems interesting for use with intensive recording, but in typical use cases there are already solutions specific to the domain (i.e., logging, telemetry).

The rest are somewhat interesting, but generally unproven. Do not do it wrong if they are intended for use in production.

Answer 9

I found this comparison of Zookeeper, etcd and Doozer: http://devo.ps/blog/zookeeper-vs-doozer-vs-etcd/

Serf (serfdom.io) is also a good solution, as it is easy! But you should keep in mind that SERF is just a cluster manager that allows you to send custom events to all nodes in the cluster. This is good, but you should write your own shell scripts (also called events). See this example: " https://www.digitalocean.com/community/articles/how-to-set-up-a-serf-cluster-on-several-ubuntu-vps "

The advantage is that you get a very simple cluster manager, and you can combine it with your favorite configuration, deployment, or continuous integration tool.

Answer 10

Corosync also seems to be similar to ZooKeeper.

Answer 11

I know this post is pretty old, but anyone looking at all possible alternatives would also like to suggest the JGroups library, which is mature enough to be used in a production environment. I successfully used it in one of my projects mainly for distributed coordination and messaging between clusters. It also supports AWS support in addition to its flexible architecture, where you can configure your stack to get what you need. I suggest you look at him