Dockers and service clusters

Question

We are exploring the use of Docker plus Mesos / Marathon or Kubernetes to host a cluster. However, one problem we really did not see answers to is how to allow clustered services to connect to each other correctly. Everything I saw should know at least one more node before they can join the cluster. Some should know about each node. However, there is no way in Kubernetes and Mesos to find out that these IP addresses are ahead of time.

So, are there any recommendations for this? If that helps, some of the technologies we consider when deploying as containers are ElasticSearch, ActiveMQ, and MongoDB. There may be others.

Answer 1

However, one of the problems that we really did not see answers to is how to allow the clustered services to connect to each other correctly.

I think you are talking about HA / replicated / fined applications here.

At the moment, in kubernetes, you can accomplish this by making an api call listing all the "endpoints" of the service; that will tell you where your peers work.

In the end, we would like to support a use case that you describe in a more first-class way.

I filed https://github.com/GoogleCloudPlatform/kubernetes/issues/3419 to maybe start something more standardized here.

Answer 2

I also wanted to set up an ElasticSearch cluster using Mesos / Marathon. Since the existing “solutions” were either simply undocumented or did not work / are outdated, I created my own container.

If you like, check out https://github.com/tobilg/docker-elasticsearch-marathon

If you have a running marathon installation installed (I use v0.8.1), setting up the ElasticSearch cluster should take several minutes.

UPDATE

The container now uses Elasticsearch v1.5.2 and it can run on the last v0.8.2 marathon.

Answer 3

For Kubernetes, it currently requires kube-controllers-manager to start with the --machines argument, given a list of mining IP addresses or host names.

Answer 4

I see no easy way to handle this correctly in Kubernetes. Yes, you can call the API, which returns a list of endpoints, but you must keep track of changes and take action when changing endpoints ...

I would rather use Mesos / Marathon, which is well prepared for this scenario. You must implement a custom Framework for Mesos. Already created Framework for ElasticSearch: http://mesos.apache.org/documentation/latest/mesos-frameworks/