I'm looking at deploying Kubernetes on top of a CoreOS cluster, but I think I ran into a deal break.
If I use only CoreOS and the fleet, I can indicate in the module files that I want certain services to not start on the same physical computer as other services (anti-affinity). This is very important for high availability. But it doesn't look like the kubernets have this functionality.
In my specific use case, I will need to run several clusters of elasticsearch machines that should always be available. If for some reason kubernetes decides to plan all my elasticsearch node containers for this ES cluster on the same machine (or even most on the same machine) and this machine dies, then my elasticsearch cluster will die with this. This cannot be allowed.
It looks like there might be workarounds. I could configure the resource requirements and machine specifications so that only one instance of elasticsearch could fit on each computer. Or I could probably use labels in some way to indicate that certain elasticsearch containers should go on certain machines. I could just provide more machines than necessary, and make more ES nodes than necessary, and assume that the tunnels would distribute them enough to be confident enough in high availability.
But all this seems uncomfortable. It is much more elegant in terms of resource management to simply specify the necessary equipment and anti-affinity and allow the scheduler to optimize from there.
, - - ? - , ?
-? ?