Why do people use Puppet / Chef with Amazon Cloud Formation instead of just using CloudInit?

Question

We plan to use AMI EC2 instances that are not “pre-baked”. That is, when they are deployed, they are bare AWS linux installations. Our boot process will involve the various installations that we need, for example. python, tomcat. We will have a minimum of 3 copies and a maximum of 8.

Given these requirements, is it helpful to use Puppet / Chef rather than using Amazon Cloud Formation (CloudInit)?

Best of all, I see that if we used Puppet, then we would have declarative programming, which would be easier to check to see what happens against the script. CloudInit also has a 16k script size limit that we may or may not work with.

Has anyone migrated from CloudInit to Puppet or Chef for the reason that they can provide here in response to my question?

Answer 1

Are there any advantages over CloudInit? Yes, absolutely, many of them!

Of course, you can write from top to bottom as soon as CloudInit scripts to provide the server. But what happens when you need to change a configuration file, add a user, upgrade a package, or install a new package? As a result, you log on to the servers or write scripts to do this, and the inevitably inappropriate state of the servers.

CloudInit is not configuration management. If you decide to start using configuration management software, use cloud-based initialization for only one task: to load the Puppet / Chef / other handler.

Puppet does not just help you automate the installation of packages, install ssh keys, or configure a bunch of Tomcat. This provides a state of things. When a developer fixes a Java application at 3 a.m. and changes the configuration of Tomcat, Puppet will change it. You can quickly change the version of Python for all or groups of nodes, and if someone installs a different version, Puppet will change it.

When your application stack changes and you start using, say, RabbitMQ or Jetty, or the new RDBMS, you can easily test and deploy the changes to tens or thousands of servers.

There are many other reasons to use configuration management software, such as reporting, auditing, and security compliance.

Answer 2

The whole point of configuration management is pushing machines predictably and consistently. CloudFormation and cloudinit are great when you are limited exclusively to AWS (although debugging CloudFormation templates is a miserable experience ), but for applications that use both data center and AWS resources, or local testing environments, or development machines?

If you exist exclusively in AWS, I suppose you could leave with cloudinit and nothing else, but I'm not sure if it is realistic for applications of any scale (Netflix, for example, pre-bakes their AMI using OSS technologies, they’re written and released to the world, consider this video for details). Highly available trans-regional applications, often based on VPCs, tend to back up to data centers via VPNs, and this does not even apply to demo, intermediate, test, or development environments. As someone who commissioned the creation of machines, the last thing I want to do is redo the job or get stuck debugging a few preparation methods.

Consequently, Chef or Puppet. They work just as well for AWS as they do for my data center, as well as for my Vagrant development machine, as well as for the demo environments I sometimes need on the fly. I'd rather run Chef or Puppet from the cloud than support both cloud and Chef or Puppet.

Answer 3

If you drop servers, say, run after an autoscale group, I would say cloudinit is probably enough. Linux shell scripts or Windows PowerShell scripts should do the trick.

If its a long server on which you plan to operate, maybe a chef, puppet or docker can give you an edge, as indicated in the accepted answer. If you don’t see the benefits after using them, you probably don’t need a tool.

Answer 4

In my experience, there are simple things that are easy to do with the out-of-the-box graphical tools that AWS provides, but as you move on to more complex things, you begin to find out that there are limits to what you can only do with their tools.

At this point, you can either stop or find other tools (such as Chef or Puppet) that will help you achieve these more complex goals, as well as do simpler things.

Your choice.