Terraform: How to run remote-exec more than once?

Question

I noticed that terraform will only run "file", "remote-exec" or "local-exec" only once. After the resource is provided, if the commands in "remote-exec" are changed or the file from the "creator" file is changed, terraform will not make any changes to the instance. So, how do I get terraform to run "file", "remote-exec" or "local-exec" every time I use terraform?

More details:

Often I had a resource, provided in part due to an error from "remote-exec", causing terraform to stop (mainly due to the fact that I enter the wrong commands when I write my script). Running terraform again after this will cause the previously created resource to be destroyed and force terraform to create a new resource from scratch. This is also the only way I can run "remote-exec" twice on a resource ... by creating it from scratch.

This is really a flaw in terraform, not a question that can do the same job as terraform, except that it is completely idempotent. When using Ansible with tasks such as "ec2", "shell" and "copy", I can perform the same tasks as terraform, only each of these tasks will be idempotent. Ansible automatically recognizes when it does not need to make any changes, when it does, and because of this, it can pick up the place where the emergency player failed, without destroying everything and from scratch. Terraform does not have this feature.

For reference, here is a simple terraform resource block for an ec2 instance that uses both remote-exec and file:

resource "aws_instance" "test" { count = ${var.amt} ami = "ami-2d39803a" instance_type = "t2.micro" key_name = "ansible_aws" tags { name = "test${count.index}" } #creates ssh connection to consul servers connection { user = "ubuntu" private_key="${file("/home/ubuntu/.ssh/id_rsa")}" agent = true timeout = "3m" } provisioner "remote-exec" { inline = [<<EOF sudo apt-get update sudo apt-get install curl unzip echo hi EOF ] } #copying a file over provisioner "file" { source = "scripts/test.txt" destination = "/path/to/file/test.txt" } } 

Answer 1

Terraform docs on provisioning explicitly indicates that it considers the use of creators for basic bootstrapping as a task only once and that it should not be used as a replacement for a proper configuration management tool such as Ansible:

Provisioners run only when the resource is created. They are not a replacement for configuration management and software changes already running the server, and instead, it simply means load the server. For configuration management, you must use Terraform Provision to invoke real configuration management solution.

and

If the resource is successfully created but not executed at the time of submission, Terraform will be an error and mark the resource as "corrupted". The resource that was corrupted was physically created, but cannot be considered safe to use, because the backup failed.

When you create your next execution plan, Terraform will delete all corrupted resources and create new resources, trying to secure again. It does not try to restart initialization on the same resource, because it is not guaranteed to be safe.

Terraform does not automatically roll back and destroy the resource during application when a failure occurs because it will happen against the execution plan: the execution plan would say the resource would be created, but did not say that it would ever be deleted. But if you create an execution plan with a corrupted resource, the plan will clearly state that the resource will be destroyed because it is corrupted.

Providing data is important for the ability to load instances. In the form of another reminder, this is not a replacement for configuration management. It is designed for just boot machines. If you use configuration management, you should use software as a way to download the configuration management utility.

Let's consider the creators as similar to the user data of the EC2 script, since it only works once at creation, and if it fails, you need to destroy the instance and try again.

The advantage of this is that Terraform does not need any knowledge on how to make idempotent changes in the operating system, since Terraform operates at a level higher than the instance itself and more when providing the entire data center.

If you need more flexibility than this, consider either using Terraform to invoke the configuration management system to provide the instance correctly (and then allowing it to retry if it fails, disconnect from the Terraform preparation stage), or use an orchestration tool like Jenkins to wrap both Terraform and an alternative configuration management tool such as Ansible.

Another option is to go further along the route of immutable infrastructure and use Packer to create AMI using Ansible or another tool and then just use Terraform to deploy AMI, as this does not require further provision of the instance.

Answer 2

I went through this thread in my searches and eventually found a solution:

 resource "null_resource" "ansible" { triggers { key = "${uuid()}" } provisioner "local-exec" { command = "ansible-playbook -i /usr/local/bin/terraform-inventory -u ubuntu playbook.yml --private-key=/home/user/.ssh/aws_user.pem -u ubuntu" } } 

You can use uuid (), which is unique for each terraform run, to run a zero resource or security tool.

Answer 3

You can use the taint command to mark a resource as corrupted, causing it to be destroyed and recreated the next time it is used.