Ansible and Terraform Integration

October 12, 2020
Tags: , , , ,

Ansible & Terraform – together or against?

I’ve worked in open source infrastructure consulting for more than 15 years and have used a fair number of the usual suspects from the automation space during this time. Home-brewn scripts, CFEngine, Puppet, to just name a few. When Red Hat acquired Ansible I was really excited, dove right in and have never looked back again. And I’m pretty sure there are many of you out there who have had the same experience. I mean Ansible is simple AND powerful… 🙂

Now working as a Red Hat Solution Architect on IT architectures with our partners and customers, especially in cloud-related discussions one tool comes up more and more often: Terraform from HashiCorp. It’s often mentioned by someone who is not working in infrastructure (or IT strategy for that matter) but in development. And the introduction is “Terraform makes it so easy to deploy new infrastructures”.

So this made me curious, being “simple to start and to use” is one of the main advantages of Ansible. I decided to get a feel for Terraform, it was time to look a bit deeper and to put in some hands-on quality time.

What is and does Terraform?

Terraform is an Open Source tool developed by HashiCorp. It is focussed on creating and changing infrastructures that are defined as code, meaning in easy to understand, declarative configuration files. Terraform uses a plugin-based (called “providers”) architecture to manage infrastructure resources. You will find a multitude of them to manage most well-known providers.

I started out with a simple AWS deployment of some RHEL instances. Some reading and understanding later I had a simple aws_instance.tf file that I could execute with Terraform and that would give me, well, some AWS instances.

So far, what’s nice is that Terraform shares the same objective as Ansible, to be simple, however, there are a number of things where Terraform works differently from Ansible. But I won’t go into syntax or abstraction layer comparison here and not even too deeply into the “procedural vs declarative” discussion, there is a good number of resources out there covering this.

But I want to address where Ansible and Hashicorp have taken different design decisions. Terraform is focused on deploying infrastructures from custom templates or images that you don’t have to touch anymore after deployment. This fits well with the “immutable” or “Phoenix Server” approach in a perfect cloud-world. And to make this mode of operation easy to consume Terraform allows you to “just declare” what you want. For this HashiCorp decided to incorporate state into their tool.

There are pros and cons to this approach but as Ansible and Terraform took different approaches, it leads to differences in behaviour and effect. Because Terraform records the state of the resources and can act on this some notable effects are:

  • After initial deployment you can change the deployment (e.g. add more instances) by just changing the configuration file. Terraform knows what it did and could just add e.g. two more nodes to get from count 5 to 7.
  • Want to update your pre-baked image from version 1 to 2? You just have to change a variable and Terraform will rebuild based on the existing configuration.
  • You can issue a simple “terraform destroy” and again, because Terraform kept track, it would without a lot of fuss remove the deployment.

And Ansible? Could do this, too?

Yes and no. Ansible was not invented to be stateful. And this is good and one reason why Ansible is so powerful. In terms of deployment you should be able to write Playbooks to match what Terraform can do. Even writing Playbooks to change an existing deployment or to delete it is of course possible. But… some of this is easier with Terraforms. It’s just there.

In my opinion it wouldn’t make sense to do a feature shoot-out and not even a “vs” type comparison. Both tools have a pretty well defined area where they excel. It might overlap, but

  • Terraform is good at easily creating and managing IT infrastructures as greenfield deployments. You basically declare “I want this” and Terraform will follow up. Generally this means it works on a rather high level of abstraction and well for the above mentioned immutable infrastructures. But the fact is we live in fast moving, most often very heterogeneous IT infrastructures. Pure greenfield, cloud-native deployments are a rare animal in the real world…
  • Ansible is the proverbial “swiss army knife” of automation and works in greenfield as well as in pre-existing environments. You might have to be a bit more verbose to describe the desired outcome, but then you can have Ansible do more or less whatever you want.

But why not combine the two and have Terraform deploy the infrastructure and then on top have Ansible deploy applications, configurations, orchestrate updates and so on? And never having to write deprovisioning Playbooks again…

Integrating Ansible & Terraform

There are a number of ways to integrate the two:

  • You could use both separately. E.g. use Terraform to deploy your cloud instances, then create an Ansible inventory from the Terraform state data and run your Ansible Playbooks. Or use Ansible’s dynamic inventory scripts to create an inventory after Terraform finished it’s work.
  • Another option would be to call Ansible from Terraform, this is well documented as well.

To me these options still mean to use two tools in a disconnected, non-cooperative mode. Which might be fine with you. But there is another great option with lots of potential: With the release of Ansible 2.5 a new module was introduced, you guessed it: “terraform”. What you can basically do is treat Terraform like any other Ansible module to deploy resources. But you still get the benefit of Terraform like state, dynamic changes to the deployment and even “terraform destroy”.

Interested? What about some hands-on?

Let’s look at a simple use case, I’ve tested this on a CentOS 8 host, YMMV. Assuming Terraform and Ansible 2.9 are already installed, let’s first configure Terraform to deploy two instances on AWS:

As your Ansible user:

  • Create a directory terraform_aws for the Terraform configuration files
  • In the directory create this Terraform configuration file as aws_example.tf:
provider "aws" {
  access_key = "<your AWS access key>"
  secret_key = "<your AWS secret>"
  region     = "us-east-1"
}


resource "aws_instance" "example" {
  ami           = "ami-6871a115"
  instance_type = "t2.micro"
  key_name      = "<your SSH key>"
  count         = "2"
}


output "address" {
  value = "${aws_instance.example.*.public_dns}"
}Code language: JavaScript (javascript)

If you’re familiar with Terraform go ahead, otherwise read the documentation on their website. Then run the example with “terraform init” and then “terraform apply” to make sure it works, change “count”, run “terraform apply” again, remove it with “terraform destroy”. You might check in the AWS console on your way.

Now that we have the Terraform part working, let’s integrate it with Ansible. Create a directory to keep your Ansible content, then inside of it create the first part of the Playbook as ansible_terraform.yml:

---
- hosts: localhost
  name: Create AWS infrastructure with Terraforms
  vars:
    terraform_dir: /home/ansible/terraform_aws
 
  tasks:
    - name: Create AWS instances with Terraform
      terraform:
        project_path: "{{ terraform_dir }}"
        state: present
      register: outputs
 
    - name: Add all instance public DNS to host group
      add_host: 
        name: "{{ item }}" 
        groups: ec2instances
      loop: "{{ outputs.outputs.address.value }}"
 
- hosts: ec2instances
  name: Do something with instances
  user: ec2-user
  become: yes
  gather_facts: false
 
  Tasks:
    - name: Wait for instances to become reachable over SSH
      wait_for_connection:
        delay: 60
        timeout: 600


    - name: Ping
      ping:Code language: JavaScript (javascript)

So what is this Playbook doing?

  • Defines a variable with the location of the Terraform files
  • Uses the terraform module to execute the Terraform configuration and register the output to a variable
  • Uses the registered output to create an inventory group with the IP addresses of the AWS instances using the add_hosts module

At this stage we have the instances created by Terraform and added to an inventory it’s easy to run whatever tasks you like. For a quick test a second play was added:

  • Waits until the instances are reachable via SSH
  • For starters just Ansible ping the instances

Finally and before running this Playbook we have to set some connection variables. Create a group_vars directory and add a ec2instances file:

---
ansible_ssh_private_key_file: "/home/ansible/.ssh/ec2key.id_rsa"
ansible_ssh_common_args: '-o StrictHostKeyChecking=no'Code language: JavaScript (javascript)

All set? Now run the Playbook

$ ansible-playbook ansible_terraform.yml

That’s it!

You are running Ansible that is using Terraform to deploy the AWS instances! Again change the Terraform configuration if you are so inclined. And at the end just change into the Terraform directory and run my favorite Terraform command: terraform destroy.

Of course this is a simple example, there is a whole lot more you can do with the terraform Ansible module. Check out the docs and happy automating!