AWS Automation: CloudFormation, Ansible, and Beyond

Daryl Robbins Daryl Robbins on AWS and Automation 7 minutes

In my first post, we explore how to automate the provisioning of an AWS environment by using Ansible in conjunction with CloudFormation. Using Ansible on top of CloudFormation gives us a much more powerful automation platform.

It acts as the glue to bind together our various CloudFormation templates. Hopefully, it will make you see CloudFormation in a new light.

I have use the techniques described in this post to develop sophisticated automation scripts for multiple different projects – involving multiple environments and regions. It is very powerful indeed to be able to stand up a complete environment by issuing a single command.

CloudFormation

For the purposes of this post, I am going to assume that you already have a basic understanding of CloudFormation. If that's not the case, please refer to CloudFormation Template Basics for an introduction.

To recap, in a CloudFormation template, I describe the resources, their desired state, and their relationships; CloudFormation then goes to work to create/update those resources to match my expectations. It seamlessly handles updates and can rollback to the last known good configuration on failure. It also maintains information about my stack, so I can refer to it later (i.e. the outputs).

Why use CloudFormation?

One could achieve the exact same results by using the AWS API or CLI; however, CloudFormation offers a powerful abstraction over the API. Where the API takes on an imperative style (do this, do that), CloudFormation offers a declarative interface (expect this state). This is analogous to the difference between a shell script and a configuration management script (like Ansible).

With Ansible 2, the built-in AWS functionality has been expanded significantly. So, one could create a lot of the same AWS resources directly from Ansible. However, I still prefer using CloudFormation with Ansible for most of my AWS automation for a number of reasons:

  • CloudFormation is faster to support new AWS functionality (as it is developed by AWS) and offer a more complete set of AWS feature. There are still a number of resources I cannot yet create with Ansible.
  • CloudFormation stacks are atomic. If something fails in the process, it will rollback the entire set of changes.
  • CloudFormation has recently introduced a new feature called change sets, which allows me to preview my changes before actually making them – helping to avoid disastrous consequences. I do not currently leverage this feature in this example, but it would definitely be feasible to extend these examples to do so.
  • CloudFormation ties in well with the rest of the AWS ecosystem to provide a very seamless automation experience. For example, I can setup SNS topics to notify me on changes or develop custom resources using AWS Lambda.
  • CloudFormation stacks show up directly in the console and can be queried via the api, offering me better visibility into the components that have been provisioned in my environment and also allowing me to refer to the resources they created in future stacks.

If I were trying to develop generic scripts that worked across multiple cloud providers, then I would likely consider using Ansible directly or potentially Terraform.

Where does CloudFormation fall short?

You may now be asking: Why can't I just use CloudFormation to automate my entire stack? Why do I need Ansible in the mix as well?

Well, CloudFormation is a very powerful tool but you start to run into some limitations, as the complexity of your automation scripts increases:

  1. Difficult to Edit: Working in JSON can be quite painful
  2. Inter-Template Dependencies: It's difficult to develop stacks that build upon other stacks, which is essential for managing the complexity of an ever-growing automated infrastructure (i.e. I can't automatically discover and inject values from dependent templates)
  3. Dynamic Templates: There is limited support for dynamic templates that can adapt to their environment and configuration. CloudFormation offers only minimal support for conditionals and no support for iteration.
  4. Missing Features: The AWS platform is growing so quickly that often times even CloudFormation has difficulty keeping up with all the new features.

A particularly common issue for me is around adjusting a template to take advantage of the availability zones offered in a particular region. In the AWS cloud, we have regions exposing 2, 3, or 4 availability zones. How can I create the right number of subnets for a given region? This is not something that is practical using CloudFormation alone. Read on to see how Ansible can help with this problem and many more that you will encounter as you try to build sophisticated infrastructure automation solutions.

Aside: You could also solve the inter-template dependency problem with custom resources, but I find using Ansible to be a more straightforward way to manage a set of interdependent templates.

Ansible to the rescue

Fortunately, we can overcome all of these limitations by leveraging Ansible. At the top layer of our automation stack, Ansible will take on two key roles:

  • Generate CloudFormation templates using a template engine (Jinja2)
  • Orchestrate the execution of those CloudFormation templates

Ansible provides native CloudFormation support. It also has standard modules for creating AWS resources directly from Ansible and allows me to easily create my own custom modules.

I generally use CloudFormation for most of my AWS automation, but there are times where it comes in handy to be able to mix CloudFormation with Ansible modules for creating some of these resources:

  • I tend to use Ansible modules for creating S3 buckets, because I do no like the fact that the bucket cannot already exist when the CloudFormation template first runs.
  • I also frequently create custom Ansible modules to manage resources or properties, which have not yet been added into CloudFormation.

Example

Without further ado, let's dive straight into our example. The full sample code for this post can be found on GitHub.

Context Diagram

Our example script is broken into two roles VPC and Network, which will create a base network in AWS, including the following components:

  • VPC Role
    • virtual private cloud (VPC)
    • internet gateway
  • Network Role
    • public and private routing tables
    • public subnets (one per AZ)
    • private subnets (one per AZ)

The project contains a playbook (setup.yml), some configuration (group_vars/all.yml) along with two roles vpc and network.

To run the automation script, all we need to do is run: ansible-playbook setup.yml

You will need to make sure you have properly setup for the AWS CLI prior to doing so. More information can be found in the AWS CLI documentation.

Here is the result of running the automation script:

Resulting VPC

Notice that there is a public and private subnet in each of the availability zones within the given region.

Playbook (setup.yml)

In Ansible speak, a playbook is the top-level automation script, defining all the steps to be undertaken. The playbook is organized into a single play (top level list) that refers to the roles to be run.

---
- name: Setup VPC and network
  hosts: localhost
  roles:
    - vpc
    - network

Variables (group_vars/all.yml)

The variables file defines the configuration to be used for setting up our environment. There are several global variables a the top that apply to both roles, followed by role specific variables.

region: us-east-1
stack_prefix: codeblog
vpc:
  cidr: 10.0.0.0/16
  slash: 24
segments:
  Public:
    azs: 3
    slash: 26   # size of AZ subnets in each segment
  Private:
    azs: 3
    slash: 26   # size of AZ subnets in each segment

VPC Role (roles/vpc)

The VPC role is responsible for creating the VPC and Internet Gateway.

Task (main.yml)

The main Ansible task renders our CloudFormation template and then uses the CloudFormation modules to stand it up in AWS. Finally, we register the output of running this module into a variable called vpc_stack for later use. It includes tons of useful information (like the outputs from the template).

- name: Render Network VPC Template
  template: src=vpc.yml.j2 dest=rendered_templates/vpc.yml

- name: Create VPC Stack
  cloudformation:
    stack_name: "{{stack_prefix}}-vpc"
    state: "present"
    region: "{{region}}"
    disable_rollback: true
    template: rendered_templates/vpc.yml
    template_format: yaml
    tags:
      Stack: "{{stack_prefix}}-vpc"
  register: vpc_stack
Template (vpc.yml.j2)

The VPC template is a jinja2 template that dynamically generates a CloudFormation template to be applied to Amazon. This is where we actually define the resources to be created.

#jinja2:lstrip_blocks:False,trim_blocks:False
---
AWSTemplateFormatVersion: "2010-09-09"

Description: Base Network Infrastructure

Resources:
  Vpc:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: {{ vpc.cidr }}
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: {{ stack_prefix | lower }}_vpc

  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: {{ stack_prefix | lower }}_ig

  VPCGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: { Ref: Vpc }
      InternetGatewayId: { Ref: InternetGateway }

Outputs:
  Vpc:
    Value: {Ref: Vpc}
  InternetGateway:
    Value: {Ref: InternetGateway}

Network Role (roles/network)

The network role builds upon the vpc role to create the subnets within the VPC. The subnets to be created are configuration-driven – determined from the variables file.

Task (tasks/main.yml)

The main task for the network role is extremely similar to that for the vpc role. Pay particular attention to the template_parameters, where we pass the vpc output from the vpc stack as an input into the network stack.

- name: Render Network VPC Template
  template: src=network.yml.j2 dest=rendered_templates/network.yml

- name: Create Network Stack
  cloudformation:
    stack_name: "{{stack_prefix}}-network"
    state: "present"
    region: "{{region}}"
    disable_rollback: true
    template: rendered_templates/network.yml
    template_format: yaml
    template_parameters:
      Vpc: "{{vpc_stack.stack_outputs.Vpc}}"
    tags:
      Stack: "{{stack_prefix}}-network"
  register: network_stack
Template (templates/network.yml)

This template is quite a bit more interesting than the previous one and starts to really show the advantages of generating our CloudFormation templates using Jinja2.

#jinja2:lstrip_blocks:False,trim_blocks:False
---
AWSTemplateFormatVersion: "2010-09-09"

Description: Base Network Infrastructure

Parameters:
  Vpc:
    Type: String

Resources:

# Route Tables #
  InternalRouteTbl:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: { Ref: Vpc }
      Tags:
        - Key: Name
          Value: {{ stack_prefix | lower }}-internal-rt

  ExternalRouteTbl:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: { Ref: Vpc }
      Tags:
        - Key: Name
          Value: {{ stack_prefix | lower }}-external-rt

# Subnets and Associations #
{% for segment_name,segment in segments.iteritems() %}
  {% set segment_cidr = vpc.cidr | ipsubnet(vpc.slash, loop.index0) %}
  {% for subnet in range(1, segment.azs + 1) %}
  # Subnets #
  {{ segment_name }}Subnet{{ subnet }}:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: {{ lookup('get_azs', region).split(',')[subnet - 1] }}
      CidrBlock: {{ segment_cidr| ipsubnet(segment.slash, subnet - 1) }}
      VpcId: { Ref: Vpc }
      Tags:
        - Key: Name
          Value: {{ stack_prefix | lower }}-{{ segment_name | lower }}

  # Associate Subnet with RouteTable #
  {% if segment_name == 'Public' %}
    {% set route_table_name = 'ExternalRouteTbl' %}
  {% else %}
    {% set route_table_name = 'InternalRouteTbl' %}
  {% endif %}

  {{ segment_name }}RouteTblAss{{ subnet }}:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: { Ref: {{ route_table_name }} }
      SubnetId: { Ref: {{ segment_name }}Subnet{{ subnet }} }

  {% endfor %}
{% endfor %}

Outputs:
{% for segment_name,segment in segments.iteritems() %}
  {{ segment_name }}Subnets:
    Value:
      "Fn::Join":
        - ","
        -
        {% for subnet in range(1, segment.azs + 1) %}
          - {Ref: "{{ segment_name }}Subnet{{ subnet }}"}
        {% endfor %}
{% endfor %}

Conclusion

In this post, we saw how we can leverage Ansible to dynamically generate CloudFormation templates and then orchestrate the provisioning of a complete environment from these templates.

These concepts can easily be adapted to support:

  • Multiple Environments (development, qa, production)
  • Multiple Regions
  • Reusability Across Projects

Thanks for reading. Stay tuned for more exciting posts to come!

Daryl Robbins

Daryl Robbins is an experienced software architect with a broad base of knowledge from cloud computing to algorithms. He is an AWS Certified Professional Solution Architect.

Ottawa, Canada https://darylrobbins.com
comments powered by Disqus