Testing AWS CloudFormation Templates with Ansible Boosts Reliability

As part of Rackspace’s Fanatical Support for AWS, we’re publishing a series of posts from some of our top AWS-certified experts to help you understand our offering and get started. Below, Justin Phelps, a DevOps Engineer at Rackspace, explains the value proposition for using Ansible to test AWS CloudFormation templates.

The many variations and combinations of AWS products and services give the platform great flexibility and opportunity for customization.

At Rackspace, we’ve evaluated these combinations and created a collection of best practices our customers can follow. These best practices include CloudFormation templates, and my team maintains a series of standard CloudFormation templates for our customers. Part of that maintenance includes updating those templates regularly and testing them for functionality.

The testing and documentation of these templates plays an important role in helping us maintain support and make sure customers’ deployments run smoothly and efficiently.

For example, changes to one template shouldn’t break another template. We keep our documentation up to date with the latest iteration of our templates and can say with confidence that the templates on our master branch will work every time. We use a combination of Ansible and CircleCI to automate this task.

Our testing process includes three main steps: template generation, auto documentation, and building and testing a stack.

Template generation

Our templates exist in a Github repository tied to CircleCI. Every change made to a template is fully tested and documented in an automated fashion. Our templates start as a jinja2 file that is then templated by Ansible. We are templating our templates! The template module in Ansible parses the jinja2 file and outputs a JSON CloudFormation template. The variables shown below get replaced when the template is generated.

This process allows us to update various parts of a template using variables defined in Ansible. This can be done at the command line to generate a unique template with customized properties. At the same time, we avoid over-complication of our jinja2 templates. Having too many “for” or “if” statements can cause confusion and make a template unmaintainable. Striking the right balance between no jinja2 usage and full jinja2 usage has been important for readability and understanding what is actually created by this process.

Auto documentation

Documenting code is a constant chore and never seems to be 100 percent accurate. To solve this problem, I wrote a custom Ansible module to automatically document the generated CloudFormation template. The module parses the JSON and outputs a markdown file to be uploaded to Github. This means you don’t have to read the JSON file to understand what parameters, resources and outputs the template provides. This includes any defaults and descriptions related to those objects. Click to see this module. Below, usage of this module:

Here is a preview of the generated markdown from this module:

Having this tool in place means our support teams and customers never have to worry about out-of-date documentation. This leads to greater confidence about what the CloudFormation template will be doing on their account. This also means that the various description and constraint description fields become more important in a template as these values are used in generating the markdown file.

Building and testing a stack

Once we generate and document the template, we launch it on a test AWS account. This ensures the CloudFormation API can validate and successfully build the template. We also use Ansible to test the results of the CloudFormation template. This can including testing a URL that should be responding with certain content using the uri module.

At Rackspace we have a standard template that creates a VPC and the associated subnets across a number of availability zones. This template is the foundation for others that are created, such as an Elastic Beanstalk template. We use Ansible to map outputs from our standard networking template into parameters for our other templates for testing. This allows us to use these templates as building blocks to design and build customer architectures. The constant testing of these templates means we will always know the template blocks fit together nicely.

With the above framework in place, we can ensure our CloudFormation templates are functional, reliable and documented. It has been relatively easy to bring contributors up to speed on our testing framework due to the lower learning curve of Ansible. Eventually we hope to use this framework to deploy our templates to a common S3 bucket or another location for consumption outside of Github.

Visit http://www.rackspace.com/aws for more information about Fanatical Support for AWS and how it can help your business. And download our free white paper Best Practices for Fanatical Support for AWS, which offers further detail regarding implementation options for AWS, including identity management, auditing and billing.

Justin Phelps was a DevOps engineer at Rackspace, where he was always searching for the latest tools and methodologies to solve customer pain points and improve processes. His mission was to make inefficient processes better through automation, testing and education. You can check out his blog for his latest talks and articles or follow him on Twitter @Linuturk



Please enter your comment!
Please enter your name here