Testing Ansible roles

We - at the dev-sec.io-team - create Ansible (and Puppet and Chef) roles that harden the security of our Linux-servers. They are meant to be used in production so we try to provide these roles to several different operating systems, including Debian, Ubuntu, RedHat and its derivates.

Manually testing the roles on many different systems is a very time-consuming task: You’ll have to set up a virtual machine, update it, install Ansible, run the playbook, fix an error, rinse and repeat. If you want to do this on many different operating systems, doing it manually is impossible.

The tools

To automate the testing, we use a multitude of programs that together perform all the above-mentioned tasks and in a fraction of time!

• test-kitchen -> Test Kitchen is a test harness tool to execute your configured code on one or more platforms in isolation.
• vagrant -> Create and configure lightweight, reproducible, and portable development environments.
• VirtualBox -> VirtualBox is a powerful x86 and AMD64/Intel64 virtualization product for enterprise as well as home use.
• serverspec -> RSpec tests for your servers configured by CFEngine, Puppet, Ansible, Itamae or anything else.
• kitchen-ansible -> Ansible Provisioner for Test Kitchen
• kitchen-vagrant -> Vagrant driver for Kitchen
• git -> a fast, scalable, distributed revision control system

To help automate testing roles I created this blog-post and the ansible-test-framework. This framework provides the necessary files and configurations to easily set up your testing environment. It is also set up to easily test your roles with travis.

The workflow

The stages of the testing-workflow are the following:

• Create - create defined instances with Vagrant and VirtualBox
• Converge - install Ansible and run your playbook on the instances
• Setup - install the testing utility and runner on the instances
• Verify - run the tests you created on the instances
• Destroy - destroy the instances

Setup

You’ll need to locally install Git, Ruby, VirtualBox and Vagrant first, all other necessary components will be installed along the way.

After installing them, create a directory for your role you want to test (called ansible_role in the following example).

The name directory and the name of the role have to be the same!

Then git-clone the testing-framework into your newly created directory and change into it. Then delete the .git-directory, so you can later upload your role to Github!

# basic setup
mkdir ansible_role
git clone --depth=1 https://github.com/rndmh3ro/ansible-test-framework ansible_role/
cd ansible_role/
rm -fr .git

The next step is to create an empty role with ansible-galaxy. Run this command inside your role-directory and replace ansible_role with the name you gave the directory.

# create empty ansible role
ansible-galaxy init -p ../ --force ansible_role

The force-option is used here because the directory is not empty as you already cloned the test-framework into it.

Now test-kitchen, serverspec, the provisioner, driver and all its dependencies have to be installed. Fortunately, this can all be done with bundler, a gem dependency handler for ruby:

# Install software and dependencies
gem install bundler
bundle install

The first command installs bundler. The second command uses the Gemfile present in the directory and installs all packages and its dependencies.

Now’s the time to customize your testing-setup. You’ll have to replace the default name ansible-test-framework with the name of your role (in my case ansible_role) in two places, the default.yml and the .kitchen.yml. In kitchen.yml the replacement should be in the roles_path. In default.yml its the first item after roles.

You can also use this sed-command to replace the occurrences. Simply replace ansible_role in the command with the name of your role.

# replace ansible-test-framework with your role-name in:
sed -i 's/ansible-test-framework/ansible_role/g' default.yml .kitchen.yml

Write the role and tests

Write your ansible role now! That’s not part of the guide.

Writing tests for your role is also not a part of this guide, but this framework provides the necessary configuration to start writing serverspec-tests: There’s already a file called test_spec.rb where you can write your tests in and the spec_helper is configured for serverspec.

For more help writing tests and using serverspec, read the docs: - http://kitchen.ci/docs/getting-started/writing-server-test - https://github.com/neillturner/kitchen-ansible#test-kitchen-serverspec

Testing

When you’re done writing your role and tests you can start testing! There are different testing methods. You can test single machines, a set of machines or all at once. See the following examples or take a look at the test-kitchen docs.

# fast test on one machine
bundle exec kitchen test ansible-latest-ubuntu-1404

# test on all machines in parallel
bundle exec kitchen test -c

# test all ubuntu machines
bundle exec kitchen test ubuntu

While developing roles it can be cumbersome to always destroy and recreate the machines if a test or the role fails at some point. To circumvent this, you can create and then converge the machine. If the role fails during converging, you can simply run the converge again:

# for development
bundle exec kitchen create ansible-latest-ubuntu-1404
bundle exec kitchen converge ansible-latest-ubuntu-1404

# ... run fails, change role, converge again

bundle exec kitchen converge ansible-latest-ubuntu-1404

If your role takes a long time to run and you want to debug a specific task, you can run the converge with the help of an environment variable like this:

ANSIBLE_EXTRA_FLAGS='--start-at-task="ansible_role | name of last working instruction"' bundle exec kitchen converge

Replace ansible_role | name of last working instruction with the name of the task you want to start at, so you can skip others.

Similarly if you want to skip certain tasks, you can use the environment variable like this:

ANSIBLE_EXTRA_FLAGS='--skip-tags=beginning' bundle exec kitchen converge

Travis tests

The framework also supports testing with Travis-CI by default. All you have to do is enable your repository in Travis and all pushes to the repository will be tested with Travis. Be aware though that Travis tests on Ubuntu, so write your role accordingly!