Test Driven Development with Ansible


I had an interesting discussion about testing on the Ansible list. I asked whether or not the Ansible developers had plans to introduce a testing framework to test the actions in a playbook. Apparently I was unpersuasive since they did not find the idea very useful.

My reasoning was that like unit testing in software development, all code must be tested. An Ansible playbook is code even though it uses a declarative format. So, I was looking for the equivalent of JUnit for Ansible. This follows the principle of No Single Point of Failure. Ansible is a great tool, but adding an external check of the resulting server configuration makes it better. Even the best coders need unit tests.

As a result of this discussion I did learn two ways to approach Ansible testing:

  1. Using a test playbook. This has the advantage of being integrated with Ansible. It uses basic Unix commands to test the server configuration.
  2. serverspec. This Ruby library is made to test server configurations. It looks very cool. It uses an easy to read declarative syntax like Ansible. Since it is a completely distinct system I think this will make a good “unit testing” framework for Ansible. Now I won’t have to write one myself.

Following the test driven development (TDD) approach, the work-flow would now be:

  1. Write serverspec test.
  2. Write Ansible playbook.
  3. When playbook passes the test its done.

I will also add this as a process step under continuous integration. Both Ansible and serverspec scripts will get checked into git. Once they are then Jenkins (my continuous integration server) will check them out and run the serverspec tests. The serverspec test will run in my integration environment. The initial Ansible playbook development will take place in my Vagrant-based development environment.



Categories: DevOps, Testing

Tags:

2 replies

  1. Right, so essentially Ansible is about infrastructure and having test for your playbooks themselves doesn’t really address if the infrastructure works. You can write a perfectly good playbook that doesn’t actually address any failings in the infrastructure. So for instance you may write a playbook that spins up lets say 4 nodes running nginx and if ansible had tests, you’d test to make sure that exist. Unfortunately one of those nodes dies because of a hardware malfunction or maybe there is a network problem. Well, now you’re test passed but the infrastructure doesn’t resemble those passing tests.

    Infrastructure as code means you need to test the actual infrastructure and not what your provisioning and orchestration tools did/are doing. Does TDD work for infrastructure? Sure, why not, but it’s an iterative process for whatever you’re building.

    Like

    • Good comment. I think you have touched on an important area. After deployment your system has to be continuously monitored to ensure that remains in the desired state. My view is this responsibility belongs to a different set of tools. Ansible, serverspec, version control tools like git, the CMDB, etc. help you define, test, deploy, and manage system and application configurations. Tools like ossec, nagios, collectd, monit, splunk, SNMP, etc. help you monitor the running, dynamic state of your system and applications. I think “test” in the monitoring context means continuous testing; that is, you are constantly checking the value of important key attributes, usually performance related. This still requires a test first strategy because you want to determine your desired values beforehand.

      The pull tools like Chef and Puppet blur the lines because they run constantly. They aren’t, however, true monitoring tools.

      Like

Share Your Ideas

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: