Infrastructure Testing With Ansible and Serverspec: Part 3 Jenkins


This is the third and final part of my guide to testing Ansible with serverspec. This part shows how to use the continuous integration server Jenkins to run Ansible and serverspec. I will explain how to install and configure Jenkins on CentOS 6.5 using Ansible (oddly circular isn’t it?). I will describe how to configure a basic Jenkins job to run both Ansible and serverspec from a git repository.

If you read my, um, commentary you may guess that I have a slightly antagonistic relationship with Jenkins. Nevertheless, I know many people want to use it so I will do my best to explain the basics and hopefully save you some headaches.

Prerequisites

I assume here that you already know how to configure and run both Ansible and serverspec. I also assume that you have a git server configured with SSH key or anonymous access. I recommend gitolite for your git server. It has worked well for me.

You will also need your Jenkins CentOS server configured to use the EPEL repository. Ansible is only available from EPEL and not from the default CentOS repository. There are many guides on the Internet explaining how to do this.

Installing Jenkins

My Jenkins playbook is very basic but easy to understand if you are familiar with Ansible. See this playbook when you are ready for a more advanced one. In this guide you will have to take some manual steps, something which I usually disapprove. My poor excuse is that I got too frustrated with Jenkins to spend the time learning the Jenkins CLI.

First create a standard Ansible role layout:

roles/
    jenkins/
        files/
            id_rsa
        handlers/
            main.yml
        tasks/
            main.yml
        templates/
            jenkins.j2

You will need to generate a key pair to authenticate the Linux jenkins user to the git server using SSH. git needs the public key. If you use gitolite then add jenkins as a read-only user to your Ansible and serverspec repositories along with its public key. Normally, you don’t want to include the private key id_rsa in your files, I do it here for training purposes only.

My handler file is dirt simple:

---

- name: restart jenkins
  service: name=jenkins state=restarted

The Ansible play is:

---

- name: Install Jenkins repo key
  rpm_key: state=present key=http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key

- name: Install Jenkins repo
  get_url: url=http://pkg.jenkins-ci.org/redhat/jenkins.repo dest=/etc/yum.repos.d/jenkins.repo owner=root group=root mode=0644

- name: Install packages
  yum: name= state=installed
  with_items:
      - java-1.7.0-openjdk
      - jenkins #jenkins repo
      - ruby
      - rubygems
      - ansible #epel repo

- name: Install ruby packages
  gem: name= user_install=no state=latest include_dependencies=yes
  with_items:
     - rake
     - rspec
     - serverspec
     - specinfra
     - bundler

- name: Change the Ansible config file
  lineinfile: dest=/etc/ansible/ansible.cfg  regexp="^#host_key_checking = False" line="host_key_checking = False"

- name: Add jenkins private key for git access
  copy: src=jenkins_private.key dest=/var/lib/jenkins/.ssh/id_rsa owner=jenkins group=jenkins mode=0400

- name: Ensure jenkins is running
  service: name=jenkins state=started

- name: Set jenkins to start on boot
  command: chkconfig jenkins on

- name: Copy conf file
  template: src=jenkins.j2 dest=/etc/sysconfig/jenkins owner=root group=root mode=0644
  notify:
     - restart jenkins

This will install Jenkins, Ansible, and the required Ruby packages. Note that the Linux jenkins user has /var/lib/jenkins/ as its home directory. All the Jenkins files will be stored there, including your Ansible and serverspec code once Jenkins checks it out from git.

The file jenkins.j2 in the templates directory is the basic jenkins configuration file. I haven’t changed it but keep it here in case I want to chnage it later. I also store files like this as templates so it is easy to add variables.

If the install worked then you should be able to open the Jenkins web interface at http://your.server.com:8080 using the host name of your Jenkins server. If it doesn’t respond then verify that it started correctly and that you have added a rule in iptables to allow incoming connection on TCP port 8080.

Jenkins manual configuration

Once you have confirmed that Jenkins is running you need only make a few configuration changes to run the tests. I will explain only the minimum needed to get Ansible and serverspec to run. If you find Jenkins useful you will want to invest the time to properly configure it; that is, add user accounts, etc.

Plugins

In order to run these tests you will need to install 2 plugins:

  • Rake
  • Multiple SCMs

You do this by selecting Manage Jenkins from the home page, then Manage Plugins. From here click the Available tab and enter the names of the two plugins in the Filter field in the upper right corner. Find and install each. Once both are installed restart Jenkins.

Verify tests

Running these tests from Jenkins involves lots of moving parts. Before you create the job described next, I recommend you first do a short test to validate your configuration. You get extra credit if you run these tests from serverspec rather than manually.

First let me warn you that there are some tricky security and permissions issues here. Ansible needs the passwords or keys to log into your servers. serverspec needs the same. In production you will have to incorporate these tools into your overall security framework. In my workflow, Jenkins makes the changes and execute the tests only in the integration testing environment and not to production servers. The Ansible plays need to be thoroughly tested before going into production and this is just a step in that process.

The steps are:

  1. Log into the Jenkins server via SSH.
  2. Test that Jenkins can check out your source from git
    1. Create a temporary directory anywhere the jenkins Linux user has write permission. For example, sudo -u jenkins mkdir /var/lib/jenkins/tmp
    2. cd into that new directory
    3. Checkout the source code for both your Ansible playbook and serverspec tests into it (assuming you have gitolite installed): sudo -u jenkins git clone gitolite@[FQDN of jenkins host]:[repo name].git
    4. If this works then you have git configured correctly.
  3. Test Ansible
    1. Now pick a simple play from your playbook and run it as jenkins. For example, sudo -u jenkins ansible -i testing.hosts [myserver] -m setup.
    2. It should run correctly.
  4. Test serverspec
    1. Now do the same for serverspec.
    2. cd into the serverspec directory and type sudo -u jenkins rake spec
  5. Now delete the temp directory you created.

Once you can run these tests manually you are ready to let Jenkins do it for you.

Add a job

In this part you will add a job to automatically run Ansible and serverspec. The trigger and frequency is up to you. You can have git trigger a Jenkins build after a push, you can have Jenkins poll git for changes, or you can simply run it on a schedule.

Create a job:

  1. From the Jenkins home screen click New Item.
  2. Give your job a name, then click the Build a free-style software project radio button, and finally click the OK button.
  3. On the job configuration screen skip down to the section titled Source Code Management. You should see a radio button labeled Multiple SCMs. This is the plugin I asked you to install earlier. Select it.
  4. Add your Ansible playbook repository
    1. Click the Add SCM button and select git. A set of form fields will appear.
    2. Enter your git URL in Repository URL. For example gitolite@server.example.com:playbook.git
    3. Click the Add button labeled Additional Behaviours. Then click Check out to a sub-directory. Enter a name for your playbook code directory.
    4. Click the Add button labeled Additional Behaviours. Then click Custom SCM Name. Enter the same name as in step c .
    5. You will need these sub-directory names for the path statement in steps 7 and 8.
  5. Add the serverspec repository
    1. Follow the same steps as in (4) but use the name of your serverspec repository instead.
  6. Skip down to the section titled Build.
  7. Add Ansible build
    1. Click the button labeled Add Build Step, then Execute shell.
    2. In the Command field enter the Ansible playbook command using full paths. Its a bit tricky but follow this example: ansible-playbook -i /var/lib/jenkins/jobs/[job name]/workspace/[ansible repo name]/[inventory file] /var/lib/jenkins/jobs/[job name]/workspace/[ansible sub-dir name]/site.yml
    3. When you have this working you might add a build step for syntax checking. Use the same command but with --syntax-check at the end. Run this before the actual play.
  8. Add the serverspec build
    1. Click the button labeled Add Build Step, then Invoke Rake. This build step comes from the plugin.
    2. In the Tasks field enter spec
    3. Click the Advanced... button.
    4. In the Rake working directory field enter: /var/lib/jenkins/jobs/[job name]/workspace/[serverspec sub-dir name]
  9. OK, that’s it! Click the Apply then Save buttons.
  10. Run the job
    1. From the home page click on the hyperlink name of your job.
    2. Click the Build now link on the left.
    3. When the build starts, click on the hyperlink for it. Then click Console Output to see the messages as the build runs.
    4. It should run without errors if all is correct.
  11. Now go back and configure the build to run periodically or by trigger.

Happy continuous integrating!



Categories: Testing

Tags: , , ,

Share Your Ideas

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: