If you are using code to set up your infrastructure, then that code demands as good or better software development practices as your application code. One of the reasons I wanted to try Chef is because I knew that people were doing automated testing of chef configurations. (The puppet community may be doing similar kinds of testing; I don’t know.) There is even a book about TDD and Chef: Test-Driven Infrastructure With Chef (and it’s even in its second edition). So I am going to try working through it. The first several chapters are background or things I have sort of done already. But they start to use Opscode’s Hosted Chef and Vagrant in chapter 4.
Setting Up a New Playground
When I did the Quick Start exercises, I downloaded the starter kit which included a .chef directory with a knife configuration file and two .pem files: ckiser.pem and ckiser-validator.pem. I explored downloading the starter kit again - but it wanted to reset my keys which would have made my example stuff not work any more. It took me a little while to figure out the instrutions on http://docs.opscode.com/config_rb_knife.html mean that one or the other file is read - not that one is read and then the second file is read, overwriting configuration variables. I was hoping to put shared information in ~/.chef/knife.rb and then project specific information in a knife.rb in the project directory. From my ‘puts’ statement, only one of the files is read - either the one in the current directory or the one in ~/.chef/.
I can still get part of what I want - reuse of my keys - but moving them to ~/.chef/ and then editing my knife.rb file to look for them there. And then I created a new directory for working the TDI Chef exercises, ~/chef-tdd/. I copied the knife.rb from the starter kit into ~/.chef/knife.rb. I also created a cookbooks directory and copied the chefignore file from the starter kit in there. And added the extensive .gitignore from the starter kit. There are some other things I think I will probably need but am going to wait until something I am doing uses them before mving them here.
Vagrant
The last part of chapter 4 is about installing Vagrant and using it to create virtual machines. Mostly this is just like what I have been doing for a while now - but it linked me to a very promising source for base boxes, Opscode’s Bento Boxes One thing that makes them especially attractive is that the don’t come with Chef preinstalled. Instead they recommend installing the most current Chef using the vagrant-omnibus plugin. From the TDI book:
The plug-in we installed into Vagrant (
vagrant plugin install vagrant-omnibus) works with Vagrant boxes that do not have Chef installed, and adds a hook to vagrant up to install Chef using the omnibus package, just as we did in “Exercise 1: Install Chef ” on page 47. This helps keep the Vagrant box slim and as close to upstream as possible, and does not require a fleet of Vagrant boxes to be created with every Chef patch release.
I want to test with a box that matches my Linode VPS as closely as possible so despite the fact I already have an Opscode Ubuntu 12.04 in 32 bit, I downloaded and added the 64 bit version:
vagrant box add opscode-ubuntu-12.04 \
http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-12.04_chef-provisionerless.box
[03:31 PM] (brazen:~/chef-tdd) $ vagrant plugin install vagrant-omnibus
Installing the 'vagrant-omnibus' plugin. This can take a few minutes...
Installed the plugin 'vagrant-omnibus (1.2.1)'!Then in ‘~/chef-tdd/ I did vagrant init opscode-ubuntu-12.04` and
then edited the Vagrant file to have the following:
Vagrant.configure("2") do |config|
config.vm.box = "opscode-ubuntu-12.04"
config.vm.box_url = "https://opscode-vm.s3.amazonaws.com/vagrant/opscode_ubuntu-12.04_chef-provisionerless.box"
config.omnibus.chef_version = :latest
config.vm.hostname = "vagrant-ubuntu-12-04"
config.vm.network :forwarded_port, guest: 80, host: 8080
endThen I did vagrant up. But I didn’t see the lines I expected
regarding installing chef. I tried destroying the VM and then tring
vagrant up again - same result. I do have warnings about the Guest
Additions not matching the version of VirtualBox that the Opscode base
boxes were built against:
[default] Waiting for VM to boot. This can take a few minutes.
[default] VM booted and ready for use!
[default] The guest additions on this VM do not match the installed
version of VirtualBox! In most cases this is fine, but in rare cases
it can cause things such as shared folders to not work properly. If
you see shared folder errors, please update the guest additions within
the virtual machine and reload your VM.
Guest Additions Version: 4.3.2
VirtualBox Version: 4.2
[default] Setting hostname...So I downloaded the latest version of VirtualBox, 4.3.6, and tried again. That wasn’t so great. I got the following error message:
$ vagrant up
Vagrant has detected that you have a version of VirtualBox installed
that is not supported. Please install one of the supported versions
listed below to use Vagrant:
4.0, 4.1, 4.2I currently have Vagrant 1.2.2 installed. Looking at vagrantup.com, it looks like the latest is 1.4.3. Let’s see if upgrading Vagrant fixes this. I downloaded the latest .dmg file and ran the installer. Then my next attempt at vagrant up gave me:
The following plugins were installed with a version of Vagrant that
had different versions of underlying components. Because these
component versions were changed (which rarely happens), the plugins
must be uninstalled and reinstalled.
To ensure that all the dependencies are properly updated as well it is
_highly recommended_ to do a `vagrant plugin uninstall` prior to
reinstalling.
This message will not go away until all the plugins below are either
uninstalled or uninstalled then reinstalled.
The plugins below will not be loaded until they're uninstalled and
reinstalled:
vagrant-berkshelf, vagrant-omnibusI unstalled both plugins. I don’t remember installing vagrant-berkshelf so, for now, I am not going to reinstall it. I only reinstalled vagrant-omnibus (1.2.1). But still no joy: no chef-client or knife when I log into the box. ARRRGGGGGG because I was editing the wrong Vagrantfile! I had put the line about vagrant-omnibus into the file in chef-repo that I had opened for reference. OK let’s destroy and recreate the VM. And now, we get chef:
04:20 PM] (brazen:~/chef-tdd) $ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'opscode-ubuntu-12.04'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] -- 80 => 8080 (adapter 1)
[default] Booting VM...
[default] Waiting for machine to boot. This may take a few minutes...
[default] Machine booted and ready!
[default] Setting hostname...
[default] Mounting shared folders...
[default] -- /vagrant
[default] Installing Chef 11.8.2 Omnibus package...
[default] Downloading Chef 11.8.2 for ubuntu...
[default] downloading https://www.opscode.com/chef/metadata?v=11.8.2&prerelease=false&p=ubuntu&pv=12.04&m=x86_64
[default] to file /tmp/install.sh.1143/metadata.txt
[default] trying wget...
[default] url https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef_11.8.2-1.ubuntu.12.04_amd64.deb
md5 3d3b3662830a44eeec71aadc098a4018
sha256 a5b00a24e68e29a01c7ab9de5cdaf0cc9fd1c889599ad9af70293e5b4de8615c
[default] downloaded metadata file looks valid...
[default] downloading https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef_11.8.2-1.ubuntu.12.04.amd64.deb
[default] to file /tmp/install.sh.1143/chef_11.8.2_amd64.deb
[default] trying wget...
[default] Checksum compare with sha256sum succeeded.
[default] Installing Chef 11.8.2
[default] installing with dpkg...
[default] Selecting previously unselected package chef.
[default] (Reading database ...
[default] 54659 files and directories currently installed.)
[default] Unpacking chef (from .../chef_11.8.2_amd64.deb) ...
[default] Setting up chef (11.8.2-1.ubuntu.12.04) ...
[default] Thank you for installing Chef!
[04:24 PM] (brazen:~/chef-tdd) $ vagrant ssh
Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.8.0-29-generic x86_64)
* Documentation: https://help.ubuntu.com/
Last login: Tue Nov 26 11:27:55 2013 from 10.0.2.2
vagrant@vagrant-ubuntu-12-04:~$ chef-client --version
Chef: 11.8.2Ch 7.2: Berkshelf
Much of configuration management (and really any software development)
is dependency management. Throughout the TDI book we were managing
that manually - downloading cookbooks, checking dependencies,
downloading more cookbooks. Finally in chapter 7 the author introduces
Berkshelf to do the managing for us. So now I am going to need the
vagrant-berkshelf plugin that I delayed reinstalling in the section
above. vagrant plugin install vagrant-berkshelf. And then the book
told me to run berks configure. This creates a default configuration
file in ~/.berkshelf/config.json. Not really sure what the implications
of some of the items in this are. I’ll have to see as they arise.
Since I didn’t make the stand alone IRC cookbook from chapter 3, I
can’t do the berks init stuff. And the discussion of Berkshelf kind
of trailed off promising more information about preferred workflows
later in the chapter.
Ch 7.3: Application Cookbooks and Test Kitchen
What are application cookbooks and why should we use them?
The application cookbook pattern is characterized by having decided the top-level service that we provide and creating a cookbook for that service. That cookbook wraps all the dependent services that are needed to deliver the top-level service. … This looks a lot like the kind of thing that might be accomplished using a Chef role, but has some significant advantages. First of all, cookbooks can be explicitly versioned and tracked in a way that roles can’t.
If there is a need to alter the behavior of an upstream cookbook, attributes can be set in a recipe, and if functionality needs to be added, tested, or tweaked, this can be achieved by wrapping upstream cookbooks in a manner that looks much like object inheritance. This has the twin advantages again of being testable, but also of avoiding constant fork‐ ing of upstream cookbooks.
The test harness tool of choice - at least of this author - is
Test Kitchen. I installed
it with gem install test-kitchen. The tests themselves are created
using a combination of cucumber, rspec, and leibniz (a library written
by the author).
Things were going OK until about page 200 where there were some missing steps - such as where are we putting our specs and how did we get our spec_helper - and for that matter, where did we get rspec since it wasn’t on the list of gems we were told to add to our Gemfile. The errata page has a complaint from someone else who noticed this - but not resolution. I spent some time wandering around, read, asked some questions on the #chef IRC channel. And then with some perspective, went back to the book to see if I could reverse engineer stuff using the code listings.
First, if we are going to run rspec -fd, we need rspec installed. So
I added it to the Gemfile and did bundle install. Now I can run
rspec --init and that created a spec directory and put a
spec_helper.rb file in there. I am kind of unclear about where to put
the integration test at the bottom of p 202 so I skipped them and
started with the unit test at the top of p 204. I typed in the ‘let’
and the first ‘it’ and then ran the spec. It complained uninitialized
constant ChefSpec so I added ‘chefspec’ to my Gemfile and a “require
‘chefspec’” to the top of my default_spec.rb. Then it complained it
couldn’t find chef. I know I have chef - but it wasn’t in my
Gemfile. So I added it to my Gemfile and reran ‘bundle’. Now I am
getting an odd error about about Chef - and a really old version of
chef. How did that get in here? And how did I end up with chefspec
0.0.1?
(brazen:~/chef-tdd/cookbooks/cnk-blog) $ rspec
/Users/cnk/.rvm/gems/ruby-2.0.0-p353@chef/gems/chef-0.8.10/lib/chef/provider/package/dpkg.rb:26:in `<class:Package>': uninitialized constant Chef::Provider::Package::Apt (NameError)
from /Users/cnk/.rvm/gems/ruby-2.0.0-p353@chef/gems/chef-0.8.10/lib/chef/provider/package/dpkg.rb:25:in `<class:Provider>'I created a completely new rvm gemset, removed my Gemfile.lock and did
a completely new bundle install. This time I only have chef 11.8.2
along with chef-spec 3.2.0 But even after fiddling with how I required
chefspec, I am still getting complaints abotu uninitialized constant
ChefSpec::ChefRunner. So I am going to give up on the example in
Chapter 7 “Acceptance Testing: Cucumber and Liebniz”.
Ch 7.5: Integration Testing: Test Kitchen with Serverspec
The next section on integration testing does a better job of showing
us the steps. But introduces one more layer - using Test Kitchen to
manage your Vagrant boxes. It is sort of nice because it gives you one
file that declares which OSs you are going to test your cookbook on -
and takes care of setting up what one needs to do that. But it is one
additional layer AND it means I am swimming in Vagrantfiles - most of
which are not used. I had thought I would test my full configuration
on a local Vagrant instance, so I created a Vagrantfile at the top of
my chef-tdd directory. When I did berks cookbook cnk-blog, that
automatically created a Vagrant file. And now when I edited the
.kitchen.ym file (which berks also created for me) and ran kitchen
create all, that created a Vagrantfile in
.kitchen/kitchen-vagrant/default-ubuntu-1204/ I think it is this last
Vagrantfile that I am actually seeing when I do kitchen list - since
I can’t ssh in when I am in ~/chef-tdd/cookbooks/cnk-blog/ but I can
if I am in
~/chef-tdd/cookbooks/cnk-blog/.kitchen/kitchen-vagrant/default-ubuntu-1204
My cookbook doesn’t yet have a run list, but if I run kitchen
converge anyway, that installs ruby and chef from the Opscode Omnibus
installer. So now, where do I put the tests. I don’t think I want to
use BATS unless I have to - I am better at Ruby than bash. So I read
that part but then started working through the Serverspec exercises
starting at p 241. I created a test file in the magic place (test
kitchen figures out it needs busser and then what kinds of tests to
run based on the directory names).
$ mkdir -p test/integration/default/serverspec/localhost
# create file cnk-blog_spec.rb
$ cat test/integration/default/serverspec/localhost/cnk-blog_spec.rb
require 'spec_helper'
describe "Cynthia's static html blog site" do
it 'should have installed apache' do
expect(package apache2).to be_installed
end
end
$ kitchen verify
-----> Starting Kitchen (v1.1.1)
-----> Setting up <default-ubuntu-1204>...
Fetching: thor-0.18.1.gem (100%)
Fetching: busser-0.6.0.gem (100%)
Successfully installed thor-0.18.1
Successfully installed busser-0.6.0
2 gems installed
-----> Setting up Busser
Creating BUSSER_ROOT in /tmp/busser
Creating busser binstub
Plugin serverspec installed (version 0.2.6)
-----> Running postinstall for serverspec plugin
Finished setting up <default-ubuntu-1204> (0m22.57s).
-----> Verifying <default-ubuntu-1204>...
Suite path directory /tmp/busser/suites does not exist, skipping.
Uploading /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb (mode=0664)
-----> Running serverspec test suite
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serversp
ec/localhost/cnk-blog_spec.rb --color --format documentation
/opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- spec_helper (LoadError)
from /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb:1:in `<top (required)>'
from /tmp/busser/gems/gems/rspec-core-2.14.7/lib/rspec/core/configuration.rb:896:in `load'
from /tmp/busser/gems/gems/rspec-core-2.14.7/lib/rspec/core/configuration.rb:896:in `block in load_spec_files'
from /tmp/busser/gems/gems/rspec-core-2.14.7/lib/rspec/core/configuration.rb:896:in `each'
from /tmp/busser/gems/gems/rspec-core-2.14.7/lib/rspec/core/configuration.rb:896:in `load_spec_files'
from /tmp/busser/gems/gems/rspec-core-2.14.7/lib/rspec/core/command_line.rb:22:in `run'
from /tmp/busser/gems/gems/rspec-core-2.14.7/lib/rspec/core/runner.rb:80:in `run'
from /tmp/busser/gems/gems/rspec-core-2.14.7/lib/rspec/core/runner.rb:17:in `block in autorun'
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb --color --format documentation failed
Ruby Script[/tmp/busser/gems/gems/busser-serverspec-0.2.6/lib/busser/runner_plugin/../serverspec/runner.rb /tmp/busser/suites/serverspec] exit code was 1
>>>>>> Verify failed on instance <default-ubuntu-1204>.
>>>>>> Please see .kitchen/logs/default-ubuntu-1204.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c 'BUSSER_ROOT="/tmp/busser" GEM_HOME="/tmp/busser/gems" GEM_PATH="/tmp/busser/gems" GEM_CACHE="/tmp/busser/gems/cache" ; export BUSSER_ROOT GEM_HOME GEM_PATH GEM_CACHE; sudo -E /tmp/busser/bin/busser test']
>>>>>> ----------------------Oops. I didn’t read carefully and missed creating the helper file in cnk-blog/test/integration/default/serverspec/spec_helper.rb
require 'serverspec'
require 'pathname'
include Serverspec::Helper::Exec
include Serverspec::Helper::DetectOS
RSpec.configure do |c|
c.before :all do
c.os = backend(Serverspec::Commands::Base).check_os
end
endNow I get properly failing tests - that is failing because I have not written the cookbook code to make them work.
$ kitchen verify
-----> Starting Kitchen (v1.1.1)
-----> Verifying <default-ubuntu-1204>...
Removing /tmp/busser/suites/serverspec
Uploading /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb (mode=0664)
Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0664)
-----> Running serverspec test suite
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb --color --format documentation
Cynthia's static html blog site
No packages found matching apache2.
should have installed apache (FAILED - 1)
have enabled the apache service (FAILED - 2)
httpd: unrecognized service
be running the apache service (FAILED - 3)
should listen on port 80 (FAILED - 4)
should have a virtual host for cnk-blog
Failures:
1) Cynthia's static html blog site should have installed apache
Failure/Error: expect(package 'apache2').to be_installed
dpkg-query -f '${Status}' -W apache2 | grep '^install ok installed$'
expected Package "apache2" to be installed
# /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb:5:in `block (2 levels) in <top (required)>'
2) Cynthia's static html blog site have enabled the apache service
Failure/Error: expect(service 'httpd').to be_enabled
ls /etc/rc3.d/ | grep -- '^S..httpd' || grep 'start on' /etc/init/httpd.conf
grep: /etc/init/httpd.conf: No such file or directory
expected Service "httpd" to be enabled
# /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb:9:in `block (2 levels) in <top (required)>'
3) Cynthia's static html blog site be running the apache service
Failure/Error: expect(service 'httpd').to be_running
ps aux | grep -w -- httpd | grep -qv grep
expected Service "httpd" to be running
# /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb:13:in `block (2 levels) in <top (required)>'
4) Cynthia's static html blog site should listen on port 80
Failure/Error: expect(port 80).to be_listening
netstat -tunl | grep -- :80\
expected Port "80" to be listening
# /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb:17:in `block (2 levels) in <top (required)>'
Finished in 0.06817 seconds
5 examples, 4 failures
Failed examples:
rspec /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb:4 # Cynthia's static html blog site should have installed apache
rspec /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb:8 # Cynthia's static html blog site have enabled the apache service
rspec /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb:12 # Cynthia's static html blog site be running the apache service
rspec /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb:16 # Cynthia's static html blog site should listen on port 80
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/cnk-blog_spec.rb --color --format documentation failed
Ruby Script[/tmp/busser/gems/gems/busser-serverspec-0.2.6/lib/busser/runner_plugin/../serverspec/runner.rb /tmp/busser/suites/serverspec] exit code was 1
>>>>>> Verify failed on instance <default-ubuntu-1204>.
>>>>>> Please see .kitchen/logs/default-ubuntu-1204.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c 'BUSSER_ROOT="/tmp/busser" GEM_HOME="/tmp/busser/gems" GEM_PATH="/tmp/busser/gems" GEM_CACHE="/tmp/busser/gems/cache" ; export BUSSER_ROOT GEM_HOME GEM_PATH GEM_CACHE; sudo -E /tmp/busser/bin/busser test']
>>>>>> ----------------------For Future Reference
-
A tutorial in the form of a git repo: RallySoftware-cookbooks/chef-tutorials