photo credit: Shapes and Angular Configurations via photopin (license)

How I run unit tests in Vagrant, in Jenkins

As some of you might know, I use Vagrant to setup my local environment, and Jenkins for my continuous integration. One major part of this setup is to automatically run tests after Jenkins is done building – and before it deploys the website to the production server.
In this article I am going to share with you a custom setup for this task that I use for my projects. Running automated tests is one of the major benefits of deploying your website with Jenkins but also one that might be difficult to setup. And if that’s not enough: there are numerous ways to set this up. But this method is mine:

The workflow

The workflow I use when it comes to deploying changes to a live server is pretty straight forward:

  1. I merge all my changes (feature branches) locally in my master branch and push it to the remote repository (the one that is watched by Jenkins).
  2. Jenkins fetches this branch, and runs a build-script. This script concatenates and minifies JavaScript and CSS files, and removes and/or optimizes some other assets.
  3. When the build is complete it checks if there is a testsuite and runs it. That’s what this article is all about.
  4. When the test suite throws no errors, the complete build is pushed to a ‘build-master’-branch which is then used by Capistrano to deploy to a remote webserver.

The testsuite

When a testsuite is found, this is basically what happens on the buildserver:

  1. It creates a virtual machine (vagrant up ).
  2. It checks if it should create or update the (test-)database in the virtual machine.
  3. It runs the testsuite.
  4. It halts the virtual machine.

This is done by 3 separate shell-scripts, runtests.sh , database.sh  and testsuite.sh  and schematically it looks like this:

Screen Shot 2015-04-10 at 16.26.42


This script is the ‘wrapper’ for the database.sh  and testsuite.sh  script and is actually the only script that is executed by the build-script when it finds a testsuite. You could also add this as an action in Jenkins so you wouldn’t have to edit your own build-scripts. That choice is up to you and depends on how you’ve got Jenkins configured for you project. Anyway, the script looks as follows:

The comments in the script speak for themselves, but the most important parts to note in this script are the set +e  and set -e  commands. Normally, when a shell script is executed and a command fails, the whole script will fail and return an exit code. Since we do a vagrant up  in our script we want to be able to detect errors in our database.sh  and testsuite.sh  scripts so we can de a proper vagrant halt  and manually return an exit code. Otherwise our virtual machine will keep running and could throw errors on a second vagrant up , or on a build server with multiple projects they could really clog up your resources.


The database.sh  script does one thing: Check if there is a SQL-file on a shared folder on the build server and import it in the virtual machine if found. The script looks like this:

We explicitly used set -e  to make sure the script stops executing and returns an error if any of the tasks fail. The only reason when the script will return an error as exit code is if the script can find an SQL-file in the shared folder on the build server, but Vagrant is unable to import it (for example, when the SQL-file is faulty or corrupt). When no SQL-file is found the script will do nothing and leave the already existing database in the virtual machine untouched.


This script is the place where the magic happens. In this script you can put all the tests you want to run. These include PHPUnit tests, but could also include other kinds of tests, like CasperJS tests for example.

Once again we used set -e  to make the script stop and return an error if any of the tests fail. This exit code is caught by runtest.sh  and will prevent Jenkins from deploying to the live server. Not that I run tests in my vagrant box by explicitly using the vagrant ssh -c “”  command. It’s important to note that all the shell scripts in this article are ran on the build server by the Jenkins user and not from within Vagrant.

Writing your tests

PHPUnit is a great tool to write tests for your code but if you’ve never written test driven code before it might look overwhelming to start with. But fear not, because it’s actually easier than you think. On the other hand, you probably already know how to write unit testable code and are reading this article because you are more interested in the whole integration-with-Jenkins-and-Vagrant-part.
One thing I do however want to point out is the whole database-testing part. I already covered this topic in some articles like:

Unit testing with a database might look tricky at first but once you already have a database in your virtual machine with actual data you’re already half-way. Now there are two things you can do:

1: Make use of a test database

Make sure that the database on your build server / virtual machine is a database filled with specific data and write your testsuite for that data. The pro’s are:

  • You don’t need to edit your code (you can use the same table names for example).
  • You have a complete setup of a working website including configurations and everything.

The cons are:

  • The database can quickly become too big to keep in your Git repository.
  • New functionality / tests might require an update on the complete test database, potentially breaking other tests in the process.
  • When you’re writing modular software (modules, plugins) chances are that you want to include the tests with your module/plugin. But when you use one big database how are you going to achieve this?

2: Include test tables with your tests

Another option is that you provide little SQL-files with test-tables (packed with testdata) with your tests that can run independent of the current installation. By using an alias-pattern when declaring table names in your query you can let your unit tests read different database tables than the production would. The pro’s are:

  • You can include the SQL-files with test tables and data in your Git repository.
  • Your test can be bundled with your module/plugin and can be ran on any given installation.

The cons are:

  • You need to keep the alias-pattern in mind when writing your SQL-queries.
  • Separate SQL-files that are only used in unit tests might be harder to maintain or update than simple exporting an already existing database table.

A simple PHP-class that could provide the alias-pattern for your table names could look something like this:

When you’re writing your SQL-queries you can use above like this:

Magento already has built-in functionality to provide something similar to this out of the box. And this article covers how you can import these small SQL-files prior before running a testsuite.

In conclusion

Unit testing is really a great help and powerful tool in writing better code. You might even say that it makes you a better programmer. It’s funny to note that almost a year ago I didn’t write any test-driven code and that today not a day goes by without me either writing unit tests or having benefits from the tests that I wrote in the past.
Having said that, unit tests aren’t the holy grail. They are a great aid in providing stability and catching problems and edge cases that otherwise might have been missed to begin with. But it’s still up to the person between the chair and the keyboard to make all the magic happen.
I hope this article was helpful for you and provided you with some insight in how you can set something like this up. On the other hand, if you have an even better solution or questions or comments on my article, feel free to place them in the comments below. I’m always eager to learn and improve myself, as well as willing to explain some topics a bit further if they need more attention.

Visitors give this article an average rating of 4.8 out of 5.

How would you rate this article?

Leave a Reply