photo credit: dissassembled via photopin (license)

Integration testing with Jenkins, Vagrant and CasperJS

Recently, I shared an article with you on how I am currently working on a Continuous Integration environment with Jenkins. I ended that article stating I am working on a full-blown example on how to combine Jenkins with Vagrant for PHPUnit and CasperJS.
At this point I can share a bit more information with you on how I’ve currently set it all up. I have a working proof of concept and in this article I’ll break it down with you.

jvc_magic guy with unicorn shirt saying magic


I hope you are already a bit familiar with Jenkins, or already using it. If not, my previous article sure is a good read-up.
While I will not spend too much time on how I’ve setup Jenkins and how the rest of a Jenkins-project looks like, there are some things you must know:

  • I use the NodeJS plugin for Jenkins to create ad-hoc NodeJS setups for my project. I tried working with a system-wide installation, but for some funky reasons I never had that running smooth.
  • As needed with this plugin, I added global npm packages for phantomJS and CasperJS, since those are used later on for integration testing.


So I assume some (or hopefully most) of you are already familiar or working with Vagrant. For those who aren’t: you should really check it out! One of the main benefits of Vagrant is that it creates a virtual machine for your project, which configuration (provisioning) can be stored as part of your project. This gives the great benefit that you can mimic the environment you are going to deploy to, but also provides other developers exactly the same development environment as you.
With Jenkins we have another great benefit: as I stated above, the development environment of your project is identical on all systems. So there’s nothing stopping Jenkins from doing a vagrant up  and running some tests on it!

The challenges

This sounds really easy, and to some point it is. But there are several challenges here to overcome:

Port 8080 vs 80

When I develop a site, I want to use the ‘real’ domain name. So I don’t want to go to something like mydomain.dev, or localhost:8080. I add an entry to my hosts-file so it gets mapped to my localhost.
But here lies a problem: Most systems have a reserved port-range that don’t allow applications such as Vagrant to forward traffic on port 80 to a virtual machine. Therefore, Vagrant is in most cases setup to listen to port 8080. But as I stated before: I don’t want that!
Luckily, there are 2 solutions to overcome this problem:

  1. ipfw is a Unix-command that allows port forwarding. A command like sudo ipfw add 100 fwd,8080 tcp from any to me 80  does the trick. Thanks go out to Marco Bac for this one.
  2. iptables is another great candidate if you don’t have ipfw (on Ubuntu systems for example). To achieve this, you can do sudo sudo iptables -t nat -I OUTPUT -p tcp -d –dport 80 -j REDIRECT –to-ports 8080 .

Needless to say, this also needs to be done on the Jenkins server, since that machine is also going to vagrant up.


I guess that most people of you know that to give Vagrant a nice little speedboost, you have to enable NFS. The drawback of this is that you’ll be prompted to enter your sudo password on vagrant up. Now, this is not a problem on your local development system, but in an automated process like Jenkins this means your build process will stall here for quite some time and eventually fail.
So in order to do deal with this, we have to disable NFS when Vagrant is going up on our Buildserver. We can use a simple sed -command prior for our vagrant up  to achieve this:

I know, you are sacrificing precious speed with this, and it will make your tests run slower, but for now this is the easiest fix I could come up with. If anyone knows a better way on how te deal with this in an automated process please let me know!


Now the next obstacle is the database. We have our Vagrant machine up and running now on Jenkins and we are ready to run some juicy tests on it, but as soon as we are going to run our tests on it, they’ll most likely fail because the site is missing the database.
But how are we going to get the database in our virtual machine? I’ve thought of this as well and discussed it with my co-workers, and this is the solution I’ve currently come up with:

  • Our buildserver has a shared folder on the network. In this folder we place the SQL-dumps (with testdata) for our websites.
  • When Jenkins has done a vagrant up , it does the following steps:
    1. It dumps the existing database.
    2. It creates a new database.
    3. It imports the SQL dump.

This way, each time the tests are done, the will work with a fresh snapshot of the database with test data. I wrote a small shell script that does just this:

One small gotcha you might have noticed in this script, is that we have to copy the SQL-dump to the workspace folder in order to import this. This is because we run mysql inside the virtual machine. So in order for mysql inside the virtual machine to read the sql-file it must be in our shared Vagrant folder.

The tests

And now we have all set this up, it’s finally time for the tests. I’ve tested with PHPUnit for code testing, and CasperJS for frontend / integration testing.
CasperJS is very easy and straight forwarded of course, since it mimics a browser and only tests the result. A simple shell command to run a CasperJS test could for example be:

PHPUnit on the other hand, needs to be run inside the virtual machine, since it needs to run PHP code and (probably) access the database. Luckily for us, Vagrant has a way to forward shell commands to a virtual machine:

All done, all green!

So, now we have a task in our Jenkins-project that creates or resumes a virtual machine for us, refreshes the database, runs some various tests on our website, and when all is done and green (because we love green), Jenkins does the rest of it’s magic to deploy our website (or changes) to the live/production server.
And the best part of it all: it does all this automagically each time we push changes to our master branch.

Visitors give this article an average rating of 3.5 out of 5.

How would you rate this article?

2 thoughts on “Integration testing with Jenkins, Vagrant and CasperJS”

    1. Giel Berkers says:

      Thanks for the tip, when I have a use case, I’ll look into it. Although I am aware that there are multiple solutions out there that can achieve the same goal, I also think it’s good to have at least once build a deployment and testing workflow from scratch, just to learn what processes and tasks are involved with it.

Leave a Reply