Things I learned about Deployment, Test Driven Development and Continuous Integration

I’m reading a lot about Continuous Integration and Test-Driven Development lately, to work out the best ways to develop code as agile and with as much flexibility for deployment as possible. I wrote a small post earlier on Continuous Integration in PHP about how to build a CI server with Jenkins. In this post I would like to go deeper into the Why of Continuous Integration and Test-Driven Development.

In the early stage of we worked live on our production codebase, changing and testing things while users were visiting the website. That provided a lot of trouble and was soon discarded. Ever since we (my team at Sugababes first and Mobypicture later) work from SVN on development environments, commit code, try to test it in a staging environment, but most of the time we just deploy. That is, export parts of the codebase to our live environment. That is not an ideal situation and proves to be very catchy for bugs that could have been prevented.

The first thing we should need is automated deployment, for example with Ant or Capistrano. Capistrano gives the option to rollback a deploy, which can be very useful. Capistrano copies the code to servers using SCP in parallel and can perform tasks before or after a deploy, like Minifying your Javascript/CSS and upload them to your CDN. This does not help in the prevention of deploying bugs and broken code. When you change a Class in one area of your code it will often break your codebase, how hard you look for all instances of your Class and change the code where necessary. Unit Testing should find most of these breaks.

So the second thing we need is Unit Testing. You write a test for every single function in every single Class. When you change Class A and Class B uses Class A, your Unit Test will tell you when Class B breaks on that change. A good way to make sure you write Unit Tests is by using Test-Driven Development. Test-Driven Development forces you to write your Unit Tests before you write any code. It thereby also forces you to better think about what you are going to code, because you define all income and outcome before writing any line of code.

Before you commit any code you should run your Unit Tests to test and make sure your code is still working. When your project grows your tests will last longer and besides Unit Tests, there could be more tests you would like to run. Like Code Duplication Detection, Javascript and HTML testing and generation of documentation. This can take a lot of time every time you would like to commit. And if you program as iterative as we do, commits happen multiple times a day. It would be nice if this could all be done automatically right?

With a Continuous Integration server you can. After someone commits any code, the CI server exports the codebase and runs all the tests automatically and will notify the person who comitted or even the whole team about errors. Those errors should be fixed as fast as possible because the codebase should be ready to be deployed at any time. And that is probably the most important reason to use Continuous Integration. I ask my team all the time if the codebase is ready for a deploy and that should not be the case. It should always be ready, because your main codebase in the repository it THE place where every developer starts to work from and new versions are build from.

Setting up a Continuous Integration server is no easy task, but Jenkins looks interesting. It supports Ant-like build files, has a nice web-based front end and has tons of plugins to provide further functionality. Because it is the leading CI server there are also a lot of tutorials about implementation for almost all development languages. Eric Hogue wrote a nice tutorial on setting up Jenkins for Continuous Integration on PHP, with PHPUnit and a lot of quality assurance. I will write more about my experiences with Jenkins and Continuous Integration when I learn more about it, because I really believe this will improve my deployment cycles in the future.

We built GroundControl to execute the ideas in this article. It puts our way of working, portfolio management, and innovation accounting in one place where innovation managers and corporate startups work seamlessly together. See How GroundControl Works.