This post is part of the Agile in small steps series. For a complete list of posts in this series, click here.

I’m sure you’ve been in some projects where integration happens in one big bang at the end of the project — and that’s when everyone discovers that the components don’t work together.

The idea with continuous integration is to learn about potential problems earlier in development, and what better way to catch integration problems than to integrate frequently and look out for errors?

Continuous integration is the culmination of the three steps we saw previously — Unit testing, Version control and One step release build. With continuous integration, you integrate all the components of your product as frequently as possible. In some cases this might be once a day. In other cases this might be once an hour.

Here is how it works: A script or tool checks out the workspace once an hour, runs the one step release build script, then verifies everything by running automated tests. Naturally version control, one step builds and unit testing are prerequisites.

Smarter scripts and tools can perform an integration after every checkin, at a prescribed schedule, and a whole lot more, but those are extras. As long as a build and test of the code in the repository happens at a regular interval, its continuous integration.

Continuous integration can also be done manually. How? After every checkin, a developer goes to the build machine, checks out the latest code, runs the build script and runs the tests. Thats continuous integration too, only a bit more painful.

The good news is that there are a whole bunch of continuous integration tools, some with a bunch of extra features and integration with various tools. Most of them save the built artifacts so that you can always get hold of a working copy of the binary at any particular time. Some have web interfaces which show you the past builds and their status, plus metrics and so on. Like I said before, these are all just extras. Useful extras, but extras nonetheless.

The main purpose of continuous integration is to build often and find problems early. Just as how unit tests provide rapid feedback at the code level, continuous integration provides rapid feedback at the integration level.

Further Reading

1. Martin Fowler on Continuous Integration
2. A comparision of continuous integration tools

This post is part of the Agile in small steps series. For a complete list of posts in this series, click here.

Is this how you currently make a release?

1. Check out code from version control
2. Execute the build script for each module
3. Execute a script to integrate all the modules
4. Execute a program to create an installer
5. Execute a script to create a CD image/Package the files into an exe/jar/war/…
6. Copy the built code to another machine/Hand over the built code to QA
7. Install/Deploy the application
8. Test the application
9. Release the application

Wouldn’t it be better if you could do something like this instead:

1. Execute release script

Impossible? Not so. The technique is to take it one step at a time.

Start out with a script that just checks out code from version control. Most build tool like ant have support for a variety of version control systems. Most version control systems have a command line interface, so even if your build tool does not have version control integration, you can write a script to execute the appropriate command. For instance, I use a python script to perform my builds.

Once that is done, add each step one by one. So the next step would be to create a script that will build your code base. Then add a line in your main build script to call this build script.

Next, make your unit tests execute from the command line. Then add that into your main build.

Keep going like this one step at a time and pretty soon you will have a one step release build.

If you are working on a standard platform, like Java for instance, you can use tools like ant which will integrate with a whole lot of standard build components like JUnit, other ant scripts and the like. Sometimes you are not working on such a system in which case you will need to do a bit more work to integrate everything.

Here is what my build system for Silver Catalyst does:

1. Copies the python application data, template files, javascript files to the appropriate locations
2. Compresses the javascript files to be smaller
3. Compiles the plugin code and copies the compiled plugin code to the plugin folder
4. Copies the default database and license keys
5. Runs the python unit tests and generates a coverage report
6. Runs the javascript unit tests
7. Combines the application module, framework module, embedded server and database into a single executable
8. Runs the install program to package everything into a single installer

It is still not complete. On the To do list:

1. Check out fresh code from the repository at the start
2. Accept the version number as a parameter and replace it in all the documents, code and installer
3. After the build is done, tag the repository

The advantages of a one step release build are

1. Easy to make releases: If you are going to be making frequent releases, then this is very important. You do not want to be spending two days on a release when you are on a two week (10 day) release cycle.
2. Less chance of human error: This is again very common. Before automating the release, there were always situations where you forgot to put in the documentation or missed a configuration file. An automated release eliminates this error.
3. You can make a release from the command line: I’ll talk about continuous integration in a future post where this point will play an important role

Further reading

1. The daily build
2. The ten minute build

This post is part of the Agile in small steps series. For a complete list of posts in this series, click here.

Okay, this is not an agile specific practice, but I had to put it in only because I’m often baffled by the answer. At first I thought this was a dumb question. Every project uses a version control system right? Right? Well, no.

This never fails to surprise me.

One of my friends who used to work in a large, well known, consulting organisation (>100,000 employees, offices in >40 countries) told me that his project didn’t use version control. All files were placed in a shared folder. To “check out”, you would copy the files to your desktop and make changes. To “check in,” you merge your changes back into the shared folder. Merging was often too much of a hassle, so they just copied the files over. If someone else had made a change… well, they can change it again right? To “update”.. well no one ever did an update anyway. What if you mess up? Simple! Everyday, a dated copy of the folder was made as a “backup”.

At first I was surprised and thought that this was an exception, until I found many people who had a habit of backing up their workspace regularly. When I enquired, I found that the companies that they had previously worked in didnt use version control, and they got into the habit of backing up their workspace.

If there is one simple thing that you can do to ease your development work, it has got to be version control. Subversion is pretty good and its free, so there is absolutely no reason not to have version control on your project. If you have multiple developers working on a project, you just have to have some sort of source control. Heck, I often use it even when I’m working alone.

Here is what version control gives you

• Store multiple versions of a file (Not much of a version control system without this…)
• Allow multiple developers to work on the same codebase concurrently
• See who wrote which part of the code
• Branch and merge codelines
• Tag versions so that you can always get back a particular set of files
• Coordinate all the developers
• See changes made between any two revisions and who made the change

Apart from storing your source code, you can also use version control to store a whole lot of other stuff like scripts, configuration files, test plans, test cases, requirements and other documents, third party libraries and so on.
With so many benefits, version control is a must have.

This post is part of the Agile in small steps series. For a complete list of posts in this series, click here.

One of the core principles of agile software development is to get rapid feedback and adjust accordingly. It goes without saying then, that the quicker you can get feedback the faster you can adapt. One of the simplest ways of reducing this time is by unit testing your own code.

Imagine this: You write some code for a few days, then check it into version control. A month later a build goes to QA. A few weeks after that a bug report is filed. Then the bug is triaged and assigned. By the time the bug comes back to you for fixing you’ve forgotten the code you were working on. Familiar?

Had you written unit tests, you would have caught the bug right when you were working on the code.

Writing unit tests have a number of advantages

1. It clarifies the requirements in your mind. You understand the expected outcome better.
2. It reduces the feedback cycle from coding to bug detection
3. It allows you to run through all the tests quickly

The nice thing about unit testing is that you are not dependent on anyone else to adopt this practice. If you are a developer, you can unit test your own code even if no one else in the company is doing it. If you are a project manager, you can adopt unit testing for your team even if none of the other teams are doing it. Most languages have a simple and standard unit testing framework that is easy to use. Learn it and use it.

If you take unit testing a step further and introduce it to the whole team, a few more advantages come up

1. Unit tests act as regression tests and can be run on a regular schedule
2. Because unit tests are automated, you can run it at a much higher frequency than manual tests
3. Unit tests help with refactoring messy code
4. Unit testing can be integrated into a continuous integration system
5. Making code testable improves the design of the code by reducing external dependencies.

So thats agile in small steps #1: Unit testing

There is a lot of literature on the Internet about adopting agile processes for software development. Most of it involves big change like changing the culture of your organisation or reconfiguring the team structure. In order for these changes to be successful, they require solid support from every stakeholder. Executives, team members, even customers have to back the changes.

To get the maximum benefits, it helps when everyone has the agile culture and mindset. But what do you do if you don’t have this support? Are there any steps that you can take to get better?

In this series, I’ll take a look at some ideas and practices that can help you start getting agile without the need for complete organisational support.