Sunday, July 26, 2009

Continious Integration

This post is a summerized version of Martin's excellant paper on CI which is availableon his homepage. I think it is better to read the whole article but if your time is limited it is sufficient to rely on this post. Mrtin starts the paper with a stroy of his first sightings from a large software company and with challenging their integration method motivates the reader to go through the whole article. The problem in company mentioned by Martin was really tangible for me since we have encountered a light version of it in our Epic Sprint # 4 whcih made us to be here! (is it correct? ). Yeah, the problem is Integration. It is time consuming and takes a few days from our exprienced resources in every sprint. So it is rational to chase a solution to resolve it and make a big improvment in our sprints.

Please note that CI is heavily based on some practices which sholud be considered by all team members not technologies. So it is reachable when these practices are obeyed by al team members and don't expect a wonderfull mechanism is introduced in this post or Martin's article.

In the below after defining the CI motives and concepts I'll list the practices to reach CI and talk about them shortly.

Whta is CI (informal)?

This defintion is from Martin which will be described next:

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage.

An informal description of how setup CI in our team

Suppose you have joined the Orbitall prestigious team. Following our protocol for configuring your system such as IDEs which you will use and where to put everything (tools, repositories, JDK version and so on) you start your work. The first work to do is checking out the SVN trunck for the projects Orbitalints are working and surely you'll get involved in some projects. The projects will be copied on your local workspace as your "working copy". [new] You'll add testing frameworks such as JUnit, DbUnit, or EasyMock based on your needs and type of your work to the project. It is mandatory that your code have enoguh unit tests to make your code self-testing [new]. You will check Jira for your tasks and start one of them. If you intend to use Mylyn the files which you modify or add to project and are out of synchronization with SVN mainline will be denoted by Mylyn. Anyway, doing your task some resources will be modified on your working copy. During completing your task you must write test (no matter after or before coding albeit it would be better to follow TDD approach and write tests before coding). [more richer than previous versions] Also you will write a build file using Ant or Cargo to build your project automatically. Be sure that build file excute all your test cases and passes them during building phase [more richer than previous versions]. Now you are thinking about commiting your works and updating Jira. The next steps are in some way different as we done previously.

Before commiting your work, you must update your projects from SVN to have the latest versions of them since somebody may change them during your task. Potentially you may have some conflicts after updating the projects and it is your responsibilty to resolve them and completely synchronize your working copy with the SVN except your modifications. Please not that you don't commit your changes at this time. After synchronizing your working copy with the SVN again run the your projects' build file to ensure its comptibility with other projects. If all the tests are passed you are authorized to commit your changes. Does commiting changes finish your work ? Without having CI it is ok but with CI definitely it is not finished. [new] At this point you run integration build but this time on the integration machine based on baseline code on SVN. Only when this build (please note this build is different from your project's build due to it is intended to build the whole product from different projects) is succeed on the integration your job is done. The integration build can be executed manually or automatically by a CI Server like Cruise, Hudson, Bamboo and so on.

The bold names are technologies which are used in CI process :

SVN: repository service

Ant, Cargo and etc: build automation tools to to write Project Build and Integration Build

Testing: JUnit, DbUnit, EasyMock and other related testing frameworks

CI Server: which runs integration build.

The following picture shows the role of CI server in CI process.


The steps in a CI scenario will typically go something like this.
1. First, a developer commits code to the version control repository. Meanwhile, the CI server on the integration build machine is polling this repository for changes (e.g., every few minutes).
2. Soon after a commit occurs, the CI server detects that changes have occurred in the version control repository, so the CI server retrieves the latest copy of the code from the repository and then executes a build script, which integrates the software.
3. The CI server generates feedback by e-mailing build results to specified project members.
4. The CI server continues to poll for changes in the version control repository.

An axact scan on the mentioned steps clearly reveals that why we strongly want to commit your code when your code has no problem in integrating with other projects and it is the responsibility of committer to be sure about it.

A one minute thinking about CI process

As you read the above defintion and compare it with our current process for integration you surely find slight differences. Frankely speaking the difference is in slicing a time consuming task [ i.e. traditional integration process] to many short intervals [ i.e. regular builds and commits] and distributing a high load on one person into low load on many persons. We use Ant, Testing frameworks, and SVN in our daily works. The testing is our best practice in Agile and TDD approach and is not belongs to CI process. The CI process only changes the way we employ these technologies and practices. We every day commit our code without considering its side effects on other developers' projects and CI process only modifies the defintion of commitiing to make our code consistent with other one's. The only novice concept is CI server which is nothing special.

So thinking scrutinizingly about beforementioned issues it is easy to understand that CI brings a novel paradigm to our daily works and only shifting a traditional paradigm and following easy practices will significantly improves the quality of our works and speeds our development process.

CI best practices

Martin counts some best practices to accompany CI process in your comapny which are listed below. I refer interested readers to study them with Martin's language albeit the titles are self-describing:

Now, I must do

1. Write unit and integration tests for my code with a good code coverage specially on critical points

2. Write a full automatic build via Ant (and maybe Cargo or other building tools) for my projects which runs tests

3. Update before starting my work and committingthe changes

4. After finsihing my work, build the project and sure all tests are passed

5. I must commit my code when I am sure that the integration build will succeedafter committing the code [by running it on my working copy]

6. Know more about CI servers such as Hudson, Bamboo, Cruise and etc

7. Before final committing check the CI server to know what is it doing?

8. Adjust myself with Agile best practices and force myself to follow them:

- My code msut have unit and integration tests [follow TDD]

- My code msut have automatic build

- Review my code and document it before making my task as done

- if the CI server has not a feedback mechasism I must communicate with other developers regarding changes I make on SVN

- The changes are committed when I am sure the integration build will not fail after submitting the changes made by me on SVN

...

No comments: