Biased to J2EE!

Sunday, July 26, 2009

Continious Integration

This post is a summerized version of Martin's excellant paper on CI which is availableon his homepage. I think it is better to read the whole article but if your time is limited it is sufficient to rely on this post. Mrtin starts the paper with a stroy of his first sightings from a large software company and with challenging their integration method motivates the reader to go through the whole article. The problem in company mentioned by Martin was really tangible for me since we have encountered a light version of it in our Epic Sprint # 4 whcih made us to be here! (is it correct?

). Yeah, the problem is Integration. It is time consuming and takes a few days from our exprienced resources in every sprint. So it is rational to chase a solution to resolve it and make a big improvment in our sprints.

Please note that CI is heavily based on some practices which sholud be considered by all team members not technologies. So it is reachable when these practices are obeyed by al team members and don't expect a wonderfull mechanism is introduced in this post or Martin's article.

In the below after defining the CI motives and concepts I'll list the practices to reach CI and talk about them shortly.

Whta is CI (informal)?

This defintion is from Martin which will be described next:

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage.

An informal description of how setup CI in our team

Suppose you have joined the Orbitall prestigious team. Following our protocol for configuring your system such as IDEs which you will use and where to put everything (tools, repositories, JDK version and so on) you start your work. The first work to do is checking out the SVN trunck for the projects Orbitalints are working and surely you'll get involved in some projects. The projects will be copied on your local workspace as your "working copy". [new] You'll add testing frameworks such as JUnit, DbUnit, or EasyMock based on your needs and type of your work to the project. It is mandatory that your code have enoguh unit tests to make your code self-testing [new]. You will check Jira for your tasks and start one of them. If you intend to use Mylyn the files which you modify or add to project and are out of synchronization with SVN mainline will be denoted by Mylyn. Anyway, doing your task some resources will be modified on your working copy. During completing your task you must write test (no matter after or before coding albeit it would be better to follow TDD approach and write tests before coding). [more richer than previous versions] Also you will write a build file using Ant or Cargo to build your project automatically. Be sure that build file excute all your test cases and passes them during building phase [more richer than previous versions]. Now you are thinking about commiting your works and updating Jira. The next steps are in some way different as we done previously.

Before commiting your work, you must update your projects from SVN to have the latest versions of them since somebody may change them during your task. Potentially you may have some conflicts after updating the projects and it is your responsibilty to resolve them and completely synchronize your working copy with the SVN except your modifications. Please not that you don't commit your changes at this time. After synchronizing your working copy with the SVN again run the your projects' build file to ensure its comptibility with other projects. If all the tests are passed you are authorized to commit your changes. Does commiting changes finish your work ? Without having CI it is ok but with CI definitely it is not finished. [new] At this point you run integration build but this time on the integration machine based on baseline code on SVN. Only when this build (please note this build is different from your project's build due to it is intended to build the whole product from different projects) is succeed on the integration your job is done. The integration build can be executed manually or automatically by a CI Server like Cruise, Hudson, Bamboo and so on.

The bold names are technologies which are used in CI process :

SVN: repository service

Ant, Cargo and etc: build automation tools to to write Project Build and Integration Build

Testing: JUnit, DbUnit, EasyMock and other related testing frameworks

CI Server: which runs integration build.

The following picture shows the role of CI server in CI process.

The steps in a CI scenario will typically go something like this.
1. First, a developer commits code to the version control repository. Meanwhile, the CI server on the integration build machine is polling this repository for changes (e.g., every few minutes).
2. Soon after a commit occurs, the CI server detects that changes have occurred in the version control repository, so the CI server retrieves the latest copy of the code from the repository and then executes a build script, which integrates the software.
3. The CI server generates feedback by e-mailing build results to specified project members.
4. The CI server continues to poll for changes in the version control repository.

An axact scan on the mentioned steps clearly reveals that why we strongly want to commit your code when your code has no problem in integrating with other projects and it is the responsibility of committer to be sure about it.

A one minute thinking about CI process

As you read the above defintion and compare it with our current process for integration you surely find slight differences. Frankely speaking the difference is in slicing a time consuming task [ i.e. traditional integration process] to many short intervals [ i.e. regular builds and commits] and distributing a high load on one person into low load on many persons. We use Ant, Testing frameworks, and SVN in our daily works. The testing is our best practice in Agile and TDD approach and is not belongs to CI process. The CI process only changes the way we employ these technologies and practices. We every day commit our code without considering its side effects on other developers' projects and CI process only modifies the defintion of commitiing to make our code consistent with other one's. The only novice concept is CI server which is nothing special.

So thinking scrutinizingly about beforementioned issues it is easy to understand that CI brings a novel paradigm to our daily works and only shifting a traditional paradigm and following easy practices will significantly improves the quality of our works and speeds our development process.

CI best practices

Martin counts some best practices to accompany CI process in your comapny which are listed below. I refer interested readers to study them with Martin's language albeit the titles are self-describing:

Now, I must do

1. Write unit and integration tests for my code with a good code coverage specially on critical points

2. Write a full automatic build via Ant (and maybe Cargo or other building tools) for my projects which runs tests

3. Update before starting my work and committingthe changes

4. After finsihing my work, build the project and sure all tests are passed

5. I must commit my code when I am sure that the integration build will succeedafter committing the code [by running it on my working copy]

6. Know more about CI servers such as Hudson, Bamboo, Cruise and etc

7. Before final committing check the CI server to know what is it doing?

8. Adjust myself with Agile best practices and force myself to follow them:

- My code msut have unit and integration tests [follow TDD]

- My code msut have automatic build

- Review my code and document it before making my task as done

- if the CI server has not a feedback mechasism I must communicate with other developers regarding changes I make on SVN

- The changes are committed when I am sure the integration build will not fail after submitting the changes made by me on SVN

...

Why Unit Testing?

TDD relies on two main concepts: unit tests and refactoring. Also Unit Tests are one the twelve best practices of XP. But many Agile developers haven't a good sense writing tests and think it is a wasting time practice. As our team has felt the integration as a major problem and time consuming task in each sprint we have decided to set up a Continuous Integration server to speedup our integration. Referring to Matrin's excellent paper on CI the self-testability of build is indicated as an important practice to reach CI. Before reaching CI all of our codes should have Unit and Integration tests to ensure make our build self-testing. Please note that in CI the focus is not on working TDD based approach and it strongly recommend that your code have automated test which could be written before coding as done in TDD or after coding. This post is tightly related to CI and Unit testing frameworks including JUnit, DbUnit, and EasyMock. So it seems necessary to read the post with the sequence of CI, Unit Testing and frameworks. In the reminder I talk about Unit testing and its justification in Agile teams.

Categorizing Tests

Unit Test

Unit tests are intended to examine the behavior of a unit of work. In a Java application UoW is often (but not always) is a method. A unit is the smallest testable part of an application. Each unit test must be independent from other unit tests. How do you know that a method doesn't need a unit test? First, can it be tested by inspection? If the code is simple enough that the developer can just look at it and verify its correctness then it is simple enough to not require a unit test. The developer should know when this is the case. If error handling is performed in a method, then that method can break. Generally, any method that can break is a good candidate for having a unit test, because it may break at some time, and then the unit test will be there to help you fix it.

The danger of not implementing a unit test on every method is that the coverage may be incomplete. Just because we don't test every method explicitly doesn't mean that methods can get away with not being tested. The programmer should know that their unit testing is complete when the unit tests cover at the very least the functional requirements of all the code. The careful programmer will know that their unit testing is complete when they have verified that their unit tests cover every cluster of objects that form their application.

Integration Test: Integration testing(sometimes called Integration and Testing, abbreviated "I&T") is the activity of software testing in which individual software modules are combined and tested as a group. It occurs after unit testingand before system testing. Integration testing is a logical extension of unit testing. In its simplest form, two units that have already been tested are combined into a component and the interface between them is tested. A component, in this sense, refers to an integrated aggregate of more than one unit. In a realistic scenario, many units are combined into components, which are in turn aggregated into even larger parts of the program.

Acceptance Test:

Acceptance tests are created from user stories.During an spirint the user stories selected during the spirint's backlog planning meeting will be translated into acceptance tests. The customerspecifies scenarios to test when a user story has been correctly implemented. A story can have one or many acceptance tests, what ever it takes to ensure the functionality works. Acceptance tests are black box system tests. Each acceptance test represents some expected result from the system. Customers are responsible for verifying the correctness of the acceptance tests and reviewing test scores to decide which failed tests are of highest priority. Acceptance tests are also used as regression tests prior to a production release. A user story is not considered complete until it has passed its acceptance tests. This means that new acceptance tests must be created each iteration or the development team will report zero progress.

Top 12 Reasons to Write Unit Tests

1. Tests Reduce Bugs in New Features

We advocate writing tests as you write new code. Tests do not eliminate bugs, but they dramatically reduce the number of bugs as you add new features.

2. Tests Reduce Bugs in Existing Features

With well-tested code, introducing new features rarely breaks existing functionality. If a new feature breaks existing functionality, existing tests fail immediately, allowing you to pinpoint the problem and fix it. Without the tests, you may introduce a bug that is not found for days or weeks.

3. Tests Are Good Documentation

A concise code example is better than many paragraphs of documentation. We see this time after time in our consulting work. Far too often, teams produce boilerplate documents that are of little practical value. When programmers need to learn an API, they search for code examples. Tests are among the best code examples because they are concise snippets of code that exercise public APIs.

4. Tests Reduce the Cost of Change

Tests make it easier to change software because you have confidence that changes do not break existing functionality. When you have good test coverage, you have confidence to explore new design ideas without fear of introducing new bugs.

Poorly-tested software becomes increasingly expensive to change as time goes on. Risk increases dramatically as the system becomes more complex because it becomes more and more likely that changes inadvertently break things that used to work.

5. Tests Improve Design

Writing tests forces you to make your code testable. Specifically, you tend to rely less on dubious patterns like singletons and global variables, instead making your classes loosely-coupled and easier to use. Code that is tightly-coupled or requires complex initialization is hard to test.

6. Tests Allow Refactoring

With tests, you are more able to change code throughout the lifetime of an application. Tests provide a safety net, allowing you to refactor at any time without fear of breaking existing code, so you can constantly improve the design of your program.

7. Tests Constrain Features

Far too often, programmers build fancy frameworks rather than deliver features customers want. When you adopt a test-first approach, you start by writing tests for the current feature. You then implement the feature. When the tests pass, you know you can stop and move to the next feature. Well-tested applications are more easily extended; therefore, you don't have to anticipate what the customer will eventually request.

8. Tests Defend Against Other Programmers

Textbook code is simple, but real-world problems are hard. We find that in real applications, you often encounter very subtle bugs due to Java bugs, quirky business rules, operating system differences, etc. These bugs may only manifest themselves under very peculiar scenarios.

Let's suppose you find that a payroll calculation routine removes a zero from everyone's salary, but only if the routine runs at 11:59 PM on New Year's Eve. Now, suppose that the bug fix involves a single-line code change.

Without a test, another programmer may come in and change that code. Unless they run the application at 11:59 PM on New Year's Eve, they won't know that they just re-introduced the bug and will cause countless bounced checks next year. With a test, however, you can ensure that when the programmer changes the code, the test breaks and informs the programmer of the problem.

9. Testing Is Fun

If you thrive on challenges, then testing is a lot of fun. Coming up with automated tests is difficult, requiring creative solutions for complex problems. Just like coding is an art, testing is an art.

In many organizations, testing is relegated to the least-experienced programmers. We often encounter the misconception that testing consists of people completing written checklists as they manually execute the application. This approach is completely unscalable, because it takes longer and longer for humans (monkeys?) to test every feature as the application grows.

Modern OO languages like Java are complex, particularly when it comes to dependencies between classes. One change can easily introduce bugs in seemingly unrelated classes. Gone are the days when each character-based screen is a standalone program. OO apps are far more complex and demand automated tests.

Writing automated tests is harder than writing the code itself, in many cases. The most expert programmers are the best testers. When faced with seemingly mundane coding tasks, coming up with creative tests provides an intellectual challenge that expert programmers thrive on.

Beginners typically need expert assistance when writing tests. This is where pair-programming helps, because experts work side-by-side with beginners as they learn the art of testing.

10. Testing Forces You to Slow Down and Think

When adding a new feature or refactoring an existing solution, testing forces you to think about what the code is supposed to accomplish. By writing tests first, you think about how to use the public API and what the ultimate outcome should be. Thus you end up with a clean and simple design that does exactly what you expect it to do.

11. Testing Makes Development Faster

On a class-by-class basis, testing slows you down. It takes time to think about and produce good tests. But as time goes on, your overall velocity increases because you are not constantly worrying about breaking existing code as you add new features.

We have also found that with good tests, we can ignore internal implementation details during the first iteration. Provided that we get the public API right, we can improve internal design and algorithms later, again without fear of breaking something. We have used this specifically for things like sorting algorithms. In the first iteration, we do something quick and dirty, like a bubble sort. We can always come back later and replace it with a better algorithm, if necessary.

12. Tests Reduce Fear

One of the biggest fears that programmers encounter is making a change to a piece of code and not knowing what is going to break. Having a complete test suite allows programmers to remove the fear of making changes or adding new features. We have found that we do not hesitate to change and improve well-tested code, whereas we fear changing untested code.

Saturday, July 25, 2009

Defined Characteristics of a good DSL

All of we have a vague imagination about DSLs and here I am intend to talk about the DSLs with more details.

What is a Domain Spesific Language and its characteristics?

Unlike a general-purpose language like Java, a DSL is fairly limited in scope and capabilities; as the name suggests, DSLs are keenly focused on a certain type of a problem or domain, and on expressing a narrow set of solutions within the context of that limited scope.

Domain-specific languages (DSLs) are "targeted at a particular type of problem." Their syntax is focused on the intended domain or problem. You don't use them for general-purpose programming like you use Java, Groovy, or C++, because DSLs have a very limited scope and capability. A DSL is small, simple, expressive, and focused on a problem area or domain. DSLs have two important characteristics: they're context-driven and fluent.

Context is one of the characteristics of a DSL. As humans, we rely heavily on context when we communicate. We're efficient, and context provides for continuity in our conversations.Context gives meaning to the syntax. Context also influences interpretation. For example, you might interpret the words "use a fork" differently depending on context. Someone could be telling a child to use a fork at the dinner table. You could be being instructed to use the fork function to create a child process on a Unix-like system. Or perhaps someone is being coached on a chess move. When it comes to interpretation, context is very important.

Fluency is another characteristic of a DSL. It helps make code readable and flow naturally. It's not easy to design for fluency, but you should do it so it's easier on your users. Please take a quick look at the following implementation of loop in different languages:

// Java way
for(int i = 0; i < 10; i++)
{
println(i);
}
// Groovy ways
for(i in 0..9) { println i }
0.upto(9) { println it }
10.times { println it }

which one do you think is fluent? It is obvious that Groovy provides fluency for looping. Fluency indicates a very good information processing speed; i.e., very low average time between successively generated messages. While typically applied to language speakers, the term fluent can also apply to a language itself. The fluency of a language can be measured by how quickly and easily it allows users to process information.

When a DSL is considered as a good DSL?

Here simplicity and conciseness are discussed as two charactersitic to judge about goodness of a DSL.

1) Simplicity is critical to the success of a DSL. A person familiar with the language's domain must easily understand it. For example, if you're creating a DSL that actuaries will use to express business rules in the domain of insurance, you don't want them to spend a lot of time learning a difficult and complicated language. You want them to focus on expressing the details associated with insurance risks in a way that they can easily understand, discuss, evolve, and maintain. The DSL you create for them must be built on their vocabulary, the terms they use every day to communicate with their peers. You want them to use the syntax you provide, but it should seem to them that they're merely specifying some discrete rules. And they should be able to do so without getting the impression that they are really programming or even using some kind of a language.

Creating a good DSL is like cooking a nutritious meal; just like you want kids to eat vegetables without realizing and fussing over them, you want clients to use your DSL without worrying about its syntax.

2) Conciseness is another part of writing a good DSL, which means choosing syntax that is both terse and expressive. Terseness within reason makes your code easier to read and maintain. Expressiveness helps to promote communication, understanding, and speed. For instance, for someone who understands matrix multiplication, matrixA.multiply(matrixB); is less expressive and concise than matrixA * matrixB. The former involves calling functions and using parentheses, and includes an intimidating semicolon. The latter is already an expression that will be quite familiar.

Why use a DSL?

Ever you asked this question from yourself that why some people feel flexible working with command than GUI?

A GUI makes an average user productive, but can slow down an expert user or a user very fluent with your application and its domain. You don't want to make the novice productive and slow down an expert in the process. So the GUI is not always the most productive alternative.

The goal of DSLs is similar -- to provide a highly effective interface that allows users to interact with your application. The interface can be graphical or textual.

A DSL is highly expressive, simple, and concise at the same time. This can help the user of an application be more productive. A DSL is designed to be very intuitive and fluent for a domain expert to use (more about this in the second article in this series). It is designed with the convenience and productivity of users -- the domain experts, within the context of the domain -- in mind.

Future of DSL?

Martin Fowlerargues that applications will eventually come to use several small and limited DSLs instead of one big general-purpose language. Ola Biniargues that "three language layers will emerge in large applications": A small, stable layer on which the rest of the application is built; a substantial dynamic layer where most functionality lies; and a third domain layer, built using DSLs.

Friday, July 24, 2009

A good interview: Lessons From a Yahoo Scrum Rollout

Who is interested in Agile and would like to know about successful teams' experience is recommended to read this interview.
The link is:
http://adtmag.com/articles/list/agile-and-scrum.aspx

Thursday, July 23, 2009

XP best practices

Are you committed or involved?

As I mentioned in the previous post Scrum primarily focuses on management issues and in contrary XP mainly focuses on best practices which should be employed by team members to reach Agility goals. The combination is a better solution as Gabrielle Benefield have mentioned in his interview
[http://adtmag.com/articles/2008/01/31/lessons-from-a-yahoo-scrum-rollout.aspx]

"Agile is an umbrella of values and practices that different methods and practices live under. The most popular current is Scrum. Extreme programming is also popular and, when combined with Scrum, makes for a very powerful combination"

So I name this post as XP best practices.

Extreme Programming has been described as having 12 practices, grouped into four areas:

Fine scale feedback

* Pair programming

Pair programming is a development technique in which two programmers work together at one work station. One types code while the other reviews each line of code as it is typed in. The person typing is called the driver. The person reviewing the code is called the observer or navigator. The two programmers switch roles frequently (possibly every 30 minutes).
While reviewing, the observer also considers the strategic direction of the work, coming up with ideas for improvements and likely future problems to address. This frees the driver to focus all of his or her attention on the "tactical" aspects of completing the current task, using the observer as a safety net and guide. Pair programming has its own drawbacks especially for novice developers.

* Planning game

The main planning process within Extreme Programming is called the planning game. The game is a meeting that occurs once per iteration, typically once a week. The planning process is divided into two parts:

Release Planning: This is focused on determining what requirements are included in which near-term releases, and when they should be delivered. The customers and developers are both part of this. Release Planning consists of three phases:
- Exploration Phase: In this phase the customer will provide a short list of high-value requirements for the system. These will be written down on user story cards.
- Commitment Phase: Within the commitment phase business and developers will commit themselves to the functionality that will be included and the date of the next release.
- Steering Phase: In the steering phase the plan can be adjusted, new requirements can be added and/or existing requirements can be changed or removed.

Iteration Planning: This plans the activities and tasks of the developers. In this process the customer is not involved. Iteration Planning also consists of three phases:

Exploration Phase: Within this phase the requirement will be translated to different tasks. The tasks are recorded on task cards.
Commitment Phase: The tasks will be assigned to the programmers and the time it takes to complete will be estimated.
Steering Phase: The tasks are performed and the end result is matched with the original user story

* Test-driven development

Test-driven development (TDD) is a software development technique that uses short development iterations based on pre-written test cases that define desired improvements or new functions. Each iteration produces code necessary to pass that iteration's tests. Finally, the programmer or team refactors the code to accommodate changes. A key TDD concept is that preparing tests before coding facilitates rapid feedback changes. Note that test-driven development is a software development method, not merely a method of testing
The sequence is as follows:
1) Add a test:In test-driven development, each new feature begins with writing a test
2) Run all tests and see if the new one fails
3) Write some code
4)Run the automated tests and see them succeed
5)Refactor code

* Whole team

Within XP, the "customer" is not the one who pays the bill, but the one who really uses the system. XP says that the customer should be on hand at all times and available for questions. For instance, the team developing a financial administration system should include a financial administrator.

Continuous process

* Continuous integration

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage.

* Refactoring or design improvement

Because XP doctrine advocates programming only what is needed today, and implementing it as simply as possible, at times this may result in a system that is stuck. One of the symptoms of this is the need for dual (or multiple) maintenance: functional changes start requiring changes to multiple copies of the same (or similar) code. Another symptom is that changes in one part of the code affect lots of other parts. XP doctrine says that when this occurs, the system is telling you to refactor your code by changing the architecture, making it simpler and more generic.

* Small releases

The delivery of the software is done in predetermined releases (sometimes called 'Builds'). The release plan is determined when initiating the project. Usually each release will carry a small segment of the total software, which can run without depending on components that will be built in the future. The small releases help the customer to gain confidence in the progress of the project. The small releases are only alpha releases and are not intended to go live. This helps maintain the concept of the whole team as the customer can now come up with his suggestions on the project.

Shared understanding

* Coding standards

Coding standard is an agreed upon set of rules that the entire development team agree to adhere to throughout the project. The standard specifies a consistent style and format for source code, within the chosen programming language, as well as various programming constructs and patterns that should be avoided in order to reduce the probability of defects^[2]. The coding standard may be a standard conventions specified by the language vendor (e.g The Code Conventions for the Java Programming Language, recommended by Sun), or custom defined by the development team

* Collective code ownership

Collective code ownership means that everyone is responsible for all the code; this, in turn, means that everybody is allowed to change any part of the code. Pair programming contributes to this practice: by working in different pairs, all the programmers get to see all the parts of the code. A major advantage claimed for collective ownership is that it speeds up the development process, because if an error occurs in the code any programmer may fix it.

By giving every programmer the right to change the code, there is risk of errors being introduced by programmers who think they know what they are doing, but do not foresee certain dependencies. Sufficiently well defined unit tests address this problem: if unforeseen dependencies create errors, then when unit tests are run, they will show failures.

* Simple design

Programmers should take a "simple is best" approach to software design. Whenever a new piece of code is written, the author should ask themselves 'is there a simpler way to introduce the same functionality?'. If the answer is yes, the simpler course should be chosen. Refactoring should also be used, to make complex code simpler.

* System metaphor

The system metaphor is a story that everyone - customers, programmers, and managers - can tell about how the system works. It's a naming concept for classes and methods that should make it easy for a team member to guess the functionality of a particular class/method, from its name only. For example a library system may create loan_records(class) for borrowers(class), and if the item were to become overdue it may perform a make_overdue operation on a catalogue (class). For each class or operation the functionality is obvious to the entire team.

Programmer welfare

* Sustainable pace

The concept is that programmers or software developers should not work more than 40 hour weeks, and if there is overtime one week, that the next week should not include more overtime. Since the development cycles are short cycles of continuous integration, and full development (release) cycles are more frequent, the projects in XP do not follow the typical crunch time that other projects require (requiring overtime).

Also, included in this concept is that people perform best and most creatively if they are rested.

A key enabler to achieve sustainable pace is frequent code-merge and always executable & test covered high quality code. The constant refactoring way of working enforces team members with fresh and alert minds. The intense collaborative way of working within the team drives a need to recharge over weekends.

Well-tested, continuously integrated, frequently deployed code and environments also minimize the frequency of unexpected production problems and outages, and the associated after-hours nights and weekends work that is required.

Coding

* The customer is always available

* Code the Unit test first

* Only one pair integrates code at a time

* Leave Optimization till last

* No Overtime

Testing
* All code must have Unit tests

* All code must pass all Unit tests before it can be released.

* When a Bug is found tests are created

* Acceptance tests are run often and the results

Scrum Teminology

The resources available for Scrum is huge in web. Here I gonna to summarize them.

Agile : an confusing word

Agile is a general term that is used to cover numerous methodologies of software development like DSDM [Dynamic system development], Extreme programming, Scrum etc. Each of these methodologies has its own terminology but all of them have a same philosophy.

Agile project management philosophy, though not very different from the traditional management practices and framework, needs to be rationalized to suit the demands of the agile methodologies. The project management practice remains the same for gathering requirements, planning, initiating and tracking the progress of the project in line with the business vision, but the focus is more on adaptability towards changing requirements, team work, collaboration and the ability to plan and deliver small chunks of usable software in short intervals of time.
So it is noticeable that the general practices of software development process remain same and an Agile team should know how to gather requirements, how to apply patterns, how to design and so on. But team is forced to promise and suppose following materilas:

1. The team should consider that the rate of changes in requirements is high. So no objection is acceptable towards instability of requirements.
changes is a fact not exception

2. The team should have a presentable product every time after after initial development phase of project. Scrum has frequent intermediate deliveries with working functionality, like all other forms of agile software processes. This enables the customer to get working software earlier and enables the project to change its requirements according to changing needs.

3. Customers become a part of the development team (i.e., the customer must be genuinely interested in the output.)

4. Frequent risk and mitigation plans are developed by the development team itself—risk mitigation, monitoring and management (risk analysis) occurs at every stage and with commitment.

5. Frequent stakeholder meetings to monitor progress—balanced dashboard updates (delivery, customer, employee, process, stakeholders)

6. Workplaces and working hours must be energized—"Working more hours" does not necessarily mean "producing more output."

So, how to design a process for software development which satisfy the mentioned requirements?

I am not going to answer this question since the response is clear. Instead I talk about the roles, artifacts in the Scrum. In the following post which be alloted for best practices I will refer to this terms frequently. Please note that Scrum focuses more on management issues and assumes that the best practices of Agile such as Refactoring, Continuous Integration, Test Driven Development, User story writing and so on are indistinguishable parts of team. In other words assuming best practices, Scrum recommends solutions for project managements. Unfortunately some people think that best practices are not important part of Scrum but I would like emphasis that they are preliminary facts which are should be considered before applying Scrum. Following materials are adopted from http://en.wikipedia.org/wiki/Scrum_(development) and other resources from web.

1) Roles

Product Owner: The Product Owner represents the voice of the customer. He/she ensures that the Scrum Team works with the "right things" from a business perspective. The Product Owner writes customer-centric items (typically user stories), prioritizes them and then places them in the product backlog.

ScrumMaster (or Facilitator): Scrum is facilitated by a ScrumMaster, whose primary job is to remove impediments to the ability of the team to deliver the sprint goal. The ScrumMaster is not the leader of the team (as the team is self-organizing) but acts as a buffer between the team and any distracting influences. The ScrumMaster ensures that the Scrum process is used as intended. The ScrumMaster is the enforcer of rules. A key part of the ScrumMaster's role is to protect the team and keep them focused on the tasks in hand. The ScrumMaster is NOT responsible for the transition from traditional methods of working to Scrum or the implementation of Scrum. .

Team: The team has the responsibility to deliver the product. A team is typically made up of 5–9 people with cross-functional skills who do the actual work (design, develop, test, technical communication, etc.).

Stakeholders (customers, vendors)

These are the people who enable the project and for whom the project will produce the agreed-upon benefit[s], which justify its production. They are only directly involved in the process during the sprint reviews.

Managers: People who will set up the environment for the product development organizations.

2) Meetings

Daily Scrum: Each day during the sprint, a project status meeting occurs. This is called a "daily scrum", or "the daily standup".

Scrum of scrums

Each day normally after the daily scrum.

These meetings allow clusters of teams to discuss their work, focusing especially on areas of overlap and integration.
A designated person from each team attends.

The agenda will be the same as the Daily Scrum, plus the following four questions:^[7]

What has your team done since we last met?
What will your team do before we meet again?
Is anything slowing your team down or getting in their way?
Are you about to put something in another team’s way?

Sprint Planning Meeting

At the beginning of the sprint cycle (every 15–30 days), a "Sprint Planning Meeting" is held.

Select what work is to be done
Prepare the Sprint Backlog that details the time it will take to do that work, with the entire team
Identify and communicate how much of the work is likely to be done during the current sprint
Eight hour limit

At the end of a sprint cycle, two meetings are held: the "Sprint Review Meeting" and the "Sprint Retrospective "

Sprint Review Meeting

Review the work that was completed and not completed
Present the completed work to the stakeholders (a.k.a. "the demo")
Incomplete work cannot be demonstrated
Four hour time limit

Sprint Retrospective

All team members reflect on the past sprint.
Make continuous process improvement.
Two main questions are asked in the sprint retrospective: What went well during the sprint? What could be improved in the next sprint?
Three hour time limit

3) Artifacts

Product backlog

The product backlog is a high-level document for the entire project. It contains backlog items: broad descriptions of all required features, wish-list items, etc. prioritized by business value. It is the "What" that will be built. It is open and editable by anyone and contains rough estimates of both business value and development effort. Those estimates help the Product Owner to gauge the timeline and, to a limited extent, priority. For example, if the "add spellcheck" and "add table support" features have the same business value, the one with the smallest development effort will probably have higher priority, because the ROI is higher.

The product backlog is property of the Product Owner. Business value is set by the Product Owner. Development effort is set by the Team.

Sprint backlog

The sprint backlog is a document containing information about how the team is going to implement the features for the upcoming sprint. Features are broken down into tasks; as a best practice tasks are normally estimated between four and sixteen hours of work. With this level of detail the whole team understands exactly what to do, and anyone can potentially pick a task from the list. Tasks on the sprint backlog are never assigned; rather, tasks are signed up for by the team members as needed, according to the set priority and the team member skills.

The sprint backlog is property of the Team. Estimations are set by the Team. Often an according Task Board is used to see and change the state of the tasks of the current sprint, like "to do", "in progress" and "done".

Burn down Chart

The Sprint burn down chart is a publicly displayed chart showing remaining work in the sprint backlog. Updated every day, it gives a simple view of the sprint progress. It also provides quick visualizations for reference. There are also other types of burndown, for example the Release Burndown Chart that shows the amount of work left to complete the target commitment for a Product Release (normally spanning through multiple iterations) and the Alternative Release Burndown Chart, which basically does the same, but allows to show clearly scope changes into a Release Content, by resetting the baseline.

It should not be confused with an earned value chart.

4) Other

Impediment: Anything that prevents a team member from performing work as efficiently as possible.
Sprint: A time period (typically between two weeks and one month) in which development occurs on a set of backlog items that the Team has committed to.
Sashimi: A slice of the whole equivalent in content to all other slices of the whole. For the Daily Scrum, the slice of sashimi is a report that something is done.
Velocity: How much product backlog effort a team can handle in one sprint. This can be estimated by viewing previous sprints, assuming the team composition and sprint duration are kept constant. It can also be established on a sprint-by-sprint basis, using commitment-based planning.
Abnormal Termination: The team can cancel a Sprint if they feel they are unable to meet the Sprint Goal. Management can cancel a Sprint if external circumstances negate the value of the Sprint Goal. If a Sprint is abnormally terminated, the next step is to conduct a new Sprint planning meeting, where the reason for the termination is reviewed.

Agile Methodolgy

The open source technology especially in Java community has many new things to learn or adapt every day doubtlessly.
In these days I am looking forward to compare my experience in Agile methodologies like Scrum with other ones to adjust myself and learn more. Moreever I am interested in knowing more about Agile best practices and their accomplishment obstacles.And here this question brings to mind:
Why Agile methodology fails?
How to build an Agile team and what is its prerequisites?
When can we assert that a team is working in Agile way?
What is the cost of switching to Agile and when is it cost-beneficent?
What is the exact meaning of " our team works with Scrum methodology"?
and many other questions............

I approximately have a 15 months experience in Scrum and have read many posts and books about it but I think in practice building an Agile team is very difficult than it seems at first glance. Comparing Agile with traditional methodologies ( I don't distinguish between words process and methodology in this post since it is not my concern here and I treat them in a same way (;- ) theoretically reveals that Agile based working is simple and easy to do but practically it is wrong. I think what makes Agile difficult to apply is its main philosophy: Agile is people oriented and traditional methods are process oriented and it is clear that working with peoples is the big concern of management during its whole life. Most of teams who switch to Agile think that bringing some terminologies ( such as Agile, XP, best pracrice, TDD, Continuous Integration, Daily scrum, burning chart and so on) and some technologies (such as Jira, Wiki, CI server, and etc) make their team Agile and here is the problem occurs. I think Agile is thought that should be injected to team members and believed by whole team not a determined framework for doing task.

Forcing team members to behave in a determined and specified framework as it is done in process-oriented methodologies is very simple than changing the way team thinks!. The former only changes the way people work but the later directly involved with thoughts of people and it is hard to change them. This is the reason why Agilitians strongly emphasize that employ Agile method in a team with committed people not involved.

In the next post I will define the Agile and its best practices. Then in different posts for each practice I'll talk scrutinizingly.

Monday, June 22, 2009

What’s a Code Kata?

These materials are extracted from following blog:

http://codekata.pragprog.com/

Suppose that you are interested in learning a new programming language. Surely you don't have a teacher to be trained by . It is evident reading a book on the topic makes you familiar with the concepts but you intended to be a good developer in the language. Which method you like to employ to reach your goal.

As a group, software developers don’t practice enough. Most of our learning takes place on the job, which means that most of our mistakes get made there as well. Other creative professions practice: artists carry a sketchpad, musicians play technical pieces, poets constantly rewrite works. In karate, where the aim is to learn to spar or fight, most of a student’s time is spent learning and refining basic moves. The more formal of these exercises are called kata.

To help developers get the same benefits from practicing, we’re putting together a series of code kata: simple, artificial exercises which let us experiment and learn without the pressure of a production environment. Our suggestions for doing the kata are:

find a place and time where you won’t be interrupted
focus on the essential elements of the kata
remember to look for feedback for every major decision
if it helps, keep a journal of your progress
have discussion groups with other developers, but try to have completed the kata first

There are no right or wrong answers in these kata: the benefit comes from the process, not from the result.

Some of the Kata's are listed below:

My goal is to implement these Katas in Groovy.

Groovy Programming Language

I have just started to learn Groovy programming language.
I 'll follow the kata method to sure going in the right way.