Special Offer from Cutter Consortium

How Agile Projects Measure Up, and What This Means to You

by Michael Mah, Senior Consultant, Cutter Consortium; with contribution by Mike Lunt

Are agile projects more successful in the sense that they deliver more functionality with fewer defects, in the words of XP visionary and Cutter Senior Consultant Kent Beck? Specifically, how do agile projects compare to waterfall or plan-based projects? If your company is considering switching to agile, what might you expect? What have other companies experienced? Managers and decision makers want to know the answers to these questions as they decide what the best strategy might be for their organization, whether it's colocating a team of paired programmers in one large room using XP or employing a Scrum approach with team members spread across multiple locations. There is no "one size fits all" approach to agile. At the same time, different variants of agile are likely to yield different outcomes.

The accompanying Executive Report reveals answers that we found when examining more than 20 agile projects across five organizations. What we discovered may surprise you. To illustrate what we mean by that, the report looks in close detail at two of the five companies, each implementing agile in a different way. You'll see what they achieved with regard to speed, effort, delivered functionality, and quality levels. These case studies are very compelling, since they depict actual projects at real companies while describing how they achieved the results that were measured. What this report also does is to remove the mystery around how to measure and benchmark projects, whether they are agile, waterfall, or any other schema for software development. It shows techniques that you might consider for your organization and how best to communicate results to team members and decision makers.

AGILE CLAIMS A BETTER WAY THAN THE OLD WAY

With the release of the Agile Manifesto in 2001, a new movement was born. What was interesting about this declaration were the values that the authors expressed in shaping how they envisioned the creation of computer software. Cutter Fellow Jim Highsmith describes that the word "uncovering" was selected for use in the manifesto to convey that the authors didn't necessarily have all the answers. What they did know was that they wanted to "do software differently," and in so doing, show others how to do it as well. The question was whether "doing it differently" would give their sponsors, clients, and organizations -- including senior management and decision makers -- what they wanted: more functionality with higher quality. If that happened, could you prove it? How would you show it?

ENTER SOFTWARE MEASUREMENT

Until recently, metrics had a bad name in the agile community. All the "religious wars" about lines of code, function points, and the like smelled of waterfall and bureaucracy and not about "real work," which was about creating software. But you need measures to share real-world outcomes and to communicate results to the world, outside of developers, programmers, and testers.

Well, it turns out that agile projects have data that can demonstrate these outcomes. More important, some agile teams are even better at keeping reliable metrics than their waterfall counterparts. The report shares the kind of measures tracked by these best-in-class organizations, including duration, effort, size (in stories, story points, and code) and defects. As to "industry norms," we're fortunate to have a large, modern, updated, and diverse database of more than 7,500 projects collected worldwide to provide some context to the agile productivity measures.1

WHAT AGILE PROJECTS LOOK LIKE

We go behind the scenes at two leading-edge companies. One is the Follett Software Company based in McHenry, Illinois, USA -- a company that implemented a full-scale XP approach: one large room with paired programmers, colocated client "proxies," all working side by side. On the other end of the spectrum is BMC Software in Austin, Texas, that used teams three times larger than Follett, distributed across four cities and multiple time zones, employing a Scrum methodology.

WHAT THE BENCHMARK TRENDS SHOW

Do agile projects produce more functionality? Do agile projects exhibit fewer defects? The answer is yes -- sometimes. It's a very strong "yes" for the two "poster children" of this study -- Follett and BMC Software. The verdict is mixed when we look at the other three companies in the sample. One thing seems to leap out: Follett and BMC each had a two-to-three-year head start on the others. That tells us this: the sooner you get started, the sooner you'll see faster schedules and fewer defects, which can give you a potent lead over your competitors.

AGILE METRICS CHALLENGES

The most mature agile organizations are highly disciplined in knowing exactly where their agile projects are at all times, including a "release retrospective," which aggregates the retrospectives of the individual project iterations and sprints. Some do this well; others less well. In the report, we describe how measurement practices around agile work better than others. Clarity is the name of the game. When decision makers know what is working, they'll throw money at a successful team to achieve more of a good thing. If things aren't working as well, it's our opinion that reliable measures are just what you need to fix things very quickly.

CLOSING THOUGHTS: WHY AGILE WORKS

We close by interpreting the measures and correlating them to agile practices. There seems to be a strong and compelling reason that agile works as well as it does when key ingredients are there. It has to do with the nature of collaboration and the "mind blending" of smart people. Put another way, it's about how software development is about manifesting the logic and thinking of talented engineers and elegantly infusing this logic and knowledge into the "dumb machine" that is hardware. Without software, a box is just a box and a network is just an information pipeline without anything flowing through it. Creating great software is about taking expert thinking and domain knowledge, then moving that information around team members rapidly and efficiently to create software that makes machines do what they do in today's modern Information Age.

ENDNOTE

1 This data comes from Quantitative Software Management (QSM) Associates, and it's the only metrics repository of its kind, with statistics from more than 500 organizations in over 18 countries. This research is collected and updated each year by QSM and is used for benchmarking engagements on behalf of clients at Cutter Consortium.

SPECIAL OFFER

Download your free copy of this Cutter Consortium Executive Report, compliments of Cutter Consortium.

To receive your free copy, simply enter the promotion code MEASUREUP when you click here and fill out our special offer form.

Share