SO WHAT IS THE STATE OF SOFTWARE ESTIMATION?

by E.M. Bennatan

An analysis of the data collected in a recent Cutter Consortium survey on the current state of software project estimation reveals neither a catastrophe nor a triumph over the proverbial dragon. However, the findings indicate that software developers are not particularly good at estimating. Why are almost 40% of companies reporting poor performance? The software estimation problem has received more than its fair share of attention for over two decades with some of the best minds devoting themselves to the problem.[1] Many tools and techniques have been developed, and they have cultivated high expectations for improvements in the delivery of software on time and within budget.

Today, we know that software estimation is one of the most difficult parts of the software development process. There are new schools of thought in the software engineering community that claim that software estimation is intrinsically limited in its ability to meet business expectations (see J.P. Lewis, "Large Limits to Software Estimation," ACM Software Engineering Notes, Vol. 26, No. 4, July 2001, pp. 54-59). They claim that program size, development time, and productivity cannot be objectively estimated. To correctly put our discussion in context, we adopted a ±10% success definition, and we assumed that the objective of any estimation methodology is to optimize the process to a reasonably achievable degree (simply put, the goal of estimation methods is to help project managers estimate as best they can).

The findings on the size of overruns were quite dramatic. Why are 21% of companies experiencing schedule overruns exceeding one year, and why are 11% overrunning their budget by more than US $1 million? One of the problems is that software estimation is not entirely in the hands of the developers; it is heavily influenced by behavioral and political factors. Software management consultant Capers Jones writes that most software projects still tend to run late because of arbitrary estimate overruling by customers and senior executives, creeping requirements, and inadequate early quality control ("Software Cost Estimation in 2002," CrossTalk: The Journal of Defense Software Engineering, June 2002, p. 4). Jones' three factors are heavily influenced by the politics and relationships between developers, customers, and senior management. Clearly, an attempt needs to be made to bring customers, users, and senior management closer to the development process, and this is, in fact, one of the characteristics of modern agile development methodologies.

Cutter Business Technology Council Fellow Jim Highsmith makes an interesting point in an E-Mail Advisor on project estimating ("Project Estimating," Cutter Consortium Agile Project Management E-Mail Advisor, 2 May 2002). Highsmith discusses the difference between senior management giving a project manager a completion date and asking when a project will be completed. He concludes, "Maybe it's not our estimating skills that need upgrading, but our negotiating skills."

We all remember the scenario from our childhood days when we would go into the neighborhood candy store and ask: "What can I get for a quarter?" We were negotiating the purchase so it would fit into our predetermined budget. This is different from our behavior as adults when we place items in our supermarket basket and then wait for the checkout person to inform us how much we need to pay for the items we chose. The problem arises when we do not have enough money to pay for the items in our basket (or when we simply do not want to spend that amount). In situations like these, we can negotiate a number of options: we can go to another store that is cheaper, we can put some of the items back, or we can do without the items.

How familiar! In software, we can negotiate the completion date and the development resources, we can negotiate functionality, or we can cancel (or abandon) the project. We can also search for ways to make our software development process and methodology more efficient and hence less costly (similar to being more efficient in the way we shop).

The negotiation of the completion date, the functionality, and the development budget is often an ongoing process.

-- E.M. Bennatan, Senior Consultant, Cutter Consortium

[1] Barry Boehm first introduced his Constructive Cost Model (COCOMO) back in 1981.