E-MailRETHINKING SUCCESS AND FAILURE IN IT
by Helen Pukszta, Senior Consultant, Cutter Consortium
I have never found the grim statistics on IT project failures particularly illuminating or convincing. These statistics and their conclusions -- basically, that IT departments are consistently failing at IT and have to try harder, again -- show up periodically from research companies and spur a flurry of funereal and pontificating commentary within the IT community.
The main reason for my skepticism is that I find the implied definitions of success and failure to be deficient: they are, by and large, based on whether a project is delivered on time, within budget, and with functionality as originally specified. Meeting all three criteria is unquestionably a desired outcome on any project but, by themselves, they do not define an organization's success with IT and therefore should not define the success of the IT department.
I recently asked a colleague whether he would prefer to deliver a project somewhat late and overbudget but rich with business benefits or one that is on time and underbudget but of scant value to the business. He thought it was a tough call, and then went for the on-time scenario. Delivering on time and within budget is part of his IT department's performance metrics. Chasing after the elusive business value, over which he thought he had little control anyway, is not.
ARE WE REALLY FAILING?
One organization of which I am familiar attempted to implement the same enterprise resource planning (ERP) package three times because business managers insisted on altering the vendor's software to fit the existing processes. They changed their minds the third time around. Was it a success? If success was measured by whether the business managers' original expectations were met, then it was a failure. If it was measured by the business impact the project finally produced, it was a success -- although the cost could have been much lower had someone decided sooner not to meet users' expectations.
Another organization cancelled a perfectly good sales management project because it decided to implement a customer relationship management (CRM) suite that would make the project redundant. Was the sales management project just another IT failure because it was cancelled and written off as sunk cost? No, its timing was wrong. The decision to start it was a good one, and so was the decision to cancel it.
Delivering a project that is late, overbudget, and with scaled-down functionality does not automatically portend failure; in fact, it could represent the best of all possible outcomes. I will venture a guess that the most successful IT projects -- those whose ratio of benefits to costs is the highest -- significantly deviate from original plans. This is because producing high business value typically implies a high degree of uncertainty, and that, in turn, significantly frustrates estimating efforts.
If we were to agree that late, overbudget, and scaled-down projects are, basically, failures, why do businesses continue to invest in IT, despite the apparently dismal rate of success? Because there is value and opportunity there. Directly measuring that value is notoriously challenging. One reason it is so difficult is that it takes time for benefits to materialize and for businesses to recognize them. IT investments transform businesses in a variety of ways, and it takes time for organizations to adapt to changes. Despite all the successful tests and preparations, the reality is that no one really knows how the new system will work out in practice and over time.
Interestingly, the rate of US productivity remained surprisingly high through 2004, despite the economic downturn of 2001 and the low rate of capital expenditures after September 11. One possible explanation is that in recent years, businesses were still digesting and producing benefits from IT investments made before 2001 (investments that in the gloomy days of the recession had been declared wasteful IT overspending). The year 2005, on the other hand, saw a significant decline in productivity. Though some of this decline can be explained by the natural disasters of that year, it could also be that businesses' IT-based investments from prior years are finally running out of steam. If that is the case, we should soon be seeing another cycle of business improvement and innovation enabled by IT.
IT innovation in the marketplace ensures that there will be new applications and technologies for businesses to consider. Steady-state IT is just not on the horizon. You could imagine instituting a rule under which businesses would not be allowed to invest in a technology until it was stable, its adoption safe and predictable, and its reliability taken for granted (sort of like a utility). However, if there were enough business opportunity in that technology, someone, eventually, would separate from the pack and go after it -- difficulties in estimating projects and keeping to schedules notwithstanding. Such is the nature of competitive markets.
BETTER WAY TO SUCCESS
We do need managers who are driven by schedules and budgets; little would get accomplished without them. But we also need managers in charge of projects who are value watchdogs and who understand that the drive to meet target schedules, budgets, and user expectations should not take precedence over the drive to maximize business value. Measures such as on-time/on-budget/in-scope statistics are only as good as their ability to approximate that business value. If tradeoffs between keeping on schedule and producing additional benefits appear, as they often do, they have to be recognized and handled appropriately.
If we are failing with IT, it is in ways much different from schedule and budget overruns. A preoccupation with timelines and budgets penalizes deviations that could prove more beneficial to the business than staying the course. Obsessed with meeting deadlines and in the name of success, IT organizations will avoid projects with significant business payoff when there's high uncertainty. They will pursue what is manageable and predictable rather than what is challenging or difficult to estimate yet has significant business potential. They will protect themselves with the aegis of project metrics and signoffs. They will avoid failure. By doing so, they will help the business avoid innovation through IT and the potential for success that can come with it.
Typical investments in IT are akin to running risky and expensive experiments. One way to lower the risk is to conduct much of the experimenting before fully embarking on the initiative and before committing significant resources. The idea is to extract much of the uncertainty up front and deal with it while the stakes are still small. Limiting an organization's exposure to large-scale IT experimentation requires running pilots, proofs-of-concept, and simulations that simultaneously address a multitude of project dimensions, including technology, process, and organization. Agile methods are ideal for this purpose.
There is a cost associated with such pre-project experimentation. For example, business resources have to be made available on a regular basis. Even in aggregate, however, the cost should be much lower than that of blindly marching into large projects and failing to produce meaningful results. Trying out a new application in the business context will not eliminate all of the project's uncertainty, but it can uncover major difficulties as well as opportunities before making hard-to-reverse decisions. These experiments should inform business cases, and they may point to new ways of using IT that are not visible under the traditional one-large-project-at-a-time approach. They will also improve project estimates and thus our success rates under on-time/on-budget metrics.
CONCLUSION
Incompetence or malpractice should never be tolerated in IT project management, and there is always room for improvement. However, there is more to success in IT investment and management than watching schedules and budgets. IT departments should define their success first by the value they provide to the business and only then by the accuracy of their estimates and ability to steer user expectations. Of course, making sure that such value is visible and recognized throughout the organization presents another challenge.