For Balance, Create Your Ideal Metrics Scorecard

by Vince Kellen, Senior Consultant, Cutter Consortium

IT shops generate oodles and oodles of metrics. IT hardware and software can spit out hundreds of operational metrics, mostly to monitor system performance and diagnose system errors. On the hardware side, routers, switches, and data devices produce metrics that let operators infer the volume and kind of usage patterns at work. On the software side, database systems, application servers, middleware servers, Web servers, Web services, and applications themselves produce a host of measures, too.

IT shops with project management offices collect plenty of data related to the size, scope, and progress made on projects, including the number of person-hours on tasks within a project. IT shops that maintain application and hardware portfolios usually produce metrics concerning the costs and benefits for IT investments. Security and risk managers produce metrics concerning the kinds of risks experienced or anticipated. IT contact centers keep metrics on the number of issues reported, how quickly they are resolved, and whether they are resolved within expected time frames. IT shops performing benchmarking reviews collect all kinds of data, including key budget, budget to revenue, or company size ratios.

The problem in IT isn't whether we can measure things but how many ways can we measure the same thing. I have worked with IT shops that maintain a portfolio of hundreds, if not thousands, of daily, weekly, and monthly metrics. Given the plethora of metrics available, what should an IT metrics scorecard look like? How many metrics should it contain? What kinds of metrics should be in it? Should there be one scorecard or many, depending on the level and area within IT? What metrics, beyond those that may be mandated, should be reported to internal business clients and externally to government, audit, or other interested parties?

Most IT shops that have a set of IT metrics have seen them evolve happenstance over time. As an IT shop matures, it may move from ad hoc measurement to the first attempts at pulling metrics together to monitor system reliability and project performance. Over time, the IT shop may grow into economic metrics that look at the returns on IT's hardware and software assets and human capital. IT shops involved in benchmarking may have competitive IT metrics that compare themselves with competitor IT shops, most likely in the area of budget and capital investment performance. IT shops within firms that have strategic scorecard programs in place (e.g., Balanced Scorecards) may have some IT metrics linked directly to the company's strategy.

The use of IT metrics tends to mirror the maturity of the organization and the history of that organization's varied interests over time. While this is necessary and useful, I also recommend companies look at some overarching metric principles that may provide some assurances that the IT metrics in place are reasonably suitable.

To help combat metrics glut, CIOs should consider establishing an IT metrics philosophy that can distinguish between what metrics the CIO needs to manage in a more deliberate and effortful way and what metrics can be managed through more automatic means, using less effort. Drawing the right balance between these two classes of metrics can be the difference between becoming a more agile IT unit versus one gripped in the paralysis of metrics glut.

I welcome your comments on this Advisor and encourage you to send your insights to vkellen@cutter.com.

-- Vince Kellen, Senior Consultant, Cutter Consortium