USING BENCHMARK FINDINGS WISELY

6 November 2001

by Michael Mah

The goal of generating benchmark findings should be to answer some clear-cut questions (such as the ones below). These questions should provide a road map for future updates to the baseline:

• How productive is the organization on its most critical projects?
  
  
• What are the findings across different areas of our business?
  
  
• Is IT application development and maintenance productivity increasing or decreasing, and at what rate?
  
  
• Are schedules getting shorter?
  
  
• Are effort and cost decreasing?
  
  
• Is reliability improving?
  
  
• How do we compare to others in the industry?
  
  

When drawing conclusions from the data, it's very important to be sensitive to careful use of language, avoiding words implying judgment or evaluation. For example, rather than saying effort metrics are getting "better" or "worse," say that effort metrics are moving "higher" or "lower." The truth is, it's difficult to make accurate judgments at first glance. The language of evaluation puts people on the defensive, causing them to worry about their reputation rather than focusing on the data and conducting a thorough analysis to understand the causes of IT project behavior.

Because of the intense pressure IT organizations are under, people are at risk of feeling harshly judged. That's not the point of benchmarking. If people get the wrong idea, it can lead to all sorts of problems, from coverups to quiet sabotage of the metrics program. Difficult conversations arise when facts are disputed because strong emotions come into play, especially if identity issues or feelings of competence are at stake.

Focusing on process and framing the results into lessons learned enables everyone to explore solutions to the problems they share. Everyone is in this thing together, and good results can be achieved with cooperation. Too often, if the numbers are not what people were hoping for, the knee-jerk reaction is to focus on who's at fault. It's important to be sensitive to this fact and present the results wisely.

An organization trying to raise itself to the next level of productivity must master not only the mechanics of acquiring knowledge about its performance, but also the ways management and staff react and respond to the information that's revealed as part of the measurement process. If you're successful at handling these people issues, you can play a vital role in elevating your organization's capabilities and improving how people interact with one another, resulting in a happier work environment.

-- Michael Mah, Senior Consultant, Cutter Consortium

[For more on metrics and the elements of negotiation, see the August 2001 issue of Cutter Benchmark Review (formerly IT Metrics Strategies), available from Cutter Information Corp. at +1 800 964 5118 or +1 781 648 8700, fax +1 800 888 1816 or +1 781 648 1950, or e-mail service@cutter.com.]