"Dashboards have not been invented to be mere data displays; their mission should be to help users make better decisions and achieve their goals."

Opening Statement

IT organizations worldwide use dashboards to provide managers with the key information they need to steer their organizations in the right direction and make important strategic business decisions. Managers must be able to understand at a glance the information presented in the dashboard and to take effective corrective actions if needed. The success of this process strongly depends on providing managers with properly designed dashboards. Indeed, a poorly designed dashboard can be confusing and may even convey misinformation.

While there are guidelines for designing dashboards, the available dashboard examples demonstrate that practitioners do not always agree on a specific design, which naturally leads to different results. This means that we do not yet have a clear definition of "properly designed" in this context.

To design an effective dashboard, there are many challenges we have to address. First, the data we measure must be meaningful if the dashboard is to have any value. We can waste considerable effort and resources tracking the wrong information. Second, all the information has to be organized to fit one screen. Thus, we must select the most effective visualizations for the data in question. Third, we need to regularly review dashboards to ensure they incorporate data from all relevant sources and show useful and up-to-date information.

In this edition of Cutter IT Journal, we will focus on the selection of the metrics that organizations should include in their dashboards to indicate how the business is performing. Moreover, we will learn best practices and guidelines for showing the information on the screen and the main requirements to keep in mind when designing dashboards. We will consider different contexts for dashboards, such as development teams and global enterprises, and we will see how different the requirements for a dashboard can be depending on their context of application.

In this issue, our authors explore a number of approaches and solutions. They come from a variety of areas and experiences -- including academia, consulting, and corporate environments -- but they share some common themes. All of them agree on the need to choose carefully both the data to be displayed and the type of visualization to be used. However, they acknowledge that there is no magic formula for doing that. We need training and experience, guidelines and examples. That's why each article offers practical applications, examples, and guidelines -- not merely theoretical discussion. By the end, you will take away some action steps you can use in your own organization.

NO DASHBOARD WILL REPLACE PROJECT MANAGEMENT SKILLS

A dashboard is a supplement and helps managers to focus their attention. It is not a substitute for managers' intuition and skills. In our first article, IT management consultant Paul Clermont states the mandatory condition for a good dashboard to succeed: a good manager. Assuming this condition to be satisfied in our organizations, how we can avoid designing dashboards that mislead good managers? Clermont suggests starting from the following three key questions: what are you measuring, why, and for whom? While finding your answers, keep in mind that dashboards must measure "what counts," and they must measure it well enough that managers can trust them to focus their questions and guide their actions. If you can answer these questions, then your dashboard will have a strong foundation. Clermont provides typical dashboard examples from IT and a list of problems inherent in measurement, along with possible solutions to those problems.

SHOW AND FORECAST (IN A TIMELY MANNER) ONLY WHAT THE MANAGER NEEDS

In our next article, Cutter Fellow Robert N. Charette asks "why a reasonably planned IT project using a dashboard would fail." He finds only one reasonable answer: the IT project dashboard doesn't provide meaningful information to the manager responsible for the project.

In his article, Charette provides a definition of "meaningful IT dashboard information" using three characteristics:

1. Dashboards should be as timely as possible and provide meaningful insights into future project deviation possibilities.

2. Dashboards should provide predictions about what information is expected at the end of the next review period, so the manager can compare expected and actual results. Making discrepancies between project perception and project reality visible enables the project manager to take more timely corrective action.

3. The information being displayed should represent the decision-driven information needs of the project manager.

If project managers are provided this meaningful information, Charette notes, "maybe, just maybe, there will be a few more IT project successes than there might have been otherwise."

DESIGNING EFFECTIVE DASHBOARDS

Dashboards have not been invented to be mere data displays; their mission should be to help users make better decisions and achieve their goals. In our next article, Andrea Janes, Alberto Sillitti, and Giancarlo Succi of the Free University of Bozen-Bolzano describe the results of their experience designing a dashboard for a software development team. The proposed dashboard was developed with a focus on two main aspects: selecting the "right" data and choosing the "right" visualization techniques. The authors discuss their approaches to these challenges so that the reader can apply them as practical solutions to the biggest issues related to dashboard design. Janes and his coauthors have developed their own model for choosing the "right" data: a GQM+Strategies model that documents measures together with the reasons why the data is being collected. To choose the most effective visualizations, the authors provide some guidelines for obtaining visualizations that minimize the time needed to understand the information that has to be communicated.

ABANDONING PERFORMANCE MEASURES: A RADICAL TREATMENT

OK, now we get it. Measures cannot be a random collection. We need rigor and expertise. For many years, our next author, David Parmenter, has been advocating the proper use of performance measures. Now he is convinced that, in many cases, a radical treatment is necessary: abandoning performance measures (and dashboards). Why? Because, he argues, "the greatest danger of performance management is dysfunctional behavior," and an organization with dysfunctional performance measures would function much better without them. Does your organization need this radical treatment? You can find out by simply using Parmenter's checklist for assessing the damage poorly designed performance measures may be causing in your organization.

If you need the radical treatment, well, this is what you should do: stop monitoring or reporting performance measures for, say, three months. During this time frame, management should find out which measures they have missed. At that point, Parmenter says the organization can gradually begin reintroducing measures -- only the necessary ones! -- to the dashboard. He concludes with further action steps you can take to instill "some intellectual rigor into your performance measurement process."

GLOBAL INFORMATION SQUEEZED ONTO A SCREEN

In today's global economy, global enterprises operate across more than one geography. As our next author, TCS's Ravi Tej Kanteti, notes, this means that they must be able to "handle different cultures, laws, languages, and timelines." Moreover, these enterprises have a distinctive structure in their IT departments: besides the global CIO, multiple local CIOs are needed. What are the challenges in designing dashboards for such global enterprises? In his article, Kanteti presents the parameters that global and local CIOs typically need to monitor. Finally, the author suggests a framework and processes for building -- and maintaining -- these dashboards over the long run.

A DASHBOARD IS NOT THE HOLY GRAIL

In our final article, Lawrence Fitzpatrick of Computech argues that "traditional PPM dashboards produce unreliable information, at high cost, frustrating CIOs and project managers alike." The solution Fitzpatrick proposes is to build an innovative PPM dashboard that considers people, process, and tools and creates a protocol for developing project management skill. Three elements are required to implement this new protocol:

1. A minimalist project management framework

2. A dashboard that automatically provides reliable, up-to-date information about the project manager's -- and the project's -- performance

3. A group of experts who evaluate project management skill through the PPM dashboard and mentor fledgling project managers

In this context, the dashboard must be designed to show data useful for both assessing the skill with which the work is performed and for communicating the work that is being done.

CLOSING REMARKS

The wide range of topics covered by our authors helps us understand when we should think about adopting a dashboard and how we should design it so it can be used effectively. Each author, in one way or another, has highlighted the importance of showing "useful" data and has proposed some strategies for achieving this goal. Another theme that has emerged is the importance of adopting effective visualizations, because the goal must be to show relevant information clearly, not merely to obtain a fancy dashboard. Other challenges include adapting dashboards to different contexts and maintaining them continuously.

To sum up, much work has been done and much more is yet to come in order to accomplish the ultimate goal of dashboards: to make the numbers talk.

ABOUT THE AUTHOR

Ilenia Fronza holds a PhD in computer science from the Free University of Bozen-Bolzano (Italy) and is currently a nontenured researcher in its Faculty of Computer Science, where she teaches the Software Engineering and Software Project course. Her research interests focus on empirical software engineering, machine learning and data mining, software process visualization and improvement, and agile methodologies. Dr. Fronza has been a program committee member of the International Workshop on Emerging Trends in Software Metrics (WETSoM) and has organized the CASE International Summer School on Practical Experimentation in Software Engineering. She can be reached at ilenia.fronza@unibz.it.