PREDICTABLE SURPRISES

by Robert N. Charette, Fellow, Cutter Consortium

Recent enterprise risk management (ERM) snafus illustrate the importance of organizational preparedness in the face of surprise. Consider the following examples.

• In 2004, Comair shut down during the Christmas season because of a hard-coded software counter in its 20-year flight scheduling software system that had been exceeded.¹ Thousands of travelers were stranded, and Comair lost millions of dollars in holiday revenue as well as significant goodwill from its customers.
  
  
• The FBI's Virtual Case File system development, meant to dramatically improve the FBI's ability to investigate terrorism and criminal cases, teetered on the verge of cancellation after US $170 million was spent on the effort. Several program assessments by the FBI's own inspector general and by the National Academy of Sciences warned of significant problems and risks in the development that were being ineffectively managed.
  
  
• Five rail executives are on trial for manslaughter in the UK following a train crash that killed four passengers and injured 102 others because of a broken rail. Twenty-one months prior to the accident, the rail was identified as defective and in urgent need of repair. At the time of the accident, a replacement rail lay next to the broken rail.
  
  

These incidents join a host of others that Harvard Professor Max Bazerman and INSEAD Professor Michael Watkins term "predictable surprises" [1]. Predictable surprises are events that create material negative consequences and that both are and are not foreseeable and preventable. Bazerman and Watkins cite the terrorist attacks on 9/ll, the collapse of Enron, and the 2004 US flu vaccine shortage as additional examples of predictable surprises. In the IT community, others that come to mind are the Denver International Airport's baggage handling system, the FAA's Advanced Automated System for air traffic control, and the Confirm airline reservation system developments.

In this Executive Update, we examine Bazerman and Watkins's concept of predictable surprises and discuss how to minimize or avoid these kinds of risks.

WHY DO PREDICTABLE SURPRISES HAPPEN?

Bazerman and Watkins argue that predictable surprises are examples of leadership failure. Whereas unpredictable failures -- the proverbial bolts from the blue -- are unavoidable, predictable failures are usually attributable to the fact that an organization's leadership has not prepared the organization to (1) recognize emerging threats or problems; (2) prioritize threats so that they receive serious attention; and/or (3) respond effectively to threats once they are identified. These three sequential activities of recognition, prioritization, and mobilization -- or in Bazerman and Watkins's terminology, "RPM" -- must be present if an organization wants to avoid predictable surprises. Furthermore, all three RPM stages must execute correctly; deficiencies or lapses in any stage can lead to an unpredictable surprise.

In the instance of the FBI Virtual Case file, threats to the successful completion of the project were well documented. FBI leadership tried somewhat to prioritize the threats, but there was a lack of will to take all the actions necessary to keep the project from sinking into deeper trouble.

SUSCEPTIBILITY TO PREDICTABLE SURPRISES

Organizations and individuals are susceptible to predictable surprises. Bazerman and Watkins identify three sources of vulnerability: cognitive bias, organizational traits, and politics. Cognitive biases influence human perception of situations.² For example, a butcher and a vegetarian view a cow differently. Cognitive bias works against our ability to recognize predictable surprises or leads us to ignore them.

For instance, overconfidence (i.e., having overly positive illusions of one's sphere of control) is a common cognitive bias. Overconfidence leads to the belief that things are better than they are, that risks won't occur, or that if they do, the consequences can be contained.

We also give greater weight to information that supports our preconceptions than to information that runs counter to preconceived notions. We view the former as "fact" and the latter as opinion, and we seek out information that confirms our preconceptions rather than information that might contradict them. Research indicates that even when a preconception is demonstrated as false, those holding onto that belief will continue to rely on it in their decision making.

We also tend to discount the future. We generally live for the present, which means that we usually don't take actions today (especially if they are difficult, costly, or painful) to avoid future problems. This bias explains why it is difficult to get organizations to perform a risk assessment or, if they do, to take action based on it, as in the FBI situation.

Bazerman and Watkins note that these and other cognitive biases share a common trait: they are self-serving. They act individually and collectively to create a perception of the world as we would like it to be rather than as it truly is.

ORGANIZATIONAL CAUSES

In addition to cognitive biases, there are organizational causes of predictable surprises. Bazerman and Watkins note that organizations are often structured into separate functional units for control and efficiency reasons, which creates information islands. These islands not only fragment information but also color the information created. Just as in the case of the butcher and vegetarian, finance and marketing organizations look at information differently. Each functional unit produces and consumes information from its own perspective, making it difficult to see how the information is interrelated. In the Comair situation, the software organization was aware of the hard-coded counter limit problem, but the likelihood of the limit being reached was not fully communicated to those in Comair's operational organization responsible for crew scheduling.

Modern organizations' structure also tends to disperse responsibility for acting on information. Few organizations have an individual, let alone a group, who is responsible for anticipating predictable surprises. The identification of predictable surprises -- such as a major shift in a market or industry -- often requires the identification of trends and patterns, but the typical organizational unit is not equipped to do such pattern recognition or matching.

Many organizations have strong internal fiefdoms where information is viewed as a resource that should not be shared beyond the organizational unit. Sensitive information, particularly that regarding potential problems, is rarely shared. Paradoxically, in such organizations, if the information is not deemed sensitive, it's not seen as important. This attitude exists in the intelligence community: if information isn't classified, it cannot be terribly important; and if it is classified, it cannot be shared. Lack of incentives to share risk information is a classic problem that plagues many organizations.

Even where there are not fiefdoms or other disincentives to share information, organizations have become markedly flatter over the past 15 years. From a cost and even a productivity perspective, the elimination of middle management may be beneficial, but part of middle management's job was to question things. Elimination of these positions removed a level of the organization devoted to scanning for external threats or problems that few companies have replaced.

Predictable surprises are even more difficult to recognize, since most corporate information is backward-looking. Little corporate information focuses on how to deal better with the future. Even when lessons learned are identified -- and these are more often in terms of activities to avoid rather than actions to take -- they are rarely widely disseminated or taken seriously by others within the organization.

POLITICAL CAUSES

Political sources of predictable surprises are probably the most difficult to overcome. They come in many guises, such as special interest groups. The airline industry is a prime example: old-line carriers have unions representing pilots, flight attendants, mechanics, and so forth, with each group trying to get the best deal for its members. There is nothing wrong with this per se, but the reality is that shifts in the airline industry have made it harder for unionized carriers to compete with nonunionized ones. Even old-line carriers that recognized the market threat of low-cost, nonunionized airlines found it difficult to mobilize until they were virtually bankrupt.

In some cases, individuals operate as a special interest group on their own, seeking personal gain over that of the organization or their peers. When these individuals are in a position of power, they may block efforts to identify, prioritize, or mobilize against the threats that create predictable surprises. Corporate governance advocates argue that actions such as granting generous stock options to senior executives can lead to self-aggrandizing behavior in the short term that may jeopardize the long-term health of a company.

In other cases, the organization itself may create a detrimental political agenda. The top priority of both government and contractor management in many government programs, for example, is to keep their program funded. In May 2004, senior FBI officials testified before Congress that the Virtual Case File system would be ready by December 2004. Did these officials lie, or was this a case of the overconfidence cognitive bias at work? You decide.

MAKING UNPREDICTABLE SURPRISES PREDICTABLE

Bazerman and Watkins describe various approaches that organizations can take to avoid predictable surprises. These approaches are aimed at improving each stage of the predictable surprise recognition-prioritization-mobilization process.

Improving Recognition

The first approach is for senior leadership to ask the question "Do we face any predictable surprises?" While seemingly obvious, Bazerman and Watkins argue that few executives bother to ask this question even once, let alone on a regular basis.

Second, organizations can improve their measurement systems and measurement incentives. What threats might exist, and what can be used to identify and measure their likelihood and consequences? Obviously, a strong risk management process that looks both within and outside the organization is an effective approach.

Third, Bazerman and Watkins recommend that specific organizational units be created that are devoted to scanning for predictable surprises and integrating information from across the organization. Unless someone -- a chief risk officer, for example -- is given the job (and budget) to do so, scanning for emerging threats likely won't happen.

Fourth, the organization must be trained to look for predictable surprises and be taught how to communicate this information to the appropriate parties. Establishing communities of practice that periodically report to a chief risk officer is one way to identify emerging threats that might not normally be communicated.

Finally, executive leadership should strive to create an environment in which lessons learned can be shared and acted upon in meaningful ways.

Improve Prioritization

One can prioritize predictable surprise signals in three ways. First, organizational leadership must foster decision-making processes that encourage the honest examination of organizational assumptions and beliefs. Intel is a company that undergoes a great deal of self-examination, and its senior management is not afraid to debate its reasons for certain actions and strategies.

Second, rigorous decision analysis -- including not only risk assessment but cost-benefit analysis, assumption analysis, and quantitative and qualitative analysis as well -- can help raise awareness of the importance of an emerging threat and the alternative courses of action available to the organization.

Third, Bazerman and Watkins recommend that wherever possible, organizational incentives should be created that align the interests of individuals with those of the corporation. Unless conflicts of interests are eliminated, individual desires will almost always trump organizational interests.

Improve Mobilization

Mobilization against predictable surprises is highly dependent on the ability to communicate persuasively that immediate action is needed. Furthermore, it is imperative for senior leadership to listen to what is being communicated. Bazerman and Watkins recommend that to improve the capability to mobilize in the face of emerging threats, executives must build formal and informal coalitions to influence decision-making behavior. Further, these coalitions must be formed before a predictable surprise occurs. To help create these coalitions, Bazerman and Watkins recommend that crisis management plans be developed. These plans help identify not only those affected by emerging threats but also those who can address them (or impede action against them).

I believe that mobilization for action is the most difficult to achieve. It is not a giant step to move from building coalitions for action to building coalitions of inaction. There is no easy solution to this problem other than to say that the effectiveness of mobilization rests heavily on senior leadership's ability to receive bad news.

Bazerman and Watkins's RPM process provides a benchmark against which to evaluate an organization's ERM efforts. While I suspect that most ERM regimes already implement many of the authors' recommendations, the RPM process is useful because it can show where an ERM implementation remains deficient. The process is also useful in that it can show how valuable the ERM actually is to an organization.

Not all surprises can be avoided, but with a strong ERM approach supporting an effective RPM process, preventable predictable surprises can be averted.

¹For more details, see the Cutter Consortium Enterprise Risk Management and Governance E-Mail Advisor " An Operational Risk Holiday Surprise," 13 January 2005.

²Humans are susceptible to -- or better put, hardwired with -- more than 25 separately recognized cognitive biases and filters. I discuss these biases and their impact on risk assessment in my book Applications Strategies for Risk Analysis.

ABOUT THE AUTHOR

REFERENCE

1. Bazerman, Max, and Michael Watkins. Predictable Surprises: The Disasters You Should Have Seen Coming and How to Prevent Them. Harvard Business School Press, 2004.