by Jim Highsmith, Director, Agile Project Management Practice
In 1979, two researchers, Spyros Makridakis and Michèle Hibon, published a paper in the Journal of the Royal Statistical Society (http://www.jstor.org/ ; a journal I know we all anticipate eagerly each month). Their research was into the accuracy of using time series analyses (moving averages, etc.) for weather forecasting purposes. The authors concluded that "The naive models (those models simply state that the next period's value adjusted for seasonality will be the same as the most recent) outperformed all of the more complex methods."
The authors' story and the quote come from a fascinating book called Strategy Bites Back, edited by Henry Mintzberg, Bruce Ahlstrand, and Joseph Lampel. After reading about half the book, I can't decide whether or not strategy is a good thing, since the various authors take a wide range of positions. But I found one article particularly interesting.
In the agile community (specifically from Extreme Programming practices) we are instructed to plan based on "yesterday's weather"; that is, plan the next iteration based on what was actually delivered during the prior iteration. The particular article in this book that sheds some light on why this is both 1) a good practice and 2) widely questioned comes from a chapter titled "Management and Magic."
Why do we seek planning forecasts in the wake of continued failures and uncertainty? It is the authors' contention "that management's enchantment with the magical rites of long-range planning, forecasting, and several other future-oriented techniques is a manifestation of anxiety-relieving superstitious behavior, and that forecasting and planning have the same function that magical rites have." In other words, our desire to predict the unpredictable is based on superstition. And yet, that superstitious behavior can have both negative and positive benefits. As John Kenneth Galbraith said, "In an uncertain subject matter such as economics or psychiatry, there is something wonderfully compelling about those who are sure" (this quote is also from the same book). As we all know, it's okay to be sure, and wrong, but it's not okay to be uncertain.
But there is an upside to superstitious behavior. The authors tell a story about Labrador Indians who had an elaborate hunting ritual in which they placed the shoulder blade of a caribou on the fire and when the bones cracked, used the crack lines as a map to the hunting grounds. Since the cracks were random, it actually benefited their hunting because they didn't overhunt a particular area. The superstition, in fact, was justification for random action. "In a random world, the best course of action is random action." But admitting to taking random actions is very difficult.
A second function of superstition is that high levels of uncertainty have a tendency to paralyze activity. People want better information before acting, but in fact, sometimes only acting will produce the very information they need. There is the story of a group lost in the Pyrenees mountains who used an old map they found to find their way to safety, only to discover that the map was of the Alps. Action, not the map itself, saved them. However, the map gave them courage that setting out in a random direction would not have.
So we need to understand that planning technology projects could often be done by reading Tarot cards or astrology as effectively as using our so-called "rational" techniques. However, these practices, rational or astral, still have benefit. The downside, as with all superstitions, is that they often are used to further the status quo and therefore discourage change and innovation.
-- Jim Highsmith, Director, Agile Project Management Practice