Building the Agile Enterprise: Myths, Perceptions, and Reality

From the Editor, Alan MacCormack, Associate Professor, Technology & Operations Management Unit, Harvard Business School

As I write this article in the spring of 2008, the need for enterprise agility is once again a topic on the minds of executives everywhere. Stock market gyrations push prices down 3% one day, up 4% the next. Firms that months before had record profits struggle to stay alive. As if the impending recession is not hard enough, new technologies continue to make inroads in traditional industries, challenging old ways of doing business: Apple is the number one US music retailer; Google, while "doing no evil," scares everyone; and a product developed by hundreds of individuals around the world -- Linux -- is forcing the world's largest commercial software firm to rethink strategy. Today, more than ever, firms must respond to a turbulent environment. How topical then that this issue of CBR examines how firms can achieve such agility.

How does one make an organization "agile?" First of all, we must recognize that agility is a capability that results from the complex interplay of many different processes throughout an organization. How fast can you introduce a product in a newly emerging market segment? How quickly can you respond to equipment failures in production? How rapidly can you find and fix a security breach in your Web site? These are all aspects of agility, illustrating the difficulty in making simple recommendations that have immediate benefits along every dimension. The search for agility requires a consistent pattern of decisions be made across different areas of the firm over time, emphasizing flexibility over efficiency and responsiveness over control.

A critical concern for firms seeking greater agility is the need for an infrastructure to support this ability. In this respect, the role of a firm's IT system is critical. These systems are integral to many of the functions a firm performs, including product development, production, operations, and financial management. The ability to respond to changing demands is influenced significantly by the functionality built into these systems. Critically, however, it is also influenced by the organization's ability to make unanticipated changes to these systems. We refer to this concept as IT agility. It captures how quickly and effectively a firm can adapt its IT system to meet new demands.

In this issue of CBR, we explore the topic of IT agility, reporting data from a survey of 124 organizations. The aim of our survey was first, to understand the extent to which IT agility is an explicit objective for firms, and second, to understand the mechanisms by which this ability is achieved. In analyzing the results, we look both at broader descriptive trends reported by firms and the underlying drivers of these trends, as revealed by the patterns of correlation among variables. While descriptive statistics provide insights into how the world looks, correlations help us to understand why the world looks this way. ¹ In short, these types of analyses help us to develop managerial recommendations in the search for improved performance.

My own interest in this topic stems from past research on a related topic: how firms develop new products in highly uncertain environments. My work in this area revealed that successful firms focus on two key areas: first, they adopt a more iterative style of development process, geared to generating early feedback on product performance that can be used to inform subsequent design decisions; and second, they emphasize the use of modular product architectures, which facilitate the ability to make changes to a design quickly and at low cost. ² In essence, I found that increased responsiveness in product development could be achieved only by rethinking some of an organization's core processes and structures. The question I have is, is the same true for IT agility?

The findings that emerge are intriguing. While almost 65% of our sample organizations identify IT agility as an explicit objective in developing IT strategy and plans (see Graph 1 in the Survey Data section), only 26% of them have developed concrete measures of this ability (see Graph 2). We must ask how they know if they are achieving their objectives? This is important given that there are significant differences between how managers perceive they perform and their "real" performance, as captured by some of our survey variables. While perceptions are strongly tied to reality, as one would hope and expect, they are also driven by the actions that firms take and the expected results from these actions. Unfortunately, not all of them work as intended. IT agility, it seems, is an elusive phenomena.

So what really counts? The bottom line is that achieving agility requires that firms make investments in application development processes that are more flexible than traditional processes. But they must also design a system architecture that facilitates flexibility by reducing the level of interdependency between different applications in the overall portfolio. To achieve this, it is not enough to adopt high-level methodologies such as service-oriented architectures (SOAs) or layered architectures. Though these methods bring the promise of better structure, we find that they are not correlated with either reduced complexity or increased agility. Instead, firms must attack complexity directly, investing in system redesign efforts to make them more "improvable."

In this article, we examine the extent to which IT agility is a concern for firms and explore why some firms seek agility while others do not. We look at the actions firms take to achieve this ability and examine the drivers of IT agility as revealed by the data, drawing insights into the differences between managerial perceptions and competitive realities. We conclude by looking at what we didn't see in the survey results, which is often just as interesting as what we do see.

IN SEARCH OF AGILITY: WHO WANTS IT, WHY, AND WHAT DO THEY DO?

In the survey we first looked at the extent to which agility is a factor in shaping a firm's approach to managing its IT infrastructure. Our survey reveals that this topic receives a great deal of attention in many organizations. Almost 65% of our sample report explicitly considering the need for agility when developing their IT strategy and plans (see Graph 1). How you view this figure depends upon whether you believe the glass is half full or half empty. While it is heartening to see so many firms addressing agility in their formal plans, it is still surprising that one-third of respondents do not.

Why then, do some firms pay so much attention to agility while others do not? One hypothesis is that the former operate in more uncertain environments, hence they need the ability to react to changing circumstances. For example, consider the situation at Apple as it moved from being a focused computer firm to a firm that sold computers, MP3 players, and smart phones, while also delivering music, movies, and software to these devices. These changes generated demands on its IT systems that could not have been predicted a few years earlier. Apple needed to be more agile, hence it would be expected to emphasize this ability much more than firms in industries with a slower "clock speed."

We used the survey data to test this explanation, with surprising results. The level of external uncertainty a firm faces -- in terms of understanding customer needs or predicting next-generation product technologies -- has little correlation with whether a firm considers the need for IT agility. By contrast, the level of internal complexity a firm faces -- in terms of the breadth of its product line and the scope of its geographic operations -- is a strong predictor of the need for agility. From an IT perspective, therefore, it seems as though agility is viewed as a way to manage an organization's internal challenges, rather than its external ones.

Why might this be so? Perhaps it is because the challenges of developing and deploying IT systems are particularly salient in firms that operate across multiple industries and geographies. If these characteristics change, through the addition of new products or locations, firms must ensure that their IT systems are quickly adapted to suit. Even so, the lack of association between the need for agility and external uncertainty still comes as a surprise. It implies that firms must pay greater attention to understanding how the demands of the external context should be met through investments in their IT infrastructures.

We next looked at actions taken by those organizations concerned with agility. One of the big surprises was in what they do not do: measure how they perform! Only 26% of our respondents report having explicit measures of IT agility that are monitored on a regular basis, less than half of those that explicitly consider the need for this ability in developing their IT strategy and plans (see Graph 2). One must ask how these organizations know if the steps they are taking lead to improved performance.

Part of the reason for the absence of metrics may lie in the complex and multifaceted nature of agility, which makes it difficult to agree on how it should be measured. Any set of measures will inevitably be viewed as incomplete. Indeed, we have observed many arguments inside firms as to what exactly it means to be "flexible" and from whose perspective this should be measured: the IT department or downstream customers of its services. But neglecting to define any measure of agility merely avoids the problem. It is far better to engage in a strategic discussion of the ways in which agility creates value, given this will typically expose a number of hidden assumptions that must be made explicit before progress can be made. Clearly, this is an area for greater attention.

Looking at the management practices adopted by organizations that emphasize the need for IT agility, we see clear patterns (see Table 1). They are more likely to use agile processes and to adopt high-level design methodologies that emphasize the need for modularity. With respect to the former, our survey reveals that firms that consider the need for agility adopt agile methods in 36% of their application development projects; whereas in firms that are less concerned with this ability, the figure drops to 14%. With respect to architectural design practices, we examined both the commitment to an SOA as well as the number of layers of abstraction identified in the IT system architecture (a proxy for the degree to which the functions within each layer have been isolated from one another). Clear differences are found on both measures. Two-thirds of the firms that emphasize the need for agility have committed to an SOA; by contrast, 25% of the remaining firms have adopted this methodology. In addition, those firms concerned with agility, on average, design an extra layer of abstraction into their IT system architectures.

Clearly, firms that seek greater agility seem to be doing the right things. They adopt processes that are more responsive than traditional waterfall models and hence are better equipped to respond to changing requirements. And they adopt architectural design methods that emphasize the need for modularity, which should allow changes to be made to one part of a system with minimal impact elsewhere. Without a way of measuring performance, however, it is hard to know if these choices are working. That is what we set out to understand in the second part of our survey.

ACHIEVING IT AGILITY: MANAGERIAL PERCEPTIONS AND COMPETITIVE REALITIES

The main aim of our study was to explore the factors that lead to improved performance with respect to IT agility. We captured data on both manager perceptions of performance, as well as several measures reflecting an organization's actual performance. Given that there are no widely accepted measures for the latter, we focused our attention on two areas: responsiveness to unforeseen problems and responsiveness to changing demands. In particular, we captured data on the time taken to resolve problem/bug reports; the time taken to implement new feature requests; and the percentage of new application development projects that are completed in less than six months. We provide descriptive data for these measures in Table 2. We found strong correlations between these measures of performance (see Figure 1). This makes us confident that we are capturing a central component of IT agility and doing it in a way that is both robust and repeatable.

We looked first at the factors that best explain managers' perceptions of their organization's level of IT agility, relative to the competition (see Graph 3). (Note that Lou and Tim have an interesting take on this issue as well.) These perceptions are strongly correlated with our real measures of agility, as one would expect. However, we also found that these perceptions are driven as much by what an organizations does as by how well it performs. That is, managers believe their IT organizations are more agile to the extent that they measure this ability and deploy practices thought to improve it. Unfortunately, not all these practices have the impact they might expect. In Figure 2, we show the correlations between perceived and real measures of IT agility as well as the factors influencing each of these measures as revealed by our analysis.

Let's take a look at those factors correlated with perceptions of IT agility but not with our measures of actual performance. The first is whether the firm has defined explicit measures for IT agility (see Graph 2). Those that have measures believe they are more agile than others, yet in reality, they perform only at the same level. Unfortunately, while measuring performance is often the first step on the road to improving it, the mere act of measurement does not mean that you are better at it.

The second factor associated with perceptions of agility, but not with actual performance, is the number of layers of abstraction identified in the IT system architecture. Respondents appear to believe that a greater number of layers implies a better structured and more modular architecture. This makes sense, given we noted above that firms concerned with agility tend to adopt more layered architectures. Intriguingly, however, committing to an SOA (see Graph 4) is not correlated with perceptions of agility, despite the fact that those organizations concerned with this ability are more likely to adopt such initiatives. Perhaps managers view SOAs as a prerequisite for attaining agility but not as a way of directly influencing this ability? The lack of correlation between adopting SOAs and our actual measures of performance suggests that this intuition may be spot on.

We next examined the factors correlated with both perceptions of IT agility and actual performance. We found only one factor with this pattern, though it is one of the most important in our survey: the percentage of a firm's application development projects that uses an agile process. Across our sample, organizations report using agile methods in 27% of projects. Organizations that report a greater use of such methods, however, have higher perceived levels of agility (3.8 versus 2.8 on a five-point scale) and higher levels of actual agility, as illustrated through the descriptive statistics shown in Table 3. ³ In sum, this variable is the single most significant factor in explaining perceived levels of IT agility and the second most significant factor in explaining our real measures of IT agility.

As we dig deeper into these results, an interesting pattern emerges. We observe that penetration of agile methods has a higher correlation with perceived levels of IT agility than with actual performance. That is, a greater use of agile methods is associated with an increase in perceptions of agility that exceeds the "real" increase that could be attributed to these methods. Put bluntly, organizations believe they are more agile because they have adopted agile processes. This is a subtle point. We are not saying agile processes don't work -- in fact our results suggest definitively that they do. What we are saying is that the mere adoption of agile processes is not what makes the firm agile. Agility is achieved only via a coherent set of actions taken across a firm. Adopting agile processes without these supporting actions is likely to lead to disappointment.

In order to tease apart these effects further, I conducted a secondary analysis of the results (indulge my academic interests for a second). I removed that part of the variation in the use of agile processes that was associated with a real performance gain and hence could be explained by the correlation between actual and perceived performance. I then examined the remaining variation -- the part not associated with real performance gains -- to assess its relationship with perception. The findings support the conclusions made above (see Figure 3). Perceptions of agility are higher to the degree that a firm uses agile methods, regardless of whether these methods lead to real improvements! The size of the correlation is almost equal to that associated with real improvements.

The implication for firms is clear. While agile processes often lead to better performance, they certainly do not guarantee it. Hence firms must not be fooled into thinking that they are agile just because they adopt such methods. Perception and reality are not perfectly aligned. Indeed, there are other factors that firms do not perceive as important that turn out to be just as powerful in explaining performance.

Finally, we turn to look at those factors that explain a firm's actual level of IT agility but that are not correlated with managers' perceptions of performance. At the top of the list is a variable we have not yet mentioned: system complexity. This captures the degree to which making major changes to one application requires that consequent changes be made to the others. That is, it measures the degree of interdependency between the applications in a firm's portfolio. Our analysis reveals that this is the strongest predictor of a firm's actual level of IT agility. Organizations that possess systems with high levels of interdependency are much less agile than others. This finding is illustrated by comparing the differences in performance between firms that report having different levels of system complexity (see Table 4 ⁴ ).

How can system complexity have such a significant impact on actual performance, yet so little influence on perceptions of IT agility? In my view, the answer to this paradox lies in managers' beliefs that the application of well-known high-level design methodologies will solve the challenges of complexity, without the need to understand what is going on "below." Consider that, in this survey, we asked managers about the use of SOAs as well as the number of layers of abstraction in the system architecture. These measures capture attempts to better structure IT systems from the top down. Indeed, the latter measure is correlated with managers' perceptions of IT agility. Yet while these methods often have compelling benefits in terms of overall performance, they tell us relatively little about potential interactions between applications at the ground level. This is especially true for the legacy systems that an organization possesses, which may have been designed to meet different priorities, are often hard to understand, and typically prove more difficult to change.

This dynamic was highlighted in a recent meeting with managers at one firm who believed their system was highly modular. To illustrate, they mapped out the design on a whiteboard, with rectangles representing each of the major modules. The design was impressive; the conceptual thinking behind it, flawless. But these managers had to admit that they had no idea of the real level of interaction between modules at the level of the actual software code. The subsequent analysis revealed major interdependencies between almost every part of the system.

The challenge is that high-level designs always look modular. But the real level of modularity emerges only from myriad individual design and configuration decisions made during implementation. We are not saying that these decisions are made with malicious intent. In fact, the situation is quite the reverse. Creating dependencies between different parts of a system often makes it better at what it does today (e.g., by eliminating redundancies or increasing its speed of operation). But in doing so, it typically makes it harder to change the design thereafter.

So what can organizations do when methods such as SOAs and architectural layering do not guarantee the level of modularity needed? Our results suggest that organizations must attack the complexity of their IT systems in a more direct fashion. Unfortunately, this is extremely hard to do. It requires that coherent actions be taken across the application portfolio to reduce dependencies between them. The challenge is magnified by the lack of visibility we have into the dependencies between applications in a modern IT system. Consider your own organization: Do you know the relative level of complexity or modularity in your IT system? How do you know? Clearly, we are in need of new tools that can reveal system structure and guide managerial actions to make systems more "improvable." I expand on this idea below, using insights from my ongoing research.

Another factor that predicts a firm's actual level of IT agility but that is not correlated with perceptions of agility is the size of the organization. Larger organizations, as measured by total firm headcount or the size of the IT staff, tend to be less agile than smaller organizations. I was surprised that size is not correlated with lower perceived levels of agility, given managers in large organizations are often concerned about the impact of bureaucracy on responsiveness. But I did not have a view either way on the relationship between size and actual agility. One could argue that large firms have more resources, hence can potentially respond more quickly to changing demands. On the contrary, more resources can often be a disadvantage, given the inertia that comes from the consequent increase in systems for planning and controlling these resources. The latter dynamic is what we appear to see in this survey. In fact, our data reveals that the negative correlation between size and agility is greater for the size of the IT staff than for total firm headcount, indicating that "diseconomies of scale" may stem more from larger IT departments than from larger firms per se. ⁵

ARCHITECTURE, DESIGN, AND IMPROVABLE SYSTEMS

The critical importance of system complexity in explaining IT agility mirrors the results I have found in parallel research on the topic of software architecture. ⁶ It is helpful to consider the insights from this work, as they inform the challenges we see here. In this research, we have developed ways to visualize and measure differences in architecture by analyzing the dependencies between components in a design. In software products, the components are defined as source files. While there are several different types of dependency between these components, we focus on the most significant one: function calls (requests by one component to access functionality in another). We can display the dependencies between system components in a square matrix (see Figure 4, and later, Figure 5). By analyzing the pattern of these dependencies, we can measure the potential for a single design change to propagate through the system. We call this measure the system's "propagation cost."

The first insight we gained in this work is that systems that perform very similar functions often have very different architectures. This is surprising given one would think that architectural choices are driven mainly by the tasks a system is intended to perform. Consider the two systems in Figure 4, which perform the same task, but have very different designs. System A has a low propagation cost, while in System B it is much higher. The implication is that it is much more difficult to make changes to System B, given each change can potentially affect more of the design. In System A, by contrast, the loosely coupled design facilitates making changes. Indeed, in our work, we have demonstrated that greater interdependency makes a system harder to adapt, harder to maintain, and harder to augment. In essence, interdependency is the enemy of agility.

The second insight we gained is that differences in architecture often stem from differences in the way that development is organized. For example, we observed that smaller, colocated teams tend to develop more tightly coupled architectures than larger, more distributed teams. While all teams doubtless wish to develop designs that perform well, the extent to which they create and leverage dependencies between different components differs. In essence, we find that products tend to mirror the organizations that develop them, an insight known as "Conway's Law" that dates back to the 1960s. ⁷ Firms must understand the biases that stem from the way that their IT organizations are structured in terms of any possible predispositions toward developing certain types of architecture.

The final insight we gained is that increasing system responsiveness often requires major redesign efforts, which have the explicit aim of reducing complexity (i.e., increasing modularity). In programming circles, this activity is known as "refactoring," wherein a program's structure is improved without changing its results. The challenge for firms is that they often believe refactoring efforts are a waste of time and effort, given no new functionality is added to the system. They fail to realize that these efforts are often a prerequisite to making a system more improvable. These efforts create value, but this value comes in the form of an option to make changes to the design in the future.

To illustrate, consider what happened when Netscape released the source code for its Mozilla Web browser in 1998, in the hope that outside contributors would help add new functionality. The problem was that the existing design was very complex, meaning it was difficult for developers to make contributions without affecting many other parts of the system. In the fall of 1998, a small team of programmers redesigned the code base with the sole aim of making it more modular, hence easier to change. The result was striking (see Figure 5). The team significantly reduced both the number of dependencies and the system propagation cost, improvements which led to an increase in the number of contributors. Their efforts eventually led to the introduction of Firefox, a Web browser that has gained both critical acclaim and market share.

"NO RESULTS" AND INTERESTING CAVEATS

It is worth noting what we did not see in our analysis, while remembering that this does not mean these factors are unimportant. We may not yet have enough data to reveal the more subtle relationships that influence agility. In addition, survey measures are notoriously noisy, making it difficult to detect all of the signals that lie within the data.

We did not see a relationship between a firm's level of IT agility and measures of its IT strategy. We examined whether a firm tends to buy packaged solutions versus develop custom (bespoke) solutions and whether it tends to adopt integrated solutions versus best-of-breed solutions (see Graph 5). I was surprised that these factors did not impact agility, given there are many arguments that they should. Yet this may be a positive sign, highlighting that achieving agility is not dependent on one particular approach to configuring a firm's IT investments.

We also did not see a relationship between a firm's level of IT agility and the percentage of application development work that is outsourced. Again, this is somewhat surprising but is consistent with the notion that outsourcing can impact agility in multiple ways, some good and some bad. Having the ability to quickly access external resources without committing to full-time staff can be an advantage when responding to varying workloads. On the contrary, effectively responding to new feature and application demands often requires a dedicated internal team with a deep understanding of the end-user context. In essence, the relationship between outsourcing and agility is complex and as such will need greater attention before we can arrive at robust prescriptions for action.

CONCLUDING THOUGHTS

Firms that compete in an increasingly uncertain world strive to be agile. In this article, we sought to understand how organizations can facilitate this capability through investments in their IT infrastructure. We discovered that while agility is an explicit concern for many firms, few have worked out how to measure it and fewer still understand the real drivers of performance. Perceptions are driven as much by what firms do as by the level of agility they actually achieve. So what really counts? Agility requires that firms invest in application development processes that are much more flexible than traditional methods. But they must also design system architectures that facilitate this flexibility by reducing the level of dependency between different applications in the portfolio. Critically, these factors are likely to be complements; doing one without the other may constrain a firm's ability to improve. Our results represent a first step in helping to make the phenomenon of IT agility a little less elusive.

So what first steps should firms take? Based on the results of this survey, let me suggest six areas for focus:

1. Begin a strategic discussion on what aspects of IT agility are most important. But center this discussion on external demands, assessing the level of responsiveness needed to cope with changing market, technology, and regulatory conditions.

2. Put a stake in the ground and begin to measure the attributes you identify as being important. You won't get it right the first time, but agility is about the need for change, so it's natural that as your abilities evolve so will the relevant measures.

3. If you haven't done so already, begin piloting agile development methods. As you do so, measure the results achieved. You should see improvements along performance dimensions you care about.

4. Dig a little deeper into understanding your "real" system architecture. While committing to SOA and architectural layering are good first steps, you must also measure their impact at the ground level. Don't be satisfied with high-level block diagrams that seem to say all is well.

5. Take corrective action where necessary. If a system is too complex, it's often worth investing in refactoring. Software systems tend to outlive the tenure of their creators, so if problems aren't fixed at an early stage, you live with the results for a long time.

6. Make agility an integral part of your decision-making processes, at both a strategic and tactical level. This is critical, for greater agility requires that firms make investments today that create the option to do different things tomorrow. Every decision has an immediate and a longer-term impact, so firms must balance the two.

Do these things and you are well on the way to building a more agile enterprise. But critically, you won't fool yourself into thinking you have built one already just because you appear to be doing the right things.

ENDNOTES

¹ The presence of correlation, by itself, does not provide evidence of causality. It is only one step in the chain of reasoning through which a causal relationship is established. In this article, I use causal terminology where I think it is warranted by our research design.

² MacCormack, Alan D. "Product-Development Practices that Work: How Internet Companies Build Software." MIT Sloan Management Review, Vol. 42, No. 2, Winter 2001, pp. 75-84.

³ We used the mean level of agile usage to divide the sample into high and low levels of use.

⁴ System complexity is measured using a five-point scale. The top two scores are used to group systems of high complexity and the bottom two scores are used to group systems of low complexity.

⁵ Note we observe the same pattern with firm revenues and the size of the IT budget. The latter has a more significant negative correlation with IT agility than the former.

⁶ MacCormack, Alan, John Rusnak, and Carliss Baldwin. "Exploring the Duality Between Product and Organizational Architectures: A Test of the Mirroring Hypothesis." Harvard Business School Working Paper 08-039, 2008; MacCormack, Alan, John Rusnak, and Carliss Baldwin. "Exploring the Structure of Complex Software Designs: An Empirical Study of Open Source and Proprietary Code." Management Science, Vol. 52, No. 7, 2006, pp. 1015-1030.

⁷ Conway, Melvin E. "How Do Committees Invent?" Datamation, April 1968.

ABOUT THE AUTHOR