The initial expression of Agile — which, for discussion purposes here, we can tie to the publication of the “Manifesto for Agile Software Development” in 2001 — is arguably among the most disruptive events to affect the software world since the industrialization of software itself. What made it so disruptive? What were the conditions that allowed these ideas to be disruptive?
Let’s suppose that effective disruption happens at the intersection of insufficient status quo and redefining fundamentals. Or in plain English, the current state isn’t getting the job done, and it turns out that what we once thought was true (or good) isn’t.
In software’s case, by the 1980s the “productivity” of the overall software industry was abysmal. Costs were skyrocketing, quality was plummeting, and morale among software professionals was desperate. In that decade, a broad-scale study of software — later confirmed in The Standish Group’s 1994 seminal work, “The CHAOS Report”1 — found that the management and engineering of software indeed underperformed in terms of the needs of the market.
In response — and all with good intentions based on tried-and-true theories, best practices, and know-how — the market filled up with management methodologies, tools, and standards. Backed by defined fundamentals and validated by the US Department of Defense (at the time, the single biggest software customer by volume of code, not to mention cost), research universities, and many of the biggest names in software, these “solutions” codified a status quo.
Against this backdrop, the contributors to the ‘Manifesto were each working (on their own, or in small cohorts) on ways of developing software and managing the software process. To some degree, each independent of the other came to a similar realization. “They” (as in, those just mentioned) had gotten the fundamentals wrong and the status quo was making things worse. The software product development industry needed a disruption and the writers of the ‘Manifesto did just that. With — grammatically — three sentences.
In practical terms, three sentences with four values embedded within them weren’t entirely sufficient to change the world. Evolving the development universe would require some help. The contributors to the ‘Manifesto offered pragmatic practices to put the transition to a higher state of consciousness within reach. Some practices were more formalized than others, some more commercialized than others, and some came with a hefty warning label.
While their respective focuses were different, each one in its own way emulated the values in the original manifesto. Despite easily visible limitations and constraints, successes were swift and impressive. By all accounts, the manifesto got it right!
The introduction of Agile practices changed how product development was organized, managed, and executed with the objectives of enhancing customer satisfaction, improving customer/market responsiveness and developer morale, and cutting out impediments to delivery. Indeed, “Agile” has been clearly popularized in much of the product development world. But strong and steady signals are emerging that those objectives are not being met. What happened?
Proponents can justifiably counter that organizations not achieving the promised results of using Agile “are not doing it right” or “don't have the environmental fundamentals” to make it work. But what if the very characteristics that made Agile successful are no longer enough? Have the underlying assumptions about team size and organization, decision-making authority, product arrangement, and customer involvement run their course?
Moreover, what’s needed to ensure that software teams can apply Agile practices in nonideal contexts, such as in larger and highly regulated organizations, and deliver results? Do the fundamentals introduced by Agile now need fundamental changes? What would it take to disrupt Agile?
It’s the year 2020. The popularized notion of “Agile development” has now been around for the better part of at least two decades. That’s a million astronomical units in Internet dog years and, to no one’s surprise, the concept of Agile has sprung new trunks, branches, twigs, and leaves and is now an entire genus, sometimes making the proto-Agile ideas unrecognizable.
These new flavors, colors, and shapes can be described as evolutions of the original concepts. Or can they? Are they really evolutions of the concepts, or evolutions of one particularly dominant branch of the Agile genus? I argue that what’s been evolving over these last couple of decades are not the concepts, but merely the practices. The concepts really haven’t changed. Indeed, I would argue the concepts are solid. Fairly indestructible, actually.
Practices, however, are context-specific instantiations of the application of concepts. The “Manifesto for Agile Software Development” is about values. Not practices. I argue that too much (if not all) of the new trunks, branches, twigs, and leaves and the entire genus have been evolutions of practices. And, it seems Darwin has spoken: this isn’t working.
This isn’t a new thought, but lately I’d been pondering the following question: “Could popularized Agile be gently coaxed back into paying more attention to the fundamental aspects of Agile before ‘Agile’ became myopically focused on practices?” The unfortunate answer to that question, I concluded, is “no.”
Well, if the answer is no, then what will it take to make good on the promise of Agile to solve broad issues in the software industry as the ‘Manifesto initially set out to accomplish? If we can’t ease the development world back toward its value-based roots, how can we make that happen? Clearly, making good on that promise needs something disruptive.
In my work and teaching, I’m noticing a number of trends. Among them are attempts to look “beyond Agile.” Some are examining other disciplines and concepts such as psychology, society, and diversity, while other vectors point to product definition, budgeting, resource allocation, and priorities. And people are also looking in the opposite direction — a more granular, DNA-level working to “rebuild” the underpinnings of Agile from even more basic roots.
I’m not alone. Using math, economics, physics, detailed statistics, various manifestations of holistic engagement, or merely appealing to Maslow’s hierarchy of needs and similar enlightened states of awareness, many of these other trajectories point to the same target. They indicate something along the lines of “practices aren’t enough.”
One of the problems addressed by the authors of the ‘Manifesto was that of a fixation on processes. And by creating a shell of practices around the embryo of Agile, it seems they inadvertently let a pandemic escape from the lab that would eventually act to harden this shell. Essentially, the very practices that were supposed to usher in a new appreciation for people, product, relationships, and responsiveness ended up with the unintended consequence of shrouding the best parts of Agile so that their brave new species would continue to fight its way out. It turns out that the practices catered to the same non-systems thinking that the ‘Manifesto was meant to smash through.
Which brings us to this issue of the Cutter Business Technology Journal (CBTJ). The status quo focus on practices is proving to be insufficient. If something drastic doesn’t push Agile off its current path, evolution doesn’t predict it will return to its initial promise on its own. I ran this idea by Cutter Consortium Senior Consultant and past CBTJ Guest Editor Alistair Cockburn, who said that he has long resisted the questions, “How do we disrupt Agile?” and “What comes after Agile?” He feels that these questions suggest that the manifesto was pointed in the wrong direction, when it seems to have stood well against the test of time. After considering how I framed the discussion, Cockburn said he liked it; it caused him to reconsider what “disruptive” might mean, how to detect it, and where to look for it. He told me he was happy to have his thoughts sent down a new path.
This CBTJ issue takes us on an evolutionary journey of our own. In a rather normalized bell curve, we start with fundamentals and progress through more advanced concepts. We then ease back with practical steps forward and wrap up with a cautionary send-off. The good news is that you’re free to take the journey, or wait for the asteroid. Your call.
In This Issue
We start with a cohort of IBMers — Matt Ganis, Michael Ackerbauer, and Nicholas Cariello — who tee up our discussion directly from where our “What will it take?” question leaves off. They look at the challenges and missteps associated with Agile, beginning with adoption, which relies on expectation setting. And there’s no expectation setting without education. Can it be that simple? Occam’s razor says, “Probably.”
Next, we move up the evolutionary scale with Eric Willeke’s look at whether we’ve missed a turn somewhere on the path. Perhaps we need to gene-splice some deliberate characteristics into our next incarnation of Agile. Forget whether we’re picking the right approach: Are we asking the right questions? Are we even asking questions? Do we know what we want to be? Are we even Agile for the right reasons?
Cutter Consortium Senior Consultant Masa K. Maeda then schools us in some hard-hitting, data-driven food for (evolutionary) thought. He helps us understand what we should be looking for when considering an agility path. It’s no simple checklist or algorithm. Maeda’s outline makes us take a holistic look at our environment; at our choice of primordial soup, as it were. The good news is that we are not completely afloat in the flotsam of the universe. We can choose how we evolve.
Our penultimate piece comes to us from Bob Galen, who picks up on our evolution theme and goes back to basics. When pursuing Agile, which comes first: the chicken or the egg? Clearly not making breakfast, Galen takes aim at whether teams or leadership “goes Agile” first. He gives us a taste for what it must look like to have teams come first and what seasonings to pepper leadership with so that leadership and teams can be “Agile-y” effective together.
Finally, Jeff Doolittle leaves us to set out on our own path to disruption. He suggests the most drastically disruptive action: don’t do Agile. At the very least, don’t do Agile the way too many others are doing Agile. Doolittle invokes the same line of thinking that started our thought experiment to begin with — what has Agile become? Has it grown in unintended ways? Have we lost what it is supposed to be? What else is there if not Agile? Should we completely abandon Agile? Wouldn’t that be disruptive!
As we work through this issue of CBTJ, several themes coalesce. There does appear to be consensus among our contributors that practices are a stumbling block. To disrupt Agile, we need focusing mechanisms around a number of non-practice principles. We hope you see what we’re seeing and are able to leverage this third-party validation to help get your Agile journey back on track or, even better, avoid being off track. Here’s a thought: maybe Agile needs practices around not using practices. No, on second thought, let’s not go there.
1 I know for a fact there was such a study in the 1980s but have not been able to find, or remember, the source.