Digital Transformation Takes a Managed “Messy” Architecture

Over the last five to 10 years, the idea of “digital transformation” has led to an increased prevalence of Agile ways of working in today’s enterprises. Agile promises to make teams more adaptable and responsive and to reduce products’ time to value. As organizations “go Agile” to reap the benefits of these new ways of working, they restructure themselves to have end-to-end feature teams. Such feature teams (e.g., Large-Scale Scrum [LeSS] or Spotify squads) theoretically are capable of taking responsibility for the design, implementation, and evolution of end-to-end, customer-centric features in digital products or platforms.

However, after trying this new organizational structure for a few months or years, organizations eventually stumble upon challenges and realize that the end-to-end fea­ture teams aren’t really working for them. Though this approach to digital transformation usually succeeds in positively addressing a number of mindset issues, a fundamental problem arises when organizations try to make feature teams work with traditional architecture (or architectural patterns). Organizations quickly realize that fundamental limitations in their technology assets — systems, infrastructure, and tooling — prevent them from reaping some of the most valuable benefits of the new, Agile ways of working. The weakest link is to be found somewhere else.

Digital transformation has hit a wall. The need for reinventing how we think about and approach archi­tecture is becoming ever more prevalent, especially if an organization is to truly become Agile.

It is important to understand that software development has changed significantly in the last few years. Software is no longer built totally from the ground up without any dependency on some form of existing software solutions, nor does software come purely from all-encompassing, off-the-shelf packages configured to one’s needs, spanning multiple business or technical domains. Software is now essentially built by “smart stitching” together already existing pieces of legacy code, new code, open source software and frameworks, and software-as-a-service (SaaS)/platform-as-a-service (PaaS) solutions, while leveraging core infrastructure-as-a-service (IaaS) components that also significantly affect functionality.

Thus, today’s best architecture is essentially minimalist, messy, and inconsistent. Architecture should no longer aim to provide robust, detailed frameworks for man­dated solutions and componentry. Rather, architecture should focus on strong general design principles and ensure that it provides guardrails for security, scalability, availability, elasticity, and maintainability (all the typical “-ity” nonfunctional requirements). Moreover, architecture needs to enable solutions that allow for rapid evolution of product and platform features, as well as experi­mentation with new technologies, by ensuring that all integrations are open as well as API- and event-driven. Architecture should also ensure that all technology decisions support safe, error-free, low-latency, contin­uous delivery to facilitate a rapid pace of change in a trusted manner.

Fundamentally, we need to recognize that time to value is the most important decision factor in today’s world. Optimizing time to value through the ability to sustain a rapid pace of quality software delivery is what archi­tecture’s contribution to digital disruption needs to be all about. As long as the solution’s reliability is assured, we should no longer care about duplication, solution “inconsistency,” or anything that conflicts with the more traditional approaches of the past that have emphasized consistency and reuse. In today’s context, it is increasingly as rele­vant to design for (unexpected) change as to design the change itself.

[For more from the authors on this topic, see “The Evolving Role of Architecture in Digital Transformation.”]