E-MailEXPERIMENTATION AND BREAKTHROUGH INNOVATION
by Stefan Thomke
Experimentation is essential to the growth of all organizations. It fuels the discovery and creation of knowledge and leads to the development and improvement of products, processes, and business models. Without experimentation, we might still use rocks as tools and live in caves. With breakthrough technologies, it is now possible to perform a greater number of experiments in an economically viable way to accelerate the drive toward innovation. In my recent book Experimentation Matters, I address how organizations can best utilize new IT-based technologies for experimentation, including simulation and computer modeling, which allow for the rapid and inexpensive generation and testing of new product possibilities.
Central to experimentation is the use of models, prototypes, controlled environments, and computer simulations that allow innovators to reflect on, improvise with, and evaluate the many ideas that are generated. Today, the ability to run massive experiments via simulation has become critical for many fields, ranging from the sequencing and analysis of the human genome to the design of modern airplanes and automobiles to understanding the flow of fluids in the development of baby diapers.
Thus far, these technology improvements have involved high product development costs, such as in the pharmaceutical, semiconductor, and software industries. My 10 years of research in these fields suggests that as the cost of computing and other combinatorial technologies continues to decrease -- making complex calculations faster and cheaper -- virtually all companies will discover that they have a greater capacity for rapid experimentation to investigate diverse concepts.
New technologies that slash experimentation cost and time also enable what-if experiments that have previously been too expensive or nearly impossible to run. What if an airplane, a car, a drug, or a business were designed in a particular way? By employing new experimentation technologies, it's possible to explore the assumptions that underlie a design, potential changes, and the consequences of such change. Since true experimentation encompasses success and failure, both results are equally important. A common relentless organizational focus on success makes true experimentation all too rare. Because experiments that reveal what doesn't work are frequently deemed failures, tests may be delayed, rarely carried out, or simply labeled as "verification" tests, implying that learning what works rather than what doesn't work is the primary goal. Under this logic, if the experiment encounters a problem (and there's always at least one), the problem will surface too late in the game. When feedback on what doesn't work comes too late, costs can spiral out of control, and opportunities for innovation are lost. By contrast, early failures can lead to more powerful successes faster.
My research also provides a cautionary tale. Detailed case studies show that tapping into the power of experimentation can be a difficult task. Just as the Internet offers enormous opportunity for innovation (far surpassing its use as a low-cost substitute for phone or catalog transactions), so does state-of-the-art experimentation. But realizing that potential requires companies to alter their mindset. Indeed, new technologies affect everything, from the development process itself, including how an R&D organization is structured, to how new knowledge -- and hence learning -- is created. For companies to become more innovative, they must recognize that the challenges are both managerial and technical.
In my research, I have discovered a set of principles that are important in the management of experimentation. We'll discuss these principles in turn.
Anticipate and exploit early information through "front loaded" innovation processes. Large companies like Microsoft, Boeing, and Toyota save precious time and millions of dollars through early experimentation. The use of new technologies tests what does and doesn't work before substantial resources are invested -- the front-loading effect. Early experimentation allows companies to explore multiple ideas, which ultimately results in better products and services.
Experiment frequently, but don't overload your organization. Although companies can save money by lumping experiments into one large test, experimenting more frequently minimizes problem-solving delays and the cost of redesign. Given that new technologies drastically reduce the cost of testing, the need for frequent experimentation becomes more vital. But companies must be prepared to handle the increasing load of information that comes with greater experimentation.
Integrate new and traditional technologies to unlock performance. New technologies like computer simulation are impressive, but they may not achieve more than 70% or 80% of their traditional counterparts' performance. Thus companies can more effectively enhance overall performance and enjoy the benefits of faster and cheaper experimentation by jointly implementing new and traditional technologies.
Organize for rapid experimentation. Integral to innovation and learning is the ability to experiment quickly. To prevent the trail of an idea or inspiration from growing cold, Thomas Edison based his West Orange, New Jersey, USA, laboratory on the concept of rapid experimentation, where storerooms had ample capacity to prevent delays in employees' work and creativity. Leveraging advances in crash safety modeling, BMW removed interfaces between functional groups in order to expedite learning through experimentation. In this case, rapid experimentation resulted in new insights that vastly improved the safety of BMW cars.
Fail early and often, but avoid mistakes. Novel ideas are bound to fail, which is why early failures are necessary to eliminate unfavorable options quickly and enhance learning. Experiments that result in failure are not failed experiments. Again, it was Edison who noted that the "real measure of success is the number of experiments that can be crowded into 24 hours." Unlike mistakes, failures generate new and useful information.
Manage projects as experiments. Projects are powerful mechanisms for managing change, knowledge creation, and the introduction of new technologies. Corporations like Bank of America, BMW, and IDEO have used factors such as fidelity, cost, iteration time, capacity, strategy, signal-to-noise ratio, and type of experiment to maximize learning from projects. These companies have achieved a balance in managing the following dual objectives: (1) finishing projects on time and within budget and (2) using projects as experiments for learning.