CUTTER BUSINESS TECHNOLOGY JOURNAL  VOL. 34, NO. 5
  

Humans have centuries of product development experience, but the recent exponential rise in product complexity means current development approaches are reaching their limit. Fortunately, artificial intelligence (AI) has the potential to help us overcome those limitations.

Product complexity is currently driven by four trends. First, products include a growing amount of embedded software. We can roughly measure product complexity by looking at number of lines of code.

Second, modern products are increasingly connected, leading to a “system of systems,” a trend also known as the Internet of Things (IoT). Such systems of systems exhibit emergent behavior — properties the individual systems do not have.

Third, complexity is compounded by larger teams, which are often distributed. This results in growing communications overhead. To cite just one example, automotive manufacturers frequently share require­ments specifications containing tens of thousands of requirements.

Fourth, regulatory compliance creates overhead. With more IoT devices entering our lives, manufacturers must demonstrate their safety. A failed audit can result in a delayed product launch.

To understand why complex products require new approaches, let’s first briefly review the existing approaches.

Systems Engineering

In the 1960s, systems engineering became an inter­disciplinary field of engineering for developing com­plex products. It was driven primarily by the space industry, as well as some high-profile engineering disasters. A key concept of systems engineering is the V-Model, which is primarily known as a development process, but it also represents the various artifacts of product development. The V-Model puts requirements, implementation, and validation and verification (V&V) into context, a key concept that is highly relevant to the rest of this article (see Figure 1).

Interpreted as a process model, the V-Model starts at the top left: defining the product requirements. These requirements are refined and decomposed, eventually leading to an implementation. The right arm indicates that on each level, V&V activities ensure the right product is being built, and that it’s being correctly built. All elements must function as designed (verification), and the result must be what the customer desired (validation). Testing is the most common approach for V&V, but there are others. Similarly, the left arm of the V often consists of more levels than the three shown.

A key takeaway is the triangular relationship between requirements, implementation, and test, which provides robustness. A failed test might indicate a problem with the implementation, but it could also point to a faulty requirement or a problem with verification.

Initially, systems engineering was paper-based; every iteration carried significant overhead because long documents had to be read, reread, checked, and aligned. For complex products like rockets, planes, and cars, iterations typically took six months to two years.

Many organizations still practice traditional develop­ment without many changes (substituting Word and Excel files for paper), even though more sophisticated tools are available. This does not necessarily result in unsafe systems, but the overhead for each iteration is increasing because of rising complexity. Furthermore, long iterations lead to a slow response to customer feedback and changes in market and technology, creating a competitive disadvantage.

Since the advent of computers, software has been used for making development more efficient, like CAD and CAE systems in the 1980s. However, these solutions helped engineers solve a specific problem, rather than addressing the overall engineering process.

In the 1990s, we saw the emergence of software tools known as require­ments management tools, which cover large parts of the development process. Rather than dealing with documents, they manage fine-grained items and allow organizations to define a data model that makes dealing with changes and traceability more effective. Key market players include:

• IBM Dynamic Object-Oriented Requirements System (DOORS) (“classical” DOORS)

• IBM Rational DOORS Next Generation (the codebase is not related to classical DOORS)

• Jama Software Jama Connect

• Siemens Polarion

• Intland Software codebeamer

• PTC Requirements Management and Validation (formally PTC Integrity)

These tools break development artifacts into smaller chunks, but most of the data is still text, written by people for people. There may be formulas, figures, and schematics, but the actual content cannot be processed by machines — they only help with content management.

Some of these tools offer AI add-ons. These typically analyze individual requirements to provide some quality feedback. Although this can be useful, it only scratches the surface of what is possible.

Another category of software tools addresses product lifecycle management, but these focus on manufactur­ing and bill-of-materials management and so are not central to the product development process.

Agile Product Development

In the early 2000s, Agile methods became popular in software development, and their application has now spread to product development and beyond.

Agile practices involve rapid iteration cycles and close collaboration with all stakeholders, including customers, in cross-functional teams. Agile addresses the problem of slow development cycles and commu­nication problems. Agile practices also tend to embrace new technologies that take friction out of development (e.g., through automation).

Applying Agile development in the context of devel­oping complex products doesn’t change that much, however. This is particularly true if the system under development involves safety. Safety regulation embraces systems engineering and the idea behind the V-Model.

There are specialized tools for Agile development, but most of them have a strong software focus (e.g., application lifecycle management). For developing complex products, all vendors of the requirements management tools mentioned above support frameworks for working in a more Agile way.

Model-Based Systems Engineering

Model-based systems engineering (MBSE) takes the idea behind the V-Model to an extreme, abandoning documents and replacing them with a formal systems model. Documents only exist in the form of specialized views of the model.

MBSE relies on a modeling language instead of human language. Prominent examples include Unified Modeling Language (UML), Systems Modeling Language (SysML), Business Process Modeling Notation (BPMN), and Event-B. SysML is by far the most visible systems modeling language, while UML is prevalent for pure software systems.

Better-known MBSE tools include Sparx Systems Enterprise Architect, IBM Rational Rhapsody, and Dassault Systèmes Cameo Systems Modeler. Intro­ducing an MBSE tool into an organization almost always requires training, coach­ing, and consulting to make it successful; the tools and methods are too complicated to use without expert help.

Modeling languages can be visual or textual, open or proprietary. But they all differ from traditional approaches in their ability to allow machines to directly reason. This opens the door to a number of useful capabilities, like consistency checks, test generation, and simulation, all of which can be automated.

MBSE has been around for 20 years and is widely used in avionics, defense, rail, and automotive. However, MBSE requires a large up-front investment with the promise of a positive ROI some­where down the road; this is particularly true for organizations that build tailored, complex systems and therefore must deal with variants (see Figure 2).¹ To date, only a few small organizations have been willing to risk this type of large investment.

There are also cultural challenges, including the fact that many stakeholders are unwilling or unable to learn a new modeling language and the correspond­ing methods. This severely limits MBSE adoption.

Applying AI to Product Development

Systems engineering and Agile both struggle to keep up with today’s exponential growth in complexity. MBSE was built to address complexity, but it requires a large up-front investment and often meets with cultural resistance. Those are two very different challenges, but both can be addressed with AI — at least in principle.

AI is already used to support specific traditional development activities like quality checking and detecting similarities. Similarly, AI is being employed to extract models from MBSE text.

However, AI in product development is in its infancy. This is partly because the market for product develop­ment is small compared to areas where AI is already adding value, such as marketing.

The most visible player right now is IBM, which has a portfolio of tools for product development that includes DOORS Next for requirements engineering and Rhapsody for modeling. IBM also has a prominent AI engine: Watson. As expected, IBM adapted Watson as an add-on for DOORS Next. Currently, this add-on analyzes individual requirements to rate their quality.

Several smaller players are also active, trying to climb their way toward wider recognition. No clear leader has emerged, but it’s early days; we’ll see a lot of activity in this area over the next five years.

Finally, it’s important to recognize that there are many AI subfields, each at a different development level, which we explore more fully next.

Natural Language Processing

By far, the most activity in AI related to product development is in natural language processing (NLP). This is not surprising, since most of the artifacts in product development are still text: requirements, risk items, test cases, and so on. This AI subfield is well suited to both traditional and Agile product development.

By far the most common application of NLP in requirements engineering is quality scoring. Two big players, IBM (DOORS) and Siemens (Polarion), have solutions for this.

Although these solutions work and provide value to practitioners, they do not address the problem of rising product development complexity. These solutions only analyze one requirement at a time, making it impossible to spot contradictions between two requirements.

There is a lot of research aimed at complete product descriptions, rather than focusing on individual items. Analyzing complete product descriptions opens the door to a number of interesting applications:

• Identifying issues that can be spotted only in context, such as gaps, contradictions, and ambiguities

• Improving traceability (traceability analysis is possible only when all items are analyzed together)

• Extracting MBSE models from natural language specifications

The last point in particular could be transformative. It would provide a bridge from traditional product development to MBSE, making MBSE’s benefits accessible to a much wider audience.

These ideas have not yet been realized in commercial products, but both academic and industry researchers are working toward that goal.

Machine Learning

Machine learning (ML) employs algorithms that use data to improve automatically through experience. In product development, a simple example is the categorization of items. For instance, you could assign a technical category like software, hardware, or electronics to an item. This is well understood but of limited value, as it does not solve the problem of rising complexity.

AI is already applied in product support — providing first-level support or analyzing the customer’s mood. A more relevant activity involves analysis and clustering of conversations to find critical or highly visible bugs or identify important feature requests or unmet needs. These can feed into the product roadmap to make future products more successful. Similar information can be extracted from marketing data or product telemetrics.

A more complex problem is the creation and analysis of traceability, a key aspect of systems engineering. Practitioners create and maintain the traceability in both traditional product development and Agile development. The company Relatics follows this approach by analyzing arbitrary data sources for traceability management. More players will certainly follow, but commercial solutions are currently scarce.

Some industries rely more on large-scale traceability systems than others. For example, the automotive industry has strict regulations regarding traceability and a large volume of requirements. It’s not unusual for manufacturers to provide suppliers with specifications consisting of tens of thousands of requirements. The scale alone makes this use case interesting for ML.

In MBSE, traceability tends to follow clearly defined rules, limiting the use of ML. However, even MBSE starts with textual requirements, and ML could support the allocation of those requirements. Once again, there are few commercially available solutions.

Interactive AI

Human-computer interaction is already used exten­sively in customer service and support, most visibly in the form of chatbots and voice-based applications, but there is unexplored potential for such systems in product development. Today’s problems are solved primarily by hiring outside experts. The goal is to raise the organization’s maturity to make it more effective with its existing resources.

Interactive AI has the potential to replace experts, at least in some areas. This could take the form of interactive wizards that help set up a development project or interactively improve the system architecture in a dialog format.

If such systems already exist, they’re most likely for internal use at large organizations. It is less challenging to build such a system in a domain-specific context. For instance, a German manufacturer of lighting systems built a system to guide users in capturing the system behavior in a domain-specific language. Using the stakeholder input, the system makes suggestions for translating the input into a machine-readable format. Building a universal system is much more challenging, primarily due to missing training data.

Training Data

Many AI systems rely on a large body of training data. This is particularly challenging in product development and is likely the reason we haven’t seen much progress compared to other sectors. Detailed product descrip­tions are often considered core intellectual property; few organizations are willing to share the descriptions, even with an NDA in place.

There are workarounds. For instance, user manuals, data sheets, and maintenance handbooks can be sub­stitutes, especially for high-level requirements. But this is not enough for a holistic body of knowledge on product development and therefore only of limited use.

This is the reason large multinationals are active in this area: they have enough training data to make AI systems useful, at least internally. It will be interesting to see what these organizations publish in the coming years.

AI Platforms

Until off-the-shelf AI product development systems are available, organizations must build their own. There are many players in this space, including established companies like IBM, Google, and Microsoft, and newcomers like Dataiku and SambaNova Systems.

There’s little question we’ll soon have an army of consultants, integrators, and service providers that promise to find areas of applications of AI technologies for their customers, in product development and beyond. Considering how specific product development is, retaining such experts will be a good idea for many organizations. As always, the key to success will be providing clear strategic direction and strong oversight.

Confidentiality is a major challenge with these platforms, as they typically operate in the cloud. On-premise systems are available but typically have limitations. For instance, they cannot access cloud-based analysis systems by third-party providers that continuously evolve their AI engines.

Conclusion

Organizations must respond to the pressure of rising complexity. Thus far, we’ve seen companies respond by adding either manpower (work harder) or exper­tise through consulting, training, and tooling (work smarter).

Adding more and better tools has been practiced successfully since the 1990s, starting with CAD systems, requirements management systems, and simulation tools. In the 2000s, we saw a certain amount of “tool-fatigue,” with practitioners noticing that simply adding a tool did not always solve the problem. This gave a huge boost to Agile methods, which require training more than new tools. However, tools continued to provide efficiency. It is surprising how much product development work is still done using Word and Excel.

Organizations that reach their limits due to complexity growth typically turn to MBSE, but today’s MBSE tools are far too complicated to be a good starting point. Successful MBSE involves a strategic, long-term objective and clear, achievable milestones.

We’ll continue to see MBSE deployment over the next five years or so, but growth will be severely limited by the number of available experts. This opens up a huge opportunity for AI-based solutions that provide some of the benefits of MBSE without requiring a long, expensive training process.

We currently see attempts to address product com­plexity by using AI to bridge the gap between MBSE and the language of the stakeholders. Most of this work is still in the research stage. Ready-to-use solutions like IBM’s Watson barely scratch the surface of what’s possible.

In the coming years, more powerful AI systems for product development will emerge, starting with writ­ten language. Initially, these systems will improve traditional or Agile development, but they will soon focus on helping with MBSE. This will reduce (but not remove) the need for companies to pay outside experts in order to benefit from MBSE.

Such systems will guide practitioners through the development process by creating and maintaining architecture, testing traceability, identifying quality issues, and more. All these help prevent waste, reduce risk, and speed product development.

Reference

¹Madni, Azad M., and Shatad Purohit. “Economic Analysis of Model-Based Systems Engineering.” Systems, Vol. 7, No. 1, 2019.