Web 2.0 and Enterprise Business Intelligence

by Langdon White

Web 2.0 has received a lot of press lately. So far, the primary use has been in creating new applications for users of the public Internet. In this article, I will discuss a potential alternative use for enterprise users. Specifically, applying these techniques to business intelligence (BI) solutions creates applications that are better adopted by users and also allows users to increase the "intelligence" created by the applications.

The subject of this article is not to introduce the reader to Web 2.0 or BI explicitly but rather to explain how the convergence of these ideas may be useful. Therefore, I will focus on which aspects of BI and Web 2.0 are relevant, how to incorporate them into new systems, and what is minimally required to support these systems in the enterprise.

BUSINESS INTELLIGENCE

According to Wikipedia, business intelligence is:

... a broad category of applications and technologies for gathering, providing access to, and analyzing data for the purpose of helping enterprise users make better business decisions. The term implies having a comprehensive knowledge of all of the factors that affect your business. [1]

While this is a good definition, one can define BI more broadly as the process of collecting information and combining it in an "intelligent" way to help people make better decisions.

The typical BI solution offers a user interface made up of static and parameter-based reports; semi-free-form, ad hoc reports; and, more recently, clunky dashboard-type components. Most BI tools allow implementers to create new user interfaces using the product's APIs, but few people/companies do. As a result, most users find the applications difficult to use and understand. An alternative solution is to leverage the ideas of Web 2.0 and modern user interface design to efficiently create interfaces that allow users to understand, use, and participate in the information they are presented, while still allowing the implementers to exploit the strong functionality provided by the BI tools as back ends.

WEB 2.0

Web 2.0 is described as many things, but Tim O'Reilly's formulation -- "there's something qualitatively different about today's Web" [2] -- is a nice summary of the idea. In this article, the key point is the idea of "continuous improvement of applications" or the "perpetual beta." When application owners and implementers accept this concept, the iterative release of less-than-perfect applications becomes possible. This prevents applications from being "defunded" because of lack of relevance; it minimizes the costs of implementation by focusing on the actual needs of users; and it enables development teams to remain mentally engaged in the maintenance of applications.

Four key aspects of Web 2.0 that are relevant to the convergence of Web 2.0 and BI are:

1. Employing public and private Web services APIs to:
   • Ease, and increase the speed of, implementation
   • Encourage new and unique ideas for applications
2. Creating Rich Internet Applications (RIAs) to:
   • Ease, and increase the speed of, implementation
   • Allow dynamic, engaging, modern user interfaces
3. Leveraging Web syndication to:
   • Deliver appropriate information to users when they need it
4. Promoting contribution of data/metadata by users to:
   • Allow improved BI via the contribution of new data and metadata
   • Enable improved access to information by allowing users to cross-reference

WEB SERVICES APIS

Web services (and other service-based protocols) allow enterprises to create applications that combine features of multiple services into new applications. Service-based enterprise environments also encourage shorter development cycles because a team can create a system by implementing only a user interface and some lightweight business logic, often allowing the delivery of a system in a matter of days.

With the popularity of Web services, many "Internet companies" are now providing public APIs to increase access to the services they offer. The functionality provided varies from blog engines to compute grids, data grids, mapping, and even dating services. These services allow companies to consider even more unusual applications because they are no longer limited to the data/functionality housed within their own enterprise. (Note: Many public providers are still only providing these APIs for noncommercial use, so it is important to review the usage rules.)

On the public Internet, these applications are generally referred to as "mashups." In a corporate setting, they are called "composite applications." To give just one example, an application might leverage mapping information in order to render sales information. My colleagues and I have used Google Maps and internal and external sales data to demonstrate this functionality (see Figure 1). ¹

Consider the implementation of such an application within a typical BI platform today. Chart/graph components would need to be configured, leveraging data that is queried and composed to meet the needs of the tools' APIs. In contrast, by employing loosely coupled services, a developer quickly composed the solution shown in Figure 1 by manipulating and transforming data in lightweight JavaScript.

RICH INTERNET APPLICATIONS

Mostly a term invented by user interface folks to refer to a "new/better" user interface technique, RIA is just a way to describe an application that is delivered like a Web application but allows a user experience more akin to a fat client. In the case of Adobe's Flash/Flex applications, the application is downloaded and then run in a virtual machine (VM) inside a Web browser. Other techniques generally revolve around Ajax, which takes advantage of the relatively new ability to make server calls from within JavaScript and retrieve information without reloading the browser window. Older techniques included using CAB files from Microsoft and delivering ActiveX controls, Java applets, and other more custom solutions.

While the RIA concept has been around for quite a while, the barrier to entry -- particularly for people exclusively focused on user interface design using Flash -- has gotten much lower. This lower barrier means that systems that use these techniques will become much more prevalent and faster to implement.

The example shown in Figure 1 leverages the Ajax concept by combining the data using JavaScript on the client side. Essentially, the two servers, Google Maps and Acme Enterprise Sales, know nothing about the application or what the developer is trying to present to the user. The application may still have its own back end, particularly for storing new information provided by the user, but the services don't have, nor do they need to have, any information about the users. There is one major caveat; authentication and authorization rules are often required for access to data and services. As a result, the server applications must be aware of the user in order to allow access. Generally, this is best achieved through a robust single sign-on (SSO) solution.

To deliver the experience in Figure 1 without such capabilities would require custom, proprietary downloads of application components. With RIA solutions, it becomes possible to compose such an interface of visual elements and services via lightweight scripting executed in VMs that are ubiquitous across browsers.

LEVERAGING WEB SYNDICATION

Web syndication, often more broadly thought of in the simple wrapper of RSS, is not typically considered an explicit part of Web 2.0. Nevertheless, the concept definitely belongs in the catalog of tools that are making a "qualitative difference in today's Web." For the purposes of this article, RSS includes Really Simple Syndication (RSS 2.0), Rich Site Summary (RSS 0.91, RSS 1.0), RDF Site Summary (RSS 0.9 and 1.0), and Atom.

The best way to describe the potential of this technology is to first understand the architecture and its capabilities and then look at some examples of potential user scenarios. Figure 2 displays an example conceptual architecture for a "Web 2.0 RSS" infrastructure.

Using such an architecture, an enterprise could:

1. Provide external RSS feeds from appropriate business and IT news sources.

2. Deploy an enterprise-wide RSS aggregator. For now, server-based, multiuser, caching RSS aggregators are not yet available on the market and, as a result, must be custom developed. The IT department should also recommend or deploy an RSS aggregator for its users.

3. Encourage internal news providers to make their information available via RSS feed to the aggregator infrastructure.

4. Allow "novel" RSS feeds into the environment. A particularly useful example is custom search agents that update changes of search results via RSS feeds (e.g., Monster job search agents, which recently added RSS). Another example would be to implement RSS feeds that update based on changes to application user interface components. Basically, any time an alert would make sense, an RSS feed should be considered.

5. Consider implementing language translation services in the aggregator. With the globalization of so many companies, the need for simple, relatively low-quality language translation is becoming increasingly important. (High-quality translation would be better, of course, but it may not be worth the expense.) Aggregating all material to a central repository allows users to request language translation for certain feeds while the aggregator only has to perform the translation once.

6. Enable semantic translation. Many enterprise search vendors are offering semantic translation as part of their solutions, either within their product(s) or via third parties. Implementing such components will not only improve the search agents, but may also allow other translations (e.g., conversion of phrasing to local, business-unit dialects) via the aggregator.

7. Implement location-aware services. The "publisher" in the infrastructure allows users to leverage the system's awareness of their physical location, the time (particularly a user's current time zone), and the multitude of devices that the users possess (BlackBerries, cell phones, printers, laptops, etc.).

Some examples may demonstrate the possibilities a Web 2.0 RSS architecture would allow:

John gets to work at 8:30 every morning and likes to read his custom "newspaper" in hard copy. Using RSS technology, we can deliver the appropriate feeds to his printer at 8:25 every morning that he is in the office (as determined from his enterprise calendar) so that he can walk in and pick them up.

Jane wants to get hourly stock and local region sales volume information during the workday (not the trading day) on her BlackBerry whenever she is working. When she is at home, she would like to receive only one update a day. With some simple rules and her GPS information, this infrastructure can respond appropriately in either case.

As you can see, the possibilities get very interesting very fast, yet the infrastructure is relatively straightforward.

PROMOTE CONTRIBUTION OF DATA/METADATA BY USERS

In the Web 2.0 world, users regularly contribute data about data. For example, the raison d'etre at del.icio.us is the tags that users create and the links that result, which create almost a shared "bibliography" for the Internet. For lack of a better term, one can only refer to this data as "metadata."

The problem with using the term "metadata," of course, is that it is widely used to denote the data that describes an organization's data in an agreed-upon way. The metadata I'm describing here is almost the complete opposite of that definition. This new conception of metadata encourages all users to contribute all data description information and relies on systems to coalesce the information in a usable way. Until the community creates a new term, I will use the term "metadata" to also describe this phenomenon.

In the context of BI, it is vital to gather as much of this metadata on your data as possible. Web 2.0 BI applications create metadata by allowing users to comment on data, correct information, "flag" useful information/application components, and so on.

Allowing users to contribute as much as possible requires that the enterprise have an open, well-defined, robust environment in which to capture the metadata. The environment must also be able to support large quantities of metadata of mixed quality. As a result, the environment must use modern techniques to coalesce the metadata around the data that it describes and develop methods to rank its quality. While there are no tools to support these needs (yet), Web 2.0 capabilities on the public Internet provide examples for solutions. Specifically, the "coalescing" may be achieved via tagging and tag clouds (e.g., del.icio.us, StyleFeeder), while quality rankings may be determined by allowing users to self-regulate through commenting on or editing other users' comments (e.g., Wikipedia).

IMPLEMENTATION

While this article is primarily focused on encouraging readers to think about Web 2.0 and how it can be applied to BI, in this section I will provide an overview of a few best practices for implementing these solutions and point out some potential pitfalls.

FLEXIBLE TECHNICAL ARCHITECTURE

A flexible, enterprise architecture, particularly one supporting a service model, is usually very important to Web 2.0 solutions. Most industry experts recommend implementing a service-oriented architecture (SOA) to support this goal. Without a services-enabled enterprise, an application team must spend a significant amount of time developing business process services, thus slowing the development of the solution they are actually tasked with.

Not that you are precluded from implementing a Web 2.0 application if you don't already have an SOA; in fact, it may be a good way to organically grow an SOA implementation. However, you should ensure that you have sufficient time and subject matter expert support to implement the required services in a reusable way.

SEMANTIC MAPPING

A common environmental issue, particularly when using a service-based architecture, is the problem of semantic collisions. Typically there are two major types: data structure conflicts and language conflicts.

Generally, data structure conflicts are a well-understood issue when working in an enterprise environment. They must be resolved, but they are beyond the scope of this article.

For language conflicts, the enterprise should consider a robust solution that supports the creation of taxonomies and ontologies and generally supports the concepts of the Semantic Web. The solution should allow "translation" for the typical equivalence, hierarchical, and associative relationships.

BUSINESS, USER, AND SYSTEM BALANCE

Implementing these types of solutions requires trading off between the needs of the business, the users, and the system (or IT department). Typically, the business has financial and time-to-market constraints. The users need to have systems that are usable and that allow them to perform their tasks in the most efficient manner possible. The IT department has constraints on human resource availability and technical infrastructure abilities. None of these groups should "get its way"; a successful solution recognizes the hard constraints versus the soft constraints of each group and strives to achieve a positive result for all constituents.

Companies have to strike the right balance between fiscal concerns, time-to-market pressures, and user requests. Because Web 2.0 enables more efficient implementation of solutions, we can often meet more of the users' demands than we could before.

AN AGILE DEVELOPMENT METHOD

Many Web 2.0 gurus and proponents suggest that agile methods are the only way to deliver Web 2.0 applications. This is not true. However, there is an aspect of agile methods that is important, which is an iterative release cycle. The iterations should be short -- say, six weeks or less -- to allow the "continuous improvement" model discussed above to really work. Agile methods are a perfectly good way to implement these solutions, but it is important to remember that they are not the only way.

ALTERNATIVE LANGUAGES (E.G., RUBY, PYTHON)

Another common myth about these types of solutions is that they must be implemented in the "modern, lightweight languages" that are getting a lot of attention lately. In fact, while Ruby on Rails, Python, and others are perfectly good languages in which to implement these solutions, Java and .NET are fine as well. Generally, the only difference in programming these applications involves implementing a richer client experience, which may leverage tools such as Ajax or Flash/Flex.

CONCLUSION

Where does all this get you? In a nutshell, Web 2.0 allows the enterprise to deliver BI solutions that reach more users in a more efficient manner than current techniques.

Modern user interfaces that allow user input and provide accurate data can reach the 95% of users who are not the power users of BI tools and give them the business intelligence they need to make better decisions. A service-based infrastructure and an iterative development method allow implementers to focus on quickly providing users the systems they need (rather than a deep understanding of the business rules involved), thereby improving the efficiency of development. Implementing a Web syndication technique allows systems to reach users in a more efficient and timely manner, thereby improving the quality of the BI information. In short, an enterprise that supports Web 2.0 solutions to enable BI improves the "intelligence" of the system through increased user engagement, more efficient development, and agile response to change.

The development of a "Web 2.0 infrastructure" may seem like a daunting task, and it is, but an enterprise may move slowly in this direction by choosing pieces and implementing them one at a time. The benefit to the business may not be as dramatic or world-changing as a greenfield system, but there are immediate benefits to implementing components while moving the enterprise to a complete solution.

NOTES

¹As a result of usage policies, this mashup is merely for demonstration, not production, purposes. Thanks to Guohu Huang, Laurie Richards, and Maksim Roslyakov for their efforts in creating the mashup.

REFERENCES

1. "Business intelligence." Wikipedia (http://en.wikipedia.org/wiki/Business_Intelligence).

2. O'Reilly, Tim. "Not 2.0?" O'Reilly Radar, 5 August 2005 (http://radar.oreilly.com/archives/2005/08/not_20.html).

ABOUT THE AUTHOR

Langdon D. White is Director of Global Engineering of Architecture Services for Keane, Inc., a $1 billion business process and IT services firm with more than 10,000 employees. Mr. White manages a global team of technical consultants and is responsible for the quality of all technical deliverables produced by Architecture Services.

Over the past six years, Mr. White has served in a variety of roles for Architecture Services, including research and development, consulting, and management. In each of these areas, he has overseen and participated in the delivery of many high-value, technical solutions for Fortune 100 clients and has delivered numerous technical industry talks and written many articles. Mr. White can be reached at Tel: +1 617 575 8212; E-mail: lwhite@keane.com.