Mashup Patterns: Designs and Examples for the Modern Enterprise
Chapter 1, Understanding Mashup Patterns
In this section you'll learn the meaning of the "long tail" and discover how mashups can help IT departments better target the long tail and meet user demand. This chapter excerpt also covers mashups and SOA, using mashups in the corporate portal and mashups and enterprise application integration.
Using mashups and SOA, EAI
The Long Tail
Although first coined to describe customers who purchase hard-to-find items, the phrase "the Long Tail" has come to have a special meaning in the world of software. Traditionally, application development dollars are directed toward those projects and enhancements demanded by the largest group of users. This practice of catering to the masses doesn't necessarily lead to an outcome with the greatest positive impact on productivity. Unfortunately, because of the huge effort involved in developing applications, it is often impractical to provide custom solutions to a lone employee or a small team, even if it would greatly increase their efficiency (Figure 1.7). Thus only the "head" of the application demand curve is ever addressed. The exact cutoff point isn't fixed and will vary by organization, although the Pareto principle or "80-20" rule suggests that 80% of application development efforts will benefit only 20% of your users.
Figure 1.7 The Long Tail
The cumulative potential of unfulfilled Long Tail opportunities exceeds that of the "head" of the curve. Alas, fulfilling the requirements of the remaining 80% of your staff might seem an impossible goal. Most technology departments do not have enough staff to meet the needs of each individual user. Unless there is a way for developers to become drastically more productive or for end users to solve their own problems, the prospects for meeting unmet demand seem bleak.
Meeting User Demand
Enter the mashup. Armed with powerful new tools that leverage the resources of the Internet, developers and power users can quickly assemble products to target the Long Tail. We are witness to the dawn of a new era in technology. Mashups are making IT more agile and empowering individuals to create their own solutions. The Long Tail is useful from an analysis standpoint only if it represents the universe of possible solutions that can be constructed. Consider the mashup example in "A Sample Mashup Use Case."
A Sample Mashup Use Case
There are countless examples where mashups can benefit an enterprise, and they needn't be complex. Consider the following example.
Every day, the employees of a firm have numerous conference calls to discuss project planning, resource management, and corporate strategy. Whenever someone new joins the conference, there is a "beep" that announces that individual's presence. The first ten minutes of every call go something like this:
"Hi, who's on the line?"
"It's me, Rob."
"Hi, who's on the line?"
"It's me, Maureen."
On each call, valuable time is wasted while the moderator takes attendance and furiously scribbles down names. Later on, he may try and match those (frequently misspelled) names to an email address or telephone number.
We can save time and expedite the meeting with a simple mashup. First, we visit the conference call Web site and grab the participant's caller ID directly from the Web page. Next, we look up those numbers in the firm's online corporate directory (where we also get the corresponding email addresses). Finally, in case someone is dialing in from his or her home telephone, we use the search form on a public Internet site (such as whitepages. com) to look up any unresolved numbers.
The entire process is hidden behind a simple Web front end with a single button, labeled "Get Attendees." No more misspelled names or missed participants. No more pausing to ask latecomers to introduce themselves. Meetings start on time and everyone is happy.
As if this capability wasn't enough of a breakthrough, it opens up new possibilities for behavior tracking (also known as reality mining). You can click the "Get Attendees" button multiple times during the call to see not only who is present, but for how long. Perhaps you can tie that "duration" data to other sources. You might find that callers drop off the line in coordination with weather, traffic patterns, or surf reports.
Although the "conference call attendance" issue was experienced by almost all employees of the firm, it was never identified as a business problem. This is because developers and business users are conditioned to view their actions in discrete, isolated chunks:
- First, I sign into Application A to locate a customer's account.
- Second, I sign into Application B to check item inventory.
- Third, I sign into Application C to create a purchase order for the client.
If you accept that Applications A, B, and C are immutable (perhaps because they were purchased from an external vendor), then you will never envision a solution where you can sign into Application D once and perform these three actions in a single step. The opportunity never appears on the Long Tail.
The greatest benefit of mashups may be their influence on our thought process. When we cast off our biases about the role of technology in the workplace, we discover the folly in applying IT to only the most obvious and wellunderstood problems. Once the blinders have been removed, you'll discover a world of missed and previously unknown challenges that you can tackle. Recognizing these opportunities is just the first stage. If you don't do something about them, then you've simply added to the tangle of unmet expectations. To achieve continuous innovation, it is essential to look outside the existing methods of measuring and meeting user demand.
Mashups and the Corporate Portal
The concept of aggregating data from multiple sites inside and outside the workplace isn't new. As companies struggled to share all of their disparate applications and information resources directly with their employees, many embarked upon a quest to create a single corporate portal. An organization's portal typically provides several features:
- Single sign-on (SSO), which allows users to authenticate only once to obtain access to multiple applications.
- Multiple "portlets" or "islands" that expose information and functionality from disparate systems.
- Interaction (or integration), which allows portals to influence one another's behavior. For example, a search portlet may cause the contents of other portlets to be filtered.
- Access control, which provides for the centralized administration of which information a user may access. A user's permissions on the portal are at least as restrictive as what the user would receive if he or she logged into the underlying application directly. Portals are unique in that they may bring content together from multiple sources wherein the user has varied entitlements.
- Personalization, which allows the user limited ability to customize the layout and presentation of the site to suit his or her own specific tastes and needs.
Of course, as our examination of the "80-20" rule suggests, portals will never meet the requirements of all users, all of the time. At best, they may meet the lowest set of common requirements across a broad audience (the 80%). The most specific requirements are typically the least general (the 20%), which explains why most corporate portals typically confine themselves to broadcasting company news, managing health and benefits information, and tracking the holiday calendar. Personalization, the latecomer to the portal infrastructure, was a desperate attempt to address this shortcoming. Unfortunately, users typically don't get a say in choosing which content can be personalized or how it can be manipulated.
At my daughters' nursery school, their teacher maintains order by telling the children, "You get what you get and you don't get upset." Those days in computing are passé. Whether we are talking about the corporate business user who wants to come to the office each day to a personalized workstation or a customer who wants to view your company's information in a certain fashion that suits his Web-based applications, this is the age of individualized construction.
When the popular social networking sites MySpace and Facebook published open APIs to leverage their data and create interfaces around it, thousands of users became bona fide developers. They quickly learned to build their own personal portals. This same demographic is just now beginning to enter the Enterprise 2.0 workforce. They won't be content to operate within the confines of a single, stoic portal that restricts how they consume and manipulate information.
A new metaphor for user interaction has recently emerged that, combined with mashups, threatens the relevance of the enterprise portal. Whether you know them as widgets, gadgets, or snippets, they are the small plug-in components that originated on the Web and have migrated to the desktop (e.g., Apple Dashboard, Yahoo Widgets, Google Gadgets, Microsoft Vista Desktop Widgets).
The tools for creating these "mini-applications" have become easier to use and more familiar to a much broader audience.
If enterprise mashups are the path to user-created data and widget platforms are the environment for presenting that information, the combination of the two represent the death knell for the corporate portal. At best, it will morph into a set of core services that provide information to mashup-powered personal environments.
Mashups and Service-Oriented Architecture
Service-oriented architecture (SOA) has come to be associated with Web Services, but at its core it is more mindset than methodology. The "service" in SOA shouldn't be thought of in terms of a particular technology, but rather as a business task. The tasks are implemented in an environment that facilitates loose coupling with other services. This combination, in turn, fosters an atmosphere where developers can create new applications that reuse and recombine existing functionality. Because the services are based on open standards, they can be consumed equally well across independent development platforms.
The promise of SOA is that it addresses the Sisyphean labor of building duplicate, siloed functionality across the enterprise. Better yet, you don't have to build services yourself; you can discover and use third-party solutions. SOA is the equivalent of a home improvement store for application development. You simply fill up your shopping cart with all the raw materials and glue and nail them together in your basement to create a shiny new product. Using a traditional development mindset would place the burden on you to chop down trees for lumber or smelt the iron for nails.
The Common Object Request Broker Architecture (CORBA) was an early stab at implementing SOA—so early, in fact, that it predates the Internet explosion of the mid-1990s and even the SOA acronym itself. The level of complexity required to work with this technology was often found to outweigh its benefits, and while CORBA struggled to find its footing, newer technologies such as SOAP, XML, and Java (Enterprise Java Beans) arrived on the scene. They began to address the problems associated with CORBA's steep learning curve and security shortcomings.
Web Services emerged as a technology-agnostic interoperable solution based on open standards such as XML, WSDL, UDDI, and SOAP. Although far from perfect, SOAP-based Web Services have become the industry-preferred method for implementing SOA. The most popular method for exposing SOAP services across the enterprise is via a custom infrastructure known as an enterprise service bus (ESB). The ESB can provide additional data transformation capabilities, security, transaction support, and scalability, all while simultaneously reducing the degree of complexity exposed to service reusers. In an attempt at product differentiation, some ESB offerings service-enabled existing corporate resources (such as databases) and were themselves progenitors of the data mashup.
One point should be clear: SOA is not a revolutionary milestone but an evolutionary one. Open communication and technology standards, combined with the ubiquity of the protocols that power the Web, have finally helped SOA reach a level of maturity where its benefits exceed its costs.
Mashups represent the next leap in reuse. They initially came about when developers combined the published APIs of different Web applications to create interesting new content. The limitation of this approach was that resources had to be "mashup ready." Robust SOA environments were a hothouse for mashup growth, as they exposed componentized functionality that could be mixed together to provide new services.
You may be wondering if mashups are the latest harbinger of SOA, or the beneficiary of it. The answer is a resounding "Both!" With most vendors now using the terms "SOA" and "Web Services" interchangeably, it has become obvious that for most corporations, implementing a successful SOA will require the service-enablement of their existing applications. Mashups are a completely valid method of accomplishing this (see the "API Enabler" section in Chapter 4 and the discussion of the Quick Proof-of-Concept pattern in Chapter 7). Most mashup products allow you to create and publish Web Services either directly or via a third-party application container (e.g., WebSphere or JBoss). Likewise, mashups are voracious consumers of Web Services. Mashups gladly leverage the Web Services that SOA-centric organizations already have in place. Because mashups can produce services with the same agility that they consume them, they are a valuable addition to any service-oriented environment.
How do SOA patterns and mashup patterns relate to each other? SOA generally focuses on server-side architecture and internal corporate resources, whereas everything is fair game with mashups. Because of SOA's maturity and association with Web Services, it has achieved greater clarity regarding its capabilities, protocols, implementation, and use. This allows SOA pattern discussions to focus on high-level abstractions. Indeed, several excellent Web sites and books discuss the process of SOA-enabling the enterprise. Mashup patterns, which remain in a nascent stage of development, must focus on more practical examples. This will drive broader adoption, which in turn should to lead to consolidation and standardization similar to what SOA has achieved.
Mashups and EAI/EII
Enterprise application integration (EAI) is the practice of connecting corporate systems at the application level rather than at the data level. EAI solutions seek to streamline business processes and transactions, whereas mashups typically combine applications with the goal of providing new functionality. EAI tools rely on support for open standards such as Web Services or CORBA. If an application doesn't expose an API, one needs to be constructed programmatically. As systems and requirements evolve, there is an inevitably large carrying cost to maintain the custom integration code. When managed and funded correctly, EAI can provide the most rock-solid method of application integration. For business-critical solutions, EAI is recommended over mashups, which permit some fragility as a trade-off for the benefit of agility.
Enterprise information integration (EII) is a data management strategy for providing uniform access to all the data within an organization. The rise of "big box" stores that sell everything from baby clothing to car tires has demonstrated that patrons appreciate the convenience of one-stop shopping. Collecting data from multiple sources and providing a single point of access has similar appeal in the enterprise. EII is often easier to achieve than EAI because it simply attempts to unify information and not applications. If you think this approach sounds similar to a data mashup, you're correct. A mature EII implementation can provide new insights into data associations and facilitate rapid solution delivery. EII tools have historically focused only on back-end databases, which limits the range of information that can be collected. By comparison, mashups surpass EII in their ability to obtain data from both structured and unstructured sources.
The knowledge requirement for successfully applying EII technology is higher than that for mashups, but as with EAI the advantage is stability. You can measure the benefits of a complex EAI/EII project empirically by developing a quick mashup-based prototype (see "Quick Proof-of-Concept," Chapter 7). This effort may help determine whether the potential benefits justify the considerable cost and time required to carry out a formal implementation.
Continue to the next section: The mashup and the user
Download Chapter 1, Understanding Mashup Patterns
Read other excerpts and download more sample chapters from our CRM and call center bookshelf