Mashup Patterns: Designs and Examples for the Modern Enterprise
Chapter 1, Understanding Mashup Patterns
In this section you'll learn about the mashup and the user, the future of enterprise mashups and how mashups may impact organizations worldwide. Discover the benefits to using mashups in your organization and find out about the fragility score. You'll also learn how mashups complement Software as a Service (SaaS) systems.
The mashup and the user
Mashups and Software as a Service
In contrast to the architectural style and Web Service implementation strategy of SOA, software as a service (SaaS) is a business model. SaaS is the latest incarnation of the Internet-boom idea of an application service provider (ASP). Under the SaaS plan, businesses do not invest money to develop and host applications internally, but instead rent the functionality they need from an external service provider. End-user interaction with applications typically occurs via a prebuilt Web interface. The customer's business data is then fed into the system manually, using Web forms, or programmatically, using a Web Service API.
To appeal to as broad a market base as possible, most SaaS providers have focused on generic services and priced them competitively (a fee of less than $100 per service is not uncommon). Exposing macro capabilities and parameterizing functionality allows customers to achieve some degree of customization.
One of the most prominent success stories in SaaS is Salesforce.com. This "zero-infrastructure" customer relationship management (CRM) platform provides services to thousands of businesses worldwide. Small and large customers alike are able to start using the hosted service almost immediately without deploying custom hardware. The success of Salesforce.com has led many to assume SaaS is particularly well suited to CRM and sales force automation. In reality, this isn't the case. WebEx, a Web-based conference and collaboration solution, has achieved adoption on an even larger scale. Google Apps is an example of a viable alternative to traditional desktop software. It serves up a business-focused mail, spreadsheet, and word processing suite at a fraction of the cost of Microsoft Office. Many commercial vendors are exploring SaaS to create new revenue streams.
Assuming SaaS products can meet technical and functional user requirements, two key challenges must be overcome before SaaS can succeed as a general distribution model. First, firms must be comfortable with the notion that their data is housed externally to the organization. It seems that there's a new story almost every day in the press about missing hard drives or accidentally leaked personal information. SaaS providers may have better security than many of their clients, but the abdication of data management to a third party is still a tough pill for many corporations to swallow. The second obstacle for SaaS is availability. For mission-critical applications, the network remains a potentially dangerous point of failure.
Mashups are a natural complement to SaaS. Perhaps there are SaaS solutions that appeal to your organization, but you have held back on implementing them because you couldn't get exactly the functionality you required from a single provider. Maybe the SaaS product is extensible, but you don't want to invest time and money in duplicating functionality you've already built internally. Mashup patterns such as Workflow (see Chapter 5) and Content Integration (see Chapter 6) can be used to link an external solution and internal products together. With SaaS and mashups, you may be able to maintain the bulk of your confidential data internally and send the hosted application only small subsets of data for processing. If the network link to the SaaS vendor fails, at least you will still have local access to your data.
If you're thinking about testing the SaaS waters as a vendor, then applying SOA via mashups can help you get started. The API Enabler (see Chapter 4) and Quick Proof-of-Concept (see Chapter 7) patterns are excellent means of creating a Web interface to your existing resources. You can use the Load Testing pattern (see Chapter 8) to see how your systems scale under heavy user activity.
SaaS shares another characteristic with mashups: It may already be in use in your company without your knowledge. Because this model requires only a Web browser and no special infrastructure, it is easy for end users to circumvent IT and obtain applications directly. It is crucial that an IT department doesn't have a monitoring and enforcement policy based solely on policing internal data centers. IT personnel need to engage with the business users and educate them about the risks and rewards of SaaS and the effects these decisions will have on future growth. Internal checkpoints with purchasing and legal departments are a necessity, too. All service level agreements (SLAs) should be reviewed and signed by appropriate parties, and attempts to expense software purchases that have not been vetted by IT should raise a warning flag. Otherwise, SaaS can sneak into your organization on a corporate credit card.
Mashups and the User
Make no mistake about it—despite the recent buzz around Enterprise 2.0, people have been creating mashups for many years. Of course, the process to this point has been overwhelmingly manual. Microsoft Excel is arguably the father of the corporate data mashup. For years, Excel end users have cut-and-pasted data to feed their calculation engines. Spreadsheet-based solutions have spread throughout the enterprise without the involvement of IT. Mashup tools enable the automation of this aggregation process, and a new clan of users is poised to run wild with the technology.
A culture of individualism is clearly emerging in today's world. People no longer plan their evenings around what TV networks schedule for them to watch, for example. Instead, they record their favorite shows onto digital video recorders (DVRs) or watch movies and shows on their computers and mobile devices. Similarly, the recording industry no longer has a stranglehold over music distribution. Newspaper readership is down, as more individuals choose to consult RSS feeds and blogs instead of purchasing the printed documents. People can even create personalized clothing and sneakers online. Members of the public have evolved from docile consumers into "prosumers." Products and services are moving away from mass markets and being shaped by the people who consume them. Likewise, a fundamental shift has occurred in software development. Armed with new tools and the skills to use them, users aren't waiting for IT to build solutions—they're doing it themselves.
Should organizations facilitate these individuals' efforts, or rein them in? For years, the mantra of professional software development was "Separate business logic from presentation logic." Programmers religiously structured their code around that principle but ignored the logical conclusion: The best shepherd of business expertise is not the IT department, but the business users themselves.
The inclination for IT departments to view user-led efforts in an adversarial light increases when IT experts believe that their "home turf"—application development—is threatened. IT needs the occasional reminder that in any development effort, it is the users who are the key to defining metrics for success. Besides, users are already creating mashups anyway, albeit humanpowered ones.
Gartner has said mashups will make IT even more critical to business operations, so a knee-jerk rejection to their emergence is not necessarily in the best interests of the firm. Rather than deny business users the tools that can increase their productivity, IT needs to embrace a new model. Besides, starting with a mashup product won't get you a business solution any more than staring at a word processor will get you the next great novel.26 Because IT personnel clearly cannot scale to meet the requirements of each particular user, they should leverage the potential of mashups and work in partnership with the business associates to train a new class of self-serve builders. This effort is akin to adding hundreds of new developers at virtually no additional cost.
It's a common assumption that the latest generation of developers is intuitively suited to filling this role. Affectionately termed the "Millennials" or "Generation Y," these individuals came of age during the Internet boom of the last decade and are inherently comfortable with technology. Millennials, green with inexperience and giddy about tinkering, question everything. This behavior stands in stark contrast to that of the entrenched workforce, whose habits of working in a particular manner condition them to no longer question the "why."
Many companies are rushing to embrace Web 2.0 ideals such as mashups, social networks, wikis, and blogs not because they have inherent value, but rather because the firms think this practice will attract the "new thinkers." In reality, instead of abdicating responsibility for innovation to a younger generation or applying technology Band-Aids, firms need to cultivate an environment of creativity and collaboration for their employees regardless of their physical age. Any firm can realize the value of mashups and Enterprise 2.0 so long as its managers are capable of taking a critical look at their workplace and realizing they don't need to settle for "good enough" any more.
The "guerrilla-style" approach of mashup development is not without its drawbacks, of course. Most business users do not fully grasp the challenges in providing scalability, reliability, business continuity, disaster recovery, security, and fault tolerance. If users are permitted to develop ad hoc solutions, IT must provide an environment that cultivates these best practices.
A Patterns Primer
The benefits of enterprise mashups are communicated through a concept known as a pattern. If you've ever baked holiday cookies, then you already have some idea of what a pattern is and how it works. Suppose you want to make a tray of chocolate-chip heart-shaped cookies. After you mix the dough, you roll it out and grab your cookie cutter. You use the cutter to press out a series of identical shapes. Afterward, you decide some oatmeal raisin hearts would be nice, so you mix a fresh batch of ingredients and cut out another series of hearts. The cookie cutter is a form of pattern. The different types of dough are the specific situations, or "use cases," where the pattern is applied. A pattern doesn't solve a problem in itself. It's just a general form that helps you think about the structure of the solution (what shaped cookie, in this example).
The remaining chapters of this book present a number of patterns, along with some examples to illustrate how they work in an enterprise context. Don't throw out the pattern if you don't like the dough! Every business has a different flavor, and the key to success with patterns is figuring out which one is yours. You can use the samples that fill out this book to help identify the mashup ingredients your organization already has. Apply the appropriate mashup pattern and you have a recipe for success.
The Fragility Factor
It may seem that the title of this book is an oxymoron. How can something as ad hoc and unstructured as Web scraping be coupled with something so formal and structured as a pattern? Ideally, the previous discussion of how mashups work under the hood will have made you more comfortable with the technology.
If you think reverse-engineering Web pages still doesn't sound like the type of rock-solid approach that a professional developer should be using, I don't blame you. One of the core tenets of software engineering is that applications should behave in a reliable and predictable manner. Web harvesting—although a great deal more reliable than screen scraping—is inherently unstable if you don't control the Web sites from which you extract data. Because you can't determine when a scrape-based solution might break, you should never employ this approach on a mission-critical system.
If you have the chance to help your firm gain a competitive advantage or reduce costs—even if just for a limited time—you should explore the opportunity. There is nothing wrong with an application that has a short lifespan, so long as you don't create a situation where the cost of remediating or retiring the solution exceeds the achieved benefit. The rapid speed with which mashups can be developed means occasional remediation isn't a time-consuming task. Plus, quick release cycles translate into more chances for exploratory development, which in turn can lead to the discovery of new uses or solutions.
The patterns in this book all adhere to this basic premise. You won't find examples of settling stock trades or sending online payments, even though mashups can facilitate those tasks. It's simply irresponsible to use the technology in this manner. Like any development effort, a mashup solution will require regular maintenance over its lifetime. Unlike with traditional applications, you may not be able to determine the time when this work will be required. Web Service APIs can change, RSS feeds can be restructured, or site redesigns may temporarily toss a monkey-wrench into your application's internal workings. Because of these possibilities, you should implement mashup-based solutions only where you can tolerate temporary downtime that may occur at unexpected intervals.
The fragility score is an ad hoc rating based on a number of factors:
- A mashup pattern that relies on a single Web site (e.g., Infinite Monkeys, Time Series, Feed Factory, Accessibility, API Enabler, Filter, Field Medic) is less fragile because there is only a single point of potential failure.
- A multisite-based pattern (e.g., Workflow, Super Search, Location Mapping, Content Migration) is more fragile with each additional site that it leverages.
- Mashups that employ Web harvesting are generally more fragile than those that use feeds (RSS, Atom). Feeds are, in turn, more fragile than Web Service APIs. APIs are the most stable integration point because they reflect a site's commitment to expose data and functionality.
- Mashups that mine data from "hobby" sources have a greater risk of failing. For example, obtaining local weather data from the U.S. governmentfunded National Oceanic and Atmospheric Administration's (NOAA) weather site (http://www.nws.noaa.gov/) is probably a safer bet than obtaining the information from your local high school or radio station.
- For-profit sites may exert legal pressure to halt mashups (see the Sticky Fingers anti-pattern).
- Mashups that use boutique data not widely available on the Internet are at high risk. What are your alternatives if the site suddenly vanishes one day?
Each pattern template described in this book contains a fragility score ranging from 1 glass (the least fragile) to 5 glasses (the most fragile). No pattern receives a score of zero, because even the most rigorously tested mashup-backed application always has some degree of brittleness.
The fragility score is ultimately intended to encourage thought about mashup stability. It's possible to have five sites in a multisite pattern that change less frequently than an individual Web page used in a single-site pattern. This is particularly true when vendor products and internally created systems are involved. The user interfaces of commercial and in-house applications aren't frequently redesigned. Public Web sites, in contrast, must constantly reinvent themselves in the battle to attract eyeballs.
If you create a mashup-based solution and don't acknowledge that it encapsulates some degree of uncertainty, you are just kidding yourself. Worse, you are deceiving your users, who will not be pleased when the system "mysteriously" fails one day.
In case you think only mashups have this Achilles' heel, keep in mind that any distributed system (which is what a mashup is) contains an inherent level of risk. Each additional component and the infrastructure that connects it represent another potential point of failure. So before you think, "Why the heck would I build something that might break?" consider how you have handled similar situations in the past. You can address many of these fragility issues by thinking about redundancy, monitoring, and notification up front.
The Future of Mashups
The primary goal of this book is for the reader to scan at least one pattern and realize, "I never thought you could do that!" The examples that accompany the patterns are aimed at both the business end user and the technical user. When you understand how mashups can be used to mine new information or automate traditionally manual activities, you'll never look at your workplace in quite the same way. The morass of daily problems suddenly becomes visible— but now you'll have the inspiration and knowledge to tackle them. As with the classic Design Patterns text, Mashup Patterns is intended to provide a general language that developers and the business can use to succinctly communicate about a solution ("Oh, we can use a Time Series mashup here").
It's not every day that we witness a groundbreaking advancement in application development. Most improvements occur gradually and can take years to snowball into something useful. They may require costly new investments in infrastructure or reengineering of existing resources. Or they may be confined to a narrow niche in our industry. Only the naive overlook the dangers that come with any great leap; only the foolish cite those risks as reason enough to ignore the potential benefits.
Don't let the hype surrounding mashups cause you to abandon the best practices that guide good development. Likewise, be open to thinking creatively about the problems that exist around you. Employees who face seemingly intractable problems or whose careers have trained them to ignore the breakdowns in their organization will be delighted to discover that practical solutions are now available. The patterns in this book will help you get started by demonstrating how mashups can help you achieve the following goals:
- Make money for your organization
- Fill gaps not met by the existing IT infrastructure
- Create a quick proof-of-concept to explore new solutions
- Gain a competitive advantage
- Avoid "information overload"
- Expose your applications to a wider audience and more!
Enterprise 2.0 is all about You. And the potential benefits from mashups are as big as anything you can imagine.
Download Chapter 1, Understanding Mashup Patterns
Read other excerpts and download more sample chapters from our CRM and call center bookshelf