AJAX DESIGN PATTERNS IS A REFERENCE FOR DEVELOPERS, DESIGNERS, AND MANAGERS WHO WANT TO know how Ajax is being used in the real world. Ajax is a new label to describe rich, desktop-like Internet applications that run in standard web browsers and do not require any special plugins. Its popularity stems from high-profile Ajax applications like Gmail, Google Maps, and 37signals' Basecamp, and it is becoming popular in the enterprise as well.

The patterns in this book fall into four categories. Foundational Technology patterns overview the raw technologies required for Ajax development, such as the

Who Should Not Read This Book?

If you haven't performed any web development work, you're probably better off looking for an introduction to the basic concepts before jumping into Ajax and these patterns.

Ajax development involves working with a broad range of technologies, including HTML, XML, CSS, JavaScript, and server-side development. This book will help you understand how Ajax relates to each of these and how they are often combined together, but if you're looking to learn any one of these in depth or you're seeking a reference on browser specifics, I'd recommend consulting a specialized text.

About the Examples

All the examples in this book—the tutorial code as well as the Ajax Patterns code examples—can be downloaded from http://ajaxify.com/run

Browser Specifics

Organization of This Book

Part I, Introduction

The first few chapters are a prelude to the patterns. Chapter 1, Introducing Ajax, overviews the Ajax phenomenon and discusses current trends. Chapter 2, A Pattern-Led Tutorial, is a quick-start tutorial on Ajax and the Ajax Patterns. The design concepts behind the Ajax Patterns are discussed in Chapter 3, Ajax Design: Principles and Patterns, along with an introduction to the patterns themselves.

Part II, Foundational Technology Patterns

The Foundational Technology patterns outline the "building blocks" at the heart of any Ajax application. Ajax itself is a pattern, hence the first pattern, Ajax App (Chapter 4), which acts as the root for all the Ajax Patterns. The next few chapters look at the three core technologies involved in creating an Ajax App. Display Manipulation (Chapter 5) patterns are about repainting the user interface, and Web Remoting (Chapter 6

Conventions Used in This Book

Italics

Used for names of patterns

Constant width

Used for code examples and fragments

Conventions in Code Examples

Design diagrams are based on Universal Modelling Language (UML) notation, though less formal in most cases.

In the code examples, a .phtml suffix is used for PHP files that output HTML; all other PHP files (such as business logic modules) end in .php.

As illustrated in Chapter 2, $( ) is aliased to document.getElementById( ) to reduce code clutter (inspired by the prototype library—see http://prototype.conio.net/).

Some code examples have been reformatted for the sake of clarify.

Safari® Enabled

When you see a Safari® Enabled icon on the cover of your favorite technology book, that means the book is available online through the O'Reilly Network Safari Bookshelf.

Safari offers a solution that's better than e-books. It's a virtual library that lets you easily search thousands of top tech books, cut and paste code samples, download chapters, and find quick answers when you need the most accurate, current information. Try it for free at http://safari.oreilly.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

O'Reilly Media, Inc.
1005 Gravenstein Highway North
Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
707-829-0515 (international or local)
707-829-0104 (fax)

Acknowledgments

Writing this book online means I received lots of excellent feedback throughout the process and have many people to thank. Which is another way of saying it's highly probable I'll leave someone out! If that's the case, please mail me any corrections.

My editor, Simon St.Laurent, has always been quick to respond to my queries and dealt admirably with a book being written in a somewhat unusual manner. Thanks also to proofreader Mary Brady for leaving no stone unturned, and illustrator Rob Romano for helping to make the content accessible at a glance. I also want to express my appreciation to O'Reilly for taking on the Ajax Patterns and especially for letting me write the book online, and blog and podcast about the content without restriction.

The book grew from a blog post on Ajax Patterns (http://softwareas.com/ajax-patterns

Part I. Introduction

The first few chapters are a prelude to the patterns. Chapter 1 overviews the Ajax phenomenon and discusses current trends. Chapter 2 is a tutorial on Ajax and the Ajax Patterns. The design concepts behind the Ajax Patterns are discussed in Chapter 3, along with an introduction to the patterns themselves.

Chapter 1: Introducing Ajax

Chapter 2: A Pattern-Led Tutorial

Chapter 3: Ajax Design: Principles and Patterns

Chapter 1. Introducing Ajax

BY NOW, YOU'VE PROBABLY USED AJAX ON SITES LIKE GOOGLE MAPS (HTTP://MAPS.GOOGLE.COM) Amazon's A9 search engine (http://a9.com), and Flickr (http://flickr.com). Despite their different domains, all these web sites make heavy use of Ajax. The technology lets them take a great leap forth towards the richness of standard desktop applications, and in a manner which still respects the established conventions of the Web.

Ajax and the Usable Web

No longer are you forced to wait five seconds a web page to reload every time you click on something. Ajax applications change in real-time. They let you drag boxes around instead of clicking on arrows and typing in numbers. They keep page content fresh instead of forcing you to keep hitting Refresh. They show meaningful animations instead of verbose messages.

At the heart of all this is a growing emphasis on web usability.[*] Perhaps you've heard the story of the dancing bear—everyone's impressed with it even though its skills quite frankly wouldn't get the bear into a dance academy;[*] it makes an impression because it can dance and not because of how well it well dances. The Web felt like that at first. Suddenly you could read news from the other side of the world, find hints on some obscure game, purchase a rare book. All valuable activities, regardless of how easy or difficult to perform them. Usability? We don't need no stinkin' usability!

The Rise of Ajax

On February 18, 2005, Jesse-James Garrett published an online article "Ajax: A New Approach to Web Applications" (http://www.adaptivepath.com/publications/essays/archives/000385.php). The Web was becoming richer and responsive, closing the gap with the desktop. Garrett introduced "Ajax" to label the architecture behind the new generation of rich web apps like Google Maps

Ajaxifying the Web: The Story of Portals

If you Google for "year of the portal", you'll find ample evidence that every year since 1996 has been the year of the portal. It's just around the corner, really. The idea has so much promise: the first thing you see when your browser opens up is a personal homepage with "My News" and "My Mail" and lots of other boxes just about "Me." In short, a page created by Me for Me. So why have they never really taken off? One big factor is that most portal interfaces, frankly, are unusable.[*]

Consider the problems you face in using a legacy-style portal, using Excite.com as an example—many other conventional portals work in a similar manner (Figure 1-1).[*]

Figure 1-1. Excite

Customizing the page is the most critical task, but you have to register first; each of the customization controls on the homepage will close the portal and take you to a completely different registration page.

Adding new "portlets" —the blocks of content—is rather painful. You have to click on Add/Delete Content, which will whisk you off to a tabbed configuration interface. There, you add and delete portlets by updating a list of current portlets.

Customizing an individual portlet—e.g., setting the stocks you're watching—will close the portal and send you to a special configuration page. You've lost the context.

Changing layout doesn't happen directly on the page, but in the configuration area. The layout is managed on a miniature model of the real portal, with titles shown only. (Some portals require repetitive clicking on arrow buttons for layout; fortunately, Excite allows drag-and-drop on the model.)

Webifying the Desktop: The Story of Office Applications

Attempts to webify office applications are almost as old as the Web itself. It's now part of computer folklore that Netscape's Marc Andreesen exclaimed, in the mid-1990s, that MS-Windows would be reduced to "a poorly debugged set of device drivers running Netscape Navigator," expecting to herald in a new era of desktop-style applications running inside the browser. The benefits of the Web over desktop apps are clear and abundant—e.g., an ability to access data from any web browser in the world, easy upgrading, no tampering of local machines, and better collaboration. However, there are serious problems too, and the most severe is interface. In the past, it's simply been impossible to produce a portable interface that's good enough to justify switching from the desktop.

This is changing quickly though, thanks to Ajax.[*] A new generation of Ajax office applications (http://innerphaze.homelinux.com/blog/?p=28) are emerging as a serious substitute for MS-Word, Excel, and their desktop contemporaries.

Characteristics of Ajax Applications

Earlier on, Ajax was defined as a technology that "builds on standard web technologies to deliver a rich, responsive, user experience." This shouldn't be seen as a binary thing, because it's useful to think of Ajax as a continuous spectrum—an application that happens to include a Flash widget, or one that avoids using any remoting technology can still be considered "partly" Ajaxian; it's useful to do so if you're designing that system as you can leverage the experience of other Ajax developers, which is the kind of experience encapsulated in the Ajax Patterns. And in documenting the Ajax Patterns, it's certainly useful to learn from applications that aren't "pure Ajax." To that end, the characteristics here are intended as a general guide, but not hard-and-fast rules, for what constitutes an Ajax application.

Applications, Not Just Web Sites

These days, you'll hear a lot more about "web applications"—or "webapps"—than about "web sites." Driving many modern web projects is the perspective of the browser as a platform and the Web as an operating system. People aren't just buying a book or browsing a manual, but are performing day-to-day work as well as socializing via the browser platform, often working on more critical, complex tasks than in the past. While Ajax can really be applied to anything running inside a browser, it comes into its own with these kinds of systems, where it helps keeps users engaged and productive.

Smooth, Continuous Interaction

Traditional web sites make you submit a form, wait a few seconds, watch the page redraw, and then start the whole cycle again. That's because the tiniest server interaction, and even the tiniest display change, requires a call to the server, and then a complete page refresh. It's a frustratingly slow and erratic sequence. Ajax changes the model in a few ways. First, JavaScript running inside the browser can manipulate the display directly—you don't have to send a whole new page from the server in order to hide an element or rearrange the page. Second, server interaction can be handled via JavaScript, so you can upload user commands and download new information without any page refresh. Third, user actions such as mouse-clicking and typing can be handled by JavaScript, so the interaction is a lot richer than just filling in a form and hitting Submit. All of these enhancements make Ajax interaction feel faster and more continuous.

Live

The Ajax Technologies

Here's a quick rundown of the various technologies involved in Ajax. To begin with, there are several that have always been popular on the Web, which are relatively well-understood by the development community. They are still used in Ajax applications, though sometimes in different ways.

HTML/XHTML

As always, HTML provides the structure of a web page. An Ajax App uses an HTML document to show the initial page, and the document is continuously manipulated to change the display and set up new events. Where possible, its XML-compliant variant, XHTML, should be used in order to make manipulation more robust.

CSS

CSS enriches the display and, thanks to stylesheets, helps separate document structure from style details. Fortunately, browsers are now reasonably consistent in their support for CSS2, so the past few years have seen many web sites shift from table-based layout, which was always something of a hack, to the cleaner, more flexible, CSS-based layout. From our perspective, the great thing about all this is that CSS can easily be manipulated with JavaScript.

Anatomy of a Server Call

At a lower level, how does an Ajax interaction look? Here's a typical sequence of events. Let's begin with the application startup sequence (Figure 1-4):

Figure 1-4. Typical startup sequence

User points browser to Ajax App. The user begins interacting with an Ajax application by visiting it in the usual way; e.g., by following a link or selecting a bookmark.

Ajax Trends

This book has certainly been a moving target. There were already quite a few Ajax applications available when the "Ajax" term was coined, and the space has since exploded, propelled by the rush of activity in what's come to be known as the "Web 2.0" movement. Each day is bringing fresh ideas to the table, as more and more Ajax applications are released. It's impossible to know where it will all lead, but this section identifies a few future trends and open questions.

Standards and Fragmentation

Better compatibility across browsers has made rich web development much easier. However, we're at a crossroads now, because the same economic boom that's fuelling Ajax application development is also fuelling innovations in the browsers themselves, leading to a serious risk of diverging technologies.

One group pushing for change is the Web Hypertext Application Technology Working Group (WHAT-WG). The key term here is "Application," as the group is pushing for the Web as a true application platform, a goal that resonates loudly with the aims of Ajax. Under current proposals, rich controls and interaction techniques such as drag-and-drop will enjoy native browser support. All this is good for Ajax developers, but will probably come at the price of compatibility. It's not clear which browsers will support which standards, and there are likely to be major discrepancies in the implementation schedules. Moreover, Microsoft is conspicuous by its absence from WHAT-WG, and it's distinctly possible IE will end up with a very different API for all this functionality. As well as standards endorsed by WHAT-WG and W3C, there will inevitably be browser-specific features to consider as well. Microsoft will continue to evolve its Atlas framework, and it's certainly possible that IE will give Atlas functionality—such as local data storage—that's not available to other browsers.

If browsers do go down the mid-1990s path of diverging APIs, developers will have to decide on appropriate strategy. The options will include: targeting a specific browser (unfortunate, but will sometimes be the most pragmatic choice), ignoring browser-specific features, and relying on compatible plugins for behavior not directly supported by a particular browser.

Browser Adoption

There may well be many browser enhancements on the horizon, but will users upgrade to take advantage of those features? Most Internet users haven't heard of Ajax and probably wouldn't care a great deal if you told them about it. They'll need a lot of convincing to upgrade their browser and even greater persuasion in order to switch to a different brand. On the other hand, there's now a large population who grew up with the Internet and are quite comfortable with frequent upgrades and browser switches. Furthermore, browsers—like a lot of desktop

Conclusions

Chapter 2. A Pattern-Led Tutorial

THIS CHAPTER INTRODUCES AJAX AND THE AJAX PATTERNS IN A THREE-PART TUTORIAL. THE FIRST section is a whirlwind tour of the key technologies, recommended if you've never worked with Ajax before. The second section shows how the higher-level patterns work to enhance a system; if you've already tinkered with Ajax, you might like to jump straight there. The final section proposes some exercises to try on your own.

The online home for this tutorial is http://ajaxify.com/tutorial/, where you'll discover running demos for each example here. To work locally, you'll need a standard PHP installation (version 5 or later) with Apache or a similar web server (no database is required). If you want the completed code, you'll find a downloadable package at the above URL—consult Appendix B for more details on installation. Each section in the tutorial begins with a summary of the corresponding online demo as well as the directory within the code package where you'll find the completed code.

Ajax Technologies in a Blink

This section includes a few exercises to get up to speed with the basic Ajax technologies. Each section is independent from the others, and each corresponds to a group in the Foundational Technologies Patterns (Part II), which are patterns about the raw technologies on which all Ajax applications are based.

Preparing the Demos

Each demo should live in its own directory, so create a tutorial directory under your server's document root, and then create three fresh directories underneath that:

cd /apache/docroot (substitute your own document root) mkdir tutorial cd tutorial mkdir display remoting dynamic

For Unix systems, ensure permissions are appropriate for your web server (e.g., make the directories globally readable and executable). Each directory will begin with the same "blank slate" HTML file, index.html. Open up your favorite editor and save the following file to display/index.html, remoting/index.html, and dynamic/index.html:

<html> <head> <title>AjaxPatterns.org - Tutorial</title> <script type="text/javascript" src="tutorial.js"></script> </head> <body> <h1>AjaxPatterns Tutorial</h1> <div id="sandbox"> </div> </body>

Remember to set file permissions according to the web server's requirements. Now point your browser to one of the new files and check that you can see the above content. The URL should be http://localhost/tutorial/display/ or, failing that, try http://localhost/tutorial/display/index.html.

The HTML file loads the JavaScript file tutorial.js, which isn't there yet. The next three demos will leave the HTML file alone and illustrate Ajax concepts with code in their respective tutorial.js files.

Display Manipulation and the DOM in a Blink

Hello World!

To begin, go to the working directory (tutorial/display/). Note that there's an online demo of the application we're going to build at http://ajaxify.com/tutorial/display.

The first thing we want to do with Ajax is update the display—change controls, run animations, and so on. We do that by manipulating the DOM so this tutorial will change the DOM as soon as the page loads. We'll do all our work in the working directory, tutorial/display, so once you're in there, save the following as tutorial.js:

window.onload = function( ) { var greeting = document.createElement("span"); greeting.style.backgroundColor = "yellow"; greeting.innerHTML = "Hello World!"; document.getElementById("sandbox").appendChild(greeting); }

Now point your browser to the display demo (http://localhost/tutorial/display/index.html) and you should see a "Hello World!" message with yellow background. As explained in Display Morphing (Chapter 5), the DOM is a tree structure representing the current page, and the browser will keep the display synchronized with programmatic changes to the DOM. You can, if you like, confirm this by visualizing the DOM after running the script above. There are numerous visualization tools, described in

Ajaxifying a Web App: One Pattern at a Time

Patterns aren't necessarily about big upfront design; more often, they're used for refactoring and enhancing existing systems. The foundational technologies from the previous section can be seen as techniques for Ajaxifying a conventional web app in addition to being useful for building a new one. In the same way, many of the subsequent patterns can be seen as techniques for improving an existing Ajax App.

Here, we'll look at how a few patterns can be progressively applied to enhance a web app. The conventional app is initially described for comparison, but we really begin building the app at Step 1, where we produce a basic Ajax version. In all, there are four steps. In the packaged code, you'll find the complete code for each step. So if you get lost in the middle of Step 2, you can start Step 3 afresh by taking a clean copy of the completed Step 2 directory. Note that all the application files reside in just the one directory.

Background: Old-School Ajaxagram

There is no working directory (no coding required here). See the location of the completed code within the installation package at /tutorial/ajaxagram/. There's an online demo at http://ajaxify.com/tutorial/ajaxagram/.

Starting on familiar ground, the initial version is a conventional web app—no Ajax here. It takes a word and lists all anagrams; i.e., all possible combinations of the letters (Figure 2-4). For simplicity, there's no dictionary comparison to look for real words, and for the sake of performance, the input field is limited to just five characters. We won't actually build this conventional version, but you might want to peruse the code (in tutorial/ajaxagram/) for comparison with the next Ajaxified version. There are two source files:

anagram.php

The server-side business logic for finding anagrams, described later in the section "Business logic: the anagram web service."

index.phtml

The application's view, a query form, and the list of anagrams. The page submits back to itself; the results area always shows all anagrams arising from the previous form submission (if there was one).

Figure 2-4. Conventional Ajaxagram

Step 1: Ajaxagram Done Ajax-Style

Ensure you're in the working directory for this demo (/tutorial/ajaxagram/; this is the same for all steps). Note that you'll find completed code for this step within the installation package under directory /tutorial/ajaxagram/ajaxified/, and an online demo for this step at http://ajaxify.com/tutorial/ajaxagram//ajaxified.

The initial Ajax version looks and feels just like the conventional version. The only difference is that there's no page refresh as the query word is passed via web remoting [see the Web Remoting (Chapter 6) patterns], and the results updated using display manipulation [see the Display Manipulation (Chapter 5) patterns]. As an overview, we'll be creating four files inside the one directory:

anagram.php

The server-side business logic for finding anagrams; described in the next section.

anagram.phtml

A server-side web service; accepts a word and outputs a comma-separated list of anagrams.

index.html

The browser-side application's view; a plain HTML file that contains the form and list of anagrams.

ajaxagram.js

The browser-side application logic; waits for a submission, passes the query word to anagram.phtml, and alters the display accordingly.

Begin the application by creating a fresh directory; e.g., tutorial/ajaxagram/ (you'll need to create tutorial/ if you didn't already do so).

mkdir tutorial cd tutorial mkdir ajaxagram

For Unix systems, ensure permissions are appropriate for your web server (e.g., make the directories globally readable and executable). All the files throughout the four steps described next should go in this directory.

Business logic: the anagram web service

We'll begin the web service by creating a backend module that knows only the business logic related to anagrams, and nothing about HTML. Since we're talking pure business logic, I'll just show you the entire file and skim over the details. Create the following as anagram.php (or copy it from the code package):

<? /* Cleans input word and passes into findAnagramsRecursive */ function findAnagrams($word) { $sorted = str_split(filterInput($word)); sort($sorted); return findAnagramsRecursive(implode($sorted)); } /* Convert to lowercase and only include letters */ function filterInput($word) { $word = strtolower($word); $word = ereg_replace("[^a-z]","", $word); return $word; } /* Assumes $word is sorted */ function findAnagramsRecursive($word) { if (strlen($word)==0) { return array( ); } if (strlen($word)==1) { return array($word); } // For each character in the word, extract it and find all anagrams // where it's the first character. $anagrams = array( ); for ($pos=0; $pos<strlen($word); $pos++) { $extractedChar = substr($word,$pos,1); // If same as previous, don't process any further as it will only // create duplicates (this is the only check against duplicates, // since the word is sorted). $sameAsPreviousChar = ($pos > 0 && $extractedChar==substr($word,$pos-1,1)); if (!$sameAsPreviousChar) { $remaining = removeCharAt($word, $pos); $anagramsOfRemaining = findAnagramsRecursive($remaining); foreach ($anagramsOfRemaining as $anagramOfRemaining) { array_push($anagrams, $extractedChar . $anagramOfRemaining); } } } return $anagrams; } /* Return word without character at pos */ function removeCharAt($word, $pos) { return substr($word, 0, $pos) . substr($word, $pos+1); } ?>

There's also a packaged test case, AnagramTest.php; it's good practice to test backend business logic in isolation. If you want to run the test, install PHPUnit2 (http://www.phpunit.de/pocket_guide/3.0/en/index.html) first and see its documentation for details on running tests.

With the business logic now in the server, we need a thin wrapper to expose it as a clean Web Service. Placing it in a separate file helps separate business logic from concerns about how it will be invoked from the browser. Thus, create the web service anagrams.phtml as follows:

<? require_once("anagram.php"); $word = array_key_exists('word', $_GET) ? $_GET['word'] : ""; $anagramsArray = findAnagrams($word); print implode(",", $anagramsArray); ?>

You can test the web service: point your browser at http://localhost/tutorial/ajaxagram/anagrams.phtml?word=ajax, and you should see a comma-separated list of "ajax" anagrams. Notice that this is a raw response intended for easy script processing and not the sort of thing you'd show a user. Our script will need to parse it in order to present the results to users.

Presentation: initial HTML

The initial HTML file contains the basic display detail—a form and a list of anagrams—and also pulls in the required JavaScript. Create index.html as follows:

<html> <head> <title>AjaxPatterns.org - Ajaxagram (Basic Ajax Version)</title> <script type="text/javascript" src="ajaxagram.js"> </script> </head> <body> <h1>Ajaxagram (Basic Ajax Version)</h1> <div> <input type="text" id="word" size="5" maxlength="5" /> <button id="findAnagrams">Find Anagrams!</button> </div> <div id="results"> </div> </body>

Point your browser to the HTML; you should see the heading and the form, but probably not the results area as it's initially empty.

There are a few things to note about the HTML, and you might like to contrast it with index.phtml packaged in the conventional version. First, the "form" is not a true HTML <form> element because we don't need the browser to submit it to the server; our script will issue the query manually. Second, as there's no HTML form here, a button element is used instead of a submit control. Third, the results div is empty. It will be populated each time our script receives a list of anagrams from the server. Finally, note the absence of any JavaScript code here; all the application logic is isolated in a separate file, ajaxagram.js.

Application logic: JavaScript

The final task is coding the application logic that acts as the glue between the server-side web service and the browser-side web page. Create ajaxagram.js with the following generic methods, explained earlier in the "In A Blink" tutorials. Keep these at the end of the file as we build the rest of the JavaScript.

// -- Generic functions -------------------------------------------------- function createXMLHttpRequest( ) { try { return new ActiveXObject("Msxml2.XMLHTTP"); } catch (e) {} try { return new ActiveXObject("Microsoft.XMLHTTP"); } catch (e) {} try { return new XMLHttpRequest( ); } catch(e) {} alert("XMLHttpRequest not supported"); return null; } function $(id) { return document.getElementById(id); }

Projects and Katas

Now it's over to you. Here are some suggestions for projects you can work on. You might like to approach these as "Katas" (http://blogs.pragprog.com/cgi-bin/pragdave.cgi/Practices/Kata/), short exercises that you can keep returning to. Alternatively, add all the bells and whistles and make them full-blown projects. Check http://ajaxpatterns.org/Katas for the latest list, and please add links to any of your online efforts.

Two-Person chat

Allow two people to chat in real-time via the browser.

Filesystem navigator

Conclusions

Chapter 3. Ajax Design: Principles and Patterns

AJAX DOES A LOT FOR WEB USABILITY, HAS ALREADY DELIVERED SOME STUNNING APPLICATIONS, AND IS clearly the "in" thing at this time. But it's no magic bullet. Careful design is always required, and it must be tailored to the technology at hand. By monitoring the state of Ajax applications, we continue to learn about what works and what doesn't, and about how developers are succeeding in their design trade-offs. This chapter explains these lessons at a high level and introduces the patterns, which discuss them in depth.

Designing for Ajax

By studying existing Ajax applications, as well as any relevant precursors, it's been possible to distill a number of important design principles for Ajax, which are shown in this section. The thinking behind the principles was a big influence on the pattern discovery process, and knowing them will help to apply the patterns.

We'll begin by looking at principles of user-centered design, followed by those of software design. Of course, you can never fully separate those concerns, and they're often in conflict with each other. Dealing with those conflicts is really a key concern of the patterns. Incidentally, it's worth checking out a good online resource that takes the opposite perspective: Ajax Mistakes (http://swik.net/Ajax/Ajax+Mistakes) is a long list of Ajax mistakes and gotchas, as well as anti-patterns originally authored by Alex Bosworth and now maintained on a wiki.

Usability Principles

Follow web standards

Try hard enough, and you can do some very confusing things with Ajax, even more so as rich graphics become commonplace. Rather than reinventing the Web as we know it, use Ajax to build a "better Web," an enhanced layer over what's already there. Respect the conventions that users are already familiar with.

The browser is not a desktop

Further to the previous principle, Ajax is a richer brand of the traditional web site rather than a webified brand of the traditional desktop. True, desktop widgets like sliders are migrating towards Ajax, but only when they make sense in a web context and often in a modified form. We're also seeing application categories like word processors head online as well, but again, the best products will be those that fit in with the Web rather than blindly replicating the desktop experience.

If it's different, make it really different

Subtle differences confuse people. If you've decided it makes sense to diverge from a standard or a common idiom, ensure your design is distinct enough to avoid confusion.

Provide affordances

Affordances (http://en.wikipedia.org/wiki/Affordance

Ajax Patterns Overview

The Ajax Patterns show how people have used the design principles effectively in real-world Ajax applications. It might seem funny that we can have so many patterns about Ajax, a term that was coined only a few months before work on these patterns began. However, the ideas are not new; there were many Ajax features on the Web before the term came about to describe them. The healthy Net economy has helped a lot too, with hundreds of new sites now using Ajax, along with powerful tools (RSS, Technorati, Google, and wikis) to locate them as soon as they're available.

With over 60 patterns, it's useful to classify the patterns hierarchically. At a high level, the book is divided into four parts, each corresponding to a different focus area—Foundational Technology, Programming, Functionality and Usability, and Development. Beyond that, each part is divided into several chapters, where each chapter includes related patterns. For instance, Foundational Technology Patterns (Part II), includes Web Remoting (Chapter 6), which includes several patterns for web remoting. Here's a summary of each part:

Foundational Technology patterns (11 patterns)

The foundational technologies are the building blocks that differentiate Ajax from conventional approaches, and this section explains typical usage.

Programming patterns (23 patterns)

Anatomy of a Pattern

All of the patterns follow the same basic format, though certain fields are left out where appropriate. This section explains the meaning of each section.

Evidence

How much real-world evidence exists for the pattern, on a 3-point scale and presented graphically? It's a rather subjective estimate, but use the following icons as a guide:

○

Suggests the idea is purely speculative

⊙

Suggests there's at least a proof-of-concept or an early usage

⊙⊙

Suggests there's a few established examples

⊙⊙⊙

Suggests the pattern is in widespread usage

Ajax Patterns Demos

The Ajax Patterns Demos appear in many patterns as Refactoring Illustrations and also in the Solution section. They're all online at http://ajaxify.com/run/, and it would be worth trying them out. In addition, the full code base can be downloaded from that location; its installation is explained in the Appendix B. It includes the demos as well as completed code for Chapter 2. Note the demos have been tested on Firefox 1.5 and IE6, though most of them work on comparable browsers as well.

Conclusions

Part II. Foundational Technology Patterns

The foundational technology patterns outline the "building blocks" at the heart of any Ajax application. The patterns are considered atomic within the pattern language, in the sense that all later patterns build on these basic techniques. To keep the Ajax Patterns concise, you will only encounter technologies that add something to conventional web development; there's no "Form Submission" pattern, for instance.

Chapter 4. Ajax App

THIS CHAPTER CONTAINS JUST A SINGLE PATTERN, THE ROOT FOR THE ENTIRE PATTERN LANGUAGE: Ajax App.

Ajax App

⊙⊙⊙ Ajax, Balanced, Client-SOA, DHTML, Fast, Fat, Interactive, Platform, RichInternetApplication, Rich, Thick, Web2.0, and WebApp

Figure 4-1. Ajax App

Goal Story

Pam's begun entering staff appraisals into a new Ajax App. She's pleased to see the data entry is much faster: fields are validated as soon as they're filled out, searches are integrated into the form rather than in annoying popups, and the remaining form fields keep mutating to reflect what she's entered so far.

Problem

How can you create a rich application?

Forces

See Chapter 1 for more details on the forces driving Ajax, which are summarized here.

Users like working and playing in the browser and are now more willing to keep their data online, but are nonetheless frustrated with conventional "click 'n' wait" interfaces.

Companies like web apps running on their Intranets because it makes deployment much easier, but they continue to be burned by unusable web apps that don't deliver the same value as a comparable desktop app.

Developers are now well-versed in the basic patterns of web architecture and ready to take on new challenges.

Technology has opened up new opportunities for the Web: broadband and beyond is becoming ubiquitous in many countries; servers can process huge quantities of requests per second; and storage is growing to the point where it's feasible for individuals to host most of their personal data online.

Solution

Produce an Ajax App—a rich application capable of running inside any modern web browser. The user fires up an Ajax App by typing in a URL, clicking a link, or activating a bookmark. The Ajax App is then downloaded into the browser and the user can begin working with it immediately.

Ajax is fundamentally a user-centered pattern. As mentioned in the first chapter, the major force driving Ajax is the needs of users and their organizations to get more out of the Web. For too long, people have endured pages that take forever to load, widgets that make no sense, and forms that conveniently "forget" their data when they take too long to fill out. There are good reasons for people to continue using a browser, as noted in Chapter 1, but it's in the interests of web site owners and employers to do a better job at making that interaction productive and enjoyable.

Ajax aims to improve user experience within the constraint of respecting browser standards. Standard-based design obviously has the benefit of portability, but also has a usability component in itself; users interact with hundreds of web sites and appreciate consistency, a quality that quickly erodes when you rely on plugins, browser-specific features, and "creative" design ideas. Sure, incorporate a novel widget or a funky layout when it works, but for most things, the standard way works best.

Non-standard technologies also break consistency; one reason Flickr (http://flickr.com) began migrating from Flash to Ajax was because users want to right-click to save images, a standard broken by Flash (http://www.ajaxian.com/archives/2005/05/ajax_summit_eri.htm). Again, don't be afraid to use a technology like Flash to achieve something standard browsers won't do—that's the whole idea behind the Richer Plugin (Chapter 8) pattern. Some people will say it's not an Ajax App if it uses Flash, but the definition here would include a mostly Ajax App that delegates to Flash (or other plugins) to fulfill some specialized task. In any event, the definition really isn't worth arguing about (after all, you can define "Ajax" however you like); the important thing is to ask what helps users—sometimes, it's better to incorporate some Flash or Java; other times, a desktop solution really is the best approach.

The standard technologies are discussed in Chapter 1. They include several well-established technologies: HTML, CSS, HTTP, CGI, and form submission, and server-side scripting. In addition, there are several more technologies that are only now becoming seriously popular and standardized in the browser: JavaScript, the Document Object Model (DOM), the Event Model (DHTML), and Web Remoting.

Decisions

Will your application be "Ajax Deluxe" or "Ajax Lite"?

There are two archetypal architectures for Ajax, and all applications lie somewhere along the spectrum between these extremities.[*]

Ajax Deluxe

Ajax Deluxe uses Ajax to the max: applications feel similar to a desktop in that the browser is driving the interaction—it mediates all interaction between the user and server, so there are no—or few—direct page refreshes. Similarly, there's no need for session tracking in the server, because in the absence of page refreshes, all relevant state can be retained inside the browser script. The server need not know about HTML at all, and might just offer generic Web Services. An example is the Ajax calendar,

Chapter 5. Display Manipulation

FOR AN END USER, THE MOST OBVIOUS THING ABOUT AJAX IS ITS VISUAL APPEARANCE. AJAX APPS TEND to look richer, and the interface tends to update smoothly, more like a desktop app than a conventional web app. This chapter describes the two main technologies for updating the user interface. Display Morphing focuses on changes to a single element, while Page Rearrangement involves changes to the page structure. Along the way, we'll look at the Document Object Model (DOM). DOM manipulation is key to many of the Ajax Patterns.

Display Morphing

⊙⊙⊙ Display, DOM, Graphics, GUI, Morph, Page, Paint, Reference, Rich

Figure 5-1. Display Morphing

Goal Story

Stuart is answering a question in an online quiz, with a countdown box showing how much time remains. The countdown label changes each second, and the color gradually shifts from green to red as the countdown proceeds.

Problem

How can you dynamically update display elements?

Forces

As the user's task and context changes, applications need to present different types of information.

As information the user is working on changes, information being displayed becomes stale.

Stale information is a big problem in conventional web apps. Because the information cannot change until it's out of date, you sometimes see disclaimers such as "May Be Out of Date" or instructions like "Click to Refresh."

Submitting and redrawing the entire page interrupts the user's flow.

Solution

Morph page elements by altering styles and values in the Document Object Model (DOM), such as text and color properties. The DOM represents the state of the web page at any time. When JavaScript manipulates the DOM, the browser notices the changes and immediately reflects them in the user interface. To morph a page element's display, you get a reference to it and change its properties. This allows you to change the display of any element on the page—headings, div elements, images, and even the document object itself. Note: an online demo (http://ajaxify.com/run/display) illustrates the code concepts throughout this Solution and the code snippets loosely follow from the demo.

To further illustrate the DOM, consider the following piece of HTML:

Figure 5-2 shows the structure that arises when the browser transforms the HTML string into a DOM model. Since each node has attributes, we can represent the model in more detail, as shown in Figure 5-3. To learn more about the DOM, experiment with the tools mentioned in DOM Inspection (Chapter 18). You can point them at a web page to view the corresponding DOM model.

Figure 5-2. DOM structure

Figure 5-3. DOM structure with attributes

Each node has a number of standard attributes and operations. Those below let you inspect surrounding DOM context:

parentNode

The element's parent node.

childNodes[]

An array of all immediate child nodes.

getElementsByTagName("tagName")

An array of all child nodes of type tagName (e.g., h1 for all level-1 headings).

The structure can be manipulated as well as inspected. Following are the key manipulation methods available on all nodes (discussed further in Page Rearrangement):

appendChild(newChild)

Appends newChild.

insertBefore(newChild, oldChild)

Inserts newChild just before oldChild.

removeChild(oldChild)

Removes oldChild.

You can also inspect the node itself; useful properties include:

The node's ID.

nodeType

Usually 1 to denote a standard element. 2 represents a tag attribute and 3 represents a text node (explained later in this section).

tagName

The name of the tag (e.g., "h1").

For a more complete set of properties and operations, a good source is available at HowToCreate.co.uk (http://www.howtocreate.co.uk/tutorials/index.php?tut=0&part=28).

Before using any of these properties, the first thing you need to do is get hold of a node somewhere in the DOM. Here's how to grab a reference to an image node with an ID of "logo" buried anywhere inside the DOM:

var logo = document.getElementById("logo");

Page Rearrangement

⊙⊙⊙ Add, Adjust, Change, Delete, DOM, Move, Overlay, Rearrange, Restructure, Remove

Figure 5-6. Page Rearrangement

Goal Story

Doc is looking at a X-ray image in an online diagnostics system. When the mouse rolls over the body, an "advice box" appears at the bottom of the page with stats to help interpret the image. In case the box gets in the way, he's able to drag it around the page.

Problem

How can you dynamically restructure the page?

Forces

As the user's task and context changes, Ajax Apps need to present different information.

Changes to page structure should be as smooth as possible.

Refreshing the page breaks continuity and also clears JavaScript state.

Refreshing the page only allows for a discrete transition from one appearance to another—it's not possible to gradually fade or move an item.

Solution

Add, remove, move, and overlay elements by manipulating the DOM. The DOM is a hierarchical structure that defines how elements relate to one another. By adjusting the DOM structure, you can adjust where elements appear on the page. There are also critical CSS styles that affect page structure—you can change these by manipulating DOM elements' style attributes. Note: an online demo (http://ajaxify.com/run/page) illustrates the code concepts throughout this Solution and the code snippets loosely follow from the demo.

Adding is normally achieved by introducing a new element as a child of an existing container element, such as body or div. For example, you can create a new div element and add it to a parent div:

var message = document.createElement("span"); $("container").appendChild(message);

Another way to insert an element is to append to the innerHTML property of the container element:

$("container").innerHTML += "<span></span>";

The opposite action is removing. By removing an element from the DOM, you take it off the display:

$("container").removeChild($("message"));

As a variant of adding and removing, you can keep an element on the page but toggle its visibility using CSS. There are two alternative style properties you can use: the visibility style and the display style. The former will always preserve layout while the latter will cause the element to squeeze in and out when it's shown or hidden. So with no visibility, it's like the element's still there but wearing a coat of invisible paint. This makes layout easy, because everything stays where it is, but it can be ugly to have a big patch of whitespace on the page. With the display style, it's more like the element's really disappeared, so the other elements squeeze in to take the place it had occupied. This makes layout a bit more complex, but is usually better visually. The visibility property alternates between visible and hidden:

$("message").style.visibility = "visible"; // Now you see me $("message").style.visibility = "hidden"; // Now you don't ("Invisible paint")

display is actually used for more than just showing and hiding an element—it also defines how an element appears when it is visible. Briefly, the main options are block

Chapter 6. Web Remoting

NOW THAT WE'VE SEEN HOW AJAX APPS CAN MANIPULATE THE DISPLAY, IT WOULD BE USEFUL IF THEY could use that capability to show data arriving from the server. We'd also like to accept input and upload it to the server. The Web Remoting patterns let JavaScript directly issue calls to the server. The point is to let the browser make a query or upload some data, without actually refreshing the page.

The first pattern is Web Service, which explains how the server side exposes functionality to be accessed by the browser.

The remaining patterns are the most useful and most common mechanisms for web remoting.[*] XMLHttpRequest Call is a clean way for the browser script to invoke and catch responses from the server. IFrame Call provides a similar use by exploiting IFrame functionality. HTTP Streaming, also known as "Push" (or "Comet"), allows the server to continue streaming new data down the pipe, without the browser having to issue new requests. On-Demand JavaScript is a fairly broad-scoped pattern and is included in this section because one style, Script Tag Generation, is a distinctive technology that's becoming a popular way to directly access external domains.

Of the four techniques, XMLHttpRequest Call is the cleanest, because XMLHttpRequest is specifically designed for web remoting. Nevertheless, all of the techniques are effective, and each has a certain set of situations where it's the superior choice. See the "Alternatives" section for XMLHttpRequest Call for a comparison of the various techniques.

Web Service

⊙⊙⊙ API, HTTP, Microcontent, REST, RPC, Share, SOAP, Platform

Figure 6-1. Web Service

Goal Story

In a financial forecasting app, Tracy runs an algorithm to forecast the next four interest rates. The browser script uploads the parameters to an "Interest Rate Forcaster" web service, which eventually outputs a concise list of future rates, free of any HTML markup: 4.5, 3.4, 4.0, 4.1.

Problem

What will Ajax Apps call on the server?

Forces

Ajax Apps can access the server directly and require well-scoped services instead of the conventional-style scripts that output an entire HTML page.

Many organizations like to expose their functionality and data for third parties to access. The API needs to be clean and easy to use.

Solution

Expose server-side functionality as fine-grained, easy-to-use Web Services. "Web Service" is an overloaded term, and the Ajax Patterns use a fairly broad definition:

A Web Service is a standard HTTP service.

A Web Service has a well-defined, consistent interface—input and output assumptions, message formats, and exception handling are all clearly understood by its developers and ideally documented in some manner.

A Web Service accepts fine-grained input parameters and outputs fine-grained responses, such as an XML document, a simple string, or a snippet of HTML. If it outputs an entire HTML document, it's probably not a Web Service.

Under this definition, there is considerable flexibility involved in creating a Web Service:

A Web Service might be open to the public, might be protected by a secure authentication mechanism, or might be hidden from the public by a firewall.

A Web Service might be built by a tool or handwritten.

A Web Service might use messages according to any protocol, such as SOAP, or might use custom messages.

Deciding on all these things is the main task involved in creating a Web Service.

XMLHttpRequest Call

⊙⊙⊙ Call, Callback, Download, Grab, Live, Query, Remoting, RemoteScripting, Synchronise, Synchronize, Upload, XMLHttpRequest

Figure 6-2. XMLHttpRequest Call

Goal Story

Reta's purchasing some items from a wholesaler's web site. Each time she adds an item to the shopping cart, the web site issues an XMLHttpRequest to save the latest cart contents. There's no form submission, so the item is added instantaneously, which saves Reta time as well as helping her understand what's going on.

Problem

How can the browser communicate with the server?

Forces

Ajax Apps require browser-server communication. User-generated information must be uploaded and new server information must be downloaded.

Because Ajax Apps should have a smooth and continuous feel, browser-server communication must be unobtrusive.

Ajax Apps should be highly responsive, so calls should involve minimal data transfer.

As the network is often unreliable and performance is inconsistent, calls should be asynchronous, allowing the user to keep working while network calls are in progress.

Solution

Use XMLHttpRequest objects for browser-server communication. JavaScript lacks a portable mechanism for general network communication, a restriction that's always been in place for security reasons and will probably remain. But thanks to the XMLHttpRequest object—now available in all major browsers—JavaScript code can make HTTP calls back to its originating server and get hold of the results. Doing so enables you to make fine-grained server calls and deal with responses as you wish, unlike conventional form submissions, which cause a complete page refresh. Note: an online demo (http://ajaxify.com/run/xmlHttpRequestCall) illustrates the code concepts throughout this Solution and the code snippets loosely follow from the demo.

This pattern uses the sum web service described earlier in Web Service, with a URL like http://ajaxify.com/run/xmlHttpRequestCall/sumGet.phtml?figure1=5&figure2=10. It returns just the sum, "15" in this case. You can test that by typing the full URL in the browser, but here we want to call it from JavaScript and catch the result. Here's a very basic example:

var xhReq = new XMLHttpRequest( ); xhReq.open("GET", "sumGet.phtml?figure1=5&figure2=10", false); xhReq.send(null); var serverResponse = xhReq.responseText; alert(serverResponse); // Shows "15"

The sequence begins by creating a new instance of XMLHttpRequest. xhReq.open( ) then prepares a call on the test service, sumGet.phtml (the code's running from the same path, so the domain and path need not be qualified). The GET signifies the request method to be used. The false argument says the call is synchronous, meaning that the code will block until a response comes back. The send command completes the request. Because the call is synchronous, the result is ready as soon as the next line is executed. The XMLHttpRequest object has saved the response from the server, and you can access it with the responseText field.

The above example shows that the fundamental technology is pretty simple. However, be aware that it's a very basic usage that's not yet fit for production. Fundamental questions remain, which are answered throughout this Solution:

How do you get hold of an XMLHttpRequest?

How do asynchronous calls work?

How do you handle errors?

What if the service requires a POST or PUT request rather than a GET?

What constraints apply to external domains?

How can you deal with XML responses?

What's the API?

As you read all this, be aware that many, many libraries are available to handle remoting (see Appendix A). Most developers shouldn't need to touch XMLHttpRequest directly. Having said that, it's good to be aware of the capabilities and limitations of XMLHttpRequest Calls, along with the other web-remoting techniques. This knowledge will help you select the most appropriate library and help with any bugs you might encounter.

Creating XMLHttpRequest objects

In most browsers, XMLHttpRequest is a standard JavaScript class, so you just create a new instance of XMLHttpRequest. However, Microsoft is the inventor of XMLHttpRequest, and until IE7, IE offered it only as an ActiveX object. To make things even more fun, there are different versions of that object. The following code shows a factory function that works on all browsers that support XMLHttpRequest:

function createXMLHttpRequest( ) { try { return new ActiveXObject("Msxml2.XMLHTTP"); } catch (e) {} try { return new ActiveXObject("Microsoft.XMLHTTP"); } catch (e) {} try { return new XMLHttpRequest( ); } catch(e) {} alert("XMLHttpRequest not supported"); return null; } ... var xhReq = createXMLHttpRequest( );

You really need to use a function like this for maximum portability. Once you have the object, its basic functionality and API are pretty consistent across browsers, but be sure to test carefully as there are a few subtle implementation differences in some browsers. (If you're curious, the "Solution" in HTTP Streaming [later in this chapter] highlights one such inconsistency.)

You can also reuse an XMLHttpRequest; it's worthwhile doing so in order to prevent memory leaks. To be safe, start a new call only when there's not one already in progress. As explained below, it's possible to inspect the status of a call, and you should only start a call if the status is 0 or 4. So if it's anything else, first call the abort( ) method to reset status.

Asynchronous calls

I previously mentioned in the "Solution" that under synchronous mode, "the code will block until a response comes back." Some hardened readers probably writhed uncomfortably at the thought. We all know that some requests take a long time to process, and some don't come back at all. Pity the user when a server script is buried in an infinite loop.

In practice, XMLHttpRequest Calls should almost always be asynchronous. That means the browser and the user can continue working on other things while waiting for a response to come back. How will you know when the response is ready? The XMLHttpRequest's readyState always reflects the current point in the call's lifecycle. When the object is born, it's at 0. After open( ) has been called, it's 1. The progression continues until the response is back, at which point the value is 4.

So, to catch the response, you need to watch for a readyState of 4. That's easy enough, because XMLHttpRequest fires readystatechange events. You can declare a callback function using the onreadystatechange field. The callback will then receive all state changes. The states below 4 aren't especially useful and are somewhat inconsistent across browser types anyway (http://www.quirksmode.org/blog/archives/2005/09/xmlhttp_notes_r_2.html). So most of the time, all we're interested in is, "Are you in state 4 (i.e., complete) or not?"

Based on all that, here's an asynchronous version of the code shown earlier:

var xhReq = createXMLHttpRequest( ); xhReq.open("GET", "sumGet.phtml?figure1=5&figure2=10", true); xhReq.onreadystatechange = onSumResponse; xhReq.send(null); ... function onSumResponse( ) { if (xhReq.readyState != 4) { return; } var serverResponse = xhReq.responseText; ... }

As shown, you declare the callback method in XMLHttpRequest's onreadystatechange property. In addition, the third argument of open( ) is now true. This argument is actually called the "asynchronous flag," which explains why we're now setting it to true. The callback function, onSumResponse, is registered using onreadystatechange and contains a guard clause to ensure the readyState is 4 before any processing can occur. At that point, we have the full response in responseText.

JavaScript also supports "closures"—a form of anonymous function—which suggests a more concise boilerplate structure for asynchronous calls:

var xhReq = createXMLHttpRequest( ); xhReq.open("get", "sumget.phtml?figure1=10&figure2=20", true); xhReq.onreadystatechange = function( ) { if (xhReq.readyState != 4) { return; } var serverResponse = xhReq.responseText; ... }; xhReq.send(null);

Use closures sparingly, because you're defining a new function each time. It's slower than referring to an existing one and might also lead to memory leaks.

Asynchronous calls are essential, but also more error-prone. If you look at the callback mechanism, you might notice the potential for a subtle, but serious, bug. The problem arises when the same instance of XMLHttpRequest is simultaneously used for different calls. If Call 2 is issued while the object is still waiting for the response of Call 1, what will the callback function receive? In fact, it's even possible the callback function itself is changed before the first call returns. There are ways to deal with this problem, and they're the topic of the Call Tracking (Chapter 10) pattern.

Detecting errors

Sometimes, a request doesn't come back as you expected it, or maybe not at all. You scripted the call wrong, or there's a bug in the server, or some part of the infrastructure just screwed up. Thinking asynchronously is the first step to dealing with these problems, because at least your application isn't blocked. But you need to do more than that.

To detect a server error, you can check the response status using XMLHttpRequest's status flag. This is just a standard HTTP code. For example, if the resource is missing, XMLHttpRequest.status will take on the famous "404" value. In most cases, you can assume anything other than 200 is an error situation. This suggests adding a new check to the callback function of the previous section:

xhReq.onreadystatechange = function( ) { if (xhReq.readyState != 4) { return; } if (xhReq.status != 200) { var serverResponse = xhReq.responseText; ... };

That's great if the browser knows a problem occurred, but sometimes the request will be lost forever. Thus, you usually want some kind of timeout mechanism (http://ajaxblog.com/archives/2005/06/01/async-requests-over-an-unreliable-network) as well. Establish a Scheduling timer to track the session. If the request takes too long, the timer will kick in and you can then handle the error. XMLHttpRequest has an abort( ) function that you should also invoke in a timeout situation. Here's a code sample:

var xhReq = createXMLHttpRequest( ); xhReq.open("get", "infiniteLoop.phtml", true); // Server stuck in a loop. var requestTimer = setTimeout(function( ) { xhReq.onreadystatechange = function( ) { if (xhReq.readyState != 4) { return; } clearTimeout(requestTimeout); if (xhReq.status != 200) { // Handle error, e.g. Display error message on page return; } var serverResponse = xhReq.responseText; ... };

Compared to the previous example, a timer has been introduced. The onreadystatechange( ) callback function will clear the timer once it receives the full response (even if that response happens to be erroneous). In the absence of this clearance, the timer will fire, and in this case, the setTimeout sequence stipulates that abort( ) will be called and some recovery action can then take place.

Handling POSTs and other request types

Up to this point, requests have been simple GET queries—pass in a URL and grab the response. As discussed in the RESTful Service (Chapter 9), real-world projects need to work with other request types as well. POST, for example, is suited to calls that affect server state or upload substantial quantities of data. To illustrate, let's now create a new service, sumPostGeneric.phtml, that does the same thing as postGet.phtml

IFrame Call

⊙ Call, Callback, Download, Frame, IFrame, Live, Query, Remoting, RemoteScripting, Upload

Figure 6-5. IFrame Call

Goal Story

Bill's ordering a car online and the price is always synchronized with his choice. This happens because of a hidden IFrame. Each time he changes something, the browser populates a form in the IFrame and submits it. The IFrame soon contains the new price, which the browser copies into the visible display. XMLHttpRequest could have accomplished the feat too, but it's not supported in Bill's outdated browser.

Problem

How can the browser communicate with the server?

Forces

Refer to the section "Forces" in XMLHttpRequest Call.

Solution

Use IFrames for browser-server communication. Much of the fuss in Ajax concerns the XMLHttpRequest object, but sometimes other remoting techniques are more appropriate, and IFrame Call is one of those (see the "Alternatives" section of XMLHttpRequest Call, earlier in this chapter, for a comparison). IFrames are page-like elements that can be embedded in other pages. They have their own source URL, distinct from their parent page, and the source URL can change dynamically. IFrame Calls work by making the IFrame point to a URL we're interested in, and then reading the IFrame's contents once the new content has loaded. Note: an online demo (http://ajaxify.com/run/iframe) illustrates most of the code concepts throughout this Solution and the code snippets loosely follow from the demo.

To begin with, you need an IFrame in the initial HTML, and an event handler registered to catch the onload event. (It would be less obtrusive to register the handler with JavaScript, but due to certain browser "features," that doesn't always work.)

We're using the IFrame as a data repository, not as a user interface, so some CSS is used to hide it. (The more natural choice would be display: none, but as an experimental demo shows [http://ajaxlocal/run/iframe/displayNone/], browser strangeness makes it infeasible.)

#iFrame { visibility: hidden; height: 1px; }

On the server, we're initially going to use the web service discussed in XMLHttpRequest Call: sumGet.phtml. To call the service, we'll change the IFrame's source:

$("iFrame").src = "sumGet.phtml?figure1=5&figure2=1";

We already registered an onload handler in the initial HTML. The handler will be called when the new content has loaded, and at that stage, the IFrame's body will reflect the content we're after. The following code calls a function, extractIFrameBody( ), which extracts the body in a portable manner (based on http://developer.apple.com/internet/webcontent/iframe.html).

HTTP Streaming

⊙ Connection, Duplex, Live, Persistent, Publish, Push, RealTime, Refresh, Remoting, RemoteScripting, Stateful, ReverseAjax, Stream, TwoWay, Update

Figure 6-9. HTTP Streaming

Goal Story

Tracy's trading application contains a section showing the latest announcements. It's always up-to-date because the announcements are streamed directly from the server. Suddenly, an important announcement appears, which triggers her to issue a new trade. The trade occurs with a new XMLHttpRequest, so the announcement stream continues unaffected.

Problem

How can the server initiate communication with the browser?

Forces

The state of many Ajax Apps is inherently volatile. Changes can come from other users, external news and data, completion of complex calculations, and triggers based on the current time and date.

HTTP connections can only be created within the browser. When a state change occurs, there's no way for a server to create a physical connection to notify any interested client.

Solution

Stream server data in the response of a long-lived HTTP connection. Most web services do some processing, send back a response, and immediately exit. But in this pattern, they keep the connection open by running a long loop. The server script uses event registration or some other technique to detect any state changes. As soon as a state change occurs, it pushes new data to the outgoing stream and flushes it, but doesn't actually close it. Meanwhile, the browser must ensure the user interface reflects the new data. This pattern discusses a couple of techniques for Streaming HTTP, which I refer to as "page streaming" and "service streaming."

Is it Practical?

Note that this pattern, though it's used in production systems, remains somewhat experimental, especially the service-streaming variant. There are issues of feasibility, scaleability, and browser portability. One particular gotcha is the effect of proxies. Sometimes, a proxy sitting somewhere between server and browser will buffer responses, an unfortunate optimization that prevents real-time data from flowing into the browser. Even worse, some proxies will, with the best of intentions, take it upon themselves to close the connection after an idle period. Another issue is server performance, given the number of persistent connections that must be handled.

HTTP Streaming works best when you can control factors such as the web server, the network, and even the browser where possible. Scaleability is often over-emphasized, so streaming might be practical on more systems than one might expect. Furthermore, some modern servers (e.g., Twisted, at http://twistedmatrix.com/) are being developed with streaming-style architecture in mind and are able to maintain many long-lived connections.

Page streaming involves streaming the original page response (Figure 6-10). Here, the server immediately outputs an initial page and flushes the stream, but keeps it open. It then proceeds to alter it over time by outputting embedded scripts that manipulate the DOM. The browser's still officially writing the initial page out, so when it encounters a complete <script> tag, it will execute the script immediately. A simple demo is available at http://ajaxify.com/run/streaming/.

Figure 6-10. Page streaming

For example, the server can initially output a div that will always contain the latest news:

print ("<div id='news'></div>");

But instead of exiting, it starts a loop to update the item every 10 seconds (ideally, there would be an interrupt mechanism instead of having to manually pause):

<? while (true) { ?> <script type="text/javascript"> $('news').innerHTML = '<?= getLatestNews( ) ?>'; </script> <? flush( ); // Ensure the Javascript tag is written out immediately sleep(10); } ?>

Each language and environment will have its own idiosyncrasies in implementing this pattern. For PHP, there's fortunately some very useful advice in the flush( ): online comments (http://php.net/flush). It turned out to be necessary to execute ob_end_flush( ); before flush( ) could be called. There's also a max_execution_time parameter you might need to increase, and the web server will have its own timeout-related parameters to tweak.

That illustrates the basic technique, and there are some refinements discussed next and in the "Decisions" section, later in this chapter. One burning question you might have is how the browser initiates communication, since the connection is in a perpetual response state. The answer is to use a "back channel"; i.e., a parallel HTTP connection. This can easily be accomplished with an XMLHttpRequest Call or an IFrame Call. The streaming service will be able to effect a subsequent change to the user interface, as long as it has some means of detecting the call—for example, a session object, a global application object (such as the applicationContext in a Java Servlet container), or the database.

Page streaming means the browser discovers server changes almost immediately. This opens up the possibility of real-time updates in the browser, and allows for bi-directional information flow. However, it's quite a departure from standard HTTP usage, which leads to several problems. First, there are unfortunate memory implications, because the JavaScript keeps accumulating and the browser must retain all of that in its page model. In a rich application with lots of updates, that model is going to grow quickly, and at some point a page refresh will be necessary in order to avoid hard drive swapping or worse. Second, long-lived connections will inevitably fail, so you have to prepare a recovery plan. Third, most servers can't deal with lots of simultaneous connections. Running multiple scripts is certainly going to hurt when each script runs in its own process, and even in more sophisticated multithreading environments, there will be limited resources.

Another problem is that JavaScript must be used, because it's the only way to alter page elements that have already been output. In its absence, the server could only communicate by appending to the page. Thus, browser and server are coupled closely, making it difficult to write a rich Ajaxy browser application.

Service streaming is a step towards solving these problems (Figure 6-11). The technique relies on XMLHttpRequest Call

On-Demand JavaScript

⊙⊙ Behaviour, Bootstrap, CrossDomain, Dynamic, JavaScript, LazyLoading, OnDemand, ScriptTag

Figure 6-13. On-Demand JavaScript

Goal Story

Bill's logging into his bank web site. The login form comes up straight away, and as he types in his username and password, the browser is quietly downloading the JavaScript necessary for the rest of the application.

Problem

How can you deploy lots of JavaScript code?

Forces

Ajax Apps make heavy use of JavaScript. Richer browser behavior means bulkier JavaScript to download.

Downloading JavaScript has a performance impact. Interaction cannot fully begin until all initial JavaScript has been loaded.

Bandwidth is also a concern with heavy JavaScript content. Often, not all JavaScript that's loaded is actually used, leading to a waste of bandwidth.

Solution

Download and run JavaScript snippets. The initial page load includes some JavaScript code, which—among other things—contains the bootstrapping code necessary to pull down further JavaScript. There are two techniques available: Script Tag Creation and Service Eval. Each will now be described, following by three main applications of the pattern.

With Script Tag Creation, use DOM manipulation to inject a new script element into the page. Perhaps surprisingly, the effect will be exactly the same as if the <script> tag had been encountered on startup: the referenced JavaScript will be downloaded and executed automatically. The tag can be attached to either the head or the body, though the former is more common as that's where you'd usually find script tags:

var head = document.getElementsByTagName("head")[0]; script = document.createElement('script'); script.id = 'importedScriptId'; script.type = 'text/javascript'; script.src = "http://path.to.javascript/file.js"; head.appendChild(script);

How will you know when the script has loaded? It seems browsers vary in their behavior here, with some loading the script synchronously and some not. In IE, you can be notified with a mechanism similar to XMLHttprequest's (http://www.xml.com/lpt/a/2005/11/09/fixing-ajax-xmlhttprequest-considered-harmful.htmlonreadystatechange). With other browsers, you might need to keep polling for some indication it's loaded (where the indication depends on what the script is meant to do). If you happen to control the server script, you could implement a notification mechanism; i.e., complete the script by calling back to a listener, if it exists. There's actually a promising proposal, JSONP (http://bob.pythonmac.org/archives/2005/12/05/remote-json-jsonp/), which aims to have a simple, flexible mechanism like this, standard across the entire industry.

Script Tag Creation doesn't let you retrieve any old text; it has to be valid JavaScript. And your script won't be able to read it directly because the browser will use it only for evaluation. So how can you get hold of some data from an external server? The most common way is for the remote JavaScript to assign a variable to the required data:

var characters = new Array("Mario", "Sonic", "Lara");

The content is usually a data structure, just like a standard JSON Message, but it requires an assignment as well, in order to be referenced in the code. It could also be a function that returns the desired value. Either way, the browser code has to use whatever name is mentioned in the script, which isn't ideal. Again, the idea of a flexible script mechanism like J is worth considering, as it would let the caller decide on the variable name.

Service Eval is the other technique for On-Demand JavaScript, though not as prominent as Script Tag Creation for reasons we'll discuss later. Under Service Eval, a Web Service is called with a standard XMLHttpRequest Call, it outputs some JavaScript as response content, and the JavaScript is then executed with an eval( ) call. We're just inspecting the XMLHttpRequest Call's responseText property, which we could manipulate before evaluating, so the body doesn't have to be a complete, valid piece of JavaScript (unlike with Script Tag Creation).

Any code not inside a function will be executed immediately. To add new functionality for later on, the JavaScript can add directly to the current window or to an object that's already known to exist. For example, the following can be sent:

self.changePassword = function(oldPassword, newpassword) { ... }

The XMLHttpRequest callback function just needs to treat the response as plain-text and pass it to eval( ). A warning: here again, asynchronicity rears its ugly head. You can't assume the new code will be available immediately after requesting it, so don't do this:

if (!self.changePassword) { requestPasswordModuleFromServer( ); } changePassword(old, new) // Won't work the first time because // changePassword's not loaded yet.

Instead, you either need to make a synchronous call to the server, add a loop to keep checking for the new function, or explicitly make the call in the response handler.

On-Demand JavaScript has three distinct applications:

Lazy Loading

Defer loading of bulky JavaScript code until later on. Works with either On-Demand JavaScript technique (Service Eval or Script Tag Creation).

Behavior Message

Have the server respond with a kind of "Behavior Message," which dictates the browser's next action. Works with either On-Demand JavaScript technique (Service Eval or Script Tag Creation).

Cross-Domain Scripting

Using Script Tag Creation, bypass the standard "same-origin" policy that normally necessitates a Cross-Domain Proxy. Works only with Script Tag Creation.

Let's look at Lazy Loading first. Conventionally, best practice has been to avoid including JavaScript unobtrusively—by including it in one or more script tags:

Chapter 7. Dynamic Behavior

With the technologies covered so far, we can change the display and talk to the server. But what will trigger these actions? Ajax Apps are generally driven from within the browser, so that's where the actions will be triggered. Broadly speaking, there are two types of triggers, each covered here: User Actions, such as mouse clicks and keypresses, and Scheduling, where actions are scheduled to be run at some point in the future.

User Action

⊙⊙⊙ Action, Change, Click, Control, DHTML, DOM, Events, Keyboard, Mouse, Move, Type, Widget

Figure 7-1. User Action

Goal Story

Pam is booking a trip on the corporate travel planner. She sees a form with the usual fields and clicks on location. Suddenly, a list of cities fades in beside the form, and Pam selects Paris. Beside the city list, a second list appears, this one showing approved hotels. Pam chooses the Hilton, and both lists disappear. Pam's pleased to proceed with the destination on the updated form, which now reads, "Paris Hilton."

Problem

How can the application respond to user activity?

Forces

A rich application allows users to interact with it, frequently and in different ways.

Responses must be as quick as possible, so as to streamline performance, keep user attention, and help the user understand the application's cause-effect model.

Using form submissions as the only response to user activity is slow and limits interactivity.

Solution

Handle most User Actions within JavaScript, using event handlers. The essence of Ajax is rich browser-based interaction, and DOM events are the technology that make it happen. DOM objects can register event handlers, functions that are notified when events occur. This callback model should be familiar to anyone who's worked with desktop GUI frameworks.

Event Handling, Behind the Scenes

If you've not worked with a modern user interface framework such as Java's Swing, you might wonder how JavaScript events work. The answer is that, behind the scenes, the operating system tells the browser what the user's doing, and the browser then passes this information to the script. For instance, the operating system—monitoring keyboard and mouse input—might notice that the user hit "X" while the keyboard focus was on the browser window. It will then notify the browser, and the browser will decide which element within the current web page has keyboard focus. If that element happens to have an associated keypress event handler, the browser will arrange for it to be invoked. Just prior to invoking it, it will construct an event object indicating the key that was pressed ("X"), the object that had focus at the time, and auxiliary information like the current mouse location and the current time. The browser will derive some of this information from the previous operating system notification, so other information might be queried there and then. When the event handler is invoked, this event object will be passed in as an argument.

User Actions are events associated closely with the DOM and use the same callback handler mechanism as many other programming environments, described by Gamma et al.'s (1995) Observer pattern. You'll see callback handlers used in other situations too:

onload and onunload are technically the same type of event as the User Actions described in this pattern because they work on DOM elements and use the same syntax. They relate to element lifecycles as opposed to user activity, which is why this pattern doesn't include them in the definition of User Actions. In most scripts, only the window (or document) onload is used; as this pattern shows, it's a good place to register User Actions, among other things. onunload is sometimes used to save data or confirm the user wants to quit.

The XMLHttpRequest Call (Chapter 6) pattern shows how you can register a callback to be notified of network activity as a remote call proceeds.

The Scheduling pattern shows how you can register a function to be executed at a certain time, or at fixed periods. There's no argument passed in, but the mechanism is similar to a DOM object event callback.

Let's say you want to run the following function when the user clicks a shutdown button:

function shutdown( ) { if (confirm("Are you sure you want to shutdown?")) { postShutdownMessageToServer( ); } }

The simplest way to set this up is to declare a button with an onclick event handler:

Now the web browser will arrange for shutdown( ) to be called whenever the button is clicked. However, we can improve on this, because the above declaration mixes JavaScript with HTML. It's cleaner to just declare the button and deal with an event handler in a separate JavaScript file. Since we always want this behavior, we should declare it as soon as the page loads. To run something when the page loads, we can use another event handler that is triggered by browser activity rather than a User Action: onload.

[HTML] <button id="quitButton"/>Quit</button> [Javascript] window.onload = function( ) { quitButton.onclick = shutdown; }

Note that we're declaring this inside window.onload instead of "out in the open"; if you do the latter, you might get an error because the script might be executed before the button is actually on the page. You'll see the window.onload idiom used in most JavaScript code, including all the Ajax Patterns demos.

Instead of referencing a callback function, it's sometimes convenient to define the callback as a closure (anonymous function), as in:

quitButton.onclick = function( ) { if (confirm("Are you sure you want to shutdown?")) { postShutdownMessageToServer( ); quitButton.onclick=null; } }

Registering events with JavaScript, as opposed to in HTML tags, is an example of unobtrusive JavaScript because it separates JavaScript from HTML. And defining the event handler in JavaScript also has another benefit: you can dynamically redefine actions in response to system events. Our shutdown( ) method could also redefine the handler to avoid a double shutdown:

function shutdown( ) { if (confirm("Are you sure you want to shutdown?")) { postShutdownMessageToServer( ); quitButton.onclick=null; // Quit button no longer triggers an event. } }

Notice the model here involves a single handler for any event type; the above commands set the handler, in a manner that will remove any existing handlers. In most cases, that's just fine, and it has the merit of being completely portable. In some situations, though, it's nice to add a handler instead as it makes the code more modular. Two separate library functions can then register for the same events, without having to be aware of each other. Likewise, a function for removing would also be nice:

addEvent(quitButton, "click", postShutdownMessageToServer); ... removeEvent(quitButton, "click", postShutdownMessageToServer);

Browsers do offer support for this functionality, but it's unfortunately varied, and a portable solution has been notoriously difficult. So much so that a competition was recently held to find the best addEvent( ) and removeEvent( ) functions, and you can find the winner, a 15-line script online (http://www.quirksmode.org/blog/archives/2005/10/how_do_i_create.html). Dojo Toolkit (http://dojotoolkit.org) also supports this behavior as part of its sophisticated event library.

It's not always enough for the event handler to know that an event has occurred; it also needs to know about

Scheduling

⊙⊙⊙ Cron, Event, Future, Loop, Periodic, Plan, Repeating, Schedule, Sequence, Timeout

Figure 7-3. Scheduling

Goal Story

Frank's panning across a map of the factory. To ensure he monitors all regions, each is color-coded according to how recently it was investigated. It's implemented with Scheduling: after 5 idle minutes, the room turns orange; after 10 minutes, it turns red.

Problem

How can you run actions in the future, or repeatedly?

Forces

Sometimes, an application needs to repeatedly run the same action; e.g., to extract new data from the server or to monitor application state.

Sometimes, an application needs to run an action at some future time; e.g., to warn a user his session is about to time out.

The server can't initiate a connection to the client, so there's no way to have the server "wake up" the client according to a schedule.

Solution

Use JavaScript timers to schedule actions. JavaScript's timer mechanism lets you schedule a one-off action or an action repeating at fixed intervals. In either case, what you specify is an action and a period of time in milliseconds. Note: an online demo (http://ajaxify.com/run/scheduling) illustrates the code concepts throughout this section and the code snippets loosely follow from the demo.

The naïve way to run an event in the future would be:

sleep(5000); // Nope, won't work. expire( );

That won't work because JavaScript doesn't have a sleep( ) capability—you can't just block in the middle of a script.[*] Instead, you need to schedule the execution.

The most basic usage is planning a one-off event in the future. For example, suppose an ecommerce application wants to expire a price offer after five seconds:

setTimeout(expire, 5000); function expire( ) { $("price").innerHTML = "Expired"; }

What if something happens and you want to cancel the timer. For example, the user starts typing in a deal quantity, and the e-commerce application wants to hold off to give the user some more time. setTimeout actually returns a timer object, which allows for cancellation:

var expiryTimer; ... expiryTimer = setTimeout(expire, 5000); $("dealQuantity").onkeypress = function( ) { // User typed something. clearTimeout(expiryTimer); };

In this example, it might make more sense to postpone, rather than cancel, expiry altogether. You can achieve this by creating a new timer:

var expiryTimer; ... expiryTimer = setTimeout(expire, 5000); $("dealQuantity").onkeypress = function( ) { // User typed something. clearTimeout(expiryTimer); expiryTimer = setTimeout(expiryTimer, 2000); // 2 secs more after a keystroke. };

So far, the future action has been a single function call (expire( )). Sometimes, it's more convenient to say what happens as part of the timeout, in which case you can wrap it all in a string. This prevents the need to create a function specifically to handle the timeout. The string will be evaluated upon timeout:

Chapter 8. Extended Technologies

Paradoxically, we round out the foundational Ajax technologies with a pattern that's all about "non-Ajax" technologies. The term "Ajax" usually relates to the technologies provided by standard web browsers—those covered in the previous chapters. However, it's pragmatic to bend that definition when there are useful "bonus" features that require extra capabilities; hence, the Richer Plugin pattern.

Richer Plugin

○ ActiveX, Applet, Extension, Flash, Greasemonkey, Hack, Java, Mashup, Plugin, ProgressiveEnhancement, Remix

Figure 8-1. Richer Plugin

Goal Story

Sasha is using an online photo-sharing web site. Since she's become a big fan, she's taking the time to install some optional plugins from the site: an in-browser photo editor, a screen capture tool, and a browser toolbar.

Problem

How can you make an Ajax App even richer?

Forces

Ajax makes web interaction much more like a desktop application, but there are limits. Many applications require functionality not available to Ajax Apps.

If you develop a desktop application, even if it connects to the Internet, you forego many advantages of an Ajax App.

Solution

Make your application "more Ajax than Ajax" with a Richer Plugin. Ajax is certainly a step up in richness, but sometimes you need more than Ajax can offer. That's when you release a Richer Plugin that users install to get even more out of your application. The concept here is "progressive enhancement"—the application works fine in standard web browsers, but by installing the Richer Plugin, users can take the interaction to the next level.

What are the things that a Richer Plugin can achieve that an Ajax App can't? Here's a summary:

Browser morphing

Adding buttons, toolbars, bookmarks, icons; changing browser behavior.

Local file access

Reading and writing files on the user's hard drive; unlike cookies, we sometimes want very large storage capacity and the ability to keep the data locally, without ever transferring to the server.

Sound

Playing music and sound effects.

Rich graphics

Providing rich graphics, changing dynamically; although this is changing with the gradual introduction of technologies like scalable vector graphics (SVG), even these can't come close to utilizing the power of a modern display.

Hardware access

Providing input from devices such as microphones, webcams, and gamepads; output to devices like printers and portable gadgets.

Extended communication

Providing communication from the client machine to locations beyond just the base server, and in protocols other than plain old HTTP.

Operating system interaction

Catching events such as shutdown initiation, changing preferences, popping up alerts, and reading hardware information.

Keyboard shortcuts

Providing a full range of keyboard shortcuts, including those that are platform-specific, while avoiding conflicts with the browser's own keyboard shortcuts.

The browser security model imposes these restrictions on web sites because it lets users freely surf the Web with no risk of being compromised (well, low risk). In its absence, users' systems would be in constant danger from malicious web site owners, complacent web site programmers, and devious users. By releasing a Richer Plugin, you allow users to say, "I trust this site to do certain things with my computer or browser," while keeping all other web sites tied down as before.

This pattern uses a loose definition of Richer Plugin to mean anything outside standard web technologies, which can take several forms:

Hooking into an existing browser plugin framework, like Flash, Java, or Real. These are third-party plugins that are very common in many browsers. They're often packaged with the browser, and even if not, there's a good chance the user will have previously installed them for another application. Your application should work automatically as long as the plugin's installed.

Building a browser-specific plugin. Users will then have to explicitly install it.

Building a companion desktop application. This is not a replacement, because the user primarily interacts through the web site, but a means for better desktop integration, with features like pop-up alerts or quick entries. Many plugins of this nature integrate with frameworks such as Konfabulator (http://konfabulator.com) and Growl (http://growl.info/).

Exploiting browser-specific functionality, such as the scalable vector graphics (SVG) capability introduced in Firefox 1.5 or IE's ActiveX components.

The main browsers all have a standard "plugin" mechanism, often used for standard third-party

Part III. Programming Patterns

Knowing the foundational technologies is one thing; applying them is another. Ajax is not rocket science, but there are still plenty of technical issues that will arise in building a serious Ajax App. The programming patterns focus on technical qualities of software—in particular:

Maintainability

Ensuring web services and browser code are easy to understand and work with

Robustness

Chapter 9. Web Services

THE CONCEPT OF A WEB SERVICE HAS ALREADY BEEN INTRODUCED IN THE WEB SERVICE PATTERN, BUT the pattern leaves open many questions—for instance, what sort of messages will be sent to and from the service? How will the URLs look? How will the service relate to backend software? These are the kind of design and programming issues covered in this chapter.

RESTful Service is an overview of Representational State Transfer (REST), a popular set of patterns and idioms for consistent and intuitive web services. RPC Service has similar goals, but with a philosophy that leads to very different services. Neither REST nor Remote Procedural Call (RPC) are Ajax-specific, because they are general industry conventions for web services. However, the next pattern, Ajax Stub, is Ajax-specific, because it provides a convenient way to invoke RPC-style services from the browser.

The remaining patterns look at popular message formats used for communication between the browser and Web Services. HTML Message is a message in HTML form, usually sent from the server to be morphed directly onto the page. In contrast, the remaining messages are raw responses that usually can't be displayed directly, and that may be transferred in both directions. They are Plain-Text Message, XML Message, and JSON Message.

RESTful Service

⊙⊙⊙ API, HTTP, REST, Standard, Universal, Web

Figure 9-1. RESTful Service

Developer Story

Devi's producing a public web site explaining pharmaceuticals to the public. The browser applications will pull down drug data from the server, and authorized users—such as drug manufacturers—will be allowed to upload new information. It's envisioned that third-party applications will also connect to the web server and provide their own user interfaces. For example, pharmacies will be able to use the information via their point-of-sale systems, and doctors will have access from their specialized consulting software. Devi opts to follow a RESTful architecture for all of the services exposed on the server. This makes it easy for the browser side of her own web app to access the data and exposes services in a standardized manner.

Problem

How do you expose services on the Web?

Forces

The browser side of an Ajax App needs to call server-side services.

There might be more than one browser-side application sharing the same server-side service.

As well, it's often desirable for third-party applications to invoke the server-side service too.

With numerous different applications—and a variety of developers—accessing the service, it ought to be easy to use.

Taken as a whole, the server-based services in a single "application" form an Application Programming Interface (API). It's important that services work consistently within the API.

The basic architecture of the Web does not force any type of service architecture—any given functionality could be exposed in a wide variety of styles.

Solution

Expose web services according to RESTful principles. Representational State Transfer (REST) is an architectural style—or "pattern"—guiding the architecture of web services. Like "Ajax" itself, "REST" is a broad term and can be broken down into many smaller patterns and idioms (enough to fill up an entire book!). The explanation here can't possibly do justice to the entire concept, but it's worthwhile being familiar with the general principles of REST. Because REST is such a large topic, be prepared for the reality that most real-world systems you'll encounter are only somewhat RESTful; they'll follow some principles and break others. Some people are now making the distinction between "High REST"—sticking closely to REST—and "Low REST"—following just a couple of core REST guidelines (http://lesscode.org/2006/03/19/high-low-rest/).

Motivating REST: many ways to skin a call

To motivate REST, let's consider a football service. Sufficiently authorized clients can read game records, upload new games, correct existing games, or delete games. We'll assume your backend's already written, but how would you offer it as a web service? How will client developers call it, and how will they learn about its interface?

To wit, following are a few random ways you might expose a game service—note that these aren't necessarily RESTful!

GET call to a special newGame service, http://example.com/newGame?id=995&blue-baggers=150&redlegs=60

http://example.com/newGame, with an XML body like <game id="995"><score team="bluebaggers">150</score><score team="redlegs">60</score></game>

POST or PUT call to a special newGame service, http://example.com/newGame, with a CGI-style body: id=995&bluebaggers=150&redlegs=60

GET call to a multipurpose gameMaintenance service, http://example.com/gameMaintenance/command=newGame&id=995&bluebaggers=150&redlegs=60

POST call to a multipurpose gameMaintenance service with a command-qualified URL, http://example.com/gameMaintenance/command=newGame, with an XML body like <game id="995"><score team="bluebaggers">150</score><score team="redlegs">60</score></game>

And we could go on. And then we could look at the other functions here. The point is this: as a service designer, you have many options for representing a single function. Is that, in itself, a problem? You might argue it's not a problem at all—just choose any old option, using some darts and a blindfold if need be. Then just create the service and document its usage—any competent developer will then be able to deal with it, right? That argument, however, is flawed on several counts:

You might assume that the browser script for your web app, and perhaps some known external clients, are the only users of your service, making it relatively easy to create and document any service. In fact, your application resides within a larger Internet ecosystem. Among the entities involved: the web browser (e.g., Firefox), which impacts issues like back-button handling and bookmarking; the web server (e.g., Apache), which impacts issues like caching and compilation; network routers and caches all along the way; robot applications such as those crawling the Web on behalf of search engines; and personal web agents that trawl through the Web on behalf of individuals. Your application will perform faster and more reliably if you follow certain conventions that are common among these entities.

Ideally, a service should be intuitive and self-documenting: if a service is based on familiar conventions, you have only to specify the bare minimum for a developer to be able to use it. That's simply not possible if everyone chooses alternatives according to their own personal taste. The standards-based interface usually trumps the optimized interface that's technically superior but based on unfamiliar conventions. There are thousands of web services available on the Internet; developers should be able to assess them and put them into action with as little effort as possible.

In the absence of any guiding principles, there's a disturbingly high likelihood you'll end up in tedious debates with colleagues over the most trivial issues, trying to arrive at an optimal API. (The blindfold and darts—or an impatient manager—would alleviate this kind of paralysis, but the other issues would still remain.)

So we've seen the argument for a convention-based approach to web services, but what's the convention? REST is one such convention, consisting of many different guidelines, that has broad industry support. Just about any functionality can be presented as a RESTful API, and by doing so, it gains the advantage of a familiar, intuitive style. Many services on the Net already follow RESTful principles, to differing degrees. Also, many real-world entities such as browsers and caches tend to assume REST. These entities work better if REST is in place—a browser, for instance, might give better feedback; a cache will retain all the data that the programmer expected it to retain. And what will happen if REST is not in place? It's probably not the end of the world, but you'll find little things will go wrong and work in unexpected ways. For instance, hitting the browser's Reload button might cause a purchase order to be resubmitted. Or a cache might end up caching data it shouldn't, or it might not cache any data at all. Why these unfortunate events might occur will become clear as we learn about the RESTful principles.

Introduction to RESTful principles

So much for abstract motivation: what exactly are the RESTful principles (http://www.xfront.com/REST-Web-Services.html) all about? REST ultimately presents the server as a big blob of "resources" such as people, cars, or football games. Clients interact by inspecting and changing that state. Putting a purchase order on the server, for example, will trigger the server to affect the purchase described in the order. This contrasts with the Remote Procedural Call (RPC) approach, where the client would call a remote purchase( ) procedure. REST is based on data manipulation, RPC is based on procedural calls.

REST sees the universe as consisting of resources and operations performed on, and with, those resources. The resources and the operations can be represented as the HTTP concepts of URLs and call methods.

Resources as URLs

A resource is something like a "business entity" in modelling lingo. It's an entity you wish to expose as part of an API. Almost always, it's a noun; e.g., a person, a car, or a football game. Each resource is represented with a unique URL. So when you say a RESTful URL, it will be a thing rather than an action.

Operations as HTTP methods

REST leverages the existing HTTP methods, particularly GET, POST, PUT, and DELETE. Note that the XMLHttpRequest object, and some wrapper libraries such as ajaxCaller (http://ajaxify.com/run/testAjaxCaller/), support all these methods. If they're unfamiliar to you, consult the W3C HTTP 1.1 Specification (http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html).

In strict HTTP parlance, there's a distinction to be made between Universal Resource Locators (URLs) and Universal Resource Identifier (URIs). URI is the more general term, as it describes a general resource. A URL is a particular type of URI that, in a somewhat vague sense, identifies a network location. You could define a protocol to represent books with URIs (e.g., isbn://12345), and those URIs wouldn't be URLs since they have no relationship with any network location; they are just a scheme for uniquely identifying objects. A string such as http://ajaxpatterns.org/RESTful_Service not only identifies an object (making it a URI), but also refers to a network location (making it a URL), one which will be used by various software to locate the resource.

Having done the Right Thing by pointing out the distinction, let me state why this pattern refers only to URLs. Most importantly, URLs are the language most developers are familiar with, even in contexts where URI is technically the appropriate term (which is arguably most of the time). Furthermore, there is confusion (http://kyseo.blogspot.com/2005/12/uri-url-urns.html

RPC Service

⊙⊙⊙ Delegate, Facade, Procedure, Proxy, Remote, Remoting, RPC, SOAP, WSDL

Figure 9-2. RPC Service

Developer Story

Dave's finished coding the backend services for some blogging software. There's an ArticleRepository object, for example, with functions to create, remove, update, and delete articles. Using a new product, he's able to effortlessly expose ArticleRepository as a web service API, with a different URL for each operation. Now he can code up the browser script for his own application to talk to the API, and there's a ready-made API available to third parties too.

Problem

How do you expose services on the Web?

Forces

Refer to the section "Forces" in RESTful Service.

Solution

Expose web services as Remote Procedural Calls (RPCs). Like REST, RPC is a broad term. It's somewhat ambiguous too, meaning different things to different people. This pattern uses a fairly general definition—a Remote Procedural Call (RPC) is a form of communication where the client invokes a remote procedure on the server. The definition implies:

RPCs are generally characterized as actions. The URL is usually verb-like; e.g., /game/createGame?gameId=995 or /game/updater?command=createGame&gameId=995.

Typically, though not necessarily, the remoting is designed to be relatively transparent. The server's backend is developed, and a remoting framework acts as a bridge between client and server, allowing the client to directly invoke backend operations. Often, the remoting framework is distributed across both tiers, wrapping and unwrapping messages at each end.

Here are some concrete forms of RPC:

Procedure-style API

In the simplest case, it's just a matter of the URL style: the server exposes URLs that look like verbs and invite action, though they don't necessarily have any direct relationship to the backend technology.

XML-RPC

XML-RPC is a simple protocol for RPCs, using tags like <methodCall>, <methodName>, and <params>. An XML document is uploaded to make the call, and another document is returned with the response. There are many frameworks available to automatically expose backend services as XML-RPC and to invoke such services, so the remoting is fairly transparent from the developer's perspective.

SOAP

Ajax Stub

⊙⊙ Delegate, Facade, Procedure, Proxy, Remote, Remoting, RPC

Figure 9-3. Ajax Stub

Developer Story

Devi begins working on the browser script for a trading application and soon decides she needs a way to execute a trade. There's already a method to do that on the backend TradeManager object, so after 15 seconds reconfiguring the Ajax Stub, the browser script is able to invoke the method.

Problem

How do you implement an RPC Service (see earlier in this chapter)?

Forces

Some services have to reside on the server, for reasons of security, complexity, and performance. Thus, you need to expose web services to be accessed from the browser script.

A web service should encapsulate only web-related logic and delegate any business and application logic to the backend.

This creates a redundancy: the web service and the backend logic are tied together. When one changes, the other must change, leading to more work and the risk that you'll forget to change one or the other.

The browser needs to access web services via XMLHttpRequest over HTTP, which is fundamentally different to the way both JavaScript and server-side scripts work. It's tedious to wrap and unwrap messages into a form suitable for transfer.

Solution

Use an Ajax Stub framework that allows browser scripts to directly invoke server-side operations, without having to worry about the details of XMLHttpRequest and HTTP transfer.. The aim is to let the JavaScript call remote operations as if they were regular JavaScript functions, facilitating a more natural program style.

There are several frameworks to support stubbing of this nature. Typically, you declare which backend operations should be exposed, and the framework takes care of the remoting. It will provide a JavaScript stub object that will delegate the call over to the server side. For example, you might have a Java method like this:

public class Book { public String getEdition(int year) {...} ... }

After configuring an Ajax Stub framework for handling remoting, it's light work to make call the class from within the browser:

editionLabel.innerHTML = book.getEdition( );

Typical facilities include:

Ability to convert JavaScript arguments and return values into server-side language constructs

Sensible handling of server-side errors and exceptions

Support for general XMLHttpRequest issues, such as timeouts

A stubbing framework usually consists of two components:

A JavaScript component marshalls the call into an XMLHttpRequest, handles the call, and later unmarshalls the response.

A server-side configuration component, with some configuration, that accepts the call, delegates to the backend operation, and marshalls the return value into a suitable response.

Decisions

How will you secure the server-side function?

HTML Message

⊙⊙⊙ AHAH, Direct, Display, HTML, InnerHTML, Message, Precise, ServerSide, Visual

Figure 9-4. HTML Message

Developer Story

Dave's identified the need for a credit history service, providing a list of transactions. JavaScript resources are limited, so the entire HTML is created server side, and the browser application has only to morph a DOM element with the entire HTML response.

Problem

What format should be used for server responses?

Forces

The browser display needs to be dynamic in Ajax Apps.

The display changes by altering the DOM, and HTML is often used to specify the new value.

The nature of the change is often complex and needs to be determined server side.

Not all developers know JavaScript, and many feel its usage is best minimized.

Solution

Have the server generate HTML snippets to be displayed in the browser. In this approach to browser-server dialogue, the server-side service outputs some HTML, which is picked up by the XMLHttpRequest callback function and an element then morphed by setting its innerHTML property to the response HTML. In general, the server side is application-specific because the HTML response is closely tied to the application's display style.

The XMLHttpRequest allows callers to retrieve responses as either XML or plain-text. With HTML, it's usually easiest to retrieve it as a plain-text string. If the HTML is a pure XHTML document, XML is also an option, but usually that's not the case because there's no need for a header section. The entire response can be as simple as "<strong>You Win!</strong>". That's not an XHTML document because there's no <xml> tag, nor are there HTML header and body sections.

HTML Messages should be used with caution because they couple server-side services with browser display. That means it's difficult to develop the tiers in parallel. In maintenance, if you change the display by altering its initial HTML or any JavaScript manipulation, you often need to change the HTML-generating service too. There's also a risk on the server side that you might be coupling business logic with HTML generation. So when might an HTML Message be appropriate?

Performance

Since browser-side parsing and rendering can be expensive, HTML Messages can make an application more responsive, especially if the HTML is cached on the server or the browser.

Server-Side Code Generation

One case where HTML Messages make some sense is with Server-Side Code Generation (Chapter 12), where the server builds all the browser-side code for you. There are strengths and weaknesses of such frameworks, and if you decide to use one, then browser-server coupling is not really an issue because all maintained code is server side anyway.

Legacy Code

Most legacy applications will use the conventional approach of publishing pages from the server. They therefore already have all the HTML generation present in the server-side environment, so if you're Ajaxifying a legacy application, a quick migration path is to retain the server-side HTML generation where possible.

Plain-Text Message

⊙⊙ Acknowlegement, Custom, PlainText, Semantic, String

Figure 9-6. Plain-Text Message

Developer Story

Devi's working on an auto-completion form field. Every few seconds, the server needs to respond with a list of suggestions. Devi codes the server side to output the suggestions as a comma-separated list, as it's convenient to output on the server and easy to parse in the browser.

Problem

How can you transfer data between server and browser?

Forces

Ajax Apps require messages to be transmitted back and forth.

Both browser and the server must be able to access the message. That usually means the format must be easily accessed in JavaScript as well as in whichever server-side language is used.

Solution

Pass simple messages between server and browser in plain-text format. This is a broad pattern, because "plain-text" is an ambiguous term. The emphasis here is on keeping things simple—XML or JSON are often more appropriate for complex data structures, but can also add unnecessary complexity.

Despite the name, XMLHttpRequest is perfectly capable of working with plain-text responses. As discussed in XMLHttpRequest Call (Chapter 6) and XML Message (later), the browser script can decide how to interpret the response in XMLHttpRequest by switching between the responseText and responseXML properties.

Plain-text formats include:

Response Code

XML Message

⊙⊙⊙ Document, Format, Hierarchical, PlainOldXML, POX, Semantic, Structured, XML

Figure 9-8. XML Message

Devloper Story

Dave is creating an Ajax search engine. The results are offered as a RESTful Service, so that external developers can create their own interface. The results are presented as XML, and Dave's own search interface pulls it down with an XMLHttpRequest call, then transforms the result to HTML.

Problem

How can you transfer data between server and browser?

Forces

Ajax Apps require messages to be transmitted back and forth.

Both browser and the server must be able to access the message. That usually means the format must be easily accessed in JavaScript as well as whichever server-side language is used.

Solution

Pass messages between server and browser in Extensible Markup Language (XML) format. It's feasible to pass XML in both directions. On the server side, it's a common way to expose structured data as text, and browser-side applications can download XML with an XMLHttpRequest Call and parse it in different ways. Where the browser needs to upload some complex data, it's fairly easy to serialize the data as XML and upload it in the body of an XMLHttpRequest Call.

XML is the definitive standard for data transfer across the industry, so just about any server-side development language has comprehensive support. The great advantage of XML over custom formats is the massive base of supporting frameworks, libraries, editors, and general tools. How you deal with XML on the server is language-specific, and a few choices are outlined in "Decisions," next. However you do it, make sure the server presents XML content with the appropriate header to ensure its handled correctly by XMLHttpRequest:

Content-type: text/xml

As the name implies, XMLHttpRequest was developed with XML transfer in mind. To retrieve an object with an XMLHttpRequest, first point it to a URL known to serve XML:

xReq.open("GET","http://ajaxify.com/run/portal/drilldown/drilldown.phtml ?categoryName=Overviews",true);

When XMLHttpRequest delivers a response, the response can either be interpreted as a DOM object or plain-text:

var domObject = xReq.responseXML alert(domObject) // Outputs "[Object]" (IE) // or "[object XML Document]" (Firefox) var xmlString = xReq.responsetText alert(domObject) // Outputs the whole string // i.e. "<category ...> ... </category>"

In most cases, you want to use responseXML because it gives you a well-structured DOM object: a hierarchical structure that can be navigated using industry-standard conventions. With recent browsers, you can rely on some basic parsing functionality to interrogate DOM objects. HowToCreate has a good list of DOM traversal functions (http://www.howtocreate.co.uk/tutorials/texterise.php?dom=1) and many of the Ajax Patterns demos perform such parsing (e.g., the Basic Wiki Demo; see http://ajaxify.com/run/wiki).

If you're parsing XML, consider delegating to a cross-browser library. Sarissa (http://sarissa.sourceforge.net), for example, has a variety of cross-browser XML manipulation routines. Another library, Interactive Website Framework (IWF) (http://sourceforge.net/projects/iwf/), simplifies XML expressions. Instead of the following standard DOM query:

JSON Message

⊙⊙ JSON, Marshal, Semantic, Serialize, YAML

Figure 9-10. JSON Message

Developer Story

Dave is creating an Ajax calendar. The browser periodically polls for new appointments, which the server is sending as JSON messages. Since JSON messages are just JavaScript code for object creation, Devi's browser script needs only to run eval against each message in order to reconstruct the appointment.

Problem

How can you transfer data between server and browser?

Forces

Ajax Apps require messages to be transmitted back and forth.

Both browser and the server must be able to access the message. That usually means the format must be easily accessed in JavaScript as well as whichever server-side language is used.

Solution

Pass messages between server and browser in JavaScript Object Notation (JSON) format. JSON is a standard serialization format, created in 2002 as a cleaner and lighter alternative to XML. As with XML, the object can range in complexity from a simple string to a deep hierarchical structure. Also like XML, JSON is language-neutral, meaning that you could marshall a C++ object into JSON notation and unmarshall it to form an object in Perl. But in practice, JSON is particularly suited to browser-server communication because it's a format based on JavaScript.

In fact, a JSON Message is JavaScript. This is a valid JSON Message:

"Homer J."

You can test browser-based JSON conversion on the Basic JSON Demo (http://ajaxify.com/run/json/). As you'll see there, the JSON Message "Homer J." maps to the JavaScript string, Homer J..

Here's a more complex example (as on the demo, reformatted):

{"houseNumber":"742", "street":"Evergreen Terrace", "city":"Springfield", "postcode":"49007", "country":"USA", "comments": ["Deliveries accepted.","Familiar address, huh?",""] }

As you can see, this JSON message is just a JavaScript object literal. You convert it like this:

var name = eval("(" + nameJSON + ")");

Or like this (http://jibbering.com/2002/4/httprequest.html) if it takes your fancy:

var name=new Function("return " + nameJSON)( );

Note that you don't even need a JSON library in the browser. The browser can pick up a JSON string using an XMLHttpRequest Call and simply run the standard eval function.

However, there is actually a JavaScript JSON library (http://www.crockford.com/JSON/js.html), which adds two important capabilities: parse( ) for safer string-to-object conversion, and stringify( ) for object-to-string conversion. The former is an alternative to manual JavaScript evaluation, because evaling arbitrary messages puts your app at risk of running malicious JavaScript code. Thus, the string-to-object conversion should be executed if you don't trust the message. To parse a string, use the following codee:

var name = JSON.parse(nameJSON);

If you need to upload something to the server, you'll need to convert a JavaScript object to JSON with stringify( ). The Basic JSON Demo (http://ajaxlocal/run/json/) shows this conversion process too. The call looks like this:

var nameJSON = JSON.stringify(name);

So far, the examples have considered only JavaScript conversion. But JSON wouldn't be very useful if you could only convert to and from JavaScript objects—you need to convert at the other end too, and your server side is probably not written in JavaScript. That's why there are JSON processors for many languages. Using these processors, you can easily share an object between JavaScript and your favorite server-side language.

A remoting modification (

Chapter 10. Browser-Server Dialogue

We've looked at the basic technologies for web remoting, which leads to questions about browser-server information flow. How will you deal with multiple calls? How will you keep the browser and the server synchronized? How will you access external domains?

Call Tracking is about tracking calls and dealing with the asynchronous nature of Ajax. The next three patterns concern synchronization. With Periodic Refresh, the browser keeps requesting fresh information from the server. The opposite is Submission Throttling, where the browser keeps uploading new information to the server. An alternative to Submission Throttling, Explicit Submission involves uploading only when the user performs some action.

All of those patterns help manage bandwidth, but keeping things in sync can still be quite complex when there are lots of entities in the server and a rich interface in the browser. Distributed Events help manage the complexity.

Finally, Cross-Domain Proxy is a technique for mediating the dialogue between the browser and external servers.

Call Tracking

⊙⊙⊙ Asynchronous, Follow, Monitor, Parallel, Tracking, XMLHttpRequest

Figure 10-1. Call Tracking

Developer Story

Dave is writing an Ajax chat client, and he's concerned the more vocal users will hog server resources, so he decides to enforce a policy of no more than three pending server calls at any time. Thus, the browser script pushes messages into a queue and tracks the progress of each XMLHttpRequest Call (Chapter 6) containing those commands. He can enforce his maximum call count policy by tracking the state of each call.

Problem

How can you control parallel XMLHttpRequest Calls (Chapter 6)?

Forces

Fast users and busy applications often stretch networks and servers to their limits.

Asynchronous interaction is the only practical approach for Ajax applications. You don't want to block all interaction just because the network happens to be slow.

The number of simultaneous requests must be controlled, because browsers can only handle a few at a time, and also to reduce the overall load.

Solution

Track XMLHttpRequest calls as they proceed from browser to server and back again. XMLHttpRequest is a fairly basic component that needs to be augmented or wrapped for better control over asynchronous dialogue. Furthermore, it is useful to keep all these wrappers in a collection. Note that this pattern is fairly low level, and the details should generally be encapsulated in a wrapper library.

The standard mechanism for Call Tracking requires an XMLHttpRequest wrapper. Consider the implementation of the Ajax Client Engine (ACE) library (http://www.lishen.name/). A Requester abstraction creates an XMLHttpRequest object upon construction:

Periodic Refresh

⊙⊙⊙ Auto-Update, Polling, Sync, Synchronise, Sychronize, Real-Time

Figure 10-2. Periodic Refresh

Developer Story

Devi's coding up a ticket sales web site. For each event, she wants to keep the browser updated with the number of tickets remaining. Thus, she introduces a timer so that every 30 seconds, it calls a web service to pull down the latest sales stats.

Problem

How can the application keep users informed of changes occurring on the server?

Forces

The state of many web apps is inherently volatile. Changes can come from numerous sources, such as other users, external news and data, results of complex calculations, and triggers based on the current time and date.

HTTP requests can only emerge from the client. When a state change occurs, there's no way for a server to open connections to interested clients.

One way to keep the browser updated is HTTP Streaming (Chapter 6), but, as the "Alternatives" section for that pattern explains, it's not always ideal. In particular, it's not very scalable.

Solution

The browser periodically issues an XMLHttpRequest Call to gain new information; e.g., one call every five seconds. The solution makes use of the browser's Scheduling (Chapter 7) capabilities to provide a means of keeping the user informed of latest changes.

In its simplest form, a loop can be established to run the refresh indefinitely, by continuously issuing XMLHttpRequest Calls (Chapter 6):

setInterval(callServer, REFRESH_PERIOD_MILLIS);

Here, the callServer function will invoke the server, having registered a callback function to get the new information. That callback function will be responsible for updating the DOM according to the server's latest report. Conventional web apps, even most of those using XMLHttpRequest Calls, operate under a paradigm of one-way communication: the client can initiate communication with the server, but not vice versa. Periodic Refresh fakes a back-channel: it approximates a situation where the server pushes data to the client, so the server can effectively receive new information from the browser. Indeed, as some of the examples show, the server can also mediate between users in almost real-time. So Periodic Refresh can be used for peer-to-peer communication too.

But before we get too carried away with Periodic Refresh, it's important to note that it's a serious compromise, for two key reasons:

The period between refreshes would ideally be zero, meaning instant updates. But that's not realistic and the browser will always lag behind. Latency is particularly problematic when the user is interacting with a representation of volatile server-side data. For instance, a user might be editing an object without knowing that another user has already deleted it.

There is a significant cost attached to Periodic Refresh. Each request, no matter how tiny, demands resources at both ends, all the way down to operating-system level. Traffic-wise, each request also entails some bandwidth cost, which can add up if refreshes are occurring once every few seconds.

So a key design objective must be to increase the average refresh period and reduce the content per refresh, while maintaining a happy user experience. One optimization is a timeout: the refreshes cease when the system detects the user is no longer active, according to a Timeout (Chapter 17) mechanism. You also want to make sure each refresh counts; it's wasteful to demand lots of updates if the network isn't capable of delivering them. Thus, the browser script can do some monitoring and dynamically adjust the period so it's at least long enough to cope with all incoming updates. Many of the Performance Optimization patterns can also be applied to Periodic Refresh—see the next section.

Decisions

Submission Throttling

⊙⊙⊙ Buffer, Queue, Performance, Throttle

Figure 10-4. Submission Throttling

Developer Story

Devi's producing an Ajax chat web site and wants to transmit text as the user types. However, she knows some users will type faster than the system can cope with so she introduces a throttling mechanism that ensures no more than one message is uploaded every 200 milliseconds.

Problem

How can information be submitted to the server?

Forces

Information is often uploaded in bursts; e.g., a chat tool incurs many hits when a user becomes passionate about the topic, or a data entry tool incurs many hits when the user responds to some new information.

It's difficult for the server to cope with lots of messages at once.

Browsers can only handle a limited number of pending XMLHttpRequest Calls at any moment.

Each message has overheads, such as packet headers and requires some processing at each stage of the browser/server round-trip.

Solution

Instead of submitting upon each JavaScript event, retain data in a browser-based buffer and automatically upload it at fixed intervals. As with many applications of buffering and throttling, the purpose is to strike a balance between responsiveness and resources. In most cases, it would be ideal to respond to every keystroke and mouse movement—so, for example, like a desktop application, tooltips could come directly from the server as the user mouses around. But that's not practical due to bandwidth considerations, and possibly server constraints too. So to ease bandwidth—and to lessen server processing, call detail is accumulated and periodically uploaded.

Exactly what the buffer holds is application-specific, but there are two general styles: data buffers and command queues.

In the first buffer style, the buffer holds some data to be uploaded. Consider implementing a Suggestion (Chapter 14) interface like Google Suggest (http://www.google.com/webhp?complete=1&hl=en), which keeps showing information based on what's been typed. The simplest thing would be to add an "onchange" listener to the text field and upload the buffer when each change occurs. However, what if you get a fast typist, one of those folks who revels in their "words per minute" metric? Banging out 100 words per minute means perhaps 10 characters per second, or a call every 100 milliseconds—feasible with a localhost web server, maybe workable on an intranet, but not scalable for most public Internet applications. So Suggestion systems, and more generally Live Command-Lines (Chapter 14), run a fixed-period timer. Every 500 milliseconds, say, the browser checks if there was a change since the last call, and if so, uploads to get some information and remembers what was uploaded to avoid doing so again. Effectively, the result for the current text field is cached, and the cache can only change on fixed periods.

A similar pattern might be used on a Live Form (Chapter 14), where the entire form is periodically uploaded to the server, even though some fields are blank, along with an indication of the user's progress. The server can then use some intelligence to critique the form data, so as to provide live feedback.

The other style of buffer is a command queue. Here, Commands (see Gamma et al., 1995) are held in a queue, and the whole queue periodically uploaded to the server before being cleared. Some kind of serialization must take place, so the Commands must be represented as Strings and the Queue must ensure they can be pulled apart by the server; e.g., by separating them with a delimiter character. It's up to the developers to agree on a protocol for defining how Commands are represented as Strings. For example, a wiki might use the string "Del Issues" to delete the Issues page. Another technique would be to store the commands as custom JavaScript objects and serialize them into JSON Messages (Chapter 9).

Explicit Submission

⊙⊙⊙ Submit, Packet, Performance

Figure 10-6. Explicit Submission

Developer Story

Devi's producing an Ajax chat web site. She decides text will only be transmitted after the user explicitly clicks a "Done" button.

Problem

How can information be submitted to the server?

Forces

It's difficult for the server to cope with lots of messages at once.

Each message has overheads, such as packet headers, and requires some processing at each stage of the browser-server round-trip.

Users often need to manipulate a small work unit privately, and then upload it as a whole atomic unit to the server.

Solution

Instead of automatically submitting upon each browser event, require the user to explicitly request it; e.g., submit upon a button click. Typically, the user performs some work for a few seconds or minutes, and then clicks a button to tell the server. It's a familiar pattern on the Web as it feels similar to standard form submission.

The most common example is a text field, which may be an input or textarea control. While it would be possible to transmit keystrokes as they happen, that's often not desirable. In a wiki, for instance, imagine what would happen if a user deleted a paragraph in order to replace it. Any other user viewing the wiki would see the paragraph disappear! Furthermore, the history would reflect that transition state.

Where there's only one input field, it sometimes makes sense to rely on onchange or onblur events to detect that the change has been made. But how about when there are several closely related fields? Even then, some automatic submission is okay, to provide some validation information, for instance. However, if the information is important enough to cause a change to the server, Explicit Submission is a good way to ensure the user intended what's been uploaded.

As well as some inconvenience, the downside of relying on the user to explicitly submit data is . . . what if the user doesn't? Have you ever quit the browser and forgot to submit a form you were working on? The consequences may be minor for a query-and-report application, but for data entry applications, large amounts of work can be lost when the user forgets to submit it or doesn't realize it's necessary to do so. For that reason, Explicit Submission can sometimes be complemented by automated Submission Throttling. For instance, Gmail (http://gmail.com) will only send mail upon an Explicit Send command, but nonetheless has an Autosave feature that will periodically upload an in-progress message as a draft.

Decisions

How will the user request the submission?

The submission mechanism should ideally be similar to those in similar non-Ajax systems.

Buttons are one common idiom, with a generic label like Submit, Done, or Go!. More meaningful names are usually clearer, being specific to the task being conducted—for instance, "Buy," "Search," or "Delete." Buttons have the benefit of making the Explicit Submission mechanism stupidly obvious: "I click the button, the info's submitted; I don't click it, it's not submitted."

Relying on a significant keystroke, usually Enter, can also work in the right context, and, by using the keyboard, it supports power users.

Listening for the onblur

Distributed Events

⊙ Events, Feed, Listener, Messaging, MVC, Notification, Observer, PublishSubscribe, RSS, Refresh, Semantic, Synchronise, Synchronize, Update, WebFeed

Figure 10-9. Distributed Events

Developer Story

Devi's producing a web app for auditors at a financial institution, aiming to highlight substantial transactions. Relevant transactions are already published on the enterprise messaging system, and Devi transforms it into an RSS feed on the web server. The browser script then checks the feed every few seconds, and updates the view whenever it detects a substantial transaction occurred.

Problem

How do you decouple code in a complex application?

Forces

Ajax Apps involve at least two tiers: a browser tier and a web server tier. In practice, the web server tier is often dependent on further tiers and external systems.

Each tier can be quite complicated, containing many stateful entities (objects, HTML controls, or regular variables).

The state of all these entities must often be synchronized, in order to keep users and external systems up-to-date. The synchronization needs to occur within a tier as well as across tiers—for example, an HTML table needs to change whenever browser-side user preferences change, but also whenever the server-side database changes.

Keeping all these objects synchronized can become complex—there are about n2 possible message paths from one object to another.

Solution

Keep objects synchronized with an event mechanism. This is a classic software pattern applied to Ajax, related to the Observer (Gamma et al., 1995) and "Publisher-Subscribe" (Buschmann et al., 1995) patterns, and also a key feature in the classic "Model-View-Controller" architecture. The DOM already provides an mechanism for low-level events, but the events discussed here are more semantic in nature; i.e., related to business and application concepts such as "account deleted." Note that the term "event" is used in a broad manner to mean any subscription-based approach that alleviates the need for direct calls from source to destination. Any publish-subscribe messaging mechanism falls under this definition.

Here's a scenario to motivate the concept of events. Say you have 10 objects with inter-dependent states. That is, when one object changes, any number of the other nine must change accordingly. The naive implementation would endow each object with an understanding of the other nine objects. Each time it's changed, an object would then tell each other object how to update. Each object now knows the other nine intimately. When one changes, the other nine must be updated—a major blow to encapsulation.

As with many programming problems, you can create a better solution by adding another layer of indirection—in this case, an event mechanism. Let each object broadcast changes instead of directly telling others how to respond. The changes should generally occur in semantic terms—rather than saying "someone's clicked me," an object should say "counting module has begin," or, "transaction has successfully completed." And let any object register to be notified whenever a message like this occurs. For larger systems, thinking in terms of events is easier as it breaks down the synchronization logic. You have one simple task to make objects broadcast events whenever they occur. And you have a separate task to decide how objects should actually respond to events, if they care about them at all.

On the Web, this pattern can be applied in different ways:

Server-to-browser

Page elements can be kept in sync with server objects.

Browser-to-browser

Page elements can be kept in sync with each other.

Browser-to-server

Server objects can be kept in sync with each other.

Server-to-server

Server objects can be kept in sync with each other.

Browser-to-server and server-to-server are both feasible, but beyond the scope of the Ajax Patterns because they are more about server-side architecture.

As we saw in Chapter 7, JavaScript offers some event-handling support, but there's no generic event API. To use distributed events then, you'll need to roll your own event system, which is easier than it sounds. Server-to-Browser propagation works something like this. There's a Server Event Manager, with two responsibilities. First, it runs a Periodic Refresh (see earlier in this chapter) or participates in HTTP Streaming (Chapter 6) to check if any server activity has occurred. Second, it offers an event propagation facility so that interested browser entities can register to discover any changes. When a server change occurs, the Server Event Manager constructs an event object based on the server's output and passes it any interested listener. To do this, the manager must retain a collection of listeners and the event types they are listening to. The minimal set of services would then be:

addListener (eventType, listener)

removeListener (eventType, listener)

These listeners can be callback functions, like the callback function used by XMLHttpRequest. Or, they can be objects that contain a function with a standard callback name for the event type being registered, such as onUpdate( ).

With Browser-to-Browser propagation, you can also have a Central Server Manager to accept events and notifications. Alternatively, each object can be responsible for creating and propagating events specific to itself. That is, each object capable of generating events needs to allow other objects to register themselves for the events.

Observer is a special case of this pattern that arises frequently. The events are not user actions but change notifications. Event listeners are observing an object and responding to its state. Often, it's used to keep state in sync. An HTML table, for example, can render the latest state of a timetable object on the server side.

Finally, note that this pattern is somewhat speculative and open-ended, but the main purpose should be clear: to add a layer of intermediation so that objects can encapsulate their own responses to system activity, rather than being told what to do by other objects.

Decisions about Distributed Events

Will you publish a history or just the current state?

Cross-Domain Proxy

⊙⊙⊙ Aggregator, Fusion, Mash-Up, Mashup, Mediator, Mix, Proxy, Tunnel

Figure 10-10. Cross-Domain Proxy

Developer Story

Dave's working on a job-hunting web site and wants to "mash up" content from various other sites. Each job ad is accompanied by a review from a corporate forum web site, recent company news, and a stock ticker. To get these details into the browser, the server queries several sites and exposes their content as a web service that the browser can query.

Problem

How can you augment your application with services available elsewhere on the Web?

Forces

There's a lot of data and useful services on the Web. Much of it is freely available. It's usually more effective to leverage that content than to replicate it yourself.

The "same-origin policy" constrains most Web Remoting (Chapter 6) techniques, meaning that the browser script can only talk to the server from whence it came, and not to anyone else.

External servers often use protocols and message formats that would be difficult for browser-side scripts to process. They may also require authentication, and such details should not be exposed in the browser.

To comply with service agreements, web site owners often have to control and monitor all traffic to and from their web service, which is impossible if their server is being bypassed.

Solution

Create proxying and mediating web services to facilitate communication between the browser and external domains. As explained in XMLHttpRequest Call (Chapter 6), the same-origin policy means the browser script can only talk to the server from whence it came, which is the "base server." Hence, any communication with external domains must go via the base server.

The simplest form of proxy is a dumb Web Service (Chapter 6) that simply routes traffic between browser and external server. The service can accept the remote URL as a parameter. That URL will then be accessed synchronously, and the service will output its response. All this follows the Proxy pattern (Gamma et al., 1995).

A cleaner type of Cross-Domain Proxy is more closely based on the Facade or Adaptor pattern (Gamma et al.). In this case, the base server presents the interface that's most suitable to the browser script, and performs whatever manipulation is necessary to interact with the external script. For example, the external server might present a complex, over-engineered, SOAP-based RPC Service, but you want the browser to deal in simple Plain-Text Messages (Chapter 9). The likely solution is to present the plain-text Web Service, dress up requests into SOAP messages, and undress responses into Plain-Text Messages.

The Facade/Adaptor approach is generally better for two reasons. First, it keeps the client simple, protecting it from dealing with the details of the protocol used to communicate with the external server. Second, it's more secure: the first approach will allow any client to call out to any server they feel like, whereas the second approach will allow you to exert much more control over what kind of communication takes place.

A Cross-Domain Proxy is implemented using some form of HTTP client library to access the remote server. Generally, the connection should be quite straightforward—specify the URL and grab the response as a string. However, it might get more complex if the remote web service relies on cookies and authentication, parameters that the proxy might have to pass on from the Ajax request that initiated the call.

Decisions

What external content will be accessed?

In theory, anything you can do with a browser can theoretically be accomplished by an automated HTTP client. However, there are likely to be legal constraints, and technical challenges too if there is serious usage of JavaScript or browser plugins. Furthermore, your script will be vulnerable to changes in the site layout, and many content providers have deliberately performed subtle changes to prevent automated access. Thus, relying on the ever-increasing collection of public APIs and structured data would make a better choice.

You can find a collection of public APIs at http://wsfinder.com. Some popular APIs include Amazon, Delicious, EvDB, Flickr, Google Maps, Google Search, Technorati, and Yahoo Maps.

How will you connect to the external server?

If you're scraping content directly from a public web site, you'll need to use the underlying web protocol, HTTP. And even if you're talking to an API, you'll probably be communicating with HTTP anyway. In some cases, API publishers will provide code to access the content, rather than just publishing the specification. Also, with more complex web services protocols like SOAP, you don't have to write a lot of low-level code yourself. In many cases, though, the easiest thing to do is talk HTTP and manually format and parse messages yourself.

Many scripting languages feature built-in libraries to communicate in HTTP and are often well-equipped for Cross-Domain Proxy implementations due to strong support for regular expression manipulation. A PHP example is featured in the code example that follows shortly.

In the Java world, for instance, the standard API provides some relatively low-level support for HTTP clients, but you can use some of the web site testing libraries to quickly extract content from external sites. HttpUnit is a good example, and it also has some support for JavaScript manipulation. To grab some content:

Chapter 11. DOM Population

WHAT DOES THE BROWSER DO ONCE IT RECEIVES A RESPONSE FROM THE SERVER? SEVERAL OPTIONS ARE covered in this chapter. XML Data Island explains how you can store an XML document within the web page's DOM instead of converting it into a custom data structure. It also touches on some browser-specific features that let you tie the document's value to the display. XML responses are also the primary concern of Browser-Side XSLT, explaining how an XML document can be converted to XHTML for inclusion on the page, or modified for uploading back to the server. Browser-Side Templating is not XML-specific; it brings to JavaScript the embedded scripting template concept popular in many server environments, such as JSPs and PHP scripts.

XML Data Island

⊙ Data, DOM, Storage, XBL, XML

Figure 11-1. XML Data Island

Developer Story

Dave's demographics viewer stores each table of statistics as an XML document. A big cache of XML documents is held within the page, one for each table. The user can rearrange the tables and switch them between visible and hidden, but all the XML metadata remains on the page. To render the tables, Dave exploits the browser's ability to translate XML into HTML tables.

Problem

How can you render incoming XML and retain the data?

Forces

Many Ajax Apps receive XML Messages and must convert them into HTML.

The incoming data often has to be retained as well.

Solution

Retain XML responses as XML Data Islands—nodes within the HTML DOM. Strictly speaking, an XML Data Island is an XML document embedded in a standard XHTML document, usually within an <xml> tag. Say the initial page looks like this:

<html> <head> ... </head> <body> <h1>Top Score</h1> <p>Here's the ... </p> ... <xml id="scoreData"></xml> </body> </html>

After an XMLHttpRequest Call (Chapter 6) brings the browser an XML Message (Chapter 9), the message is retained in the tag, using techniques described in a moment. The result is a web page DOM like this:

<html> <head> ... </head> <body> <h1>Top Scores</h1> <p>Here's the ... </p> ... <xml id="scoreData"> <score> <title>Galaga"</title> <player>AAA</player> <score>999610</score> <score> </xml> </body> </html>

How do you do something like that in JavaScript? You might assume it's pretty trivial, given that HTML itself is XML (or close enough to it). But unfortunately, it's not as easy as you'd expect and also is browser-dependent. If you have the XML string, you can set scoreData element's innerHTML as the string value. If you have only a DOM node, you can convert it to an XML string by inspecting its innerHTML property. However, this won't work on all browsers and might lead to portability problems as browsers interpret the string in different ways. An alternative is to avoid string manipulation and directly graft one DOM node onto another. The document object has two useful methods here: cloneNode( ) and importNode( ). Again, there are plenty of portability issues to consider—see Peter-Paul Koch's discussion (http://www.quirksmode.org/blog/archives/2005/12/xmlhttp_notes_c.html

Browser-Side XSLT

⊙⊙ Presentation, Render, Style, Stylesheet, Transform, View, XML, XPath, XSLT

Figure 11-3. Browser-Side XSLT

Developer Story

Dave has just received a request to change the weather portlet on the homepage. The image must now be on the bottom, not the top, and the wording should be more precise. The browser receives periodic weather updates in the form of an XML specification, and it uses Browser-Side XSLT to get HTML for the portlet. The stylesheet is embedded inside the static homepage HTML file. So Dave just has to tweak the XSLT to get all the browsers rendering the new design.

Problem

How can you render incoming XML?

Forces

Many Web Services (Chapter 6) output XML Messages (Chapter 9), and Ajax clients have to convert them into HTML.

While modern browsers provide good support for XML parsing via the JavaScript's DOM support, the direct interface is lengthy and tedious to use.

Code gets complex and error-prone when you mix HTML generation with application logic. JavaScript is not well-suited to programmatically building up HTML strings from input XML.

Solution

Apply XSLT to convert XML Messages into XHTML. XSLT—Extensible Stylesheet Language Transformations (http://www.w3.org/TR/xslt)—is a language for transforming XML documents into other forms. Here, XML is transformed to XHTML, a type of HTML that conforms to the XML standard. (You could get away with converting to plain-old HTML, but things works better if you stick to XHTML.) XSLT is fairly well-supported in modern browsers and, although browser APIs vary, there are good cross-browser toolkits available.

XSLT is well-suited to rendering XML. An XML document is designed to encapsulate a data object, and an XSLT stylesheet is a strategy for presenting such objects. Previously, XSLT has been used as a server-side technology—the server grabs some XML from a repository or another process and creates a web page by transforming it into XHTML. All that happens on the server. More recently, browsers have incorporated XSLT functionality, so the browser can automatically create an HTML page by marrying together an XML document with an XSLT stylesheet.

Browser-Side XSLT is slightly different. Here, the XML document does not constitute the contents of the entire page, but it does constitute a response from an XMLHttpRequest Call (Chapter 6). The transformation to XHTML can take advantage of the browser's built-in XSLT support, or alternatively an XSLT processor can be implemented in JavaScript, building on the browser's more primitive XML features. However the XSLT occurs, you're unlikely to be implementing it yourself. There are a couple of good cross-browser libraries available, discussed in later in this chapter in "Real-World Examples."

It's beyond the scope of this pattern to explain XSLT in any detail; however, here's a quick overview:

An XSLT stylesheet is itself an XML document.

An XSLT processor transforms an XML document into another form, by parsing the original document and using the stylesheet as the transformation strategy.

XSLT expressions are specified in another language, XPath. Among its capabilities, XPath provides a powerful mechanism for expressing a position within an XML document. For example, the XPath expression /category/items/category will match twice when the following document is processed:

Browser-Side Templating

⊙ Pages, Presentation, Render, Template, Transform, View

Figure 11-5. Browser-Side Templating

Developer Story

Devi has just been assigned to make the user profile more verbose. The original version takes an XML document about the user and renders it all with a jungle of JavaScript code that accumulates a big HTML string amid if-then conditions and loops. Devi decides the first step is to refactor all that into a Browser-Side Template, isolating the HTML generation. As a result, introducing the new content becomes trivial.

Problem

How can you separate presentation from logic?

Forces

Generating HTML in the browser is a good way to isolate all presentation logic in one tier.

To render context-specific information within the browser, you need to rely on a dynamic mechanism. It's not feasible to just set an element's innerHTML property to point to a static HTML page.

Code gets complex and error-prone when you mix HTML generation with application logic. JavaScript is not well-suited to programmatically building up HTML strings.

Solution

Produce templates with embedded JavaScript and call on a browser-side framework to render them as HTML. The template contains standard HTML and allows context variables to be substituted in at rendering time, which gives it more flexibility than a static HTML page. In addition, you can also intersperse JavaScript code. For instance, generate an HTML table by running a loop—one iteration for each row.

The templating idea has been used for a long time on the Web, which is evident in frameworks like Perl's HTML::Mason, Java's JSPs, and the layout of languages like PHP and ASP. All of these are syntactic sugar of the "killer app" variety. That is, they technically don't add any new functionality, but make life a whole lot easier for web developers. In the Java world, the standard complement to JSPs is servlets. Here's how you'd write a message in a servlet:

package com.example.hello; import java.io.IOException; import java.io.PrintWriter; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; public class HelloServlet extends HttpServlet { protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { response.setContentType("text/html"); PrintWriter out = response.getWriter( ); out.println("<html>"); out.println("<head><title>Hi Everyboddeee!</title></head>"); out.println("<body>"); out.println(" Your name is <%context.name %>\"" + request.getAttribute("name") + "\"."); out.println("</body></html>"); out.close( ); } }

Not pretty. There are printing commands all over the place, escaped special characters, quote marks, and error handling. What happened to the simplicity of HTML? Code like this is best avoided because it mixes presentation with logic. The following JSP, using the standard JSTL library, achieves the same thing in a more lucid, maintainable style:

<%@ taglib prefix="c" uri="http://java.sun.com/jstl/core" %> <html> <head><title>Hi Everyboddeee!</title></head> <body> Your name is <c:out value="$name" />. </body> </html>

With many Ajax Apps, it's deja vu. Often, the browser receives a raw response such as a Plain-Text Message (Chapter 9) or an XML Message (Chapter 9) and must then render it as HTML, ruling out server-side templates. A common approach is to manually traverse the structure programmatically and create some HTML to be injected via some element's innerHTML property. The old "bucket of print statements" strikes again.

The solution here recapitulates in the browser what all those server-side templating frameworks offer. A templating framework, while not immensely difficult to implement, is probably not the sort of thing you'd want to write yourself. This pattern, then, is only made possible by the fact that frameworks already exist, and some are mentioned in the following examples. Here's a quick example of a template in Ajax Pages format (http://ajax-pages.sourceforge.net/):

<html> <head><title>Hi Everyboddeee!</title></head> <body> Your name is <%= name %>. </body>

The JavaScript template is in similar format to the JSP above and follows standard conventions of templates. That is:

Any code is contained inside <% and %> tags.

Any expression to be substituted in the HTML is contained inside <%= and %> tags.

To apply the template, create a processor and pass it a context. With Ajax Pages:

Chapter 12. Code Generation and Reuse

THERE ARE TWO GENERAL PROGRAMMING PATTERNS IN THIS CHAPTER. SERVER-SIDE CODE GENERATION IS one technique people are using to avoid working directly with HTML and JavaScript. Cross-Browser Component is a component that's been developed for portability across the major browsers.

Server-Side Code Generation

⊙⊙ AutoGenerated, CodeGen, Framework, Library

Figure 12-1. Server-Side Code Generation

Developer Story

Dave's creating a web app with a trendy new server-side framework, ZeroWeb. The framework lets him write everything in Java as if he's writing a standalone GUI application. It's capable of using that code to generate a conventional edition of the application, An Ajax edition, a desktop edition, and—for an extra .95—a smartphone app too.

Problem

How do you avoid using HTML and JavaScript?

Forces

HTML and JavaScript are the only code recognized by standard browser—any Ajax App must be represented in a combination of these formats.

Ajax is increasing the complexity of HTML and JavaScript. Whereas JavaScript was previously used for simple tasks like navigation support or form validation, it's now driving the interaction in many applications.

Most Ajax Apps also have a substantial backend, which is rarely implemented in JavaScript.

It's difficult to completely separate the roles of browser-side developer and server-side developer. In fact, it's often more desirable to have the same developer working on a vertical slice of functionality; i.e., contributing a little to all tiers in order to deliver a new requirement. Thus, the same developer is often expected to work across tiers, where the environment differs significantly.

Server-side development has certain advantages over JavaScript development. In some cases, there's better support for code editing, refactoring, testing, and so on.

Sometimes, you want different user interfaces for the same application—a non-Ajax version or a desktop version, for example.

Solution

Automatically generate HTML and JavaScript from server-side code. You write code in a standard server-side language like Java and it's converted to HTML and JavaScript. Depending on the framework, you might not need to code directly in HTML or JavaScript at all. Since you're unlikely to create such a framework yourself, this pattern relies on a certain category of publicly available Ajax framework (see Appendix A).

In the case of all code being auto-generated, the code is something like a conventional desktop application. Widgets and event handlers are defined, and the browser generates the appropriate HTML, along with XMLHttpRequest Calls (Chapter 6) and general JavaScript handling. As well, the framework will often generate Ajax Stubs to deal with incoming calls and pass them to appropriate application code.

In other frameworks, the approach is more mixed. The framework might let you create a Slider with just a single tag, for example. But in the tag, you'll specify a JavaScript function to be called whenever the slider value changes.

There are several arguments for Server-Side Code Generation:

It may be more productive, as the server-side language and framework may be more expressive and also because the framework might remove redundancy across the tiers (such as a data structure having a JavaScript representation as well as a server-side representation).

Developers don't have to know HTML and JavaScript; they can work in the server-side language and environment they're most familiar with.

The framework can help with graceful degradation. In other words, it can free the developer of worrying about browser portability, JavaScript, or cookies being turned off, or XMLHttpRequest Calls timing out.

The framework can, in theory, deliver multiple platforms simultaneously. A single code base can be used to generate a web application (Ajaxian or not), standalone desktop application, and maybe even a mobile application.

However, code generation does comes at a cost to usability. Look at it this way: anything you could generate from the server, a skilled developer could also hand code. The reverse is not true; given enough time and talent, it's always possible to build a custom solution

Cross-Browser Component

⊙⊙⊙ Agnostic, Common, Component, CrossBrowser, Independent, Portable

Figure 12-4. Cross-Browser Component

Developer Story

Dave has developed a cross-browser sprite engine, allowing users to easily drag a sprite around the screen. The library provides a browser-independent interface, taking into account browser incompatibilities regarding detection of mouse position and positioning of icons. When a new drag-and-drop requirement comes along, programmers can code to Dave's library in blissful ignorance of the underlying browser incompatibilities.

Problem

How can you manage cross-browser incompatibilities?

Forces

A major selling point of Ajax is portability—the ability to run an application on any platform with a modern browser.

There are numerous browsers on the market, and while IE still dominates, the market is becoming increasingly fragmented. IE currently has around 85-90 percent share on average web sites, but within certain user groups, Mozilla is actually far more popular. A recent survey of AjaxPatterns.org traffic showed 47.5 percent hits from Firefox, 30.1 percent from IE, and 2.7 percent from Safari.

Browsers vary from each other in subtle and not-so-subtle ways. Sometimes, a browser simply doesn't support certain functionality. Other times, two browsers offer the same functionality, but it must be accessed in different ways. While the W3C (http://www.w3c.org) establishes standards for browsers, not all are followed.

When creating a web program, you want to focus on business and application logic, without being distracted by idiosyncrasies of underlying browsers. Cross-browser compatibility is a cross-cutting concern, best dealt with as a separate task at a separate time.

Solution

Create Cross-Browser Components, allowing programmers to reuse them without regard for browser idiosyncrasies. A Cross-Browser Component abstracts away from underlying browser issues, offering a single API that can be used on any supported browser.

Following are a couple of examples of cross-browser incompatibilities. For simplicity, only the two most popular browsers are discussed: IE and Firefox. Other browsers are also important, and not surprisingly, introduce further incompatibility issues. The following are two examples:

XMLHttpRequest

XMLHttpRequest, a critical Ajax component, is constructed in a browser-specific way, as established in XMLHttpRequest Call (Chapter 6). Most browsers include a specific XMLHttpRequest class, but with IE prior to version 7, it's created as an ActiveX component instead. Furthermore, there are some subtle behavior differences. For example, XMLHttpRequest on Opera lagged behind in terms of support for features like authentication (http://www.scss.com.au/family/andrew/webdesign/xmlhttprequest/). An even greater problem is that many older browsers simply don't support this object, even though similar functionality can nonetheless be achieved using IFrames.

opacity

opacity—or transparency —is becoming important in Ajax applications, as discussed in Popup (Chapter 15) and in One-Second Spotlight (Chapter 16). However, portability is poor (http://www.sitepoint.com/newsletter/viewissue.php?id=3&issue=102&format=html

Chapter 13. Performance Optimization

THE PERFORMANCE OPTIMIZATION PATTERNS IMPROVE THROUGHPUT AS WELL AS APPLY SOME USER-centered techniques to ease the pain of waiting for the server. Browser-Side Cache talks about implementing a custom cache within the application and explains why it's sometimes more suitable than the standard web browser cache. Predictive Fetch extends the cache concept by proposing that likely actions be anticipated, so that required data is already sitting in the browser by the time it's required. Another very different way to achieve zero network delay is to simply take a stab at the required value, the idea behind Guesstimate.

Multi-Stage Download proposes parallel or scheduled downloads so as to reduce bottlenecks and grab the most critical details as early as possible.

Fat Client is one of several possible Ajax architectural styles that helps optimize performance by pushing as much logic as possible—and potentially storage—over to the browser side.

Browser-Side Cache

⊙ Auto-Update, Memoise, Memoize, Sync, Synchronise, Sychronize, Real-Time

Figure 13-1. Browser-Side Cache

Developer Story

Devi's produced a physics simulation and, though it's functional, she's disappointed that it keeps pausing to request calculations from the server. To speed it up, she sets up a cache that will retain the calculations locally.

Problem

How can you make the system respond quickly to user activity?

Forces

The application should respond to user actions quickly—ideally, instantaneously.

Many user actions require a response from the server.

Responses from the server can be noticeably latent due to data transfer and server processing overheads.

Solution

Retain server results in a Browser-Side Cache. The cache is a JavaScript map-style object that holds query-result pairs; the queries are cache keys and the server results are server results. So whenever the browser needs to query the server, first check the cache. If the query is held as a key in the cache, the corresponding value is used as the result, and there's no need to access the server. Caching like this is performed in many systems, but there are some interesting Ajax-specific issues, such as the relationship with the browser's built-in cache and the asynchronous nature of calls.

XMLHttpRequest Call (Chapter 6) explains that call results can already be cached by the browser, so you might wonder why this pattern exists. After all, the browser not only handles caching for you, but can cache huge quantities of data using the filesystem, and, unlike a Browser-Side Cache, this data will usually live beyond the current session. The reason you'd use a Browser-Side Cache is to exert more control over caching. You get to choose how much data is cached and how long it lasts, whereas those things are set by the user in a web browser cache. Moreover, you bypass the notorious portability issues associated with cache control, choosing exactly what will be and what will not be cached. Another advantage of a Browser-Side Cache over the web browser's cache is that you can save JavaScript objects you've built up, thus saving the effort of reconstructing them later on (a process known as "memoisation").

What exactly is the format of the key and value? In the simplest case, the query would just be the URL, and the result would be the response body. However, that's not so useful because if you make calls correctly, as discussed in XMLHttpRequest Call (Chapter 6) and RESTful Service (Chapter 9), the browser will handle caching on your behalf, and as it's backed by the filesystem, it can cache much greater quantities of data. However, more useful are caches that hold high-level semantic content—typically the results of processing server responses.

A cache data structure requires some way to access the key-value pairs "randomly"—that is, directly, without having to traverse the entire data structure until the desired key-value pair is found. In JavaScript, a cache can be created using an associative array (in JavaScript, this is the same as an object). So an empty array is created at startup and gradually accumulates server results. Of course, an Ajax App lives indefinitely, so the array could continue growing if not controlled. There are a few ways to handle this, as discussed later in "Decisions."

It's common in many systems to treat the cache as a Proxy, as shown in Figure 13-2

Predictive Fetch

⊙ Anticipate, Fetch, Guess, Prefetch, Preload, Prepare, Ready

Figure 13-3. Predictive Fetch

Developer Story

Dave's tuning a music web site and the logs tell him that users who listen to a song for more than a minute are quite likely to click the Purchase button. So he introduces a new timer—at the one-minute mark of each song, the browser will quietly download pricing data just in case the user wants to buy it later on.

Problem

How can you make the system respond quickly to user activity?

Forces

The application should respond to user actions quickly; ideally, it should feel instantaneous.

Many user actions require a response from the server.

Responses from the server can be noticeably latent due to data transfer and server processing overheads.

Solution

Pre-fetch content in anticipation of likely user actions. Pre-fetching attempts to remove the delay altogether for certain user actions.

The obvious motivation for instantaneous feedback is efficiency: the things can happen faster because the user's not sitting around waiting for results. In reality, though, the proportion of overall time waiting might actually be quite small. The more severe problem is distraction, because delays will break the user's concentration, along with the sense of frustration at not being in control. The other problem is that for real-time applications, the user is slower to respond to state conveyed by the server. A chat session will be more strained, and a remote device under browser control will be more erratic.

Here are some occasions when Predictive Fetch might be used:

The user's navigating a Virtual Workspace (Chapter 15) such as a large table. Pre-fetch the results of moving in each direction.

The user's converting between two currencies. Pre-fetch the major currency rates.

The user's reading some articles. Pre-fetch all stories in her favorite category.

Predictive Fetch usually requires a Browser-Side Cache (earlier in this chapter) in order to accumulate the pre-fetched content. In addition, you can exploit the web browser's built-in cache in a couple of ways—see Browser-Side Cache , earlier, for a comparison of these techniques. To get something in the web browser's built-in cache, one technique is to issue XMLHttpRequest Calls for content you'll need later on, using the techniques mentioned in that pattern to encourage the content to be cached. The response handler does nothing here—the point is simply to bring the content into the browser, where it will be cached for later on. The other technique is for images: create a dummy image (document.createElement("img")) object and set its src property to whatever image URL you want to preload. Again, we're only doing this to ensure the image goes in the cache, so the image isn't actually attached to the page.

It's rarely possible to pre-fetch for all actions, so the designer has to be smart about anticipating which actions are most likely. Higher priority will be given to content that is more important or more likely to be used.

One problem with pre-fetching is the application will be a bit erratic. The user might be surprised that some commands respond instantaneously, while similar commands take a long time. While some variation is always expected on the Web, the main problem comes when the data arrives immediately. The user will reasonably question whether the app was really taking their action into account (http://www.baekdal.com/articles/Usability/usable-XMLHttpRequest/). For that reason, a useful but counter-intuitive trick is to actually fake a small delay and use a visual effect like

Guesstimate

⊙ Approximate, Estimate, Extrapolate, Fuzzy, Guess, Guesstimate, Interpolate, Predict, Probabilistic, Sloppy, Trend

Figure 13-5. Guesstimate

Developer Story

Devi's producing a taxi tracker so the head office knows where its fleet is at any time. The taxis only transmit their location every 10 seconds, which would ordinarily lead to jerky movements on the map. However, Devi wants the motion to be smooth, so she uses interpolation to guess the taxi's location between updates.

Problem

How can you cut down on calls to the server?

Forces

To comprehend system activity and predict what might happen next, it's useful to have frequent updates from the server.

It's expensive to keep updating from the server.

Solution

Instead of requesting information from the server, make a reasonable guess. There are times when it's better to provide a good guess than nothing at all. Typically, this pattern relates to dynamic situations where the browser periodically grabs new information, using Periodic Refresh. The aim is help the user spot general trends, which are often more important than precise figures. It's a performance optimization because it allows you to give almost the same benefit as if the data was really arriving instantaneously, but without the bandwidth overhead. For this reason, it makes more sense when using Periodic Refresh than HTTP Streaming, because the latter doesn't incur as much overhead in sending frequent messages.

One type of Guesstimate is based on historical data. There are several ways the browser might have access to such data:

The browser application can capture recent data by accumulating any significant observations into variables that last as long as the Ajax App is open.

The browser application can capture long-term data in cookies, so it's available in subsequent sessions.

The server can expose historical data for interrogation by the browser application.

Equipped with historical data, it's possible to extrapolate future events, albeit imprecisely. Imagine a collaborative environment where multiple users can drag-and-drop objects in a common space, something like the Magnetic Poetry Ajax App (http://www.broken-notebook.com/magnetic/). Using a Periodic Refresh of one second, users might see an erratic drag motion, with the object appearing to leap across space, then stay still for a second, then leap again. A Guesstimate would exploit the fact that the motion of the next second is probably in the same direction and speed as that of the previous section. Thus, the application can, for that second, animate the object as if it were being dragged in the same direction the whole time. Then, when the real position becomes apparent a second later, the object need not leap to that position, but a new estimate can be taken as to where the object's moving, and the object can instead move smoothly toward the predicted location. In other words, the object is always moving in the prediction of its current predicted location. Dragging motion is an example where users would likely favor a smooth flow at the expense of some accuracy, over an erratic display that is technically correct.

How about long-term historical data, stretching over weeks and months instead of seconds and minutes? Long-term data can also be used for a Guesstimate. Imagine showing weather on a world map. The technically correct approach would be not to show any weather initially, and then to gradually populate the map as weather data is received from the server. But the philosophy here would suggest relying on historical data for a first-cut map, at least for a few indicative icons. In the worst case, the Guesstimate could be based on the previous day's results. Or it might be based on a more sophisticated statistical model involving several data points.

Historical data is not the only basis for a Guesstimate. It's also conceivable the browser performs a crude emulation of business logic normally implemented server side. The server, for example, might take 10 seconds to perform a complex financial query. That's a problem for interactivity, where the user might like to rapidly tweak parameters. What if the browser could perform a simple approximation, perhaps based on a few assumptions and rule-of-thumb reasoning? Doing so might give the user a feel for the nature of the data, with the long server trip required only for detailed information.

Multi-Stage Download

⊙ Download, Incremental, Multi-Stage, Parallel

Figure 13-8. Multi-Stage Download

Developer Story

Dave's been informed an e-commerce homepage is turning many users away with its absurdly slow initial download. He refactors it so the initial download is nothing more than a banner and the key navigation links. News and featured items are still on the page, but they're pulled down separately, after the initial page has loaded.

Problem

How can you optimize downloading performance?

Forces

Content can be time-consuming to load, especially when it contains rich images.

Users rarely need to read all the content on the page.

Users often have to click through several times to locate a particular item.

Users sometimes browse around by exploring links between pages.

Solution

Break content download into multiple stages, so that faster and more important content will arrive first. Typically, the page is divided into placeholders (e.g., div elements) with the content for each placeholder downloaded independently. XMLHttpRequest Calls can be issued simultaneously or in a serial sequence, and the page will gradually be populated as the calls return.

The pattern applies when an application needs to download a lot of content from the server. Typically, this occurs on startup and also when a major context switch occurs. By breaking up the call, you can deliver faster and more important data earlier on. You avoid bottlenecks that occur when a single piece of content blocks everything else from returning.

The initial download of a page might be very small, containing just a skeleton along with navigation links and enough context to help the user decide if it's worth sticking around. That's the critical content for a fresh visitor, and there's no need to make him wait for the rest of the page to load. The browser can request the main page content as a separate call.

This pattern is in some respects an Ajaxian upgrade of the old technique of filling up a page gradually, where the server continues flushing the buffer and the user can see the page as it loads. Ajax makes the whole thing more powerful, because the latter downloads can attach to any existing part of the page.

One resource risk is having too many requests at once. Thus, consider a (purely speculative) Multiplex Call variant. Establish a browser/server protocol that lets the browser wrap multiple requests and the server wrap multiple responses. Then, issue only a single call with all required data inside. The immediate response might return a few results only, so the browser waits a bit, and then makes a further request for outstanding data. The whole process repeats until the browser has all the data.

Decisions

How will the page be divided into blocks?

The trickiest decision is how to divide the page into blocks, each of which will be downloaded individually. Since the blocks are downloaded in parallel, the main advice is to create small blocks of initial content to ensure the initial download is quick, and to group together content that's likely to be ready at the same time. Also, too many blocks will give the initial load an ugly appearance and may have the undesirable effect of causing already displayed elements to move around. For a fresh visitor, one example would be to create blocks as follows:

A block of general information about the web site; e.g., name, summary, and a small icon.

A navigation block showing links within the site.

Fat Client

⊙ BrowserSide, Decoupled, DHTML, Fast, Responsive, Rich, Thick

Figure 13-11. Fat Client

Developer Story

Dave's new mortgage application is very quick as it avoids server calls wherever possible. After some benchmarking, he discovered that optimized JavaScript is actually fast enough to run the financial algorithms in the browser, so that users now receive almost immediate feedback as they tweak parameters.

Problem

How can you architect the system to feel truly rich and interactive?

Forces

The application should respond to user actions quickly—ideally, instantaneously.

Responses from the server can be noticeably latent due to data transfer and server processing overheads.

Solution

Create a responsive Ajax App by performing remote calls only when there is no way to achieve the same effect in the browser. Whereas the server is traditionally the realm of more complex processing, this pattern suggests harnessing the browser for all it's worth. In the extreme case, it means running just about everything inside the browser, but that's not the main message here. The main message is to reconsider the traditional server-centric view of web architecture; for developers to be more conscious about decisions on matters regarding where the various components live. Placing more logic inside the browser gives you the opportunity to make the application more responsive.

Web apps usually have several layers:

External system communication

The user interface is what the user sees and interacts with. The application logic concerns the dialogue between the user and the computer, the flow of information back and forth—for example, given a particular user action, what should the computer do next? The business logic concerns knowledge about the domain and the web site owner's practices and policies. The data access layer concerns reading and writing data to a persistent store. External system communication is also necessary in many enterprise systems, where outside systems play the role of information sources and sinks.

Conventional web apps concentrate UI code in the browser, using a mix of HTML and CSS, with some JavaScript used to perform a few basic UI tricks like placing keyboard focus or showing some animation or offering a dynamic menu. Everything else is usually managed on the server. This pattern reconsiders the balance by suggesting that some applications are better served pushing application and business logic into the browser (Figure 13-12).

Figure 13-12. Responsibilities of a Fat Client

With application logic in the browser, an inversion of control occurs in which the browser now controls the flow of activity. When the user performs some action, it is the browser that decides how to respond, and it calls the server only if necessary, and for very specific services.

The advice to hold business logic in the browser is anathema to most literature on serious web site development and should not be taken lightly. Business logic in the browser has numerous problems:

Programming in JavaScript can be difficult because many features familiar to mainstream developers are unavailable. The standard API is minimal and object-oriented facilities are based on prototypes rather than classes. Indeed, the usage of dynamic typing is also uncomfortable for many developers in the Java and C# camps.

Portability is a further complication due to the number of platforms that must be supported, and their subtly inconsistent behaviors.

Development is also a challenge. Coding, refactoring, debugging, automated testing, and monitoring, are standard in modern IDEs like IntelliJ Idea, but support at this level is minimal for JavaScript.

Security is another constraint that forces logic server side. Savvy users can peruse code and data in the browser and can even tamper with what's sent back to the server. Obfuscation—making the code difficult to read—is a very weak defense.

Business and application logic should be held on the server, where they can be reused by other clients.

Web data generally can't be saved on the local machine.

Are any of these showstoppers? Let's look at how we can cope with each of these problems:

JavaScript may be different from server-side languages, but many programmers are discovering it's a lot more powerful than previously assumed (http://www.crockford.com/javascript/javascript.html). Until recently, the main resources on JavaScript were "cut-and-paste this code" web sites, but there are an increasing number of books and resources that take the language seriously. The language has undergone a renaissance in parallel with Ajax. New idioms and patterns are emerging, along with a pile of cross-browser frameworks to augment the relatively bare native API. The recent introduction of JSAN (http://openjsan.org)—a repository of JavaScript libraries like Perl's CPAN—will only further reuse. Object-oriented concepts, including inheritance and aggregation, are indeed possible, if not in a manner familiar to all server-side developers. Dynamic versus static typing is a matter of taste that shouldn't affect a language's credibility; the dynamic approach of JavaScript may not be everyone's preference, but JavaScript is hardly an outsider here.

Browsers are more standard nowadays, and many of the new libraries abstract away browser dependencies (see Cross-Browser Component [Chapter 12]). For example, over a dozen libraries offer a browser-independent factory method to retrieve XMLHttpRequest.

Recent tools are making life a lot easier; some are discussed in Development Patterns (Part V).

Security remains an issue, but it's possible to deploy the application so that security-

Part IV. Functionality and Usability Patterns

Ajax is fundamentally about delivering value to users and their organizations. The previous parts have mostly covered technical issues that support developers, and now it's time to consider the user experience: how Ajax user interfaces will look, and what users will be able to do with them.

Many of the patterns combine display manipulation with the web remoting capability of Ajax. Others, such as Slider

Chapter 14. Widgets

THE WIDGETS PROVIDED BY STANDARD HTML—TEXT INPUTS, RADIOBUTTONS, AND SO ON—HAVEN'T changed much for ten years and are getting a little tired. So many people have been building their own widgets on top of the standard HTML offerings, and a few patterns have emerged. It's unlikely you'll need to build these widgets yourself, because library implementations are available for almost all of them. Nevertheless, it's worth it to consider when to use them and how to configure them.

The chapter starts with smaller, isolated widgets and builds up to more complex components. Slider is "the widget HTML forgot"—a typical slider widget so useful it's surprising it hasn't yet become standard HTML. Progress Indicator is a simple message or animation shown while waiting for a web remoting call. Drilldown is a control for selecting an element within a hierarchy.

Traditional HTML widgets are turbo-charged in the next couple of patterns. Data Grid is a "table on steroids," with tools for querying and restructuring that will be familiar to users of spreadsheets and database report tools. And Rich Text Editor is a "textarea on steroids," with options such as font size and color similar to those of a modern word processor.

Suggestion is similar to the traditional combo-box, a mixture between browsing and searching that relies on web remoting to locate a set of options against a partial text input. Live Search works similarly but shows search results rather than helping to complete a text field. It also works with nontext controls.

Finally, Live Command-Line and Live Form are higher-level patterns that tie together various widgets and patterns. The former is largely speculative and explores techniques to help people learn and use command-line interfaces within the Web. The latter is a more proven Ajax Pattern—a form that keeps changing in response to user input.

Slider

⊙⊙⊙ Continuous, Lever, Multiple, Range, Slider

Figure 14-1. Slider

Goal Story

Stuart is rating his lecturer online. Each answer lies somewhere between "Strongly Disagree" and "Strongly Agree." Unlike conventional surveys, the scale is continuous, because Stuart uses a Slider to set a precise value along the spectrum.

Problem

How can users specify a value within a range?

Forces

Most data is constrained to reside within a range.

Prevention is better than cure; the user interface should be designed such that the user is forced to choose a value within the range instead of the interface checking and providing an error message later on.

Solution

Provide a Slider to let the user choose a value within a range. The user drags a handle along one dimension to set the value. As a variant, two handles on the same Slider can allow the user to choose a range instead of a single value.

Standard HTML contains two related input controls: radiobuttons and selectors. Each of these lets you specify a value from a fixed list of choices. The biggest advantage of a Slider is that the data range can be continuous as well as discrete. Of course, "continuous" is an approximation, since you can only have as many choices as there are pixels in the Slider range. But with most Sliders, that means you have hundreds of unique values, which is continuous for most intents and purposes.

A Slider can also show discrete data by identifying several points along the range. When the user releases the handle, the Slider jumps to the nearest point. Why use a Slider when radio buttons and selectors already support discrete data? A Slider provides a better visual indication that the data resides in a spectrum. Also, it's often faster because the user can click anywhere in the region as opposed to aiming precisely at a radiobutton or opening up a dialog box.

A further benefit of Sliders is their excellent support for comparing similar data. When several Sliders share the same range, they can be placed in parallel to show how the variables differ. For example, a product survey could ask questions such as "Were you happy with the price?" and "How easy was it to start using?" The answers lie on a different scale but ultimately map to the same range from "Unhappy" to "Happy." Placing these horizontal rows in parallel helps the user stick to the same scale and compare each factor to the others.

Progress Indicator

⊙⊙⊙ Activity, Feedback, Hourglass, Meter, Progress, Waiting

Figure 14-4. Progress Indicator

Goal Story

Reta has made a sale and is typing in the details. She completes the credit card details and proceeds down the page to delivery dates. Meanwhile, the status area has appeared beside the credit card details. While she completes the rest of the form, the status area continues to update with the current verification stage—initially "submitted," then "responding," then "verified."

Problem

How can you provide feedback while waiting for server responses?

Forces

Ajax Apps often make XMLHttpRequest Calls (Chapter 6) to query and update the server.

To provide good feedback, you should provide an idea of which page elements are affected by any update.

Users like to feel in control, and that control is taken away when the user is waiting for a response. It's not just the delay that should be minimized, but the perceived delay.

Solution

Indicate the progress of server calls. You can't always reduce delay, but you can include a Progress Indicator to ease the pain. A Progress Indicator helps maintain the user's attention, improves the user's understanding of how the system works, and also communicates that the system is still alive even if a response hasn't yet occurred.

The Progress Indicator is typically introduced to the DOM once an XMLHttpRequest Call begins and removed when the call has returned. The easiest way to detect whether the call has returned is using the XMLHttpRequest callback function. An indicator need not relate to a single call—it can show progress for a sequence of related calls.

Sometimes it's a Popup (Chapter 15) element instead of a new element directly on the page. A popular idiom is a small opaque Popup on the corner of the page showing just a word or two (i.e., "Saving . . . ", "Loading . . . ").

For shorter delays, typical Progress Indicators include:

A small message like "Updating document"

An animated icon (there's now a library of open source Progress Indicator icons available at http://mentalized.net/activity-indicators/)

For longer delays, the following can be used:

A meter showing how much progress has been made

An estimate of time remaining

A sequence of messages indicating what's happening at present

Content that's engaging but not directly related to the progress, such as "Tip of the Day" or a canned graphical depiction of system activity

Of course, you can combine these approaches. Generally speaking, some form of unobtrusive animation is worthwhile in any Progress Indicator, because it at least tells the user that something's happening, even if progress is temporarily stuck. In addition, longer delays should usually be completed with a visual effect such as One-Second Spotlight (Chapter 16), since the user's focus has probably moved elsewhere by that stage.

Note that one form of indicator to avoid is changing the cursor. Many traditional GUIs switch over to a "rotating hourglass" or related icon during delays. That's probably inappropriate for Ajax because it's something the actual browser software will do too—e.g., while loading a new page—so it's likely to create confusion.

Decisions

What sort of Progress Indicator will you use?

A well known set of guidelines is summarized in Jakob Nielsen's Usability Engineering (http://www.useit.com/papers/responsetime.html). Following is a quick summary:

If the delay is less than 0.1 second, the user will feel it's instantaneous. No feedback necessary.

If the delay is between 0.1 second and 1 second, the user will notice it but it won't break their flow of thought. No feedback necessary.

If the delay is between 1 and 10 seconds, the user's flow of thought is interrupted as he awaits the response. Basic feedback necessary; i.e., an indication that a delay is occurring. Ajax examples include animated icons and simple text messages.

Drilldown

⊙⊙ Drilldown, Menu, Progressive

Figure 14-6. Drilldown

Goal Story

Pam is booking a trip on the corporate travel planner. She sees a form with the usual fields and clicks on location. Suddenly, a list of cities fades in beside the form, and Pam selects Paris. Beside the city list, a third list appears, this one showing approved hotels. Pam chooses the Hilton, and both lists disappear. The location field now contains "Paris Hilton" as Pam had intended.

Problem

How can the user select an item in a hierarchical structure?

Forces

Applications and web sites are often arranged in hierarchies. To navigate, users need to choose a page from within the hierarchy.

Hierarchy navigation should be fast.

Solution

To let the user locate an item within a hierarchy, provide a dynamic Drilldown. The Drilldown allows for navigation through a hierarchy and ultimately for an item to be chosen. At each level, there are several types of elements in the Drilldown:

Current category

A read-only name for the present level.

Individual items

Items the user can choose in order to end the interaction.

Child categories

Deeper levels the user can drill down to.

Ancestor categories

Parent category and above that let the user "drill up."

In some hierarchies, items and categories are mutually exclusive: items only exist at the edges—or leaves—of the hierarchy. Even when that's the case, items and categories should be distinguished for the sake of clarity. The upward navigator goes by different names, but a general guideline is that it should tell the user which category that she is going back to.

Typical applications include navigating and filling out a field by drilling down a hierarchy of candidate items.

As the user drills down the hierarchy, will you show each progressive level? There are three main options (Figure 14-7):

Keep the Drilldown in a fixed area. Each time the user clicks on a category, the Drilldown morphs to show only that category. This has the benefit of preventing page rearrangement.

Show all levels at once. Each time the user clicks on a category, expand out the Drilldown region to add a new list for each category.

As a variant on the second approach, all levels are shown, but the submenus are rendered in front of the rest of the document, so the menu is effectively modal: nothing else can be done from the time the menu is opened to the time the selection is made. This approach is similar to choosing an application from the Windows Start menu.

Figure 14-7. Drilldown styles

The first two options are modeless; the third is modal. Modeless interaction works well with Ajax, where it's possible to dynamically update, without page refresh, the menu as well as any surrounding content related to the user's selection. Specifically:

Each navigation through the Drilldown can result in a server call to fill the next level with the latest content or even auto-generate new content.

The display throughout the application can change to reflect the state of the Drilldown, giving the user a "tour" through the hierarchy. As a user drills down from the top level to a deep category, the application is always synchronized with the current level, so the display reflects the medium-depth categories along the way. Because the interaction is modeless, the user is then free to stay on those categories.

The user can iterate between drilling down and performing other tasks. Imagine the user is a human resources clerk who needs to use a Drilldown to select an "employee of the month." She's already decided on the region and drills down to show all of the employees there. Now that she can see each candidate, she can go perform some research in another region of the page, or on another web site. When she returns, the Drilldown is still there, waiting to be completed.

Decisions

Will you call on the server each time the user navigates through the Drilldown?

Sometimes the entire hierarchy is loaded as a one-off event. Other times, categories can be pulled down as required using an XMLHttpRequest Call (Chapter 6). The choice is governed by two things:

How big is the hierarchy? The more items, and the more information per item, the less desirable it is to transfer and cache the data.

Data Grid

⊙ Database, Query, Report, Summary, Table

Figure 14-11. Data Grid

Goal Story

Reta is doing a little data mining on this season's sales figures. She's scrolling through a table showing a row for every transaction. Noticing that higher sales figures tend to come in the morning, she sorts the table by sales figure. There's too much data there, so she filters out any transactions below ,000, then groups the transactions by time of day.

Problem

How can you help users explore data?

Forces

Ajax Apps have their own databases, as well as access to external systems.

Users need a way to explore all this data, in order to verify it, understand how it works, predict future trends, and so on.

You can't anticipate what users will need to do with data.

Solution

Report on some data in a rich table and support common querying functions. A Data Grid has the familiar database client interface: a table with a row for each result and a column for each property. Think of it as a traditional table on steroids. Typical database client functionality is as follows (and it's feasible to achieve each of these in an Ajax interface):

Sorting

Each column can be usually be sorted by clicking on the header. To let the user sort by more than one category, use a sorting algorithm that preserves the order of equal values. Then, users can click on one header followed by the other.

Filtering

The user can filter in to retain data matching certain criteria, or filter out to exclude that data.

Grouping

Data can be grouped by similarity. A large table is essentially broken into smaller tables, in which each item in a smaller table is similar to the other items in that table.

Aggregate calculations

Calculations can be performed across the whole table—for example, a sum or average for a column, or a sum of the products of two columns in each row.

Editing

Some Data Grids allow fields to be edited.

With the magic of Web Remoting (Chapter 6), the grid can become a Virtual Workspace (Chapter 15, giving the illusion of holding a massive set of data. In reality, the data lives on the server, and each User Action leads to a new server call. Most queries lead to a structured response, such as an XML Message or a JSON Message, containing a list of results for JavaScript rendering, or alternatively an HTML Message with the specific view the user will see. The semantic style has the advantage of encouraging performance optimizations such as Predictive Fetch,

Rich Text Editor

⊙⊙⊙ Editor, Formatting, Fonts, Rich, Text, Textarea, Toolbar, Write, WordProcessor, WYSIWYG

Figure 14-14. Rich Text Editor

Goal Story

Pam is working on a presentation for the steering committee; style will count here. From the toolbar, she sets the font to Arial, the size to 24 pt, and switches italics on. She then types out the heading, selects it, and moves to the toolbar again to set the color scheme.

Problem

How can users create and maintain rich content?

Forces

Many Ajax Apps let users create and edit substantial chunks of content.

Rich content for the Web needs to make the most of HTML and go well beyond a string of plain-text.

Most users don't know HTML; even "easy" markup substitutes for HTML are complicated and inconsistent.

Solution

Incorporate a Rich Text Editor widget with easy formatting and WYSIWYG[*] display. Typically, the widget looks like a mini word processor: a toolbar on top with a rich editing area underneath. The editing area is usually a div rather than a textarea, meaning that any HTML content is possible.

Typical features include:

Flexible font styles, sizes, boldfacing, etc.

Flexible color schemes

Embedded images

Embedded tables

Bullet-point and numeric lists

Indenting and flexible text alignment

All of these features are usually accessible by the toolbar, as well as via keyboard shortcuts. It would also be possible to make them available from drop-down menus, though the main examples to date have avoided doing so. In addition, the toolbar sometimes offers other operations too:

Undo, Redo

Cut, Paste, Copy

Save (Explicit Submission [Chapter 10])

Spellcheck

Rich Text Editors are a great tool for nontechnical users, but as with GUI word processors, they can slow down power users. If power users are important to you, a few guidelines apply:

Offer an alternative "WYSIWYN" (What You See Is What You Need) interface, where the user can enter raw HTML and/or some other text-based markup such as Markdown (http://daringfireball.net/projects/markdown/).

Offer keyboard shortcuts and advertise them well, e.g., as tooltip Popups (Chapter 15) on the corresponding toolbar icons.

Offer personalized toolbars and keyboard bindings.

The major browsers do have some support for rich text editing. Firefox has Midas (http://kb.mozillazine.org/Firefox_:_Midas), an embedded text editor, and IE (http://msdn.microsoft.com/workshop/author/dhtml/reference/properties/contenteditable.asp) has a similar editor available on items flagged as contentEditable. However, neither mechanism is portable, and with both versions, you're stuck with whatever version the user's browser has. For that reason, the best solution right now is probably the Cross-Browser Component

Suggestion

⊙⊙ Auto-Complete, Fill, Intelligent, Populate, Predict, Suggest, Wizard

Figure 14-18. Suggestion

Goal Story

Doc is typing out an electronic prescription. He intends to prescribe "Vancomycin 65 mg," but as soon as he types in "V", the system has already detected a handful of likely choices, based on the patient's history and Doc's prescription style. Doc selects "Vancomycin 65 mg" from the list and proceeds to the next field.

Problem

How can you improve throughput?

Forces

Free text remains the most powerful way for humans to communicate with computers. The trend seems to be toward more typing than ever before due to instant messaging, blogging, and email. Even search engines are undergoing a transformation to become general-purpose, as described in Live Command-Line later in this chapter.

When presented with a free text area, people don't always know what they're meant to type in.

Though many users are now quick on the keyboard, there are still many users for whom the Web is a click-mostly experience.

Typing speed remains a bottleneck even for fast typists, most of whom think faster than they can type.

People make mistakes when they type.

Solution

Suggest words or phrases that are likely to complete what the user is typing. The user can then select the Suggestion and avoid typing it in full. The results usually come from an XMLHttpRequest Call (Chapter 6)—the partial input is uploaded and the server responds with a collection of likely matches.

Suggestion has its roots in "combo boxes"—fields on traditional GUI forms that combine a text input with a drop-down list. The elements are kept synchronized. The user is free to type any text, and the current selection in the list will track what's been typed so far. The user is also free to choose an element from the list, which will then be posted back to the text field. In some cases, the list constrains what the user types; in other cases, it's there to provide Suggestions only.

It's the latter style that has become popular in recent years—free text entry, with some Suggestions for completion at any time. Auto-completion became popular with Internet Explorer 5, which auto-completed fields based on user history. Most users are comfortable with the approach, as all the modern browsers offer a similar feature, and the technique is also popular in mobile phone text messaging and East Asian text entry.

In an Ajaxian context, Google set the standard with its introduction of Google Suggest (http://www.google.com/webhp?complete=1), which suggests the most popular terms to complete the user's search query. Its release surprised many, as Google had managed to completely replicate conventional combo-box behavior, but this time, the terms were dynamically fetched from the server instead of being present when the form was created.

The mechanics of a Suggestion usually work like this:

A standard input field is used. At the same time, an initially invisible div element is created to contain the Suggestions as they appear. The input field needs an event handler to monitor the text it contains so as to ensure that the list always highlights whichever Suggestion matches.

Instead of requesting a Suggestion upon each keypress, Submission Throttling is usually adopted. Thus, every 100 milliseconds or so, the browser checks whether anything has changed since the last request. If so, the server is passed the partial query as a GET-based XMLHttpRequest Call.

The server then uses some algorithm to produce an ordered list of Suggestions.

Back in the browser, the callback function picks up the Suggestions and does some Display Morphing (Chapter 5) to show them to the user in a format that allows them to be selected. Each entry will have an event handler, so that if the entry is clicked, the input field will be altered.

A combo-box is not the only way to render results, but it has the dual virtues of efficiency and familiarity. "Code Example: Kayak," later in this section, covers Kayak's implementation.

You might think the main benefit of a Suggestion is to cut down on typing, but that's not the case. If anything, it's often slower to enter a single word using Suggestion, as it's a distraction and probably requires some mousing. The main benefit is to offer a constrained set of choices—when there are more choices than would fit in a standard option list, but still a fixed set. For Google Suggest, it's a list of choices that are considered probable based on search history. In tagging web sites, it's a list of tag names that have been used in the past. In travel web sites, it's a list of airports you can include on your ticket.

Decisions

Live Search

⊙⊙ Feedback, Immediate, Live, Real-Time, Search

Figure 14-21. Live Search

Goal Story

Browsing a trade magazine, Reta has just spotted the "Curiously Costly" line of shoes and declared them a "must-have" for the upcoming season. She heads over to her favorite wholesaler's web site and sees an empty search form with an assortment of products underneath it and the message, "50,000+ items." First, she selects "shoes" from a drop-down and watches as the products below morph into a collection of shoes; the message is now "8,000+ shoes." Then, she begins typing. As she types "C," she now sees all 500 shoes beginning with "C." This continues until the search field contains "Curio," at which point only three items remain. One of those is "Curiously Costly," which is what Reta clicks on to successfully conclude the search.

Problem

How can users quickly search for an item?

Forces

Ajax Apps often require a search capability. Users search for business objects, other users, web sites, and more.

Sometimes the user is searching for a needle in a very large haystack. In the case of search engines, a haystack of over a billion items.

To help deal with the complexity, users should be free to experiment.

Solution

As the user constructs and refines his search query, continuously show all valid results. The search query is a combination of controls that lets the user narrow down the collection of items. It may be a straightforward text field, or a complex arrangement of Sliders (see earlier in this chapter), radiobuttons, and other controls. The results appear in a separate region and are continuously synchronized with the query that's been specified. The search concludes when a result is chosen.

There are several benefits to using Live Search instead of the conventional style:

Browsing and searching can be combined.

Searching proceeds more quickly because no explicit submission is required. The results are ready almost immediately, so the user can experiment more easily.

There's no page reload, so the interaction feels smoother; results are updated, but the input query is completely unaffected.

The most common form of Live Search right now closely mirrors traditional searching. The user types into a free-text search field, and results are frequently updated as the user types. So, searching for "cataclysmic," the user will see results for "c," "ca," and so on. (With Submission Throttling (Chapter 10) in place, the sequence might skip a bit, so a fast typist would see results "c," "cata," etc.)

Search and browse have often been considered mutually exclusive on the Web, which is unfortunate because both have their benefits. Search is good for finding something you already know about, while browse is good for exploring all of the items and stumbling across things you didn't know about. Live Search provides a way to combine the two. Some controls—such as selectors, radiobuttons, and Sliders—let the user choose an item from a small collection. If the search form can change dynamically (see Live Form later in this chapter), you can support browsing through a hierarchical structure.

Imagine how a web site search engine might achieve this. The initial control is just an empty text box and a selector with ten categories—Arts, Tech, News, and so on (Figure 14-22). The user chooses News on the selector and a new selector appears—World, Business, Science, etc. At any point, the user can change any selector. All the while, the results below are updating to show a sampling of results in the most specific category that's been specified. Furthermore, the text input is there all along, acting as an additional filter. So if you search at the top level, the interface degenerates into a standard live text search. But if you search while the selectors are set to a category, only the matching results from that category will be shown.

Figure 14-22. Live Search with Live Form

Live Search makes use of event handlers on the input controls, which listen for user activity and update the results. A typical sequence works like this:

The user changes a control; e.g., types a new character or clicks a radio button.

The entire input query—all of the input fields' settings—is transmitted to the server in an XMLHttpRequest Call.

The server decides which—if any—results should be shown and returns the list.

The browser replaces the list of results with this new list.

In the preceding scenario, the XMLHttpRequest Call need not occur in direct response to the user input. Often, Submission Throttling is used to delay the response.

Live Command-Line

○ Assistive, CLI, Command-Line, Grammar, Supportive

Figure 14-25. Live Command-Line

Goal Story

Frank is using an advanced query tool to run a report on current output. He begins entering an SQL-like query: "list all." At this point, a suggestion list appears with all the things that can be listed, and he chooses "products." Continuing on, the query now reads "list all products where" and a new list appears: "creationTime," "quality," and so on. He chooses "creationTime," and. then enters the rest manually, preferring to ignore further suggestions. The query ends up as "list all products where creationTime is today and quality is faulty." There's a tick mark next to the query to indicate that it's valid.

Problem

How can a command-line interface be supported?

Forces

Users can only say so much with point-and-click interfaces. The command line is more expressive and remains the tool of choice for expert users.

URLs and search fields are becoming general-purpose command-lines (http://www.codinghorror.com/blog/archives/000296.html).

Solution

In command-line interfaces, monitor the command being composed and dynamically modify the interface to support the interaction. The interface changes according to data, business logic, and application logic. When the command line is to be executed on the server, as is often the case, the browser usually delegates to the server the task of evaluating the partial command.

The command line is the quintessential expert interface and it's worth bearing in mind that most users are "perpetual intermediates" (http://www.codinghorror.com/blog/archives/000098.html). That is, they learn enough to get by and generally learn new things only as and when required. So, while a powerful command line may be a superior choice in the long run, most users simply don't have the time to spend learning it in advance.

Traditional command lines present a problem because they offer the user a blank slate. Everything the user types must be based on prior, upfront, learning, and the only feedback occurs after the command line has been executed. That's not only inefficient for learning, but downright dangerous for sufficiently powerful commands. The model is: human enters command, computer executes command.

The solution is to be more interactive: with a smarter, more proactive command line, the construction of a command—its quick evolution from nothingness to an executable string—becomes something of a human/computer collaboration. For example, you can:

Indicate whether the command is valid before submitting it. If it's not valid, prevent it from executing and inform the user what's wrong and how it can be rectified.

Offer to fix invalid commands.

Offer Suggestions to help complete part of the command.

Help the user predict what will happen when the command is executed. For a query, this means hinting at the result—e.g., estimating number of matches (see Guesstimate [Chapter 13]).

Provide graphical widgets to help the user construct the command. For example, build up a mathematical expression using a calculator-like interface to introduce numbers and functions. The query is still represented as plain-text, but the text doesn't need to be entered by hand.

Provide graphical widgets to help the user refactor the command. For example, you could provide a "delete this" button next to each condition in a query.

Provide visualization tools to render the query in other formats. For example, certain numerical queries might be visualized graphically.

The benefits apply to novices and experts alike. In supporting novices, the technique has sometimes been referred to as a "training wheels" approach. Like training wheels on a bike, you get a feel for the more powerful interface while still being productive.

Often, the command goes to the server, so it usually makes sense for the server to advise the use regarding partial commands. Basically, the procedure goes like this:

Submission Throttling (Chapter 10) periodically uploads the partial command string as an XMLHttpRequest Call.

The server then processes the command and returns supporting information.

The browser updates the user interface accordingly.

As an Ajax feature, this pattern is largely speculative: I'm not aware of any production-level Ajax Apps, but the command-line interface is growing at the same time as Ajax; it's inevitable that these things will collide, and as the discussion here suggests, there are plenty of synergies. One source of inspiration here is the evolution of IDEs over the past five years, particularly in the Java space. Environments like Eclipse (http://eclipse.org) and IntelliJ Idea (http://www.jetbrains.com/idea/

Live Form

⊙⊙ Dynamic, Live, Real-Time, Validation

Figure 14-27. Live Form

Goal Story

Tracy is using a Live Form to apply for a new online brokerage account. Upon declaring her place of residence, some additional regulation-related questions specific to her region appear. Several asset classes can be traded. She clicks a box to indicate that she wants to trade in bonds, and further questions specifically for that asset class appear. She then clicks on another box, for options. A moment later, a warning appears that new users can trade in either options or bonds, but not both. After further refinement, she successfully submits the form and sees it fade into a welcome message.

Problem

How can the user submit data as quickly as possible?

Forces

Most data submission tasks require some flexibility. A particular answer for one item may necessitate further questions. Often, this must occur server side, where business logic and data usually reside.

Most data submission requires validation. This must occur server side to prevent browser-side manipulation of inputs.

Users get frustrated waiting for data to be validated and continuously refining data upon each explicit submission.

Solution

Validate and modify a form throughout the entire interaction instead of waiting for an explicit submission. Each significant user event results in some browser-side processing, often leading to an XMLHttpRequest Call. The form may then be modified as a result.

Modifying the form is similar to responding to a Microlink (Chapter 15)—there's usually a server call followed by some Page Rearrangement (Chapter 5). Typical modifications include the following:

New controls appear.

Controls become disabled.

Information and error messages appear.

The result is a more interactive form. The user doesn't have to wait for a page reload to find out if a form field is invalid, because it's validated as soon as she enters the value. Due to the asynchronous nature of XMLHttpRequest Call, the validation occurs as a background process while the user continues entering data. If there's an error, the user will find out about it while filling out a field further down on the form. As long as errors don't happen too often, this is no great inconvenience.

Validation doesn't always occur at field level though. Sometimes it's the combination of data that's a problem. This can be handled by checking validation rules only when the user has provided input for all related fields. A validation error can result in a general, form-level error message or a message next to one or more of the offending fields.

A Live Form usually ends with an Explicit Submission (Chapter 10), allowing the user to confirm that everything's valid. Note that unlike a conventional form, the Explicit Submission does not perform a standard HTTP form submission but posts the contents as an XMLHttpRequest Call.

Real-World Examples

WPLicense

Chapter 15. Page Architecture

THE "PAGE ARCHITECTURE" PATTERNS ARE A VARIETY OF INTERACTION STYLES AND STRATEGIES FOR structuring content. In many cases, it is the possibility of web remoting that makes these patterns worth using.

Drag-And-Drop has been a fixture on the desktop for a couple of decades but is only now gaining currency in the browser. One situation in which it's used is with the Sprite pattern. A Sprite is a little icon that lives "in front of" the main document and can move around freely like a character in a video game. The next pattern, Popup, uses the same overlay technique, but for larger, dialog-style content.

Malleable Content is content that appears to be read-only but becomes editable upon a gesture such as a mouse rollover, a fairly new idiom because it relies on web remoting. Similarly, a Microlink is a link that uses remoting to conjure up content for insertion in the page.

A Portlet is a small component of the page that acts like a mini-application, capable of conducting a conversation with the server independent of the main page flow. A Status Area provides information about current and past activity and is relevant to Page Architecture because it helps reduce the space occupied by each individual element. Update Control allows the user to control information flow on the page. The final pattern, Virtual Workspace, lets the user explore a large server-side structure, yet it makes the remoting transparent, creating the illusion that everything's happening in the browser.

Drag-And-Drop

⊙⊙ Drag, Move, Pull, Rearrange

Figure 15-1. Drag-And-Drop

Goal Story

Stuart is purchasing books online. Each time he wants to buy one, he drags the book icon into a shopping cart.

Problem

How can users rearrange objects on the page?

Forces

Object relationships are important and the user's task often involves changing those relationships.

Ajax Apps often represent object relationships visually.

Rearranging the visual structure from a separate form is messy—i.e., trying to map between the form and the visual structure is confusing and error-prone. It's easiest if users can directly manipulate the visual structure.

Solution

Provide a Drag-And-Drop mechanism to let users directly rearrange elements on the page. Drag-And-Drop has proven itself to be a powerful control mechanism in conventional desktop applications; it is certainly achievable using standard web technologies. The basics are straightforward: the user holds down the mouse button while the mouse hovers over a page element, then moves the mouse with the button still depressed. The element follows the mouse around until the user finally releases it.

Sprite

⊙⊙ Bitmap, Buffer, Character, Layer, Overlay, Shape, Sprite, Z-Index

Figure 15-2. Sprite

Goal Story

Not for the first time, Pam is negotiating to eliminate a stack of requirements from the project goals. Having successfully convinced the client that "Security Features" was never really worthwhile, she drags a "Magic Eraser" Sprite from the Tool Palette to the "Security Features" requirement, jiggles the mouse a couple of times, and the eraser is animated for a second as the requirement fades away.

Problem

How can you ensure that visual content is flexible?

Forces

Rich visual displays are often the most effective way to present information.

Information frequently changes as the user interacts with and new data is received from the server; hence, the visual display needs to update frequently.

It's expensive to continuously download new visual content.

Solution

Augment the display with Sprites—i.e., small, flexible, icon-like blocks of content. Typically, the Sprites are images, but they may also be div elements containing text or images or a combination of both. The name refers to the Sprites used in traditional graphics programming, particularly gamin—those little graphics representing players, enemies, and other objects capable of moving around a scene and animating themselves to reflect current activity. The entire scene does not need to be recomputed each time, only the Sprites. DOM elements are not (yet) capable of more advanced features such as arbitrary rotation, but many aspects of Sprites can actually be translated to a web context.

Google Maps is a case in point. If you search for "museum new york," a set of thumbtack Sprites will appear on the map, one for each museum. Then, if you click on a thumbtack, a new speech balloon Sprite will show up with the precise address. Using Sprites, Google spares itself the burden of preparing images for every possible configuration, and from generating images on the fly. All it needs to do is ensure that there's a basic display of the map in question, then overlay the thumbtacks and speech balloons.

Sprites are often implemented as div elements or simply as images. They usually appear "in front of" the rest of the document and are sometimes partially transparent. Following are the relevant CSS properties (also see Page Rearrangement [Chapter 5]):

zIndex

Determines how elements are stacked. Sprites usually have a high value in order to appear in front of other elements.

left, right, top, and bottom

Often used to position a Sprite. position can also be set to influence how the browser interprets these settings.

opacity

Often used to make the Sprite partially transparent. (IE requires the alpha filter workaround; see http://www.domedia.org/oveklykken/css-transparency.php.)

backgroundImage

Used to select a background image when the Sprite is a div. As long as there is no foreground content, the background image is shown. The image doesn't need to be rectangular because a transparent GIF file can be used to show any arbitrary shape.

You can also animate the Sprite by rapidly changing its appearance. Unfortunately, JavaScript cannot easily control animated GIFs, so that's not a viable option. However, animation is easy enough with some Scheduling (Chapter 7). One simple technique is to set up a recurring changeImage( ) action, in which the element's backgroundImage is continuously run through a cycle of different images. Ideally, you should preload the images to avoid a slow cycle the first time round. Several tricks are available for preloading, the most common being set an invisible image's source to point to the new image. An alternative animation technique is outlined in the section "Code Example: DHTML Lemmings," later in this chapter.

Decisions

What file format will the Sprite image be?

Popup

⊙⊙ Balloon, Dialog, Hover, Popup, Transparent, Tooltip

Figure 15-5. Popup

Goal Story

Frank is monitoring the factory equipment online and notices the T1000 machine is acting a little funny. He clicks on the icon, holds the mouse button down for a second, and up pops an information sheet about the T1000 model. The sheet is partially transparent and overlayed on top of a region of the plant diagram.

Problem

How can the user perform a quick task without distracting from the main flow?

Forces

Users often need to make a quick change, which requires that they have some extra controls with which to interact.

Users often need to perform a quick lookup, get further details about an object in the application, or just find some general information. This requires that extra information appear.

Screen real estate is limited, and minor, quick changes don't warrant a major change to the user interface.

Solution

Support quick tasks and lookups with transient Popups, blocks of content that appear "in front of" the standard content. Usually, the content is partially transparent, is about the size of a standard dialog, and hovers above the page until the user explicitly closes it. A more subtle variant is the tooltip, a solid block usually containing content between one word and a paragraph in length. The tooltip itself has a trendy new variant, a cartoonish balloon. For Progress Indicators (Chapter 14), a popular idiom is to use a small label in the corner with a message like "Saving . . . ." Figure 15-6 illustrates several popular styles.

Figure 15-6. Popup styles

Popup is suitable for particular situations:

You expect the task to be quick

The Popup is limited in space and its transparent appearance makes it difficult to work with for a long time.

You expect the user to perform the task immediately

The rest of the document is partially blocked, making it difficult to work with while the Popup is around.

Often, a Popup has no permanent state; it becomes visible, and then is destroyed upon close. With many Popups, there can be only one of its kind at any given time.

Popups are usually divs. To ensure that only one Popup of a particular kind can appear, you can ensure that Popups are always created at the time they are required and deleted on close. On opening, you just need to check whether the Popup already exists on the page. Alternatively, you can simply create the Popup on startup and toggle between showing and hiding it—show it when the user needs a Popup and hide it afterwards. The only catch here is that the state won't be cleared, and often it's more effective to treat each Popup as a new entity.

The following CSS styles are often useful for creating a Popup.

opacity (alpha filter in IE)

Used to produce a transparent feel.

zIndex

Used to place a Popup "in front of" the rest of the document. A solid (opacity=1) element will completely occlude everything that its zIndex is higher than.

visibility

Used to toggle whether the Popup is being shown.

Malleable Content

⊙⊙⊙ Affordance, Collaborate, Edit, Hint, Input, Microcontent, Mutable, Outline, P2P, Selection

Figure 15-9. Malleable Content

Goal Story

Reta is logged in and is revamping the catalogue web site. It looks clean, a replica of the read-only page that public users will see. But as she runs the mouse down the screen, she sees that many regions can be edited.

Problem

How can you let users change content?

Forces

The Web is not just about consuming information. Administrators, and increasingly everyday users, spend a lot of time adding and editing content.

All users, no matter what their roles are, need to know what they're allowed to change.

To encourage contributions and improve user experience, editing should be simple and free of barriers.

Editable content pages are often ugly, with an overload of form controls, labels, and buttons. Worse, they create a divide between the contributors and the readers; while editing content, it's often difficult to imagine how a reader will see it.

Solution

Compose the page of Malleable Content blocks—small chunks of content that can be edited in page. The page is structured in chunks of small "microcontent," each editable in isolation. There might be a way to edit the whole page too, but the usual style of editing is to view the read-only display and carve up little pieces as you see fit.

The reload problem is one reason why conventional web sites must present large forms in which everything can be set at once. It's often the only practical way to edit content. However, users have difficulty relating the editing interface to the readable interface. Worse, it discourages editing; users are subjected to a sequence of at least two-page reloads in order to change a single character.

Ajax technologies lead to a smoother editing process. Display Manipulation (Chapter 5) and Dynamic Behavior (Chapter 7) help the user locate and open up editable Malleable Content. Display Manipulation also shows you how to conjure up a new editable area, or morph read-only Malleable Content into an editable form. Once the user has made the edit, you can seamlessly upload the response, as described in Web Remoting (Chapter 6), at which point the content can revert to read-only again.

In most cases, each block of Malleable Content is a div or textarea element. It can become editable in one of several ways:

The display of the original element is adjusted and augmented with editing controls.

A new div or textarea is created in its place.

The original element becomes hidden and a hidden div or textarea is revealed.

Is that the sound of your inner usability critic banging his fists on the table and exclaiming "this defies everything that users have come to know about the Web"? Fair point. The forms are gone, so where's the affordance to let the user know what can be edited? Affordances can indeed be provided, as discussed in the following section.

Even if sufficient affordance can be provided, you might object that the whole thing is a complete departure from the usual forms-based approach. You've got me there. It certainly is a departure, but it's a departure from a ten-year-old approach that's remained frustrating since day one. Consistency is certainly an admirable goal—one that shouldn't be neglected because you have a few "one-percenter" improvements—but Ajax represents a new approach, and the benefits of in-page editing are just too big to ignore in the name of consistency. Better to adopt the approach and use sensible design to guide users through it. In any event, Malleable Content is likely to take off and will become familiar to users in the not-too-distant future.

Decisions

How will users locate Malleable Content and open it up for editing?

A read-only page will contain several editable Malleable Content blocks and various other regions that cannot be edited. Some of those regions are program-generated; others might be Malleable Content that this user is not permissioned to edit. So the user needs to know what can be edited. The static appearance should provide some cues, and events like mouse rollovers can refine that information.

The static appearance should make the location of the Malleable Content apparent. There are a few techniques to consider, some which are based on the Highlight pattern:

When there are multiple Malleable Content blocks, increase spacing or alternate fonts between them to keep them distinct.

Present the Malleable Content with a distinct background.

Apply an explicit border around the Malleable Content.

Expose the edit controls (e.g., Undo button), or the controls which initialize the edit (e.g., Edit button), but faded out.

Add a label and auxiliary information beside each Malleable Content block. (See the following.)

These provide enough information for the user to detect that Malleable Content is present. However, too many of these cues will defeat the purpose of this pattern by making the whole thing hard to read. So you might use only a few subtle cues, or even none at all, and rely instead on dynamic behavior. Mouse movement is the most important activity to monitor; fields can be morphed to advertise their editability as the mouse rolls over them. Here, onmouseover

Microlink

⊙⊙⊙ Appear, Conjure, Emerge, Hyperlink, Insert, Introduce, In-Page

Figure 15-13. Microlink

Goal Story

Frustrated with a developer's productivity, Pam logs into the version control system and sees a "Project Committers" table. Each row includes a developer's name and photo, and Pam immediately clicks on one of the names. The row expands out to reveal a mini-summary of the developer's project activity, and a bunch of links to further content.

Problem

How can the user quickly navigate through content?

Forces

Ajax Apps often respond to user requests by providing new content.

To encourage interaction with the system, content should be shown quickly and without distraction.

Page reloads slow down interaction, break concentration, and obscure the nature of whatever changes have occurred.

Solution

Provide Microlinks that open up new content on the existing page rather than loading a new page.Microlink is an Ajaxian upgrade of the traditional hyperlink. Most often, it entails fetching content from the server with an XMLHttpRequest Call and inserting it onto the page. The content is usually a tightly scoped block of Microcontent, though it need not be—a Microlink could be used to open a whole page of content, for example.

The term Microlink should not be taken here as a literal translation of textual hyperlinks. It's used to represent any form of content being inserted into the page. The trigger might be something as subtle as a form field focus event.

Microlinks can be used in many contexts:

They can open a Popup (earlier in this chapter) to augment a particular word or phrase. Some conventional web sites provide glossary lookups in a pop-up window. A Microlink could instead be used to place a small explanation directly next to the term.

They can be used to drill down by expanding out successively deeper levels of content. You could show all teams in a football league, each expanding to a list of players, in turn expanding to a list of games, and so on.

They can let the user switch content around on the page, which is useful if the Microlinks are included in a Drilldown (Chapter 14).

The name "Microlink" refers to the subtle effect it has on interaction flow. Microlink carries several benefits over the full-blown page refresh caused by a standard hyperlink:

Content appears faster

There is much less to transfer because only the new content is loaded rather than the whole page. Also, the browser needs only to make a few changes to the DOM as opposed to rerendering the whole page.

Application flow is smoother

To the user, a page reload is distracting. Microlink avoids this distraction.

Changes are more salient

The user is able to watch as an element appears on the page. Visual effects can be used to further improve change detection.

Internal state is maintained

The browser-side application's state—as held in the DOM—is retained, whereas a page reload destroys it. This is a technical benefit.

Microlink usually involves an XMLHttpRequest Call (Chapter 6) followed by a Page Rearrangement (Chapter 5). Often, a new div

Portlet

⊙⊙ Block, Conversation, Dialogue, Gadget, Independent, Parallel, Portal, Portlet, Widget

Figure 15-15. Portlet

Goal Story

The sidebar of Tracy's stock trading system contains a few independent blocks down the side: a currency Portlet, a calculator, and a miniature chart generator. Right now, she is checking on the USD/EUR exchange rate in one of those blocks. She types "USDEUR" into the text field, and the block updates a moment later to show the rate. Everything else on the page remains the same.

Problem

How can the user conduct rich conversations in parallel?

Forces

A complex application contains a range of loosely related functionality. The user often needs to work on these things in parallel, gathering knowledge about one thing and using it to influence another.

It isn't just different functionality, but different data that users need to work with in parallel.

These conversations need to be "rich." They require conversational state to keep track of what's occurred so far and what can happen next, and they require interaction with the server.

It's important to retain the state of each conversation between the user and the application. You don't want to invalidate one activity just because the user chose to initiate another.

Solution

Introduce Portlets—isolated blocks of content with independent conversational state. A Portlet is like a mini-application inside a regular application. It presents a little bit of information and usually some controls. The user can communicate via the controls, which causes the Portlet to update.

Usually, the Portlet state is completely independent of the rest of the application. For example, a travel web site can include a weather Portlet. Users are able to search for locations and drill down from a result to an extended forecast. The conversation can be as rich and stateful as the display allows. However, the conversation has no effect on the rest of the application. If the main site is in the middle of a booking transaction, it will stay that way while users spend a few minutes checking the weather. Browser/server communication will take place, but it won't affect the conversational state of the booking transaction, nor will it affect the state of any other Portlets.

In some cases though, actions on the Portlet can affect the general application state, and vice versa. A search Portlet can allow a user to locate content in the application. While locating the content, the user conducts a rich conversation alone with the Portlet, but once located, the user can click on the search result to open it up in the main content area.

If you aggregate Portlets together on the same page, you will end up with a Portals. A news Portal for example, contains a Portlet for each news category. But Portlets need not reside in Portals. A single Portlet can also be embedded in a regular web app (Figure 15-16), as the preceding example illustrates.

Figure 15-16. Portals and embedded Portlets

Portlets have been around for a long time, and Portals were once touted as the next big thing after search engines. However, they've never really taken off, and the interface is part of the problem. Chapter 1 explains in detail the problems with traditional portals and how much they gain from Ajax. The various Ajax Patterns help in different ways:

XMLHttpRequest Calls

Status Area

⊙ Console, Log, Message, Status

Figure 15-18. Status Area

Goal Story

Pam wants to revise the list of project issues. As she begins checking off those that are now resolved, she notices the Status Area below update, which is showing the number of issues resolved and the number still outstanding.

Problem

How can you show auxiliary information?

Forces

The same data can be represented in different ways.

Users often benefit from redundant summary information.

Screen real estate is limited; you can't augment each field with its own summary information.

Solution

Include a read-only Status Area to report on current and past activity. The Status Area is usually auto-generated text based on some aspect of system state. The main purpose is to save space by occupying a region with information from different sources. This is done by dynamically altering the information according to current context.

Applications of the Status Area include the following:

Summarizing information about elements the mouse is hovering over.

Summarizing information about the element being edited.

Summarizing information about the overall application state.

Capturing past events in a log.

Offering a preview.

Often, there's no server-side processing involved—the browser has enough information to maintain the Status content itself. For example, the browser can easily show a count of selected elements or a log of past data that's been retained. A Status Area can be particularly valuable for monitoring the state of dynamic objects. For example, an e-commerce system can use Periodic Refresh (Chapter 10) to continuously update the state of an order within a user's profile ("Submitted," "Credit Card Verified," "Stock Available," and so on).

The Status Area is usually a div element, with changes triggered by events such as mouse rollovers and form editing.

Note that this pattern is mostly speculative and is based on analogies from conventional desktop systems (where Status Areas are indeed commonplace).

Update Control

⊙ Backwards, FastForward, Forwards, Freeze, Pause, Push, Refresh, Rewind, Reverse, Speed, Sticky, Update

Figure 15-20. Update Control

Goal Story

Sasha is entranced by a world map that continuously updates with events from around the world. She plays around with the velocity controls, first speeding it up so there's a new event every second (no matter how trivial), then backtracking through previous events, and finally pausing on something that catches her eye.

Problem

How can the user deal with continuous information entering the browser and updating the page?

Forces

Many Ajax Apps continuously grab data from the server and place it on the page.

The web page has a limited area, so you'll need to remove or relegate older content.

The optimum speed of updating depends on the user and his task. It should be fairly quick if the user is actively monitoring, for example, and probably quite slow if the content is a bit of eye candy at the edge of a page.

Solution

Let the user control the rate and criteria of updates. When using patterns such as HTTP Streaming (Chapter 6) and Periodic Refresh (Chapter 10), it's possible to keep grabbing fresh content from the server, so you can show news updates, system events, and more. The trend is increasing, with photo slideshows, news updates, and so on, and it can easily lead to information overload. This pattern is about giving the user control over the incoming stream of information. It takes several forms.

First, the user can control the rate of change, by pausing, rewinding, and fast-forwarding. Pausing is important for several reasons: it gives the user an ability to reflect on the content, it lets him keep the content open while perform work related to it, and it lets him interact with the content in the case that it contains links or other forms of control. This is a serious issue in interfaces like Digg Spy (http://digg.com/spy) that stream new links every second or so. Go to click a link and—BAM!—it's already been replaced by a new link by the time your mouse pointer gets there.

Rewinding is useful too, because it lets users see things they didn't catch the first time, or lets them revisit something in the light of new information.

Virtual Workspace

⊙⊙ Camera, Desktop, Illusion, Infinite, InfiniteScrollbar, Lens, Move, Pan, Portal, Scroll, Solipsism, Viewport, Virtual, Visible, Window, Zoom

Figure 15-23. Virtual Workspace

Goal Story

Bill is using a meta-search engine to receive insurance quotes. The browser can't retrieve all 500 results at once, but Bill sees a table containing what looks like all the results. He can scroll up and down as if it were a regular table, and the newly appearing rows are populated on demand.

Problem

How can the user navigate through a large workspace?

Forces

Ajax Apps often require data from the server.

On the server side, data storage is virtually unlimited—search engines, for example, store billions of web pages. Even on corporate intranets, query results are often huge; try searching for every retail transaction on a given day.

The application should respond to data requests as quickly as possible.

Bandwidth constraints mean it's not feasible for an interactive application to download the entire search result. Yet users should be free to explore the whole thing.

Solution

Provide a browser-side view into a server-side workspace, allowing users to navigate the entire workspace as if it were held locally. The illusion is that the entire workspace is already in the browser, but the reality is that the server actually provides content on demand. At any time, the user is looking at an "opening" or "portal" into the entire workspace. He can pan across, jump to a different region, and zoom in and out. Each of these actions requires a view change, so the browser transparently fetches the data for the new portion of the workspace and renders it accordingly.

Here are some examples of navigable workspaces:

Lists

These include search results and user preferences.

Tables

These include Data Grids and spreadsheets.

Physical images

These include diagrams, photos and maps.

Documents

These include web content.

3D Models

These include computer-rendered landscapes and visualizations of virtual models such as chemical molecules.

Time-dependent artifacts

These include any artifact that changes with time—world population, for instance—and can be "navigated" by moving across time.

Users move through the space in different ways. Often they use a combination of the tools described next.

Scrollbars

Users can make the content scroll incrementally or jump to a completely different region.

Dragging tool

Gives the user the impression of dragging the document around while keeping the view fixed.

Keyboard shortcuts

Users can scroll incrementally using directional keys. Page Up and Page Down are often used to completely replenish the current view with an adjacent region. Home and End are often used to jump to the start and end.

Direct command

Users can use a separate control to specify an exact region—for example, typing in the dimensions they want to view. Or, for an image, placing a bounding box on a thumbnail view of the entire document.

There are also different zooming techniques:

Zoom slider

This is an adjustable Slider that represents the current zoom level.

Keyboard shortcuts

A common choice for zooming in and out is - and + (or =, since + usually requires pressing Shift-=).

Selecting a region

This is used to zoom further into a region within the current view.

Depending on the mechanisms you're supporting, you'll need to add different types of event handlers. So, for keyboard shortcuts, watch for events like keydown; for dragging and selecting a region, watch for mousedown, mousemove, and mouseup. In some cases, you'll need a control separate from the view itself, as is the case with a zoom slider or a thumbnail sketch of the entire workspace.

Whatever the event mechanism, the upshot is that the browser script will sometimes receive notifications that there is a new desired region or zoom level, if that's applicable. At that point, a Web Remoting (Chapter 6) call must occur, passing the server the details of the new region. Upon reply, the old data is either replaced or shifted along and the new data is rendered.

This pattern is often very bandwidth-intensive, considering that the entire workspace can be massive and the interaction complex. For that reason, there are several important performance optimizations—for more details, see "Related Patterns" later in this chapter.

Decisions

How will you handle panning?

Chapter 16. Visual Effects

ARICH INTERFACE IS MORE THAN STATIC WIDGETS; THERE'S A DYNAMIC COMPONENT THAT HAS A BIG impact on usability. Used carefully, visual effects can guide the user's attention and help him understand what the application is doing. And don't underestimate the aesthetic benefits. It might be eye candy, but good eye candy (not just "how good am I" hacks) makes passionate users.[*]

The "One-Second" effects are fleeting effects that sometimes convey a permanent transition from one state to another, and other times are used just to highlight something. One-Second Spotlight is a more general version of the popular "Yellow Fading Technique" and involves changes to brightness and color. In One-Second Mutation, the element changes shape, and in One-Second Motion, the element moves around.

Highlight is a more permanent effect where one or more elements are highlighted, typically because the user has selected them.

One-Second Spotlight

⊙⊙⊙ Attention, Effect, EyeCandy, Fade, Fading, Graphics, Refresh, Update, YellowFadeTechnique

Figure 16-1. One-Second Spotlight

Goal Story

Tracy is checking out her portfolio, which shows each stock in its own row. Suddenly, "Neverland Mining Co." stock plummets, and its row lights up to reflect the change. Tracy's attention is drawn instantly to this visual effect, and she immediately begins to offload her stake in the company.

Problem

How can you direct the user's attention to spots on the page?

Forces

To ensure that the user is working with current data, the browser display must be frequently updated.

With Ajax, it's possible to update only the portion of the screen that has changed, and if nothing has changed, the display will remain exactly as is.

The screen can get cluttered with a lot of information, much of which might change at any time.

While human vision is good at spotting changes, it's easy to miss a sudden change, especially if it's a subtle one.

Solution

When a display element undergoes a value change—or any other significant change—dynamically increase its brightness for about a second. As a device for communicating system activity, the One-Second Spotlight accomplishes several things:

It highlights changes so the user doesn't have to actively monitor for changes and try to remember what—if anything—has changed.

Because the spotlight lasts for a few seconds, the user has enough time to redirect attention to that item and study the details of the event—e.g., looking at an item's new value. In contrast, a standard transition would afford her no time whatsoever—if the user happened to blink, she could easily miss the change.

When the animation technique follows certain conventions or mimics real-world phenomena, it can provide hints to the user about the nature of the event.

This pattern is a generalization of the technique 37signals introduced as the Yellow Fade Technique (http://www.37signals.com/svn/archives/000558.php). With that, an item lights up for a second after the user has edited it, and the lighting quickly fades away. Unlike a conventional application, in which a page refresh would occur, it's often unclear when an XMLHttpRequest Call has occurred. So a quick visual effect is almost essential for the user to understand what's happening.

From a technical perspective, the Yellow Fade Technique is usually achieved by suddenly switching the element's background-color style from white to yellow, then using Scheduling to run through a continuous spectrum of background colors, starting with yellow and ending with white.

In more general terms, One-Second Spotlight encompasses several types of animation. One or more of the following CSS styles are affected during the spotlight period:

color

Usually, the element's background changes, but font color is also a possibility. Though commonly assigned with Red-Green-Blue hex codes such as #ff00ff, animation algorithms can benefit from the percentage form, e.g., (100%, 0%, 100%). Often, the algorithm cycles through one or more of the Red, Green, and Blue components, e.g., gradually raising Red from 50 percent to 100 percent and lowering it back to 50 percent while holding blue and green constant. You might also prefer to represent the color as a combination of Hue-Saturation-Value (HSV) and cycle through one or more of those components. In this case, calculate the HSV values and convert them to RGB for display.

opacity

This style determines how transparent the element is; an opacity of zero means the item is solid, and an opacity of 50 percent means the browser background—or any items underneath the element—will be partially visible.

visibility

This style determines whether the element can be seen. Visibility does not affect page layout, so the element still occupies the same space on the page.

The possibilities are endless, but there are a few common effects. Firstly, there are straightforward transitions from one style setting to another:

Fade Out

An element suddenly brightens and then fades back to its original appearance. This is useful for general highlighting and may also indicate that the element has been "sucked into" the server, i.e., the data is now safely stored server side. Since many pages use a white background, fading the background out to white is a natural choice for general attention grabbing.

Fade In

An element is suddenly dimmed and then brightens up to its original appearance. This is useful for general highlighting and may also indicate that the element has been "pushed out" of the server.

Fade Away

An element fades further and further out until it disappears altogether. If the element is already somewhat dim, it may be helpful to brighten it at the start of the animation. This effect often indicates that the element (or its value) has been suspended or destroyed.

Materialize

From nothing, an element fades into its regular appearance. If its regular appearance is somewhat dim, it may be helpful to fade it in to a higher level, then fade back out a bit. This can indicate that the element has been created or retrieved.

Switch

An element gradually switches its appearance from one setting to another. Instead of fading or brightening, though, an element might shift from red to blue. This effect is usually used to indicate a state change.

The other category of effects involves rapid oscillation and includes the following:

One-Second Mutation

⊙⊙⊙ Auto-Update, Sync, Sychronize, Real-Time

Figure 16-4. One-Second Mutation

Goal Story

Doc monitors a patient's condition, watching some histograms shrink and grow to reflect changes in the patient's body.

Problem

There's a lot of activity in an Ajax App; how can you help the user digest it all?

Forces

To ensure that the user is working with current data, the browser display must be frequently updated.

With Ajax, it's possible to update only the portion of the screen that has changed; if nothing has changed, the display remains exactly as is.

The screen can get cluttered with a lot of information, much of which could change at any time.

While human vision is good at spotting changes, it's easy to miss a sudden change, especially if the change is a subtle one.

There are different types of changes, and those differences are important to users.

Solution

When a page element undergoes a value change or some other significant event, dynamically mutate its shape for about a second. In most cases, the mutation is used to reflect a permanent change. Instead of flipping from one form to another, the element gradually transitions into another form. The effect serves a few purposes:

The mutation and the extended transition time make it likely that the user will notice that something has happened.

The type of mutation hints at what sort of action has taken place.

The gradual shift in appearance feels more natural than an instant transformation.

Mutation combines Display Morphing with Scheduling. A loop runs while the second of mutation progresses, with each transition occurring perhaps once every 50 milliseconds. Each transition will likely involve a change to at least one of the following CSS properties:

width

An object can be thinned or fattened by changing its width value.

height

An object can be heightened or shortened by changing its height property.

backgroundImage

Instead of just resizing the element, it's possible to swap in a completely different image. Just like a movie, the DOM element quickly flips through various images to give the illusion of mutation. See the DHTML Lemmings walkthough in Sprite (Chapter 15) for more details.

position

Manipulating position with position, left, right, top, and bottom is possible for certain mutation effects.

overflow

Overflow can be set to hidden to enable the entire element to be present on the page while showing only a certain portion of it.

The first category of mutation involves making an object appear. The following are effective ways to communicate that the object has just been created or retrieved from the server.

Form

An item "forms" from nothingness into its standard appearance—typically, by incremental, outward growth. It might grow from the center of the object, a corner, or some other position entirely. Typically, width and height increase at the same time.

Slide Out

An item slides out. If the item slides in from the side, it appears to be pushed out like a sliding door. If the item comes in from the top, it appears like a garage door closing. If coming in from the bottom, the item appears to rise from the ground. Usually, either width or height is fixed, while the other dimension gradually increases. The item's position changes simultaneously.

Reveal

An item is gradually revealed. It appears that the item has been there all along, and that a cover is gradually being lifted off. Here, one or both of width and height incrementally increases, while overflow is set to hidden so that part of the element is covered.

The next category of mutation involves some common disappearing acts—the opposite of the effects above—and a special effect for disappearing:

One-Second Motion

⊙ Copy, Displace, Duplicate, Jump, Leap, Move, Movement, Motion, Rearrange, Transfer

Figure 16-7. One Second Motion

Goal Story

Bill is adding a new monthly bank order. The existing orders are shown in the usual way—as a table with one row per order, sorted by cost. Once he is verified the new order form, it morphs to become a table row, then slowly drifts down the table. It soon reaches its rightful place in the sorted table, which is ordered by amount, and comes to rest after bouncing up and down a little due to reverberation.

Problem

How can you communicate that an element is shifting from one position to another?

Forces

An element's location is important, as it can provide cues about its prominence and its relationship to other elements.

An element sometimes needs to move from one place to another.

When an element changes position, the user often needs to know about it.

Solution

Incrementally move an element from one location to another, or temporarily move it, to communicate that an event has occurred. In most cases, the motion of the element involves a permanent change in its position. It can also be used to show a temporary displacement or gesture, such as a vibration effect.

There are several benefits of showing the motion explicitly:

The motion and the extended transition time make it likely that the user will notice that something has happened.

The motion will highlight the place where the element is moving to (it won't get lost on the page).

The motion will feel more natural than an abrupt change.

The effect combines Page Rearrangement (Chapter 5) with Scheduling (Chapter 7). A loop runs for the duration of progresses, with each transition occurring perhaps once every 50 milliseconds. Each transition will likely involve a gradual change to the positioning style settings, usually to top and left. The position style can be set as absolute or relative, depending on the overall application design.

When an element has to move from point A to point B, in most cases it's logical for the trajectory to be the shortest distance from A to B, i.e., for the element to move as the crow flies. The algorithm in this case uses interpolation—for example, one-quarter of the time into the effect, an element will be one-quarter of the way between A and B—so, when moving from left to right, the left style of the element will be (0.25*x(A) + 0.75*x(B)), where x( ) represents the source and destinations' horizontal coordinate.

This assumes a linear interpolation. You can also experiment with other algorithms, e.g., you might like to accelerate the element as it proceeds, which is often a more realistic model.

Note that you probably won't have to hand code motion, because libraries like Scriptaculous (discussed in the "Real-World Examples") are quite flexible and usually easy to incorporate into typical scripts.

Decisions

What events will trigger a motion effect?

One-Second Motion is not very common (yet), but there are several potential applications. First, a permanent movement can show any of the following:

A straightforward change from one place to another.

One element being spawned from another, and then jumping out to a new location.

An element being introduced into a space. Instead of making the element just appear, or fade in, you could make it arrive from a logical starting point. For example, a new data point in a chart can "fly in" from the corner.

A temporary movement has its own applications:

Attention-grabbing

Highlight

⊙⊙⊙ Auto-Update, Sync, Sychronize, Real-Time

Figure 16-8. Metaphor

Goal Story

Sasha notices that a few spam comments have appeared on an old blog posting. When she clicks on the comments, they are successively Highlighted—the background color switches from white to yellow. Because she's already logged in, there's a Delete Spam button below the comments, which she clicks to send them where they belong.

Problem

How can you make elements stand out?

Forces

An Ajax App can enable a lot of information to be shown at once—often information with many with different states.

There is usually one element which the user is working on or about to activate.

In addition, the user often needs to group several elements together to perform a common function. These elements must stand out from the crowd.

Solution

Highlight elements by rendering them in a consistent, attention-grabbing format. This pattern has been applied to dynamic web sites prior to Ajax but is particularly important in the context of rich displays and interaction.

Consistency is important here; when items are selected, they should look the same. One straightforward way to achieve this is with selected and deselected CSS classes. By dynamically switching the element's className property, you can easily Highlight and de-Highlight.

Highlighting is useful in the following ways:

To show which particular element has input focus.

To show which elements are selected.

When the user rolls over an element, to indicate its boundaries and hint that some action will occur by clicking on it.

To indicate that an element is particularly important.

To indicate that an element is undergoing change.

To prompt the user to perform some kind of work on the element.

Decisions

Chapter 17. Functionality

THIS CHAPTER CONTAINS A NUMBER OF HIGHER-LEVEL PATTERNS THAT ARE MORE GEARED TOWARD offering new functionality than modifying the user interface. Technical barriers have prevented these from being used much on the Web, but Ajax technologies are helping to break down the barrier.

Lazy Registration addresses the issue of managing the lifecycle of a user profile. Instead of a binary "you're registered or you're not" perspective, Lazy Registration advocates a more gradual accumulation of user data and authentication credentials. A related pattern is Direct Login, which explains how to let the user securely log in without forcing a page refresh. It incorporates browser-side encryption, a theme at the heart of the next pattern, Host-Proof Hosting. The purpose of this pattern is to encrypt and decrypt data within the browser so that it can't be inspected by the hosting organization.

The next two patterns address monitoring the user's activity level. Timeout uses event-handling and scheduling to decide when the user is no longer active. Heartbeat is similar, but it brings the server into the equation; the browser keeps sending a heartbeat message to tell the server that the user is still active and that the application is still sitting in the browser.

Finally, Unique URLs restores some functionality that Ajax, to some degree, takes away. Bookmarkable URLs and browser history have always been an issue on the Web, but Ajax exacerbates the problem when developers choose not to use page refreshes. The Unique URLs pattern identifies a range of tricks to handle bookmarking, deep linking, the Back button, and related concerns.

Lazy Registration

⊙⊙ Account, Authentication, Customisation, Customization, Incremental, Login, Password, Personalisation, Personalization, Profiling, Registration, User, Verification

Figure 17-1. Lazy Registration

Goal Story

It's Saturday afternoon, and Stuart is busy planning the evening's activities. He visits a band listings web site, where he clicks on a map to zoom into his local area. Even though he has never visited the site before, a profile, which he can see on the top of the site is already being constructed. At this stage, the profile guesses at his location based on his actions so far. As he browses some of the jazz bands, the profile starts to show an increasing preference for jazz bands, and some of the ads reflect that. Since the jazz thing is a one-time idea, he goes into his profile and tweaks some of those genre preferences but leaves the location alone since the system's guess was correct. Finally, he decides to make a booking, at which point he establishes a password for future access to the same profile including his address, which is posted back to the profile.

Problem

How can the user customize the site while deferring formal registration?

Forces

Public web sites thrive on registered users. Registered users receive personalized content, which means that the web site is able to deliver greater value per user. And registered users can also receive more focused advertising material.

For nonpublic web sites, such as extranets used by external customers, registration may be a necessity.

Most users don't like giving their personal information to a web server. They have concerns about their own privacy and the security of the information. Furthermore, the registration process is often time consuming.

Many users spend time familiarizing themselves with a site before registering. In some cases, a user might interact with a site for years before formally establishing an account. There is a lot of valuable information that can be gained from this interaction that will benefit both the web site owner and the user.

Solution

Accumulate bits of information on the user as they interact while deferring formal registration. As soon as the user visits the web site, a user account with auto-generated ID is immediately created for her and set in a cookie that will remain in the browser. It doesn't matter if she never return; unused IDs can be cleared after a few months.

As the user interacts with the application, the account accumulates data. In many cases, the data is explicitly contributed by the user, and it's advisable to expose this kind of information so that the user can actually populate it. In this way, the initial profile may be seen as a structure with lots of holes. Some holes are eventually filled out automatically and others by the user himself. The user is also free to correct any of the filled-in data at any time (Figure 17-2).

Figure 17-2. User Profile

Two particularly notable "holes" are a unique user identifier and a password. It is this combination of attributes that allows the user to access the profile from another machine or a different browser. They will also preserve the profile in the case that cookies are deleted from the user's browser. So, while this pattern is generally about gradual accumulation of profile data, there remains a significant milestone in the user/application relationship—the moment at which user ID and password are established.

Do the user ID and password have to be provided simultaneously? No. Even that can be incremental as long as you make the email address the unique identifier. In fact, this is pretty common nowadays. Email is usually required anyway, and it's unique, so why not make it the user ID? In the context of Lazy Registration, though, there's an additional benefit, as the email might be accumulated in the natural flow of events—the site might add the user to an announcements list, for example. In some cases, the email might even be verified during this process.

Sceptics may wonder why a user would want to actively work with her profile. The answer was formulated in a web usability pattern called "Carrot and a Stick" (http://jerry.cs.uiuc.edu/~plop/plop99/proceedings/Kane/perzel_kane.pdf):

Determine what users consider to be a "valuable" carrot. Offer the end user a portion of that carrot before you request personal information. The content is withheld ("the stick") until the requested information is provided.

Thus, users will only enter information if there is a perceived benefit to them. There is plenty of evidence that this occurs—witness the social bookmarking phenomenon, where thousands of users make public their personal links. By exposing their profiles, many of those users are hoping the system will point them in the direction of related resources they have not yet heard of.

Some web sites have used this pattern for years, so what does it have to do with Ajax? Lazy Registration aims for a smooth approach in which the barrier is low for each new user contribution. For instance, you sign up for a web site's mailing list, and your email is automatically added to your profile and shown on the side of the page. With Ajax, there's no need to break the flow. No more "just go over there for a few minutes, then come back here, and if you're lucky, you might be looking at something similar to what you can see now." That's a big win for web sites aiming to drop the barrier of registration, and it's great for users, too.

It's standard practice for web sites to collect data about users. The aim of this pattern is to empower them to contribute to this. Instead of covertly building up a corpus of data on a user, you invite him to add value to his own experience by contributing and maintaining the data himself.[*]

Several technologies are involved in Lazy Registration:

Database

You clearly need a persistent data store in order to retain user profiles.

XMLHttpRequest

Passing profile information back and forth with XMLHttpRequest Calls is the key to the smooth interaction mode you are seeking to achieve with this pattern.

Cookie manipulation and session tracking

A cookie, associated with your domain and identifying a unique session ID, must reside in the user's browser. The session ID can serve as a key on the server side to locate details about the user each time she accesses the web site. In conventional applications, the cookie is pushed from the browser to the server as a header in the response. That's fine for Ajaxian Lazy Registration when the user first accesses the system, though sometimes it may be convenient to use a more Ajax-oriented approach. The first such approach is to manipulate the cookie in JavaScript (http://www.netspade.com/articles/javascript/cookies.xml). The second is to set the cookie using the response from an

Direct Login

⊙⊙ Access, Account, Authentication, Login

Figure 17-7. Direct Login

Goal Story

When Stuart logs in to perform an online exam, he is presented with a standard username-password form. He enters his username and password, but a few seconds later, the form becomes red and shows an error message underneath. He switches off Caps Lock and re-enters his credentials. This time, they're accepted. The form morphs to show his name and balance, and a new menu appears alongside it.

Problem

How can users present their credentials to the server?

Forces

Casual users may not bother to log in if the process interrupts their browsing experience.

The password should not travel in plain-text; it could be intercepted while traveling to the server.

Solution

Authenticate the user with an XMLHttpRequest Call instead of with a form-based submission, hashing in the browser for improved security. The essence of this pattern is a routine transformation from a submission-based approach to an Ajaxian, Web Remoting (Chapter 6) interaction style. But, in this pattern, I also discuss a very useful, though optional, technique that involves JavaScript-based hashing and is specific to the login process.

Conventional authentication usually requires the user and password to be uploaded as a standard form submission. The server usually converts the password to a hash (or "validator") value and checks it against a stored hash value in the database. If they match, the user is in.

There are two problems with this approach. Firstly, flushing the page can be a distraction. It might not take long, but it will usually leave the user in a different context, which will discourage her from logging in. Even more troublesome is the stream of pages that ensue when a password must be recovered (e.g., having to provide your maiden name, last purchase date, and the name of your favorite pet canary), especially in this security-conscious era. The other problem relates to security of the transmission; if the password is uploaded as plain-text, there's a risk of interception.

Direct Login addresses the page refresh problem and, optionally, the transmission problem too. In the simplest approach, you can implement Direct Login by simply sending the username and password in plain-text to the validation service using an XMLHttpRequest POST. Then, the server behaves similarly to a conventional server: it checks whether the password matches using a hash function and prepares for session management. XMLHttpRequest deals with cookies as it does regular form handling; this allows the session to be established in the same way as a conventional form submission. The only difference is the response content: instead of outputting a new HTML page, the server outputs an XML or plain-text acknowledgment, as well as any personalized content.

Passing the credentials over with XMLHttpRequest will improve usability, but as long as the password is being transferred in plain-text, there's still a security threat. Ensuring the whole transaction runs overs HTTPS is always the best measure, as this generally makes the transaction secure from interception. However, many web sites don't provide such a facility. Fortunately, there's a compromise that can prevent transmission of plain-text passwords. The technique, strictly speaking, is orthogonal to the Direct Login approach; you could apply it to conventional submission-based authentication as well. But since it makes heavy use of browser-side processing, it fits nicely with Direct Login.

Host-Proof Hosting

○ ASP, DataCloud, Key, Secure, Untrusted

Figure 17-9. Host-Proof Hosting

Goal Story

Reta is dismayed to learn that a malicious hacker managed to download a chunk of the company's database containing personal details about all the customers in her store. Fortunately, she also learns that the pass-phrase she always entered was actually used to encrypt all that data, so the hacker won't be able to make sense of any of the contents.

Problem

How can you mitigate the effects of unauthorized access to your application data?

Forces

Web apps require that some form of persistent data hold information about users, business state, past events, and so on.

Security restrictions—using standard web technologies—prohibit users from storing web apps on the user's own hard drives. Even with Richer Plugins that allow it, many benefits of using a web app in the first place are lost. This means that persistent data is usually stored server side.

Cookies allow some data storage within the browser, but cookie data is also transmitted to the server where it's vulnerable.

Server-side storage is open to abuse: the administrators, along with anyone who is able to gain access to it, are able to extract sensitive information by reading the data, as well as effect malicious changes by tampering with it. The abuse can occur within an organization's own IT department or be inflicted by a third-party hosting company entrusted with the data.

Solution

Host sensitive data in encrypted form so that clients can only access and manipulate it by providing a pass-phrase that is never transmitted to the server. The server is limited to persisting and retrieving whatever encrypted data the browser sends it and never actually sees the sensitive data in its plain form. All encryption and decryption takes place inside the browser itself.

Just what does secure hosting have do with Ajax? The Ajaxian twist comes in the maintenance of the pass-phrase. You could use the browser-side encryption with a conventional application, but the pass-phrase would have to be entered upon each page refresh, since no JavaScript state survives a reload. With page refreshes occurring every few seconds, the pass-phrase is completely unusable. However, using Ajax to avoid page refresh means you can retain all session state in the browser, so the pass-phrase only needs to be entered at the start. After being entered, it can be retained as a standard JavaScript string and will disappear from the browser when the user quits the browser or visits another site. Suddenly, Host-Proof Hosting becomes usable.

Incidentally, don't take this pattern to be a new "Ajax-HTTPS" protocol. The issue here is how the data is actually stored, not how it's transmitted. In theory, the data itself need not travel over a secure connection because it's already encrypted. In practice, a secure connection might be worthwhile in order to reduce some of the vulnerabilities described later in this section.

Before you rush off to upload all your trade secrets to Shonky Hosting Inc., you should be aware that this idea isn't foolproof. On the one hand, the host is assumed to be inherently untrustworthy. But on the other hand, the script for the browser application is held right there on the server, and the browser runs whatever scripts come down from that URL. This leaves open the possibility that the host will tamper—to evil ends—with either the code itself or the outgoing HTML and JavaScript.

What if a rogue administrator from the hosting company decided to quietly add a small monitoring function to a JavaScript file and append its execution to a window.onload function. Then, the evil monitoring function could be made to run once a minute within the browser. It might, for instance, serialize the entire DOM and upload a summary back to the server with Web Remoting (Chapter 6

Timeout

⊙⊙ Logout, Screensaver, Session, Suspend, Timeout

Figure 17-11. Timeout

Goal Story

Tracy was so keen on leaving the office Friday afternoon that she forgot to shut down the market news web site. Fortunately, the web site monitors mouse movements, meaning that if she doesn't interact with the page for an hour, it will suspend itself, saving a lot of wasted refreshes. On Monday morning, Tracy walks in and sees a screensaver-like animation in the browser, with a message indicating that updates have ceased. She clicks a mouse button and sees the screen update with the latest prices.

Problem

How can you tell whether the user is still working with the application?

Forces

Users often leave an application in a browser window for hours or days without interacting with it. Many times, they abandon it altogether. The practice is becoming more prevalent as all major browsers now support multitab browsing.

Ajax Apps often use Periodic Refresh or HTTP Streaming to keep informed about server state. The user may not care about the application anymore, but if it's still in the browser, a naïeve implementation will continue to poll the server indefinitely—a massive waste of bandwidth and resources.

Ajax Apps often contain sensitive data which is at risk if the user forgets to log out when leaving a public terminal.

Ajax Apps often involve collaboration and communication between users. To cultivate an awareness of other users and their activities, the server must track the state of individual browsers.

Solution

Have the browser Timeout the user after a period of inactivity and, optionally, inform the server. After a period of inactivity, the application is either suspended, requiring the user to manually resume it or shut down, requiring the user to restart.

This pattern raises the whole question of sessions in Ajax. Conventional applications usually have server-side session objects which represent the user's current session. Typically, this means short-lived information such as shopping cart contents or recently viewed items.

In Ajax, server-side sessions are not so useful. In fact, if you rely solely on RESTful Services, you may not need those session objects at all. In browser-centric Ajax Apps, the browser is a full-fledged application with all of the session state held within. So the browser state is the session. The server-side session, if used at all, is only relevant to authentication and has no impact on the response to any particular query. See RESTful Service (Chapter 9) for more details.

How is all of this relevant to Timeout? Well, consider this familiar nightmare scenario:

The user opens up a form, spends two hours populating it while researching the answers, and clicks submit. The browser then responds with "Your session has expired" and adds insult to injury by presenting a new blank form.

Stories like this are all too common because server-side session Timeouts ignore what's happening in the browser. If the user hasn't submitted a form in, say, 30 minutes, the session times out and all the data is lost. That's a problem if the user has been actively working on the browser side. With an Ajax App, the server has a better chance of staying in sync with the browser thanks to XMLHttpRequest Calls (Chapter 6). In most environments, the server-side session will be automatically refreshed when an XMLHttpRequest Call comes in. However, this is not optimal either because the server still doesn't know why browser calls are occurring. For applications using Periodic Refresh (Chapter 10), for example, a server-side session will stay alive indefinitely even if the user has left the building.

By relying instead on browser-based Timeout detection, you can be more intelligent about how Timeouts will occur. Browser-based Timeout detection is based on Scheduling (Chapter 7). A timer is established to count down to Timeout state. Each significant activity cancels the timer and starts a new one in its place.

What activities are significant enough to restart the timer? Mouse movement is one candidate. You can monitor each mouse movement on the page; one of the following Code Refactoring illustrations below does exactly that. Mouse movements are a very broad indicator, and you might be concerned about "false positives": These may be suggestions that a user is still working with an application when they're in fact not, e.g., they just happened to move the mouse across the browser window while cycling through each open window. You'll also get some "false negatives": A user working only with the keyboard is liable to be timed out. If you'd prefer to keep the session going only if the user is actively changing data, you can instead catch events such as button clicks and keypresses within input fields.

Sudden Timeouts can be frustrating for users, so consider these feedback measures:

Unobtrusively show Timeout status, e.g., number of minutes remaining. You might highlight the status as Timeout approaches, and also offer some background and instruction on avoiding the Timeout.

When Timeout is approaching, offer a warning and an opportunity to explicitly restart the timer.

Once you've detected that the user is inactive, what do you do? Timeout can be used in several ways:

Stop Periodic Refresh

Conventional applications tend to do nothing when the user is idle, but many Ajax Apps continuously poll the server. Thus, the Timeout should trigger cancellation of any Periodic Refresh timers.

Clear data

Sometimes an inactive application is a hint that the user may no longer be in control of a terminal. There's a risk that someone else could gain access to the user's data and permissions. After a Timeout, document.url can be pointed to the main application URL in order to refresh the page. In addition, you might decide to delete cookies holding sensitive data.

Save data

If you clear data, you might also consider first saving it on the server. There's nothing more annoying than arriving back from lunch and finding that a partly filled-out form has been cleared for security purposes. With an XMLHttpRequest Call, you have the option of saving the user's progress as a "draft."

Inform user

Heartbeat

○ ACK, Announcement, Flash, Heartbeat, Monitor, Signal

Figure 17-14. Heartbeat

Goal Story

For auditing purposes, Frank must run the factory decision support system all day long, so the server needs to ensure that it's always alive in the browser. But requests won't come in very often because it's a fat client, with the business logic in JavaScript. To ensure that the server keeps getting requests, the browser explicitly uploads a Heartbeat message every 10 minutes.

Problem

How do you know the user still has an application in the browser and is actively working with it?

Forces

It's useful for the server to track whether the user is active, with several applications highlighted in the Solution of Timeout.

Due to the stateless nature of HTTP, the server doesn't know when the user has quit the browser, experienced a browser crash, or surfed away to a different URL. Note that you could use the JavaScript onunload event to catch the last of these, but it won't work for the first two. In tracking the user's activity, the server may assume the user is still active when he has in fact abandoned the application.

The user sometimes spends a long time working in the browser, but in a manner which yields no calls to the server—for example, filling out a long form or playing a game. The server has no way of knowing that the user's still around if no calls are made.

Solution

Have the browser periodically upload Heartbeat messages to indicate that the application is still loaded in the browser and the user is still active. The server keeps track of each user's "last Heartbeat." If the Heartbeat interval is 10 minutes and the user's last Heartbeat was longer than 10 minutes ago, the server knows that the user is no longer active. The objective is to help the server track which users are active. Session tracking isn't essential in Ajax Apps, where it's possible to hold all session data as JavaScript state, but, as described in the Solution of Timeout, there are still reasons to do it.

Heartbeat messages are uploaded to a special "Heartbeat service" using XMLHttpRequest Calls. Because they affect server state, they should be POSTed in. The Heartbeat service will update the user's last Heartbeat record, but how does it associate a Heartbeat message with a user? Heartbeat relies on some form of session management. One approach is to use cookies, either directly or via a cookie-based session framework. However, if you do that, the conversation will be stateful, thus violating a fundamental RESTful Service principle. A cleaner approach is to explicitly include the session ID in the Heartbeat body. Note that you shouldn't just upload the user ID, as others could easily fake the user ID.

Heartbeat is closely related to Timeout, but they work in different ways. You can use either independently, but Heartbeat works best as a supplement to Timeout. The point of Timeout is to stop the browser application after an idle period for security and bandwidth reduction. Notifying the server of Timeout events yields extra benefits, but it only works if the application is still sitting in the browser and working fine. That's why we use Heartbeat messages, which are, in a sense, the inverse of Timeouts, and therefore a good supplement. Whereas a Timeout message has the browser announce when a timeout has occurred (Figure 17-15), a Heartbeat message has it continuously announced that a Timeout hasn't occurred (Figure 17-16). As soon as the server detects a missed Heartbeat, it can assume the application is no longer running.

Unique URLs

⊙ Address, Bookmarks, Cut-And-Paste, Distinct, Favorites, Hyperlink, Location, Link, Mail, Share, Unique, URL

Figure 17-18. Unique URLs

Goal Story

Sasha is exploring a recent news event by browsing a map of the area. There's a Unique URL for each location, so she's able to link to it in her blog and add it to her social bookmarking account.

Problem

How can you assign distinct URLs to different parts of your application?

Forces

Expressive URLs are vital. The ability to link from one site to another is arguably what made the Web successful. Furthermore, web surfers like to bookmark pages for later on. And these bookmarks are now a social phenomenon, with the advent of "linkblogs" and social bookmarking software like del.icio.us (http://del.icio.us).

Browsers retain URLs in their histories, and the Back button will only work if URLs vary. The Back button is a big habit for many users. It can mean "Go back to where I was a minute ago." For many users, it can also mean "Undo my last action," and that mental model must be respected even though some experts deem it to be technically incorrect.

A single address for your whole site is not enough; you need to support deep linking for links to be effective. That is, each distinct component of a web site should have its own URL.

In conventional web sites, the URL changes when the page is refreshed—e.g., when the user submits a form or follows a link. But in Ajax Apps, the page never refreshes, so the browser won't change the URL.

Solution

Provide Unique URLs for significant application states. Each time a significant state change occurs—e.g., the user opens a new product in an e-commerce system—the application's URL changes accordingly. The objective is straightforward, but as discussed later in this section, the implementation requires a few unusual tricks. This Solution is fairly complex, because it delves into the details of building Unique URLs. Increasingly, though, reusable libraries will save you the effort, so be sure to check out what's available before directly applying any of the techniques here. The Real-World Examples identifies a couple of resources, and there will doubtless be others by the time you read this.

In Ajax, most server communication occurs with XMLHttpRequest. Since XMLHttpRequest Calls don't affect the page URL, the URL is, by default, permanently stuck on the same URL that was used to launch the Ajax App, no matter how many transfers occur. The only option you have to manipulate the URL is to use JavaScript. And, in fact, it's very easy to do that with the window.location.href property:

window.location.href = newURL; // Caution - Read on before using this.

Very easy, but there's one big problem: under normal circumstances, the browser will automatically clear the page and load the new URL, which sort of defeats the purpose of using Ajax in the first place. Fortunately, there's a cunning workaround you can use. It was originally conceived in the Flash world and has more recently been translated to an Ajax context. Mike Stenhouse's demo and article (http://www.contentwithstyle.co.uk/Articles/38/fixing-the-back-button-and-enabling-bookmarking-for-ajax-apps) provides a good summary of the approach, and the Solution here is based heavily on his work.

The cunning workaround relies on fragment identifiers —those optional components of URLs that follow the hash character (#). In http://ajaxpatterns.org/Main_Page#Background, for example, the fragment identifier is Background. The point of fragment identifiers has traditionally been to get browsers scrolling to different points on a page, which is why they are sometimes called "in-page links" or "named links." If you're looking at the Table of Contents on AjaxPatterns.org (http://ajaxpatterns.org#toc) and you click on the Background link—http://ajaxpatterns.org/#Background—the address bar will update and the browser will scroll to the Background section. Here's the point: no reload actually occurs. And yet, the browser behaves as if you clicked on a standard link: the URL in the address bar changes and the Back button sends you back from whence you came (at least for some browsers; more on portability later).

You can probably see where this is going. We want Ajax Apps to change the page URL without forcing a reload, and it turns out that changes to fragment identifiers do exactly that. The solution, then, is for our script to change only the fragment component. We could do this by munging the window.location.href property, but there's actually a more direct route: window.location.hash. So when a state change is significant enough, just adjust the URL property:

window.location.hash = summary;

Throughout this pattern, summary refers to a string representation of the current browser state—at least that portion of the state that's worth keeping. For example, imagine that you have an Ajax interface containing several tabs: Books, Songs, and Movies. You want to assign a Unique URL to each tab, so the summary will be the name of any of these tabs. If there were other significant variables, they would be integrated into the summary string. To keep the fragment synchronized with the current state, you need to identify when state changes occur and update the fragment accordingly. In the following tab example, you can do this by making the onclick handler set the hash property in addition to changing the actual content:

songTab.onclick = function( ) { window.location.hash = "songs". // ... now open up the tab }

So now the URL changes from http://ajax.shop/ to http://ajax.shop/#songs. That was easy enough, but what happens when you mail the URL to a friend? That friend's browser will open up http://ajax.shop and assume songs is an innocent, old-fashioned, in-page link. It will try to scroll to a songs element and won't find one, but it won't complain about it either; it will simply show the main page as before. We've made the URL change after a state change, but so far we haven't built in any logic to make the state change after a URL change.

State and URL need to be in sync. If one changes, the other must change as well. Effectively, there's a mini-protocol specific to your application that defines how state and URL map to each other. In the preceding example, the protocol says "The fragment always represents the currently open tab." Setting the fragment is one side of the protocol; the other side is reading it. After your friend opens http://ajax.shop/#songs, and after his browser unsuccessfully tries to scroll to the nonexistent songs element, we need to read the fragment and update the interface accordingly:

window.onload = function( ) { initializeStateFromURL( ); } initializeStateFromURL( ) { var initialTab = window.location.hash; openTab(initialTab); }

Good. Now you can mail the link, bookmark it, and link to it from another page. In some browsers at least, it will also go into your history, which means the Back button will work as expected. For a simple implementation, that's good enough.

However, the solution still isn't ideal, because what will happen if the application is already loaded? The most likely situations occur when clicking the Back button to revert to a previous state and when retrieving a bookmark while already on that application. In both cases, the URL will change in the address bar, but the application will do absolutely nothing!

Remember, we're making the application manipulate fragments precisely because the browser doesn't reload the page when they change. When it comes to Back buttons and bookmarks and the like, we're talking about changes that are initiated by the user, yet the browser treats these with exactly the same inaction as when our script initiates them. So our onload won't be called and the application won't be affected in anyway.

To make Unique URLs work properly, we have to resort to something of a cheap trick: continuously polling the fragment property. Whenever the fragment happens to change, the browser script will notice a short time later and update state accordingly. In this example, we could schedule a check every second using setInterval( ):

window.onload = function( ) { initializeStateFromURL( ); setInterval(initializeStateFromURL, 1000); }

In fact, we should only perform an action if the hash has recently changed, so we perform a check in initializeStateFromURL( ) to see if the hash has recently changed:

var recentHash = ""; function pollHash( ) { if (window.location.hash==recentHash) { return; // Nothing has changed since last polled. } recentHash = window.location.hash; // URL has changed. Update the UI accordingly. openTab(initialTab); }

The polling adds to performance overhead, but we now have a URL setup that feels like the real thing.

One last thing on this technique. I mentioned earlier that you can set window.location.hash manually after state changes. Now that we've considered the polling technique, you can see that it's also possible to do the reverse: change the URL in response to user events, then let the polling mechanism pick it up and change state accordingly. That adds a delay, but one big benefit is that you can easily add a hyperlink on the page to affect the system state.

Here's a quick summary of the technique just described, which I'll call the "page-URL" technique:

Ensure that window.location.hash always reflects application state.

Create a function, e.g., initializeStateFromURL( ), to set application state from window.location.hash.

Run initializeStateFromURL on startup and at periodic intervals.

But wait, there's more! I mentioned earlier that each URL goes into history, but only "for some browsers." IE is one of the outliers here, so the above technique will fail IE's history mechanism, and the Back button won't work as expected. The good news is there's a workaround for it. The bad news is that it's a different technique and won't work properly on Firefox. So if you can live with everything working but IE history, go with the hash-rewrite technique. Otherwise, read on . . . .

Part V. Development Patterns

The patterns in this part are not "things" you'd see in an Ajax App—as in the previous parts—but processes you can use to aid development.[*]

Chapter 18. Diagnosis

IF THERE'S ONE CERTAINTY IN THE LIFE OF A DEVELOPER, IT'S BUGS. AJAX PROGRAMMING ISN'T ROCKET science, but you can still expect the odd moment of pain during development and maybe even in production. Fortunately, there are plenty of techniques, tools, and libraries available to help you isolate problems as they arise. This chapter breaks them into four categories.

Logging is just good old logging applied to the browser, where log messages can be shown in a popup, uploaded to the server, or shown in some browser-specific way. Likewise, Debugging is general-purpose debugging applied to JavaScript that is running within the browser.

The next two patterns are more Ajax-specific. DOM Inspection is about analyzing the dynamic state of the DOM. Traffic Sniffing is about watching traffic between browser and server.

Logging

⊙ Capture, Log, Message, Monitor, Record

Figure 18-1. Logging

Developer Story

Dave's search engine keeps giving a blank result for certain queries. Fortunately, he previously added some Logging commands to see how the query is processed, so he makes the console visible and sets log level to verbose. After entering the query again, he now sees that one of the regular expressions is not matching the query as expected.

Problem

How can you track program process?

Solution

Instrument your JavaScript with log messages. Typically, the page contains a div element especially for Logging, and each significant action is appended there, e.g.:

$("log").innerHTML += "User searched for " + query + ".<br/>";

Even this simple implementation offers several benefits over the common alternative, debugging with an alertbox. Most importantly, the log element can be a Popup (Chapter 15): you can easily toggle visibility by dynamically switching CSS properties such as display, or even make it partly transparent. Further, Logging is unobtrusive—that is, there's no impact on application flow. Another benefit is that you have a history to consult if something goes wrong; there's no need to try replicating the problem.

Inspired by libraries such as Java's log4j

Debugging

⊙ Break, Debug, Deduce, Fix, Inspect, Repair, Step, Test

Figure 18-3. Debugging

Developer Story

Devi is scratching her head, wondering why a particular visual effect is so jerky; instead of growing gradually, the icon transitions from small to large in just a few, erratic, steps. Code analysis is getting her nowhere, so she fires up her JavaScript debugger, which allows her to step through the effect in her own time. After adding the box dimensions to a watchlist, she quickly diagnoses the cause as a number-rounding issue.

Problem

How can you diagnose errors and strange behavior?

Solution

Diagnose problems with a JavaScript debugger. JavaScript Debugging used to be as sophisticated as adding a few alert messages, but it has come a long way. A tool such as the Venkman debugger (http://www.mozilla.org/projects/venkman/

DOM Inspection

⊙ DOM, Dynamic, Explore, Inspect, Interrogate, Investigate, Presentation, Render, Snapshot, State, View, Visualise, Visualize

Figure 18-6. DOM Inspection

Developer Story

Devi keeps clicking on the Clear button, but the document text remains stubbornly unaffected. She fires up a DOM Inspection tool and discovers that the button's onclick event handler isn't present, which explains why nothing was happening.

Problem

How can you inspect the DOM's state at any time, taking into account DOM manipulations that may have occurred since the page was loaded?

Solution

Use a DOM Inspection Tool to explore the dynamic DOM state. The initial HTML for an Ajax App is often minimal and, in any event, likely to change over time due to DOM Manipulation. A DOM Inspection tool traverses the browser's current DOM model, rather than the initial HTML source, to build up a presentation of the current DOM. Many times, the tool is interactive, allowing you to drill down the hierarchy, search for keywords, and so on. All of this is very useful for checking assumptions and diagnosing problems, since many Ajax bugs arise because the programmer misunderstood the DOM state at a particular time.

Incidentally, DOM Inspection is useful beyond the fact that the DOM is dynamic. Even for static applications, an interactive exploration tool is often a good complement to the mound of text that makes up a complex HTML document.

Tool Support

Firefox DOM Inspector

Firefox ships with a DOM Inspector tool, accessible from the tools menu (Figure 18-7).[*] The inspector is a popup window showing the DOM hierarchy on one side and facing a summary of the selected node. The summary shows node attributes such as registered event handlers, as well as CSS style information.

Traffic Sniffing

⊙ Capture, Intercept, Log, Monitor, Network, Record, Sniff, Traffic

Figure 18-12. Traffic Sniffing

Developer Story

Dave hears a report that critical form data isn't being saved on the database. He turns a sniffer on and tries uploading the same data himself. The sniffer shows him an XML message being uploaded via XMLHttpRequest. When inspecting the message in an XML editor, he discovers that the JavaScript hasn't formatted it correctly.

Problem

How can you diagnose errors and strange behavior?

Solution

Diagnose problems by sniffing Web Remoting traffic. Many Ajax problems arise because Web Remoting (Chapter 6) messages are either wrong or are not sent at all. Debugging (see earlier in this chapter) and general Logging on either side of the network might help you infer what's being transmitted, but it's better to grab the traffic directly. There are various ways to do this:

In the browser, use XMLHttpRequest Calls (Chapter 6) in a wrapper that logs the request and response (see Logging earlier in this chapter). This may be the most convenient technique, because the programmer is probably already looking at the browser. The downside is that you'll miss out on non-Web Remoting (Chapter 6) traffic, e.g., full page refreshes.

Chapter 19. Testing

THANKS TO THE AGILE MOVEMENT, TESTING IS NO LONGER SEEN AS THE BIT AT THE END WHERE QA gives its stamp of approval. It's a core development technique, and if you're creating complex web apps, it's worth testing within and across all tiers.

Because of remoting, Ajax browser scripts usually rely on web services. But it's not much fun working on both sides at the same time, so Simulation Service explains how to create a "dummy" or "stub" web service to call from the browser. You can use it for tests or for prototyping. Browser-Side Test is simply a test of the JavaScript code sitting in the browser. The hardest thing is to exercise it in an automated fashion; the pattern also looks at that issue. Service Test involves no JavaScript, because it actually takes the place of the browser; an HTTP client is used to test the web service. System Test effectively simulates a user to exercise the application across all layers.

Simulation Service

○ Canned, Double, Dummy, Fake, Masquerade, Mock, Service, Shunt, Sim, Simulation, Stub

Figure 19-1. Simulation Service

Developer Story

Dave is beginning to work on a new email application. Eventually, the server will need to expose a mail API, so the browser can query for recent messages and upload new ones. However, that will require email expertise, and Dave wants to focus on the user interface in the browser for now. Thus, he hardcodes the responses he'd like in a "fake" server-side service. This allows him to proceed with user-interface development while his colleague takes care of the real server implementation.

Problem

How can you develop the browser script in parallel with the services it uses?

Solution

Develop the browser application against "fake" web services that simulate the actual services used in production. A Simulation Service is typically a hardcoded response that ignores any input parameters. So instead of:

$taxRate = calculateTaxRate($userId, $year); print $taxRate;

a Simulation Service would just say:

print "0.1235";

It's also possible to do all this in a browser. If you use a library to perform Web Remoting (Chapter 6), you could make the implementation produce a hardcoded response, and ignore the server altogether.

In some cases, you might make the service more flexible; e.g., it can expose an operation to let the browser tell it what to output later on.

Simulation Service decouples the server and browser components of the application. Once the interface is defined, each can be developed in parallel. Moreover, this pattern paves the way for an agile process for building the application, based loosely on concepts such as Test-Driven Design (TDD)

Browser-Side Test

⊙ Automated, IntegrationTest, JavaScript, Regression, UnitTest, TDD, Test

Figure 19-2. Browser-Side Test

Developer Story

Devi is about to embark on upgrading a blog reader and realizes she will be tinkering with the JavaScript RSS parser. The parser will need some refactoring, but fortunately there's already a JavaScript unit test for it. She proceeds to refactor the parser, ensuring the test still passes, and continues to enhance the test as she builds in further functionality.

Problem

How can you test JavaScript components?

Solution

Create automated tests of browser-side JavaScript components. This pattern is a mapping from automated testing in other languages, where frameworks like the xUnit series (http://en.wikipedia.org/wiki/XUnit—e.g., JUnit; see http://junit.org—are now popular. Several frameworks do the same for JavaScript. As with other languages, you can unit test an individual JavaScript component (e.g., a function) you've created, and also perform integration tests and system tests that work across several objects. And also like other languages, you can accumulate tests into a test suite object, so it's possible to run a full regression test.

This pattern is heavily influenced by the "xUnit" testing frameworks (http://en.wikipedia.org/wiki/XUnit), which includes frameworks like JUnit for Java, PHPUnit for PHP, and NUnit for .Net. If you haven't used any of these before, it's worth learning the basic principles involved in xUnit-style testing, as these apply universally to all xUnit frameworks including the JavaScript ones. Kent Beck's Test-Driven Development By Example (Addison Wesley) is an excellent overview.

The test is usually some JavaScript that performs some task and includes assertion to check the state is as expected. Usually, JavaScript frameworks include a Test Runner that will run a given set of tests, then morph the DOM to show results in a table. You usually build an HTML page, with test code embedded, and perhaps some inputs to control the test; e.g., Start and Stop buttons.

Service Test

⊙ Automated, JavaScript, Remoting, UnitTest, TDD, Test, WebService, XMLHttpRequest

Figure 19-4. Service Test

Developer Story

Dave is creating a trading application. On the server, there are services to give quotes and accept orders. To help develop and test the web service, he builds a suite of Service Tests. One test, for example, submits an order and checks that the database registered it. These tests are automated and re-executed every time the project changes.

Problem

How can you test web services?

Solution

Build up automated tests of web services, using HTTP clients to interact with the server as the browser normally would. The browser-server interaction of an Ajax App is usually a sequence of XMLHttpRequest Calls (Chapter 6). That's fortunate, because testing the service is usually as easy as simulating the browser—passing a certain request and checking the response. Sometimes, verification might also involve checking what happened on the server, e.g., a database state change or a message to an external system.

System Test

⊙ Automated, FunctionalTest, JavaScript, PerformanceTest, Regression, Robot, UnitTest, TDD, Test

Figure 19-5. System Test

Developer Story

Devi has completed the first cut of a new Ajax App and wants to build an automated test of the whole thing. With the help of a testing application, she records a typical usage so it can be played back later. The software takes note of the final state, which will be used for verification during automated playback.

Problem

How can you test an entire Ajax App?

Solution

Build automated tests to simulate user behavior and verify the results. A System Test exercises a complete system in much the same way as a user would. The most common category of system test is a functional test—verifying the functionality—but system testing also encompass qualities such as performance and robustness. While they are often conducted manually, with users clicking and mousing around, it's highly preferable to automate as much as possible in order to promote continuous integration.

Automated system tests remain somewhat difficult, but tools are improving. They can be broadly categorized as follows:

Browser Controller (Figure 19-6)

A specialized, "robot"-style testing tool will fire up the browser, then use the underlying operating system's API to create the effect of a user interacting with the browser. Several commercial controllers are able to record user actions for later playback.

Figure 19-6. Browser Controller

Embedded Browser Script (Figure 19-7)

Run the application in a standard browser and embed a Browser-Side Test on the page. The test can use some JavaScript trickery to simulate user activity. Don't let the browser aspect put you off, because it's feasible to have a continuous integration tool like Ant coordinate Browser-Side Tests, as pointed out earlier in this chapter.

Figure 19-7. Embedded Browser Script

Browser Simulation (Figure 19-8)

Part VI. Appendixes

There are several appendixes.

Appendix A, Ajax Frameworks and Libraries, is a summary of frameworks and libraries supporting Ajax, ranging across many programming languages and licensing models. Appendix B, Setting Up the Code Examples, offers, instructions for setting up the tutorial and demo code on your system. Appendix C, Patterns and Pattern Languages, provides a background on patterns, pattern languages, and Ajax Patterns in that context. Finally, Appendix D, References, lists the texts that are referenced throughout the book.

Appendix A: Ajax Frameworks and Libraries

Appendix B: Setting Up the Code Examples

Appendix C: Patterns and Pattern Languages

Appendix D: References

Appendix A. Ajax Frameworks and Libraries

An explosion of libraries and frameworks has been one of the fortunate consequences of the Ajax phenomenon. They come in all shapes and sizes and in any licensing model you'd care to name. Throughout the Ajax Patterns, the importance of using frameworks and libraries has been emphasized—it's worth knowing the low-level details, but often not worth implementing them yourself. For this reason, the listing here makes a good companion to the Ajax Patterns.

The initial list was published back in June 2005, when I documented what seemed like an overwhelming 20 products. Maintained in an openly editable wiki, it's blossomed to 160-odd products, and doubtless many still to come—stay tuned to http://ajaxpatterns.org/Ajax_Frameworks for the latest updates.

Some of these were around long before Ajax was "Ajax," but all are here because they support Ajax development. This section is a lightly edited snapshot of the wiki in January 2006—many of the descriptions come straight from project owners and users. It begins with two sections on pure JavaScript frameworks, where you have to provide your own server-side integration. Following that are frameworks for a large number of server-side environments, many including some JavaScript library API as well as a server-side API. Note that there's a multilanguage section, since some frameworks address multiple server-side environments. Therefore, consult the multilanguage section in addition to the section for any particular server-side environment you're interested in.

People getting into Ajax often ask me for a "quick-fix" framework to help get started. So at risk of offending many worthy projects, each section points out a few of the frameworks that have generated buzz. This is, of course, a subjective measure, and it is one of popularity more than intrinsic value; do investigate further for any serious development work. In addition, look at the patterns, many of which discuss specific frameworks.

JavaScript Multipurpose Frameworks

These are pure-JavaScript libraries used to achieve a range of functionality in the browser—usually in a manner that protects you from dealing with browser compatibility issues, as described in Cross-Browser Component (Chapter 12). The Prototype framework has been very popular for general JavaScript infrastructure, and many frameworks mentioned here build on it. At a higher level, Dojo and Scriptaculous are probably the two best known products here and are good options if you're looking for a multipurpose library. Along those lines, Mochikit and OpenRico have also been popular as well. In the commercial space, there's been a lot of interest in Backbase and Tibco General Interface, though the products here are so varied, it really depends where you look.

AjaxFace (from 2005)

AjaxFace (http://www.vertexlogic.com) is an Ajax, JavaScript UI framework.

High-level JavaScript API for building UI

Open architecture for updating and retrieving data from the backend

Automatic synchronization of data and UI

Easy extensibility

High performance

Easy customization of look-and-feel through well-defined stylesheets

Online demo (http://www.vertexlogic.com/demo.html)

Commercial license by VertexLogic

Backbase (from 2003)

Backbase (http://www.backbase.com) is a comprehensive browser-side Ajax framework with cross-browser support. The Backbase Ajax framework is compatible with any server-side platform (e.g., Java, .NET, PHP, or Coldfusion).

Free community edition for noncommercial/noninstitutional use

Advanced tooling for .NET developers—Visual Studio 2005 plugin

Advanced tooling for Java developers—JSF taglib/Eclipse integration

Over 80 Ajax widgets/components at Backbase Explorer (http://www.backbase.com/demos/explorer/)

Active Ajax developer community at Backbase DevNet (http://www.backbase.com/#dev/home.xml)

Online Ajax demos/starter kits at Check Out (http://www.backbase.com/#home/pulldown/demos.xml)

Commercial license by Backbase (formed in 2003)

Bindows (from 2003)

Bindows (http://www.bindows.net) is a software development kit (SDK) that generates highly interactive Internet applications with richness that rivals modern desktop applications using the strong combination of DHTML, JavaScript, CSS, and XML. Bindows applications require no downloads and no installation on the user side—only a browser is required (no Java, Flash, or ActiveX are used). Bindows is probably the leading object-oriented platform for developing Ajax applications.

The Bindows Framework provides you with the following:

A class-based, object-oriented API

A complete windowing system with a wide array of supported widgets including menus, forms, grids, sliders, gauges, and more

The leading toolkit for developing zero-footprint SOA client-side applications

Native XML, SOAP, and XML-RPC support

Single-user to enterprise-level support

Built-in support for Ajax

The Bindows development environment includes:

Enterprise scale projects support

Cross-browser, cross-platform support

Server independent architecture

Interoperability with new and existing resources

Uniform development methodology

Commercial license, by MB Technologies (based in GA, USA, with the main development center in Sweden, formed in 2002)

Dojo (from September, 2004)

Dojo (http://dojotoolkit.org/) offers comprehensive widget and browser-server messaging support:

Extensive deployment support such as dependency-based packaging and compression of required libraries into a single download (see On-Demand JavaScript [Chapter 6]).

Framework for creation of custom JavaScript widgets

Library of pre-built widgets

Solid drag-and-drop, effects, and generic animation support

Browser-server messaging support such as XMLHttpRequest and other mechanisms

Event management (see Distributed Events [Chapter 10])

Support for bookmarkability and manipulating URLs in the browser

Open source license (Academic Free License 2.1); led by Alex Russell (http://alex.dojotoolkit.org/) of JotSpot (http://www.jot.com/)

eBusiness Applications (EBA) Ajax Components (from 2002)

EBA (http://www.ebusiness-apps.com/) offers self-contained Ajax components enabling data entry and lookup:

JavaScript Remoting Frameworks

As many of the Ajax Patterns establish, Web Remoting (Chapter 6) is error-prone, tedious, and broad in functionality, which makes it ideal to encapsulate in a library or framework like those covered here. The multipurpose frameworks covered earlier will do it too, but if you're looking for a standalone simple option, SACK is a popular choice. HTMLHttpRequest is a good option for compatibility as it supports IFrames as well as XMLHttpRequest. A more feature-rich API is offered by Ajax Client Engine (ACE).

AjaxCaller (from May 2005)

AjaxCaller (http://ajaxify.com/run/Lib/js/ajaxCaller.js) is a basic, threadsafe wrapper around XMLHttpRequest that is mainly for Ajax newcomers packaged with the Ajax Patterns demos. See TestAjaxCaller (http://ajaxify.com/run/testAjaxCaller).

RESTful calls to the server (GET/POST/PUT/DELETE) with plain-text or XML routed to a callback operation

Aimed at Ajax newcomers—instead of optimizing performance or footprint, the library aims to be a readable code base and provides debugging support

Open source license (Creative Commons) by Michael Mahemoff (http://softwareas.com), with some ideas from John Wehr and Richard Schwartz

Ajax Client Engine (ACE) (from December 2005)

Ajax Client Engine (http://www.lishen.name) is a powerful remoting wrapper.

Object-oriented API

Cross-browser support

Flexible features: request options; request parameter validation; callback argument; callback option; callback timeout; tracing service; caching service; polling service; common callbacks; and exception handling

Open source license (MIT) by Li Shen

AjaxGear (from November 2005)

AjaxGear (http://www.ajaxgear.com) is a simple and cross-platform Ajax toolkit.

Enables a web browser to make asynchronous call to the web server without the need to refresh the whole page.

Use the Ajax class to communicate with the web server.

A complete web site is available for download to see AjaxGear in action.

Use the AjaxGear.PageEngine.js file to learn how a web site can easily use the toolkit.

Three objects are currently being developed as part of the AjaxGear toolkit—namely, progress bar, autocomplete, and form validator.

JavaScript Effects Frameworks

Fade Anything Technique (FAT)

FAT (http://www.axentric.com/posts/default/7) handles the Yellow Fade Technique (see One-Second Spotlight [Chapter 16]).

Open source license (stated as Creative Commons in the page comments) by Adam Michela

Moo.fx (from Oct 2005)

Moo.fx (http://moofx.mad4milk.net) builds on a lite version of the Prototype library to provide an extremely compact effects package.

Visual effects

Cookie memory for effects

36 KB total file size

Open source license (MIT) from Volerio Proietti

JavaScript Flash Frameworks

The Richer Plugin pattern points out how Flash can benefit Ajax development; these frameworks support Ajax-Flash crossovers.

AMFPHPKit

AMFPHP (http://amfphp.sourceforge.net/) is an open source implementation of the Flash Remoting framework.

JavaScript XML Frameworks

These libraries support patterns such as XML Message, Browser-Side XSLT, and XML Data Island.

Google AJAXSLT (from June 2005)

Google AJAXSLT (http://goog-ajaxslt.sourceforge.net/) is a JavaScript framework for performing XSLT transformations, as well as XPath queries.

Builds on Google Map work

Open source license (BSD) by Google

Sarissa (from February, 2003)

Sarissa (http://sarissa.sf.net) is a JavaScript API that encapsulates XML functionality in browser-independent calls.

Portable XMLHttpRequest creation

Portable XPath queries

Portable DOM manipulation

Portable XSLT

Portable serialization to XML

Open source license (GPL 2.0 or LGPL 2.1, your choice) by various contributors

JavaScript Specialized Frameworks

These are a couple of useful libraries that don't fit well in other sections.

Drag-Drop

Walter Zorn's Drag-Drop library (http://www.walterzorn.com/dragdrop/dragdrop_e.htm) is a well-featured drag-and-drop library (see also Drag-And-Drop [Chapter 15]).

Open source license (LGPL 2.1 or later) by Walter Zorn

Giant-Ass Image Viewer (GSV)

GSV (http://mike.teczno.com/giant/pan/) lets you place a massive image on the server and allow the user scroll around as if it was all stored locally (like scrolling around Google Maps; see Virtual Workspace [Chapter 15]).

Open source license (custom) by Michael Migurski

Multilanguage Ajax Frameworks

Each library here integrates browser-side JavaScript with several server-side languages. Usually, an individual project will only need to integrate with one server-side language, but the library includes several to make it applicable to a broader market. JSON has had a big impact, because it is as much a popular data format as it is a collection of open source libraries for remoting. SAJAX has also received considerable attention in this area, in part because it supports so many languages. CPAINT is popular as well.

Cross-Platform Asynchronous INterface Toolkit (CPAINT) (from May 2005)

CPAINT (http://cpaint.sourceforge.net/) is a true Ajax implementation and a JSRS (JavaScript Remote Scripting) implementation that supports both PHP and ASP/VBScript. CPAINT provides you the code required to implement Ajax and JSRS on the backend, while the returned data is manipulated, formatted, and displayed on the frontend in JavaScript. This allows you to build web applications that can provide near real-time feedback to the user. See Ajax Stub (Chapter 9).

Supports both PHP and ASP

Unified JavaScript file for all functions

Supports both remote scripting and XML

Supports both local and remote functions

Single or multiple XMLHTTP objects

Returns backend data as text or as a JavaScript XML/DOM document object

Can support both POST and GET requests

Backend proxy functions for accessing remote functions and data

Tested with all major browsers

C++ Ajax Frameworks

Wt

Wt (http://witty.sourceforge.net/) is a C++ widget library, which uses Ajax to render changes to the widget tree when available.

API inspired by existing GUI libraries. It targets C++ application developers.

The widget library makes complete abstraction of the underlying technology (JavaScript, HTML, Forms, CGI, Ajax).

Documentation is at http://jose.med.kuleuven.ac.be/wt/doc/index.html.

Open source license (Affero General Public License) by Koen Deforche.

ColdFusion Ajax Frameworks

AjaxCFC

AjaxCFC (http://www.robgonda.com/blog/projects/ajaxcfc/

.NET Ajax Frameworks

In the .NET world, all eyes are on Microsoft with their in-progress Atlas framework. It's not yet clear exactly what it covers, how portable it will be, and how it will affect the existing frameworks here. In any event, Michael Schwartz's Ajax.NET is one popular choice, and for a broader-scoped framework, the Monorail project—similar to Ruby on Rails—has a lot of interest too.

Ajax.NET for ASP.NET 1.x/2.0

Ajax.NET (http://ajax.schwarz-interactive.de/csharpsample/default.aspx) is an Ajax framework for ASP.NET 1.x/2.0.

A basic Ajax library for ASP.NET that provides the very basic Ajax capability to make XMLHTTP callbacks. Does not have any Ajax-enabled controls or support for Viewstate, etc.

Open source license by Michael Schwartz.

Anthem.NET for ASP.NET 1.x/2.0

Anthem.NET (http://anthem-dot-net.sourceforge.net/) is Jason Diamonds' new version of a long-established Ajax library, now a SourceForge open source project.

Athem.NET provides a set of Ajax-enabled controls inherited from ASP.NET server controls, with most Ajax behaviors pre-built, avoiding lots of client JavaScript. It supports Viewstate so that you can actually modify server controls (either Anthem controls or ASP.NET controls inside an Anthem control—e.g., panel) in your server-side code, in C# or VB without having to mess about in JavaScript. Still in its early days (DEC 2005) but promising!

Open source license by Jason Diamonds.

AjaxAspects

AjaxAspects (http://www.mathertel.de/AJAXEngine/) is an engine that uses JavaScript proxies to call server-side web service methods.

Use Ajax with web services.

Built upon JavaScript client proxy methods for standard web services. (Just call a regular JavaScript method to call a method on the server.)

Standard SOAP and WSDL is reused for the communication between client and server.

No need for special implementations in web services.

Multiple types, arrays and XML objects are supported as parameters and return values.

Caching on the client and server.

Queuing actions.

Delaying actions.

Many Ajax controls available that integrate in standard ASP.NET web forms.

Supporting ASP.NET 2.0.

Full source code available.

Demo web site available (http://www.mathertel.de/AjaxEngine/).

Supported by a blog in English (http://ajaxaspects.blogspot.com/) and German (http://ajaxaspekte.blogspot.com/).

Java Ajax Frameworks

Java has always had many wide and varied frameworks. With Ajax, we've seen new ones emerge, and we've also seen add-ons for the earlier projects. Two popular libraries have been the Ajax JSP Tag Library and Direct Web Remoting.

AjaxAnywhere (from September 2005)

AjaxAnywhere (http://ajaxanywhere.sourceforge.net) returns any set of existing JSP, JSF, Struts, Spring, etc., components into Ajax-aware components without JavaScript coding. (see the Quick Start Demo at http://ajaxanywhere.sourceforge.net/quickStart.html).

Does not break existing server-side MVC architecture.

Less JavaScript to develop and maintain. Absence of commonly accepted naming conventions, formatting rules, and patterns makes JavaScript code messier then Java/JSP. It is extremely difficult to debug and unit-test in multibrowser environments. AjaxAnywhere eliminates all these complexities.

Easy to integrate. AjaxAnywhere does not require changing the underlying application code.

Graceful degradation. Switch whenever you need to between Ajax and the traditional (refresh-all-page) behavior of your web application. Your application can also support both behaviors.

Open source license (Apache 2).

AJAX JSP Tag Library

The AJAX JSP Tag Library (http://ajaxtags.sourceforge.net/) is a set of JSP tags that simplify the use of Asynchronous JavaScript and XML (Ajax) technology in JavaServer Pages. This library eases development by not forcing J2EE developers to write the necessary JavaScript to implement an Ajax-capable web form. The tags are:

Autocomplete

Retrieves a list of values that matches the string entered in a text form field as the user types.

Callout

Displays a callout or pop-up balloon, anchored to an HTML element with an onclick event.

Select

Based on a selection within a drop-down field, a second select field will be populated.

Toggle

Switches a hidden form field between true and false and at the same time switches an image between two sources.

Update Field

Updates one or more form field values based on response to text entered in another field.

Open source license by multiple contributors.

AJAX Java Server Faces Framework

AJAX-JSF (http://smirnov.org.ru/en/ajax-jsf.html) will make any existing Java Server Faces applications use Ajax functionality. Most existing components can be used as is or easily converted to support Ajax.

See a worked example called MyFaces JSF Tree, with table scroller and a tabbed pane as Ajax components (http://smirnov.org.ru/myfaces-ajax/ajax.jsf).

Minimal differences from JSF specifications. This is proposal to the MyFaces project.

Open source license (Apache Software License 2.0) by Alexander Smirnov.

Direct Web Remoting (DWR) (from 2005)

Direct Web Remoting (DWR)(http://www.getahead.ltd.uk/dwr/) is a framework for calling Java methods directly from JavaScript code.

Like SAJAX, can pass calls from JavaScript into Java methods and back out to JavaScript callbacks

Can be used with any web framework: Struts, Tapestry, etc.

Follows Spring-like KISS/POJO/orthogonality philosophy

Being incorporated into next WebWork release (http://www.opensymphony.com/webwork/)

Open source license (Apache see http://www.apache.org/LICENSE.txt) by Joe Walker (http://www.getahead.ltd.uk/sg/space/joe/)

Echo 2 (from March 2005)

Lisp Ajax Frameworks

CL-Ajax

CL-Ajax (http://cliki.net/cl-ajax) directs JavaScript calls directly into server-side Lisp functions.

The export function script is as follows:

(export-function #'my-function)

Generates a JavaScript stub with arguments.

Can callback to a JavaScript function or DOM object.

May be integrated into SAJAX.

Open source license (custom, very flexible, license) by Richard Newman (http://www.holygoat.co.uk/).

Perl Ajax Frameworks

CGI::Ajax—Export Perl Methods to JavaScript for Ajax

CGI::Ajax (http://pjax.sourceforge.net/

PHP Ajax Frameworks

PHP frameworks tend to focus more on remoting than HTML generation. XOAD is one popular remoting library, as well as CPAINT and SAJAX (covered earlier in "JavaScript Multipurpose Frameworks").

AJASON

AJASON (http://ajason.sourceforge.net/) is a PHP-based framework.

Uses JSON (JavaScript Object Notation) to encode/decode data between server and browser

Open source license by multiple contributors

AjaxAC (from April, 2005)

AjaxAC (http://ajax.zervaas.com.au/) encapsulates the entire application in a single PHP class.

All application code is self-contained in a single class (plus any additional JavaScript libraries).

Calling a PHP file or HTML page is very clean. All that is required is creating the application class, and then referencing the application JavaScript and attaching any required HTML elements to the application.

Built-in functionality for easily handling JavaScript events.

Built-in functionality for creating subrequests and handling them.

Allows for custom configuration values, so certain elements can be set at runtime.

No messy JavaScript code clogging up the calling HTML code—all events are dynamically attached.

Easy to integrate with templating engine.

Easy to hook in to existing PHP classes or MySQL databases for returning data from subrequests.

Extensible widget structure to be able to easily create further JavaScript objects (this needs a bit of work though).

Open source license (Apache 2.0) by Zervaas Enterprises (http://ajax.zervaas.com.au/).

Cajax

Cajax (http://sourceforge.net/projects/cajax) is a PHP object-oriented framework that creates and handles server-side requisitions though JavaScript remote calls. Some features are:

Simple server-side programming

Almost no client-side programming (less JavaScript writing is possible)

Handler for server-side events

A suggest handler like Google's

Plugin for multiple selects (like country/state/city)

Python Ajax Frameworks

CrackAJAX

CrackAJAX is a Python framework (http://www.aminus.org/blogs/index.php/phunt/2005/10/06/subway_s_new_ajax_framework) that does not require JavaScript skills.

Open source license by Peter Hunt

Turbo Gears

Turbo Gears (http://turbogears.org) is a "mega-framework" combining many libraries and frameworks.

Open source licence (MIT license) and multiple licenses from incorporated frameworks by multiple developers

Ruby Ajax Frameworks

Ruby On Rails

Ruby On Rails (http://www.rubyonrails.org/) is a general web framework with strong Ajax support.

Rails was still in its early days when Ajax hype began, so Ajax may become increasingly important to the Rails framework.

Generates most or all of the JavaScript for widgets and animation in the browser.

Supports calling server side.

Provides scheduling support.

Open source license [MIT or similar (http://wiki.rubyonrails.com/rails/show/License) by David Heinemeier Hansson and contributors (http://www.loudthinking.com/).

Appendix B. Setting Up the Code Examples

A SINGLE PACKAGE CONTAINING ALL CODE FOR CHAPTER 2 AND THE AJAX PATTERNS DEMOS AS WELL IS at http://ajaxify.com/run. Inside there is a INSTALL.txt file containing installation instructions. These instructions are reprinted below, but if you have any trouble, please do check the electronic version as it may include updates.