THERE'S AN OLD JOKE THAT COMPUTERS ARE ACTUALLY EASY MACHINES TO SECURE: just turn them off, lock them in a metal-lined room, and throw away the key. What you end up with is a machine that is very secure, just not very usable.

Of course, people need to use computers, not just think about them. So while this secure computer is safe in its metal can, people who need to get their jobs done will use other computers with significantly weaker security properties. They may have their passwords recorded by keystroke loggers and sent to bad guys in Russia. They may go to web banking sites that happen to be run by illegal cartels in South America. They may use portable laptops that are targeted and stolen at trade shows. And when they are done, they may format their hard drives and throw them away—unaware that their computer's "format" command doesn't delete any data at all.

Audience for This Book

In creating the first book to be focused entirely on the subject of usability and security, we had a difficult decision to make. Did we want an academic book, one focusing on the growing amount of research in this burgeoning field, or did we want a book for practitioners, one with a heavy emphasis on practice and many recommendations for specific actions?

Structure of This Book

This book is divided into 6 parts consisting of 34 chapters.

Part I, Realigning Usability and Security

In this part of the book, we state our premise: that security and usability can be synergistic. The chapters in this part argue that, with careful attention to user-centered design principles, significant progress can be made toward this goal:

Chapter One, Psychological Acceptability Revisited, by Matt Bishop, takes a new look at the question of how to align security and usability: although the need to consider usability in the design of security systems is recognized more now than it was in the past, designers still need to create systems that are easy to install, provide adequate protection mechanisms, and are unobtrustive to use. This is a solvable problem, and there is much work to do.

Chapter Two, Usable Security, by M. Angela Sasse and Ivan Flechais, lays the groundwork for our volume. It argues that the actual security provided by a computer system is the product of human factors, policies, and security mechanisms. Ignore any one of them, and security suffers.

Chapter Three, Design for Usability, by Bruce Tognazzini, states a truism that is ignored all too frequently: the goal of computer security professionals must be to build systems that are actually secure, rather than to build systems that are theoretically secure. Many security "compromises" in the interest of usability aren't compromises at all—they are frequently improvements, because the systems that are "theoretically secure" are so hard to use that people avoid or sabotage them in practice.

Chapter Four, Usability Design and Evaluation for Privacy and Security Solutions, by Clare-Marie Karat, Carolyn Brodie, and John Karat, introduces tools for performing usability evaluations and shows how they can integrate into the product development life cycle. The chapter then describes how these tools were applied to two different security products at IBM.

Chapter Five, Designing Systems That People Will Trust, by Andrew S. Patrick, Pamela Briggs, and Stephen Marsh, examines the issue of trust for security and privacy systems. The interface with which the end user interacts plays a central role in building or breaking that trust. It is the interface—whether it is a computer screen, a web site, a standalone kiosk, or a telephone system—that must convey all the features and limitations of the underlying service to the user. The authors show how successful trust designs can have a positive impact on both products and services.

Part II, Authentication Mechanisms

The chapters in this part of the book take an in-depth look at techniques for identifying and authenticating computer users to systems that are both local and remote:

Chapter Six, Evaluating Authentication Mechanisms, by Karen Renaud, considers the range of authentication systems that are currently available and presents a framework for evaluating their strengths and weaknesses.

Chapter Seven, The Memorability and Security of Passwords, by Jeff Yan, Alan Blackwell, Ross Anderson, and Alasdair Grant, presents the results of a study of password usage among university students. The study finds that some conventional wisdom given in the choice and maintenance of passwords is correct, and other advice is "bunk."

Chapter Eight, Designing Authentication Systems with Challenge Questions, by Mike Just, considers the role of questions like "what is your mother's maiden name" and "who was your favorite teacher" for authenticating users. Challenge questions can be used very effectively for self-service password resetting and as an additional identifier—especially on systems that are rarely used. On the other hand, a poorly implemented challenge system can compromise security while simultaneously decreasing usability. Once again, careful design and analysis are required for favorable outcomes.

Chapter Nine,

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Used for URLs, file and directory names, emphasis, and the first occurrence of terms

Constant width

Used for code examples and literals

Safari Enabled

When you see a Safari® Enabled icon on the cover of your favorite technology book, that means the book is available online through the O'Reilly Network Safari Bookshelf.

Safari offers a solution that's better than e-books. It's a virtual library that lets you easily search thousands of top technical books, cut and paste code samples, download chapters, and find quick answers when you need the most accurate, current information. Try it for free at http://safari.oreilly.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

O'Reilly Media, Inc.1005 Gravenstein Highway NorthSebastopol, CA 95472(800) 998-9938 (in the United States and Canada)(707) 829-0515 (international/local)(707) 829-0104 (fax)

There is a web page for this book, which lists errata and any additional information. You can access this page at:

http://www.oreilly.com/catalog/securityusability/

To comment or ask technical questions about this book, send email to:

bookquestions@oreilly.com

For more information about books, conferences, software, Resource Centers, and the O'Reilly Network, see the O'Reilly web site at:

http://www.oreilly.com

Acknowledgments

Part I. Realigning Usability and Security

Chapter One. Psychological Acceptability Revisited

Matt Bishop

IN 1987, BRIAN REID WROTE "PROGRAMMER CONVENIENCE IS THE ANTITHESIS OF SECURITY, because it is going to become intruder convenience if the programmer's account is ever compromised."[*] This belief of the fundamental conflict between strong computer security mechanisms and usable computer systems pervades much of modern computing. According to this belief, in order to be secure, a computer system must employ security mechanisms that are sophisticated and complex—and therefore difficult to use.

Today a growing number of security researchers and practitioners realize that this belief contains an inherent contradiction. The reason has to do with the unanticipated result of increasing complexity. A fundamental precept of designing security mechanisms is that, as the mechanisms grow more complex, they become harder to configure, to manage, to maintain, and indeed even to implement correctly. Errors become more probable, thereby increasing the chances that mechanisms will be configured erroneously, mismanaged, maintained improperly, or implemented incorrectly. This weakens the security of the system. So the more complex a system is, the more secure it should be—yet the less secure it is likely to be, because of the complexity designed to add security!

Finding ways to maximize both the usability of a system and the security of a system has been a longstanding problem. Saltzer and Schroeder's principle of psychological acceptability [2] (see the sidebar) says that a security mechanism should not make accessing a resource, or taking some other action, more difficult than it would be if the security mechanism were not present. In practice, this principle states that a security mechanism should add as little as possible to the difficulty of the human performing some action.

THE PRINCIPLE OF PSYCHOLOGICAL ACCEPTABILITY

"It is essential that the human interface be designed for ease of use, so that users routinely and automatically apply the protection mechanisms correctly. Also, to the extent that the user's mental image of his protection goals matches the mechanisms he must use, mistakes will be minimized. If he must translate his image of his protection needs into a radically different specification language, he will make errors."

Jerome Saltzer and Michael Schroeder

Applying this principle raises a crucial issue: difficult for whom? A programmer may find setting access control permissions on a file easy; a secretary may find the same task difficult. Applying the principle of psychological acceptability requires taking into account the abilities, knowledge, and mental models of the people who will use the system. Unfortunately, on those infrequent occasions when the principle is applied, the developers often design the mechanism to meet their own expectations and models of the system. These are invariably different from the expectations and models of the system's users, no matter whether the users are individuals at home or a team of system administrators at a large corporation.

As a result, security mechanisms are indeed cumbersome and less effective than they should be. To illustrate the problem, I focus on three examples in this chapter: passwords , patching, and configuration.

Passwords

Patching

Correcting problems such as default or empty passwords poses problems when the vendor distributes the correction. As an example, for many years, Microsoft's SQL Server was distributed with an empty password on its administrator account.[12] This changed when the SQLSnake/Spida Worm exploited that empty password to acquire access to servers running that database. Microsoft issued a patch to update the server and add a password.

A patch is an update to a program or system designed to enhance its functionality or to solve an existing problem. In the context of security, it is a mechanism used to fix a security problem by updating the system. The patch, embodied in a program or script, is placed on the system to be patched, and then executed. The execution causes the system to be updated.

Ideally, patching should never be necessary. Systems should be correct and secure when delivered. But in practice, even if such systems could be created, their deployment into various environments would mean that the systems would need to be changed to meet the needs of the specific environment in which they are used. So, patching will not go away. However, it should be minimal, and as invisible as possible. Specifically, the principle of psychological acceptability implies that patching systems should require little to no intervention by the system administrator or user.

Unfortunately, several considerations make invisible patching difficult.

The first difficulty is collecting all of the necessary patches . In a homogeneous network, only one vendor's patches need to be gathered, but a single vendor may offer a wide variety of systems. The patches for one system likely will not apply to another system. If the network has systems from many vendors, the problem of gathering and managing the patches is severe. Various tools such as Cro-Magnon[13] and management schemes using several tools[14] attempt to ameliorate this task. All require configuration and maintenance, and knowledgeable system administrators.

The second difficulty is system-specific conflicts. When vendors write and test a patch, they do so for their current distribution. But customers tailor the systems to meet their needs. If the tailoring conflicts with the patch, the patch may inhibit the system from functioning correctly.

Configuration

Building a secure system does not assure its security: the system must also be installed and operated securely. Configuration is a key component of secure installation and operation, because it constrains what users and the system processes can do in the particular environment where the system is used. For example, a computer configured to be secure in a university research environment (in which information is accessible to everyone inside the research group) would be considered nonsecure in a military environment (in which information is accessible only to those with a demonstrated need to know). Different configurations allow a system to be used securely in different environments.

The decisions about configuration

Conclusion

The lesson that we draw from the three illustrations provided in this chapter is that the solution to the problem of developing psychologically acceptable security mechanisms

About the Author

Matt Bishop is a professor in the Department of Computer Science at the University of California at Davis. He studies the analysis of vulnerabilities in computer systems, policy models, and formal modeling of access controls. He is active in information assurance education, is a charter member of the Colloquium on Information Systems Security Education, and has presented tutorials at many conferences. He wrote the textbooks Computer Security: Art and Science and Introduction to Computer Security (both from Addison Wesley).

Chapter Two. Why Do We Need It? How Do We Get It?

M. Angela Sasse and Ivan Flechais

SECURITY EXPERTS FREQUENTLY REFER TO PEOPLE AS "THE WEAKEST LINK IN THE CHAIN" OF SYSTEM SECURITY. Famed hacker Kevin Mitnick revealed that he hardly ever cracked a password, because it "was easier to dupe people into revealing it" by employing a range of social engineering techniques. Often, such failures are attributed to users' carelessness and ignorance. However, more enlightened researchers have pointed out that current security tools are simply too complex for many users, and they have made efforts to improve user interfaces to security tools. In this chapter, we aim to broaden the current perspective, focusing on the usability of security tools (or products) and the process of designing secure systems for the real-world context (the panorama) in which they have to operate. Here we demonstrate how current human factors knowledge and user-centered design principles can help security designers produce security solutions that are effective in practice.

Introduction

The need for people to protect themselves and their assets is as old as humankind. Peoples' physical safety and their possessions have always been at risk from deliberate attack or accidental damage. The increasing use of information technology means that individuals and organizations today have an ever-growing range of physical (equipment) and electronic (data) assets that are at risk. To meet the increasing demand for security, the IT industry has developed a plethora of security mechanisms that can be used to make attacks significantly more difficult or to mitigate their consequences.

A series of surveys has shown that—despite ever-increasing spending on security products—the number of businesses suffering security breaches is increasing rapidly. According to the United Kingdom's Department of Trade and Industry's Information Security Breaches Surveys,[1]

Product: Human Factors, Policies, and Security Mechanisms

It is unfortunate that usability and security are often seen as competing design goals in security, because only mechanisms that are used, and used correctly, can offer the protection intended by the security designer. As Bruce Tognazzini points out in Chapter Three, a secure system needs to be actually, not theoretically, secure. When users fail to comply with the behavior required by a secure system, security will not work as intended. Users fail to show the required behavior for one of the following two reasons:

They are unable to behave as required.

They do not want to behave in the way required.

Impossible Demands

The current situation with computer passwords provides a good example of the first case: most users today find it impossible to comply with standard policies governing the use of computer passwords (see Chapter Seven in this volume). Remembering a single, frequently used password is a perfectly manageable task for most users. But most users today have many knowledge-based authentication items to deal with. We have multiple and frequently changed passwords in the work context, in addition to passwords and personal identification numbers (PINs) outside work, some of which are used infrequently or require regular change. The limitations of human memory make it impossible for most users to cope with the memory performance this requires.[4] As a result, users behave in ways forbidden by most security policies:

Users write passwords down. Externalizing items we have to remember is the most common way of dealing with memory problems. In office environments, users stick notes with passwords onto their screens, or maintain a list of current passwords on the nearest whiteboard.

Similarly, many bank customers write their PINs on their cards. A less common remedy is to write or scratch the PIN on the ATM or its surroundings.

ANECDOTAL EVIDENCE

A reality TV show set in a police station in the UK featured a whiteboard behind a PC used to log the movement of prisoners with a prominent reminder: “The password is custody.”

The customer relations manager of a UK building society received irate phone calls after a major re-branding exercise, in which ATMs and the surrounding environments had been restyled. The customers did not object to the new corporate color scheme, but rather, to the fact that the panels and surroundings onto which they had written or scratched their PINs had been replaced, and as a result they were unable to withdraw cash.

Users share passwords with other users. Another common way of preventing loss of data due to the vagaries of human memory is by sharing the information widely, so if you cannot remember the password, you are likely to find a colleague who can.

Users choose passwords that are memorable but not secure (when the mechanism allows this).[5] Many users choose passwords or PINs that are memorable but easy to crack (names of spouses or favorite sports stars, birth dates, 1234, 8888).

The standard password mechanism is cheap to implement and—once recalled—executed quickly. But in the preceding examples, users are knowingly breaking the rules, and the examples give a feeling for the despair that the ever-growing number of passwords and PINs induces in many users. A key human factors principle is not to impose unreasonable demands on users ; in fact, designers should minimize the physical and, especially, the mental workload that a system creates for the user.

Frequently used passwords—that is, passwords used on a daily basis—are not a problem for the average user in an office context. Infrequently used passwords and PINs, however, can create significant problems—for instance, many people withdrawing money once a week have problems recalling a PIN. There are a number of ways in which the memory demands of passwords and PINs can be reduced:

Provide mechanisms that require users to recognize items rather than recall them. Recognition is an easier memory task than recollection, and designers of graphical user interfaces (GUIs) have applied this principle for decades now. Recognition of images[6],[7] has already been used for security mechanisms; but even text-based challenge-response mechanisms (see Chapter Eight

Process: Applying Human Factors Knowledge and User-Centered Approaches to Security Design

The process of building a secure system is vital to its effectiveness. The process is the means by which security needs are assessed, policies are elaborated, and countermeasures are designed. As with any software development project, the right mix of participants, expertise, and methodology is vital in ensuring a system that is actually secure. To achieve this, designers of secure systems need to consider that security is not the primary goal of users and organizations. The role of security is a supporting one—to protect assets and activities that users care about or that are part of the production activity of business organizations.

Security Is a Supporting Task

Two further concepts that are key to designing successful security applications are goals and tasks. Human behavior is essentially goal driven, so the effective and efficient execution of tasks that help users attain goals is a key principle for designing successful systems. Human factors analysts distinguish between production tasks (those that are required to achieve the goal or produce the desired output) and supporting tasks (those that enable production tasks to be carried out in the long run, or be carried out more efficiently, but are not essential to achieving the goal). Security—like safety—is a supporting task . Production tasks are the reason why a system exists, and if production tasks cannot be completed effectively and efficiently, the system will cease to exist. Users put production tasks first; organizations, sensibly enough, do the same from a higher-level perspective. This understanding leads us to a number of insights for security design :

Security tasks must be designed to support production tasks

Panorama: Understanding the Importance of the Environment

The environment surrounding the process of developing security is also extremely important to the effective operation of the product (security mechanism). During the design of the security, a number of factors not necessarily related to any security needs can influence the final product. The personal responsibility of participants for the resulting security, the enthusiasm of high-level management for security (even their presence in the design process), pressure to achieve functional features, time-to-market, personal agendas, ethical constraints, industrial involvement, and legal requirements—all influence security design in one way or another.[23]

The cultural panorama surrounding security does not stop affecting it after the design is complete, but continues even after it has been put to use. An analysis by Flechais, Riegelsberger, and Sasse[24] has identified the influences and mechanics of trust relationships in the operation of secure systems. In most current cases, existing trust relationships in an organization facilitate the breaking of security policies and practices. In fact, given the right (and currently widespread) environment, simply adhering to existing security policies can undermine social relationships within a group of peers. The authors argue that trust relationships are beneficial for organizations by promoting social capital [25] (i.e., trust based on shared informal norms that promote cooperation)[26] and that the organizational culture and the actual security should be designed to support both trust relationships and adherence to policy.

The Role of Education, Training, Motivation, and Persuasion

While a well-designed security mechanism won't put off users, it also won't entice them to make the extra effort that security requires. In many home and organizational contexts, users lack the motivation to make that extra effort. User education and training can be used to explain the need for the effort that security requires, but changing users' knowledge and understanding does not automatically mean they will change their

Conclusion

About the Authors

Chapter Three. Design for Usability

Bruce Tognazzini

THE GOAL OF SECURITY IS NOT TO BUILD SYSTEMS THAT ARE THEORETICALLY SECURABLE, but to build ones that are actually secure. This requires a combination of the theoretical and the practical. It requires close examination not only of the technology, but also of the human beings that will use it.

Balance Security and Usability

Balance is key to all security efforts. The same phenomenon that happens with cars happens with computers. Unless you stand over them with a loaded gun, users will disable, evade, or avoid any security system that proves to be too burdensome or bothersome. Preventing your system from becoming the victim of such "empowered" users requires a combination of engineering and education. The engineering builds security systems that are safe and usable, and the education informs users about the actual risks so that they will be motivated to use your security (or at least so that they won't disable it).

How do you find the right balance? You begin by examining and exploiting, in each new situation, the differences between the two groups.

Exploit Differences Between Users and Bad Guys

Let's return to those "marching dots" that I mentioned in my introduction. The purpose of those dots is to protect a password from "shoulder surfing" as it is being entered. In theory, if you have a strong password that's sent over an encrypted link, shoulder surfing is the main threat that passwords face.

But the user who actually types the password is in a fundamentally different position from a potential shoulder surfer. Consider:

The user knows what he is typing—he is only looking for errors.

The user is close to the screen where the password is being entered—he can read text that is printed with relatively low contrast.

The user is always present.

Now, compare that with a potential attacker:

An eavesdropper needs to accurately reconstruct every character in the password.

An eavesdropper is probably several feet away from the screen—if that.

An eavesdropper might not even be present—the user might be entering the password in the privacy of his own home, for example, or on a desert island.

You can take advantage of these differences to produce an interface that promotes complex passwords, while still leaving the eavesdropper in the dark. We did that when we designed Tresor 2.2 (see the upcoming sidebar). Alternatively, you can design an interface that allows the user to choose the amount of security that he wants when entering a password. As shown in Figure 3-1, the designers of GNU Keyring followed this approach.

Exploit Differences in Physical Location

One user may be working in an airport lounge, surrounded by people who eavesdrop from either boredom or darker motivation. Another user may be working in her private study at home, with nothing but two blank walls behind her. However, our current "one size fits all" security systems tend to ignore that difference. They arise from a single assumption: the bad guy may be standing behind you this minute!

Vary Security with the Task

The task at hand is a vital component in security decision-making. Security practitioners call this threat analysis. Different kinds of security measures are called for protecting information that is in transit versus information that is to be stored permanently on a hard drive. Sure, both data streams might be protected with 128-bit AES encryption. But in the

Tresor is a file encryption application that offers very high security. The application makes it easy to type a long "passphrase"—even for users who perpetually make typographical errors. Designed by Roland Blaser and myself, Tresor 2.2 's passphrase entry design replaces "marching dots" with a new metaphor: the rolling blackout.

Users in the light; eavesdropping in a blackout

Like most web browsers, Tresor 2.2 uses those cute little dots to replace the characters in the user's password. But in Tresor 2.2, the characters are a bit slow in acting, always revealing the last few characters for a few seconds as the user types, long enough for the user to catch a typo.

Balance Privacy and Security

We often say "privacy and security" in the same breath, but they can often be at odds with each other.

Build a Secure Internet

We are now engaged in the transition from isolated one-person computers to the concept of a personal network, enabling people to tap into their personal cyberspace from any point on the globe.

The most fundamental requirement of such a personal network is that individuals be able to maintain the most private of information without worry, no matter where their travels might take them. They further need to be able to monitor and control access of others both to their information and to themselves. Before that can occur, we will have to build a multiplicity of new, secure walls.

The original ARPANET , forerunner to today's Internet, was the exclusive domain of university scientists and the U.S. military. The world was neatly divided into "We" and "They." "We," the "good guys," had access to ARPANET. "They," everyone else in the world, did not. Security, in those days, consisted of keeping the "bad guys" from gaining physical access of any kind to the network.

DESIGN CHECKLIST

Achieving balance:

Are you exploiting the differences between users and attackers?

Users know what they are typing in—they are only looking for errors.

Eavesdroppers, meanwhile, have to reconstruct every character accurately.

Users are closer to the screen and therefore can read with lower contrast and can differentiate better between green and blue (unless they are colorblind).

Are you detecting and exploiting differences in physical location?

Home

Office

Airport and other public venues

Are you providing a way that your software can track location changes or the user can casually indicate such changes?

Are you varying security with the task?

Temporary versus archival security

"Here" versus "there" versus "en route"

"Hide from co-workers" versus "Hide from competitors"

Does your design exploit the special skills of your user population?

Does it serve to reduce the user's burden?

Authentication:

Are your passwords restrictive because of:

Low upper-limits on password length

Passwords that are system generated

Other password rules that encourage posting the password on a yellow sticky note

Is it quick and easy to obtain or replace a password?

Methodology:

Conclusion

The evidence of fatal flaws in today's security approach is all around us, gracing the edges of monitors across the world. We need to take the initiative. We need to look outward, to the way things "really work" once people are in the mix.

About the Author

Bruce "Tog" Tognazzini has been part of personal computing from the beginning. He built his first electro-mechanical computer in 1959, and was employee #66 at Apple, where he spent 14 years before going on to be a Distinguished Engineer at Sun Microsystems, then Chief Designer at Healtheon/WebMD, and finally becoming the third Principal at the Nielsen Norman Group. He publishes http://asktog.com, has written two books, has coauthored three others, and currently has 49 patents issued in HCI, automotive safety, and aviation.

Chapter Four. Usability Design and Evaluation for Privacy and Security Solutions

Clare-Marie Karat, Carolyn Brodie, and John Karat

THIS CHAPTER SHOWS HOW YOU CAN INCORPORATE USABILITY DESIGN AND EVALUATION into the life cycle of privacy and security solutions. Here we provide you with an overview of critical-path human-computer interaction (HCI) activities that occur during the development of successful solutions and their maintenance after release. We will point you to key publications and references that you can use to gain a more in-depth understanding of HCI methods and practices in evaluating the usability of your security and privacy systems. And we will walk you through the use of these methods in a case study of a security application as well as a case study of two years of research on privacy tools for authoring and implementing privacy policies.

Of course, this chapter alone cannot make you proficient in using HCI methods. But it will guide you in understanding the value that HCI work can contribute to the success of your product.

We recommend that most projects bring an HCI expert on board early in the project's development, and that the HCI expert be a full team member. Working together, the project team should define the usability activities as a critical development path. We have seen many successful examples where the HCI lead's expertise was leveraged to train and educate the product team so that team members completed parts of the HCI project plan with consultation and collaboration from the HCI expert. We have also seen several examples of skilled computer scientists and programmers who gradually became HCI specialists, frequently with great success. One of this chapter's co-authors, Carolyn, has, in fact, followed such a career path!

Usability in the Software and Hardware Life Cycle

Properly applied, HCI methods and techniques elicit and identify the useful information in each phase of the project life cycle . These activities can facilitate a more accurate and complete project definition during the requirements phase and an improved project design during the design phase. During the development phase, they can improve solution performance and reduce development time and costs. The benefits of HCI and usability activities continue after release, including increased user productivity and satisfaction, reduced training, support, and error recovery costs, increased sales and revenue, and reduced maintenance costs.

Unique Aspects of HCI and Usability in the Privacy and Security Domain

Although many HCI techniques are general, there are unique aspects in the design of privacy and security systems that present challenges and opportunities.

First, a key issue to consider is that security and privacy are rarely the user's main goal. Users value and want security and privacy, of course, but they regard them as only secondary to completing primary tasks like completing an online banking transaction or ordering medications. Users would like privacy and security systems and controls to be as transparent as possible. On the other hand, users want to be in control of the situation and understand what is happening. Therein lies the rub. As a consequence, the display of information and the interaction methods related to security and privacy need to be accessible if and when desired by the user, but they shouldn't get in the way.

Second, as more of people's interactions in daily life involve the use of computing technology and sensitive information, disparate types of users must be accommodated. Security solutions in particular have historically been designed with a highly trained technical user in mind. The user community has broadened extensively as organizational business processes have come to include new types of roles and users in the security and privacy area. Many compliance and policy roles in organizations are handled by legal and business process experts who have limited technical skills. Moreover, the end user base includes virtually everyone in the population. The functionality provided by the system for people with different roles must accommodate the skill sets of each. Security and privacy are requirements for doing business as an organization and must be done well, or the organization may lose the user as a customer—or worse.

KEY REFERENCE BOOKS IN THE HCI FIELD

Please consult these books for additional information; there is a wealth of knowledge and experience contained in them.

Bias, R., Cost-Justifying Usability: An Update for the Internet Age, 2nd Edition (London: Academic Press, 2005). This book provides methods for calculating the value of HCI work and many case studies and perspectives across a variety of domains.

Beyer, H. and Holtzblatt, K., Contextual Design (New York: Morgan Kaufmann, 1997). This practitioner guide outlines a method for identification of user requirements through unobtrusive observation of users in the field and instruction on analysis of that data for the design of usable systems.

Diaper, D. and Stanton, N., The Handbook of Task Analysis for Human-Computer Interaction (Hillsdale, NJ: Erlbaum, 2003). This handbook provides instruction on how to complete task analysis in a variety of domains and information on user tasks identified in key past research.

Helander, M., Handbook of Human Computer Interaction (Amsterdam: Elsevier, 1988). This reference book provides a synthesis of HCI theory, methods, and research in many different domains and from many perspectives. It contains seminal chapters not found elsewhere.

Helander, M., Landauer, T., and Prabhu, P. (eds.), Handbook of Human-Computer Interaction, 2nd Completely Revised Edition (Amsterdam: Elsevier, 1997). This is an update of this key reference book.

Jacko, J. and Sears, A. (eds.), The Human-Computer Interaction Handbook (Hillsdale, NJ: Erlbaum, 2003). This is a key reference book with the most up-to-date synthesis of HCI theory, methods, and research and its application to a wide variety of domains.

Mayhew, D., The Usability Engineering Lifecycle: A Practitioner's Handbook for User Interface Design (San Diego: Academic Press, 1999). This is a practical guide on how to conduct HCI work as part of the development life cycle.

Preece, J., Human Computer Interaction (Wokingham, U.K.: Addison Wesley, 1994). This book provides an introduction and overview of the HCI field. This text is used in university courses.

Rubin, J., Handbook of Usability Testing—How to Plan, Design, and Conduct Effective Tests (New York: John Wiley & Sons, Inc., 1994). This is a practical guide for conducting usability tests of systems.

Shneiderman, B. and Plaisant, C., Designing the User Interface (Reading, MA: Addison Wesley, 2004). This is a key text in the field of HCI. It provides an introduction and overview of the field for university students and others interested in the topic.

Third, the negative impact that usability problems can have is higher for security and privacy applications than for many other types of systems. Complexity is at the very heart of many security and privacy solutions; from an HCI point of view, that complexity is the enemy of success. If a system is so complex that whole groups of users (e.g., technical users, business users, and end users) cannot understand it, costly errors will occur. There is a saying in the HCI field: "If the user cannot understand functionality, it doesn't exist." In the case of security and privacy, sophisticated systems that are badly designed may actually put users at more risk than if less sophisticated solutions were used. So, the increased risk of errors in this domain provides an even greater incentive to include HCI work in system research and development.

Fourth, users will need to be able to easily update security and privacy solutions to accommodate frequent changes in legislation and regulations. Different domains (e.g., health care, banking, government) frequently have unique requirements. Systems must be designed to enable easy and effective updates to them.

Case Study: Usability Involvement in a Security Application

Clare-Marie Karat was the HCI lead on the development of a security application in 1989. She provides a firsthand account of the process she used to understand the user requirements and improve the usability of this product.[4]

The project's goal was to improve the dialog of a large data entry and inquiry mainframe application used by 23,000 end users (IBM employees in branch offices across the U.S.). This large mainframe application was composed of many subapplications. The goal was to eliminate the recurring need for security authorization while performing discrete but related tasks that composed a business process. However, the project had entered the design and development phase: coding had already begun. So, before I actually joined the team, the project manager and I agreed upon the approach that I would use. I would quickly complete the HCI activities that should have been done in the requirements phase, then move ahead with the design and development phase HCI work.

Prior to the development of this security application, working with the mainframe application was like having a series of disconnected conversations for the end user. Every time a user wanted to perform a transaction, he had to re-identify, provide a password, select the appropriate application, and type a command known as a mnemonic that indicated the type of transaction to be performed. Further, the use of the actual mnemonics was controlled: users were only told about the mnemonics relevant to their individual job responsibilities and were instructed not to write down the mnemonics or share them with anyone.

I met with the project team to understand the goals of the project and define the overall usability goal. They were ambitious. The old security application was to be taken down on a Friday, the new application was to be installed over the weekend, and employees needed to be able to walk up and use the new security application the next Monday. The transition had to be smooth, without disrupting the end users or business. End users signed on to the target mainframe application about a dozen times a day.

After interviewing key team members, I drafted the measurable usability objectives for the security application and discussed them with the team during a regular status meeting. Consensus was reached that the usability objective for the security application was that "95% of the users will complete the sign-on task error-free within the first three attempts at the task." No specific usability objective was set for the time to complete the sign-on task itself; however, the team members thought users ought to be able to complete it within about 8 seconds.

The Field Study

I went into the field (a field study of end-user work context) to several branch offices and observed the general workplace setting and the way the employees worked with the current system. I met with and talked to a number of the target users, collected necessary information, and was able to create the user profile definition. The users had an average of two years of work experience in their current jobs, and five years overall with IBM. They all had college degrees, good computer skills, and reasonable experience with the major business applications they used.

I observed their work environment and the transactions they needed to complete on the mainframe application. These users accessed the application to complete a variety of tasks and transactions related to customer relationship management, sales, and fulfillment. Employees in the branch offices were working at close quarters in open office settings with no partitions. Phones rang constantly; people carried on conversations. When sales were completed, big bells were rung to celebrate. It was a busy, exciting, fast-paced, and sometimes rather loud environment in which to work. The end users were typically attending to their terminal screens while juggling a variety of other tasks simultaneously, including communicating with other staff and customers through telephone calls and talking to people stopping by and requesting information. It was clear that the user interface to the security application needed to take these social and physical environment factors into account in the end-user requirements for the user interface and interaction methods for the security application.

Interviews and observation of the employees identified that the number of mnemonics that they needed to remember (for some employees, this number approached 60, with an average in the high 30s) was more than the human memory can handle. As might be expected, the users had devised their own methods of keeping track of the mnemonics they needed for their tasks. I conducted a task analysis to identify the key security sign-on tasks the users completed related to the various transactions they worked on each day, and then defined the set of core user scenarios that the security application would have to handle. These scenarios also identified a number of end-user requirements.

The User Tests

Over the course of the next three months, I completed three iterations of design and usability testing

Case Study: Usability Involvement in the Development of a Privacy Policy Management Tool

In 2002, we became involved in a research project concerned with the development of approaches for helping organizations protect the personal information (PI) that they used and stored about their customers, constituents, patients, and/or employees. Most organizations store PI in heterogeneous server system environments. In our initial review of the literature and interview research, we found that organizations do not currently have a unified way of defining or implementing privacy policies that encompass data collected and used across different server platforms.[10] This makes it difficult for the organizations to put in place proper management and control of PI that allows the data users to access and work with the PI inline with organizational privacy policies. Our task was to collect data to help us understand the problem and propose solutions that would meet the needs of organizations for privacy policy management.

This work involved a wide range of user-centered techniques (discussed earlier in the chapter). This project differs from the security application presented earlier in at least three important ways; it therefore highlights the application of different usability and evaluation techniques to the creation of a privacy application:

The case study involves the creation and use of new software technologies rather than the addition of an incremental improvement to a system using existing technology.

The targeted user community is broader and includes different user groups across a variety of domains and geographies.

The solution requires integration with the organization's heterogeneous configurations, including valuable legacy applications.

To create technology to support all aspects of organizational development and use of privacy policies, we needed to understand the concerns of a large and diverse user group. The research included four steps:

Identify the privacy needs within organizations through email survey questionnaires.

Refine the needs through in-depth interviews with privacy-responsible individuals in organizations.

Design and validate a prototype of a technological approach to meeting organizational privacy needs through onsite scenario-based walkthroughs with target users.

Collect empirical data in a controlled usability laboratory test to understand the usability of privacy policy-authoring methods included in our proposed design.

We briefly describe these activities in the following sections, and provide a reference to a more complete description of this ongoing work.

Step One: Identifying Privacy Needs

Our initial work involved gathering data from people about their needs related to privacy-enabling technology. Interviews and surveys are generally appropriate methods for this stage of a project. We completed the initial interview research in two steps:

An email survey of 51 participants to identify key privacy concerns and technology needs, and

In-depth interviews with a subset of 13 participants from the original sample to understand their top privacy concerns and technology needs in the context of scenarios about the flow of PI through business processes used by their organizations

The goal of both of these activities was to create an initial draft of the requirements definition and the user profile definition. For the first step, we recruited participants from industry and government organizations who were identified as being concerned with privacy issues. These participants and their organizations represent early technology adopters concerning privacy, and we welcomed their assistance in the early identification of issues and requirements for product development. They were recruited through a variety of mechanisms including follow-ups on attendance at privacy breakout sessions at professional conferences and referrals from members of the emerging international professional privacy community.

The email survey was designed to give us an initial view of the privacy-related problems and concerns that are faced by organizations. We recognized that our target user group was made up of people with highly responsible jobs and very busy schedules, so our email survey consisted of three questions carefully chosen to help us understand their perceptions of how their organizations handled privacy issues. These were:

What are your top privacy concerns regarding your organization?

What types of privacy functionality would you like to have available to address your privacy concerns regarding your organization?

At this time, what action is your organization taking to address the top privacy concerns you listed above?

Participants were asked to rank-order their top three choices on questions 1 and 2 from a list of choices provided, and were also allowed to fill in and rank "Other" choices if their concern was not represented in the list. For question 3, they could choose as many options as were appropriate, or choose "Other" and explain the approaches to privacy their organizations were currently using.

Conclusion

About the Authors

Clare-Marie Karat is a Research Staff Member at the IBM T.J. Watson Research Center. Dr. Karat conducts HCI research in the areas of privacy, security, and personalization. She is the editor of the book Designing Personalized User Experiences in eCommerce, published in 2004 (Springer). She is an editorial board member of the ACM

Chapter Five. Designing Systems That People Will Trust

Andrew S. Patrick, Pamela Briggs, and Stephen Marsh

TRUST IS A FUNDAMENTAL BUILDING BLOCK OF SOCIETY,[1] a means of making decisions about conferring authority or responsibility in unfamiliar or uncertain situations,[2] a method of understanding how decisions are made in context,[3] and one of the most important concepts in the security arena. Unfortunately, it also remains one of the most poorly understood concepts. A lack of trust will result in systems being ill-used at best, and not used at all at worst. A lack of understanding of trust, in both user and system, will result in the wrong decision—or no decision at all—being made in security contexts. Too much trust can be at least as dangerous as not enough, and not enough trust can be dangerous enough.

This chapter examines the issue of trust in security and privacy systems. These systems purportedly help users make decisions about whom to trust with access, information, or data. For example, how much, when, and for what purposes can specific information be used? They can also help make decisions for the user when the user is not available. These decisions are based on a foundation of trust.

Introduction

Current security systems are often seen as difficult to use, or as getting in the user's way. As a result, they are often circumvented. Users should not have to delve into arcane issues of security to be able to allow access to a part of their personal information online: they don't have to in the real world, after all. In the real world, they rely on trust, an understanding of fiduciary responsibilities, and common sense. So it should be online.[4]

Fundamental questions arise when considering trust, including how to reliably represent trust in different interactions and interfaces, how to transform trust-based decisions into security decisions while maintaining the meaning of the trust-based decisions (in other words, attaining computational tractability without sacrificing meaning), how to transform in the opposite direction, and what the building blocks of trust really are in such contexts as information sharing or secure access to systems. Finally, because trust is fallible, what are its failings, how can they be addressed in this context, and what means of controlling the fallibility exist or should exist? Through investigating prior and current work in the area, this chapter arrives at recommendations for future systems and guidance for how they can be designed for use in a context of trust.

In the next section, we discuss the definitions of trust, and in the following section, we examine the context of trust, its relation to risk, and the fundamental building blocks of trust online that have arisen from e-commerce research. Later, we present formal models of trust and describe what can be learned from these models. We conclude with a set of guidelines addressing how trust can be used in security systems, and concrete suggestions for system developers.

Definitions of Trust

Trust has not always been a subject of mainstream consideration.[5] In fact, prior to the Internet boom and bust, trust was a poor sibling to other sociological and psychological constructs. The Internet boom changed things, as people began to realize that, with trust, people will buy things, and without it, they will not.[6] As simple as this observation may seem, it remains profound. What's more, the realization that imperfect designs can affect the trust of a user has had an equally profound effect on how people have gone about implementing user interfaces, web sites, and interactivity in general.[7] The result has been an increasing amount of well-designed, well-thought-out interfaces, and a great deal of discussion in fields such as Human-Computer Interaction (HCI) and Computer-Supported Cooperative Work (CSCW) about how to encourage, maintain, and increase trust between people and machines, and between people and other people.[8]

Unfortunately, given all of this interest in trust, a deep and abiding problem became evident: everyone knows what trust is, but no one really knows how to define it to everyone's satisfaction. Thus, we now have a great many different definitions, almost as many as there are papers on the subject, all of which bear some relation to each other, but which have subtle differences that often cannot be reconciled. Trust, it seems, is a lot of things to a lot of people.

Looking at the literature, this state of affairs is understandable because trust is multifaceted, multidimensional, and not easy to tie down in a single space.[9] The problem remains, however, that to discuss trust, one must in some way define terms. We suggest the following definition: "Trust concerns a positive expectation regarding the behavior of somebody or something in a situation that entails risk to the trusting party."[10] Problems remain with this and other definitions,[11] but it will do for our purposes.

The Trust-Risk Relationship

Trust is intimately associated with risk—indeed, it is possible to argue that in the absence of risk, trust is meaningless.[18] Let's take an everyday example: I could ask a stranger to look after my seat on a train (low risk) and not feel any need to engage in an evaluation of the trustworthiness of that stranger. However, if I leave an expensive video camera or even my baby behind on the seat (high risk), a more careful trust judgment would ensue. But this example raises other issues in relation to the trust-risk relationship . In particular, it seems that the characteristics of trust are dependent upon the types of underlying risk. To pursue the example, if I would trust someone to watch my video camera, does that imply that I would trust them to look after my infant? Not necessarily, as the two trust judgments are related but somehow distinct, with the latter relying more heavily on judgments of competence and kindness and the former on judgments of honesty. So, to add to the argument made earlier, we may need to be able to phrase trust not just in terms of "I trust you this much" but also in terms of "I trust you this much to do this thing."

The same complexities occur in e-commerce. An online consumer's decision to trust an e-vendor may reflect beliefs about honesty, but is also likely to tap into decisions about competence and expertise, and it is further informed by judgments about the extent to which any information provided will remain private. Thus, a seemingly simple act of trust invokes a complex set of judgments. Once again, the risk assessment involved is crucial—there is no doubt that people are more willing to trust a site if the perceived risk is low. This was shown very clearly in a study of more than 2,500 people who said they had sought advice online.[19] Those that sought advice in relatively high-risk domains (e.g., finance) were less likely to trust and subsequently act on the advice than those who sought advice in low-risk domains (e.g., entertainment). Similar findings can be found in the well-known Cheskin/Sapient report on trust in e-commerce,[20] where, for lower-risk purchases such as books or groceries, trust was strongly associated with familiarity, whereas for high-risk purchases, such as drugs or financial services, trust remained low, even when the companies themselves were well known.

Even though some e-commerce transactions may seem to be low risk (say, involving small amounts of money), they usually involve high-risk elements such as the threat to privacy or credit card fraud. Furthermore, a typical exchange is complicated by uncertainties about whom or what is being trusted. Thus, in situations where perceived risk may be low, actual risks may be high, and the assessment of actual risk is complex. For example, when a person logs into a secure web site to do a transaction, who are they trusting and on what are they basing their trust decision? In terms of people, they are trusting the writer of the web browser, the owner of the computer system, the web host operator, the e-commerce vendor, all the network operators who handle their data, and the certificate authority that registered the web site—but each to a different extent.

Technology Factors

Technology can alter the trust equation. When properly implemented, SSL encryption reduces the amount of trust that needs to be placed in network operators by limiting the opportunity for them to eavesdrop on TCP/IP connections, but operators must still be trusted to deliver packets to their intended destination. On the other hand, SSL does not help to protect against a keystroke logger that may be running on an Internet kiosk—a risk even when the kiosk's browser displays a secure "lock" icon in the status bar.

Customers must be prepared to place their trust not only in the people, but also in the technology that underpins an interaction. Understanding the context for trust, therefore, involves understanding issues of encryption and data security as well as understanding the development of a psychological bond. Bollier argued that it is vital to distinguish between issues of "hard trust ," involving authenticity, encryption, and security in transactions, and issues of "soft trust ," involving human psychology, brand loyalty, and user friendliness.[21] But as the earlier example demonstrates, hard and soft trust can easily overlap or be confused.

The Time-Course of Trust

The research on trust reviewed in earlier sections suggests a need for more explicit consideration of the ways in which trust develops over time. It is certainly worth distinguishing between the kinds of trust that support transient interactions and those that support longer-term relationships.[33] A number of authors[34] have suggested that three phases are important: a phase of initial trust, followed by a more protracted exchange, which then may or may not lead to a longer-term trusting relationship. If one considers trust in this developmental context, some of the findings in the literature make more sense. In particular, consideration of a developmental context helps to reconcile the tension between those models of trust suggesting that trust is a concept grounded in careful judgment of vendor expertise and experience, process predictability, degree of personalization, and communication integrity,[35] and those models suggesting that trust decisions depend much more heavily on the attractiveness and professional feel of a site.[36]

The importance of visual appeal in the early stages of interaction with a web site is not unexpected given that in face-to-face interaction, we often make judgments on the basis of the attractiveness of an individual, giving rise to the well-known halo effect.[37] Other influences on first impressions in face-to-face conversation include the small talk that strangers engage in. Some trust designers have tried to capture this in the design of relational agents that promote early trust. Thus, Bickmore and Cassell describe the use of small talk to build "like-mindedness" between interlocuters in the early stages of an interaction.

Models of Trust

Researchers have developed a variety of models of trust components, antecedents, and/or consequences.[42] The advantage of models is that they may make fuzzy concepts clearer by defining terms and concepts. They can also provide structure where none existed before. More practically, developing a model may lead to specific metrics of interest that can be measured in research studies using questionnaires or other instruments. Models of trust can also lead to specific development advice. Some researchers working in the trust area, such as Egger, have used their models to develop criteria or checklists that practitioners can use to evaluate and improve a web site or similar service. In this section, we review some of the models of trust, pointing out the similarities and differences, and we conclude with some specific lessons that the models can provide for developers.

Early Work on Modeling Trust

Some of the earliest work on modeling trust focused on different components of the concept. Mayer et al.[43] proposed that trust is based on a set of beliefs about trustworthiness , and that the most important beliefs concerned ability, integrity, and benevolence:

Ability is the capacity for a trustee to be able to fulfill a promise made in a trusting relationship.

Integrity relates to the promises made by the trustee—does he promise more than he can deliver?

Benevolence refers to acting in another's best interest.

Gefen[44] operationalized this model of trust components by developing a questionnaire that addressed the three concepts of ability, integrity, and benevolence. Students who used the http://Amazon.com web site were asked questions related to Amazon's ability (e.g., "http://Amazon.com knows about books"), integrity (e.g., "I expect that http://Amazon.com will keep promises they make"), and benevolence (e.g., "I expect that http://Amazon.com has good intentions toward me"). Analysis of the results showed that these concepts are reliable, statistically independent, and valid for predicting past shopping behavior and future intentions.

Bhattacherjee's Model of Trust

Bhattacherjee took a different approach and focused on the antecedents and consequences of trust for e-commerce situations.[45] That model consists of three components and, like many others, Bhattacherjee uses a flow diagram to illustrate the proposed relationship between the components, illustrated in Figure 5-1. The component of familiarity is defined as knowledge of the trustee based on prior interactions or experiences. Trust is assumed to be made up of beliefs in ability, benevolence, and integrity, based again on the pioneering work of Mayer et al. In this model, familiarity can lead to trust, which in turn can lead to a willingness to transact. In addition, familiarity can lead to a willingness to transact directly, even without feelings of trust. Such a situation might occur if a customer continues to transact with a vendor out of habit or convenience, even though there may be a lack of trust. Like Gefen, Bhattacherjee developed questionnaire items to operationalize each of the components in the model, and then demonstrated in an empirical study that the concepts were related in the expected statistical manner.

Figure 5-1. Bhattacherjee's model of trust

Lee, Kim, and Moon's Model of Trust

A similar model of trust for e-commerce was developed by Lee, Kim, and Moon.[46] The model, illustrated in Figure 5-2, also describes antecedents to trust, this time focusing on three concepts: comprehensive information, shared values, and communication. In a way, these antecedents are describing the things that might be learned in the familiarity component proposed by Bhattacherjee, so the two models are similar in that respect. What makes the Lee et al. model unique is the addition of a transaction cost component that is seen as being in opposition to trust. In this model, trust and cost are combined, in opposite directions, when customers make their decisions about e-commerce behaviors (in this case, customer loyalty). Lee, Kim, and Moon describe three antecedents to transaction cost: uncertainty, the number of competitors, and specificity (the nature of the store or transaction).

This model is important because it describes both trust and cost as being independent, opposing factors. According to the model, customers will choose to continue a relationship with a vendor if factors leading to trust are strong and factors leading to transaction costs are weak. We have recently adapted the model to replace transaction costs with the more general concept of perceived risk, and found it to be useful for explaining trust in a different domain.[47]

Corritore's Model of Trust

Corritore et al. also included trust and risk in their model (see Figure 5-3), although they proposed that increased perceptions of risk lead to decreased trust, instead of having trust and risk be independent factors.[48] This model also includes perceptions of credibility as a concept related to risk, and as we have seen, assessments of credibility are seen to be related to perceptions of honesty, expertise, predictability, and reputation. Corritore et al. also include ease of use in their model, and this is meant to measure how easy it is for a truster to achieve his goals (e.g., find the desired goods or complete the transaction). They propose that ease of

Figure 5-2. Lee, Kim, and Moon's model of trust

Trust Designs

There are some examples available of successful designs that have promoted trust in online users. For example, gambling over the Internet using an off-shore, unregulated casino is an act that requires a great deal of trust. Such sites require that the gambler trust the casino operator to provide fair odds and to handle money securely and properly. Shelat and Egger examined factors that online gamblers use when deciding to trust Internet gambling sites.[53] Conducted within the framework of the MoTEC model, the study revealed that:

Informational content was the most important factor. People were most trusting when they could easily find information about the casino, its staff, and its policies.

The second most important factor was relationship management, and trust-building attributes were an ability to contact the casino and rapid, high-quality responses and payments.

The third most important factor was interface properties, which included usability and the ease of finding information.

Future Research Directions

We began this chapter with a discussion of some of the reasons why considerations of trust will be important for future privacy and security systems. Let us end the chapter with some explicit considerations of the trust issues raised by future technologies. We know that researchers and developers are increasingly excited about the concept of Ambient Intelligence

About the Authors

Part II. Authentication Mechanisms

Chapter Six. Evaluating Authentication Mechanisms

Karen Renaud

THE END USER PLAYS A VITAL ROLE IN ACHIEVING SYSTEM SECURITY. If a security system is designed to accommodate the average user's needs and limitations, it is more likely that the system will succeed. Bear in mind that computer users are primarily goal directed and engaged in carrying out some task—and that maintaining security is usually not an integral part of that task. Hence, security systems are sometimes seen as an intrusion to be dealt with as quickly as possible so that users can continue with their primary task. Jonathan Grudin[1] found that humans would subvert any technology that did not directly benefit them in a group-based technological environment. This finding appears to apply to authentication mechanisms too: people often work around these mechanisms, which are put there explicitly to protect them, because they do not fully understand the benefits that will accrue from observation of security guidelines. Of course, security mechanisms do benefit end users, but they sometimes have a limited understanding of the whole security arena and do not have an insight into the benefits of taking the time to behave securely.

Password-based authentication is currently the most common authentication mechanism, but passwords are notoriously weak, mostly because of human information-processing limitations. People have too many passwords and PINs to remember, so they resort invariably to choosing easily remembered weak passwords, writing down the strong passwords, or using the same password with multiple accounts. All of these strategies weaken the password authentication mechanism.

Biometrics is another common authentication mechanism. Besides being usually more expensive than passwords, biometrics is also somewhat unreliable because human beings are, by their very nature, variable. For example, fingerprint readers can misread if given dirty or damaged fingers; they can also be confused by users who don't place their fingers correctly on the reader. Some biometrics can also be readily compromised; for example, people can also unknowingly leave enough skin oil residue on the reader to enable an attacker to duplicate their fingerprints.

This chapter provides an overview of the different authentication mechanisms and proposes a technique for comparing these mechanisms to support developers in making an informed choice for their systems.

Authentication

Security systems are designed to let authorized people in (the permission problem), and to keep unauthorized people out (the prevention problem[2]). This involves three distinct steps: identification, authentication, and authorization.

Identification. The identification step asks a person to identify himself—usually by means of a token or an identification string such as an email address or account number.

Authentication. Once the identification token has been tendered, the person will have to provide evidence of his identity—the authentication step.

Authorization. Authorization allows an authenticated person to take a set of actions once permission has been granted.

People authenticate themselves by what they know (memometrics [3] ), by what they recognize (cognometrics), by what they hold, or by what they are (biometrics) (see Figure 6-1). In the case of the first three, the system and the person share a secret[4] (the authentication key). At enrollment, the user and the system agree on what the secret is; at authentication time, the system determines whether the person being authenticated has possession of the pre-agreed secret. If the user proves knowledge of the secret, the system will authenticate her. In the case of biometrics, the system records a digital representation of some aspect of a person's physiology or behavior at enrollment, and this is confirmed at authentication time.

Authentication Mechanisms

This section discusses authentication mechanisms grouped according to whether they are based on what users are, what users know, what users recognize, or what users hold.

What the User Is—Biometrics

Biometrics mechanisms (Figure 6-4) fall into two distinct categories:

Behavioral biometrics. These can be based on mouse usage patterns, keystroke latencies or dynamics, or signature dynamics.

Physiological characteristics. These can be based on fingerprints, voice, iris or retina, vein pattern, face, hand, or finger geometry, or even ear shape.

Biometrics, while appearing infallible, can actually suffer from some potentially insuperable flaws, depending on the actual biometrics. In the first place, they are easy to forge in an uncontrolled environment. For example, facial recognition systems can be fooled by a photograph of the individual held up to the camera—and in some cases, even by a drawing.

They also present problems in terms of convenience because the user's biometrics has to be captured securely at enrollment and at each authentication attempt, potentially a time-consuming process that requires a controlled environment to ensure that people are not being forced to authenticate themselves for someone else's benefit.

Biometrics are covered in more detail in Chapter Ten.

Figure 6-4. Biometrics

What the User Knows—Memometrics

There are two types of passwords:

Random. The random approach uses a random sequence of characters and digits (which may be generated randomly or selected by each user), and is called a password (if it is a word), a PIN (if it is composed only of digits), or a passphrase (if it consists of more than one word).

Cultural. The cultural approach relies on the memory of a concept or a phrase, and is sometimes also called a cognitive or a semantic password.

Random passwords (uncued recall)

Currently, the most pervasive and popular authentication mechanism is the random password. Passwords have the potential to be very secure, but this potential is seldom realized in the real world, especially in an uncontrolled environment such as the Web. There is evidence that users remember passwords better if they choose them,[10] and even though the system-assigned password will probably be stronger, most systems allow users either to choose their own passwords or to change a system-assigned password. The reason is that users frequently will resort to writing down random passwords that are difficult to remember.

Unfortunately, many end users do not choose strong passwords.[11] This is primarily a result of their being overloaded with too many passwords and PINs to remember. They choose easily guessed passwords and reuse those passwords out of self-defense; it is the only way to cope in a world that places increasing burdens on users without any consideration given to the cumulative effects of similar demands being placed upon them by other security systems. (For further discussion of password memorability, see Chapter Seven.)

An example of the use of a passphrase is demonstrated by Spector and Ginzberg.[12] Their approach compensates for differences in syntax to determine whether users remember the semantics of the passphrase at authentication, even if the user has forgotten the actual words used at enrollment. This is a good alternative to passwords because it addresses both the memorability and the predictability issues, but it does require special software and is potentially more time consuming at authentication than simple passwords.

Cultural passwords (cued recall)

Quality Criteria

Authentication mechanisms often have deficiencies in one or more areas. To support meaningful comparison of authentication mechanisms , we propose a set of evaluation criteria that can be used to assign relative deficiency values to each authentication mechanism. Once these values have been assigned, the characteristics of the environment within which the authentication mechanism will be used are identified and used to group the criteria into four different groups—critical, vital, significant, and incidental—indicating their relative importance to the developer. The assigned deficiency values in each group can then be used to support an informed decision as to which authentication mechanism to use in a particular situation.

Three fundamental authentication mechanism deficiency categories can be isolated from the previous discussion: accessibility , memorability, and security. A cost category will also be included, because a cost-benefit analysis is part of any decision process. Each of these categories has several dimensions, which are discussed in the following sections. In our discussion, we describe how these criteria can be applied generally to several types of authentication mechanisms. The deficiency values for specific instances of each of these mechanisms may be somewhat different. However, after reading this section, you should be able to use these criteria to evaluate any authentication mechanism yourself.

Accessibility

Figure 6-8 depicts the various aspects of this dimension, which reflects how easy it is for users to use a particular authentication mechanism. The aspects are described in the list that follows.

Figure 6-8. Accessibility

Special hardware and software requirements

This aspect refers to the minimum hardware, software, or technical expertise required to support the authentication mechanism. If only one of these is required at the user's machine, only a minor deficit can be applied; if two are required, it is obviously more serious; and if all are required, it can be considered to be a major deficit.

Many authentication systems can have special requirements. Most obvious are biometrics such as fingerprint- or iris-based identification, which naturally requires a fingerprint or iris scanner. On the other hand, biometrics systems such as voice recognition or identification by keystroke dynamics can often be performed using standard PC hardware. And biometrics or smart card readers may be built into the terminal used by the target population—for example, there are mobile phones with integrated fingerprint or smart card readers—and thus the special requirements may not pose any problems at all.

Even a token-based authentication system that displays a code may require that the server be equipped with special software or hardware to verify the tokens. Smart cards further require that the user have access to a smart card reader, and USB tokens require that the user have a computer that has an available USB connector.

Special requirements must always be carefully considered for authentication mechanisms that are likely to be accessed from small, handheld devices .

Convenience

There are three aspects of convenience to be considered: enrollment time, authentication time, and key replacement time. Authentication time is the most relevant (it will mount up), so a large deficiency can be applied if this is time consuming. A smaller deficit can be applied if the mechanism is time consuming only at either enrollment or replacement. A large deficit results if all stages are time consuming.

Only random passwords are fast and convenient both at enrollment and at authentication , with replacement being potentially time consuming depending on how it is handled. Graphical passwords, both recognition-based and position-based, are considerably more time consuming at all stages, but less so than cultural passwords where users have to answer a succession of questions. The most time consuming are biometrics, where the user has to potentially spend a substantial amount of time enrolling and being authenticated (depending on the biometrics mechanism and the level of control applied).

Inclusivity

This aspect addresses the issue of the exclusion of users. Three kinds of disability are considered here—cognitive, physical, and sensory—and the deficit should be assigned based on whether users in the disability categories are excluded. The deficit gets larger as more categories of disabled users are excluded. If users in all categories are excluded, that constitutes a maximal deficit.

Cultural passwords do not exclude users with any type of disability—assuming that an accommodation has already been made so that the user can enter a response—whereas random passwords affect users with cognitive disabilities, such as users with memory difficulties.

Recognition-based graphical authentication systems affect the same disabilities, but also exclude people affected with cognitive disabilities such as dyspraxia.

Biometrics possibly affects users with physical disabilities, such as amputees in the case of fingerprint devices. Physiological changes, such as those that occur in retinas during pregnancy, may affect users' use of retinal screening devices. Position-based devices may exclude users with both sensory and physical disabilities—or even people who are simply too short or too tall, depending on how the biometrics system is mounted.

In general, this deficit depends on the actual type of authentication strategy being used and should be tailored accordingly.

Environmental Considerations

This section explains how to determine a ranking of the criteria introduced in the previous section. To help the system developer make a decision, it is necessary to rank the criteria so that deficiencies in the more important criteria will tend to carry more weight. We will use the characteristics of the environment to classify criteria as critical, vital, significant, or incidental. All criteria will be initially assigned to the significant category, and moved into the other categories based on an environmental analysis.

There may be certain situations where a large deficiency of a particular aspect or dimension can completely disqualify the authentication mechanism for use in a particular environment. For example, the developer may decide that the system controls access to such critical data that only a completely nonpredictable mechanism will suffice. These criteria are termed critical. Mechanisms with unacceptable deficiencies in these criteria can be withdrawn from consideration if requirements are stringent.

Environmental factors are identified in terms of the category in which they fall, provided in the following subsections. The list of environmental factors given here does not attempt to be exhaustive but, rather, is intended merely to give an example of the kinds of environmental factors that can affect the relative importance of the different criteria.

Accessibility

Control of environment

Choosing a Mechanism

To evaluate an authentication mechanism, it is useful to divide the selection criteria into four categories:

Critical

Used to disqualify mechanisms with an unacceptably large deficit.

Vital

Used to come up with a cumulative deficit for authentication mechanisms across these criteria, to identify candidates for use in the system.

Significant

Used to confirm the decision made by using the vital criteria only. If a chosen mechanism happens to have a maximal deficiency in a significant criterion, we should consider other options based on the vital criteria.

Incidental

These criteria are not important and usually would not be weighed as part of the decision-making process.

Clearly, different applications will put different criteria into different categories.

An Online Banking Example

Conclusion

Random passwords are currently the most popular user authentication mechanism, but passwords are inherently weak, and it is becoming necessary for system developers to consider other authentication mechanisms. Using the techniques explained in this chapter, an objective and informed choice can be made based on the mechanism, the environment within which it will be used, and customer preferences.

About the Author

Karen Renaud is a Senior Lecturer in the Department of Computing Science at the University of Glasgow in Scotland. She previously lectured at the University of South Africa. She does research in a variety of areas including usability of security systems, distributed systems, Internet technology, design patterns, and issues related to recovery from interruptions. She is also actively involved in developing and testing alternative web authentication mechanisms.

Chapter Seven. The Memorability and Security of Passwords

Jeff Yan, Alan Blackwell, Ross Anderson, and Alasdair Grant

MANY THINGS ARE "WELL KNOWN" ABOUT PASSWORDS, such as the fact that people can't remember strong passwords and that the passwords they can remember are easy to guess. However, little research on the subject would pass muster by the standards of applied psychology.[1]

In the study presented here, we confirmed some widely held folk beliefs about passwords. However, we also observed a number of surprising phenomena that run counter to the established wisdom. Our study shows that the methods of applied psychology can bring new insights and solid results for security research and development.

Introduction

Many of the deficiencies of password authentication systems arise from the limitations of human memory. If humans were not required to remember the password, a maximally secure textual password would be one with maximum entropy: it would consist of a string as long as the system allows, made up of characters selected from all those allowed by the system, and in a manner that provides no redundancy — that is, a totally random selection.

Existing Advice on Password Selection

Adams and Sasse[8] note that users are not enemies of security, but collaborators who need appropriate information to help maintain system security. They observe that users, when not told how to choose good passwords, make up rules for password generation, resulting in insecure passwords. They therefore recommend that organizations "provide instruction and training on how to construct usable and secure passwords."

Later research by Sasse, Brostoff, and Weirich, based on a survey of system users, found that 90% of them had difficulty with standard password mechanisms and that they welcomed advice on password generation.[9] The authors conclude that, "instructions for constructing and memorizing a strong password...should be available when a password needs to be chosen or changed."

Many large organizations do give specific advice to new users about how to select a "good password." A good password, in terms of the preceding discussion, should aim to be reasonably long, use a reasonably large character set, but still be easy to remember. There are some subtleties about whether the attacker is going to try many passwords over a network or whether she has obtained a copy of the password file and is cracking it offline, but we propose to ignore these for the purposes of the present study.

We made an informal survey of advice given to new users at large sites by searching on the Web for the terms "choose," "good," and "password." Many sites did not recognize the importance of memorability, merely emphasizing resistance to brute force search. Some typical pieces of advice were:

One recommendation that seems increasingly popular is the "passphrase" approach to password generation. A typical description of this is as follows:

Of course this informal survey does not include sites where no advice at all is given on password selection. We believe that many sites simply tell new users the minimum requirement for a valid password (length and character set), and give no further advice regarding security or memorability. Others, in our experience, enforce rules such as:

Passwords must be at least eight characters long and must contain at least two non-letter characters. They must also be changed at least once a month.

Experimental Study

Method

In October 1999, students attending the introductory lecture were told that they would be subjects (with their consent) in an experiment on password selection. At the tutorial session, they were then asked for consent and were randomly assigned to one of three experimental groups. Each student was given a sheet of advice depending on the group to which they had been assigned. The three different types of advice were:

Control group. Students in this group were given the same advice as in previous years, which was simply that "Your password should be at least seven characters long and contain at least one non-letter."

Random password group. Students in this group were given a sheet of paper with the letters A-Z and the numbers 1–9 printed repeatedly on it. They were told to select a random password by closing their eyes and picking eight characters at random. They were advised to keep a written record of the password, but to destroy it once the password was memorized. This is very similar to the advice given by banks when issuing PIN numbers, which are issued in written form, but with the user advised to destroy the written slip as soon as it has been memorized.

Passphrase group. Students in this group were told to choose a password based on a mnemonic phrase.

The text of the instructions given to the three groups is reproduced in Figure 7-1

Results

Of the 300 students we asked, 288 consented to participate in the experiment. They were allocated randomly to experimental groups as follows:

Control group

Random password group

Passphrase group

The selected passwords were, on average, between seven and eight characters long (7.6, 8.0, 7.9, respectively) with no significant difference between the three groups. All experimental groups chose slightly longer passwords than did the comparison sample of 100 students who had not attended the introductory lecture. Mean length of password in that group was 7.3, with analysis of variance (ANOVA) statistically significant at F=8.3, p<.001.

The most successful cracking method was the permuted dictionary attack. Cracking based on user information was not successful in any case, probably because of the very limited amount of user information available in these password files (they do not include first names or forenames, for example). All six-character passwords were cracked successfully using a brute force attack. Table 7-1 summarizes these results (treating brute force attacks separately).

Table 7-1. Results of password crack, by test group

Group

Passwords cracked using first three attacks

Passwords cracked using brute force attacks

Number

Percent of total

Control group

Random password group

Discussion

Our study confirms a number of widely held folk beliefs about passwords, and debunks some others:

Users have difficulty remembering random passwords. This belief is confirmed, both in users' subjective reports and in their actions—the latter in a manner that has worrying parallels to common practice in the issue of bank PIN numbers.

Passwords based on mnemonic phrases are harder for an attacker to guess than naively selected passwords are. This belief is confirmed.

Note

Theoretically, random passwords should provide the maximum security. This result highlights the importance of studying systems as they are used in practice.

Random passwords are better than those based on mnemonic phrases. However, each type appeared to be just as strong as the other. So this belief is debunked.

Passwords based on mnemonic phrases are harder to remember than naively selected passwords are. However, each appeared to be reasonably easy to remember, with only about 2%-3% of users forgetting passwords. So this belief is debunked.

By educating users to use random passwords or mnemonic passwords, we can gain a significant improvement in security

Acknowledgments

We thank Sacha Brostoff, Lorrie Cranor, Simson L. Garfinkel, and anonymous reviewers whose comments helped us improve this chapter.

About the Authors

Jeff Yan is a lecturer in the School of Computing Science, Unversity of Newcastle, in England. He is interested in most aspects of information security, both theoretical and practical. His recent research is largely about systems security, applied crypto, and human aspects of security. He has a Ph.D. in Computer Science from Cambridge University.

jyan

Chapter Eight. Designing Authentication Systems with Challenge Questions

Mike Just

THAT IS YOUR MOTHER'S MAIDEN NAME?" "What is your date of birth?" Such questions are often used to authenticate an individual. The answers often represent information well known to the individual, but (one hopes) not so widely known so as to be available to a potential impersonator. These challenge questions require an individual to recall and present previously registered answers when authenticating.

In this chapter, I review the design and evaluation of authentication systems that use challenge questions and answers to identify or authenticate individuals. I pay particular attention to ensuring that the design satisfies the security, usability, and privacy requirements of the authentication system.

While systems today use challenge questions for recovering forgotten passwords, they can be used more broadly for other forms of authentication, such as routine user login. This chapter focuses on password recovery but considers other applications as appropriate.

Challenge Questions as a Form of Authentication

Criteria for Building and Evaluating a Challenge Question System

We begin our introduction to the design of challenge question systems by introducing criteria that are helpful in both their design and evaluation. These criteria relate to the privacy, security, and usability of the challenge question system.

Privacy Criteria

In environments that use personal information, it should be common practice to follow recognized privacy principles to protect answers to challenge questions.[1] For the use of challenge questions and answers to authenticate users, one principle in particular seems relevant: collection limitation. This criterion serves to limit the collection of personal user information to what is necessary for the purpose of authenticating an individual. Adherence to this principle helps to ensure that only information necessary to support a suitable level of security and usability is maintained.

Designers should give particular caution to using questions that ask for personal information, such as "What is your mother's maiden name?", because the answer, while possibly obscured (hashed), will be stored at the account server. Preference should be given to asking nonpersonal questions, provided that they offer sufficient security and usability.

Types of Questions and Answers

In this section, we present a candidate classification of questions and answers. For both questions and answers, three different types are described: fixed, open, and controlled. Depending on the system needs, various combinations of questions and answers can be used.

Question Types

The two types of questions that are likely to be most familiar are fixed questions and open questions. A fixed question provides a list of preset questions to a user, where the user's choice of question can be taken only "as is" from this list. At the other extreme is an open question, where a user has complete choice and control over the question; guidance as to the question construction may be provided to the user, but the user enters the question in free-form text.

Designing a Challenge Question Authentication System

Previous sections presented options for question and answer types and criteria upon which to evaluate a challenge question system design. This section discusses the design of a complete authentication system.

Determining the Number of Questions to Use

Usability tends toward requiring fewer questions and answers. This lessens the recall requirements for an individual, and also introduces fewer repeatability mistakes. For reasons of security, however, it is often necessary that more than one question-answer pair be registered by a user. This is to ensure a sufficient difficulty for either guessing or observing the answer. To ensure a sufficient level of protection against guessing, the entropy for the answers should provide a level of security similar to that for routine authentication (that may be performed with a password). In situations in which no complementary security measures are used (see the later section, "Complementary Security Techniques"), the entropy for the answers should be at least that for the routine authentication. In terms of guessability considerations, the strength of a challenge question system can be measured explicitly against password-based authentication. For example, an 8-character password constructed from the set of 52 upper- and lowercase characters, 10 numbers, and 32 punctuation characters, has approximately 252 possible passwords. An 8-character answer to a question that uses only lowercase characters has only 238 possibilities.

Unfortunately, this number is misleading. Answers to questions cannot be expected to conform to the same, strict rules as for passwords; otherwise, the answers effectively become passwords. Instead, we would expect many answers to be dictionary words. This can be a problem, as most dictionaries have only between 216 and 220 words, while studies show that many adults have vocabularies between 215 and 217 words. Thus, even with these extremely optimistic values, at least two questions would need to be asked to ensure security similar to that provided by an 8-character password. In addition, as discussed earlier under "Security Criteria," observability is important, but unfortunately is less quantifiable.

When more than one question is asked, both the interface and the administrative storage for the answers should ensure that multiple answer attempts are all validated before an indication of success or failure is given. Through the interface, for example, if two questions are asked, an indication of success or failure should be given only when both questions have been answered. If not, an attacker can guess answers to one question at a time, even though the entropy level of only one question might not provide a sufficient level of security. Similarly, as with the storage of passwords, answers should be obscured (hashed), but additionally, when multiple answers are used, they should be obscured in such a way that if the obscured answers are compromised, an attacker would have to guess all answers before determining the success of his guess. This can be achieved, for example, by inputting all answers to a single hash rather than by separately hashing each answer.

Variations exist where the number of questions presented at recovery is less than the number of questions registered. There are at least two models:

The user registers n questions, but is presented only t ≤ n questions upon recovery. All t questions must be answered properly in order for the recovery process to continue.

The user registers n questions, and is presented t ≤ n questions upon recovery. Differing from the previous model, only r < t questions must be answered in order for the recovery process to continue.

The first model is an attempt to offer a level of security equivalent to that of n questions, but to provide a usability benefit at the time of recovery, with fewer questions to the user. However, the usability benefits appear only to reduce the time required for recovery and do not affect the arguably more important concerns of memorability and repeatability (the user still has to remember the answers for n questions, as it is not known what questions will be posed at recovery). Yet there is some benefit for users who register n questions, but after a period of time happen to forget the answers to some of these questions. The purpose of the second option is to tolerate mistakes upon answer presentation. However, it seems that an additional question is being used to tolerate such mistakes whereas a more usable system might attempt to reduce the number of questions used.

Some Examples of Current Practice

Chapter Nine. Graphical Passwords

Fabian Monrose and Michael K. Reiter

A GRAPHICAL PASSWORD IS A SECRET THAT A HUMAN USER INPUTS TO A COMPUTER WITH THE AID OF the computer's graphical input (e.g., mouse, stylus, or touch screen) and output devices. In this chapter, we review the arguments supporting graphical passwords as being potentially superior to text passwords, present several graphical password designs, and discuss some analyses of graphical password memorability and security.

Introduction

The ubiquity of graphical user interfaces and input devices, such as the mouse, stylus, and touch screen, that permit other than typed input, has enabled the emergence of graphical passwords. Graphical passwords are particularly useful for systems that do not have keyboards. In addition, they offer the possibility of addressing known weaknesses in text passwords. History has shown that the distribution of text passwords chosen by human users has entropy far lower than possible,[1] , [2] , [3] ,

A Picture Is Worth a Thousand Words

One of the most compelling reasons for exploring the use of a graphical password scheme stems from the fact that humans seem to possess a remarkable ability for recalling pictures, whether they are line drawings or real objects. The so-called picture effect—the effect of pictorial and object representations on a variety of measures of learning and memory—has been studied for decades.[7] , [8] , [9] , [10] , [11] , [12] For the most part, cognitive scientists and psychologists have shown that there is a substantial improvement of performance in recall and recognition with pictorial representations of to-be-remembered material over verbal representations.

The picture effect has itself been the focus of numerous debates related to perception, and the most prevalent theory in its support is known as the dual-code theory, which postulates that language and knowledge of worlds are represented in functionally distinct verbal and nonverbal memory systems. Examples of dual process theory can be found in experiences that we have all had at some time or another: we meet someone, know them to be familiar, but do not know who they are; we recognize a melody, but fail to remember its name or when or where we heard it before; we read a line of a poem, know it, but do not know where we have read it before, much less the title or author of the poem. In all these cases, we experience a sense of familiarity, but we have, at least at first, no access to any contextual information.[13] , [14]

Whatever the underlying reason, our ability to recall images may lend itself naturally to building robust human authentication and key generation technologies. In fact, much of the work that we see today on graphical passwords is motivated in part by the scientific literature in psychology and perception. Although we do not provide a complete survey of proposed schemes here, the following subsections present a set of representative examples in the three main areas: image recognition, tapping or drawing, and image interpretation.

Image Recognition

The most widely explored paradigm for graphical passwords is that of image recognition—that is, the user enters her "password" by recognizing the images that comprise it from among many more images. These images might be those of persons, everyday objects, or abstract images such as those in Figure 9-1

Picture Perfect?

In this section, we summarize several analyses that have been performed to evaluate the security and usability of graphical passwords. While the old adage that a picture is worth a thousand words might indeed be true, it's not entirely clear that user-chosen graphical passwords of the type suggested to date offer additional security over text passwords. An alternative may be to utilize only system-chosen passwords, although we might expect that this would sacrifice some degree of memorability; we do not explore this avenue here, as we are unaware of empirical results to evaluate this conjecture in general.

Security

As with usability analyses, which we examine later in this chapter, the most compelling security analyses for graphical password schemes permitting user password selection are those performed on the basis of extensive user experiments. After all, the security weaknesses of text passwords were revealed only by their use in practice. That said, to date there are few such user studies, and so graphical password design efforts have appealed to surrogate analyses in an effort to reason about the security of particular proposals.

Key generation

The graphical password scheme that has been the topic of most such analyses is the Draw-a-Secret (DAS) scheme.[19] In the paper that originally proposed this scheme, Jermyn et al. reason about the size of the memorable password space, giving a counting argument that the number of memorable DAS passwords (i.e., those having a simple algorithm to generate them) quickly outpaces the number of text passwords that are commonly chosen, as measured by the size of dictionaries commonly applied to break them. As discussed previously in this chapter, the password space is particularly important when considering the use of this scheme to generate cryptographic keys.

Recently, however, Thorpe and van Oorschot have postulated that the memorable DAS passwords are those that exhibit mirror symmetry. If true, they show that the security of DAS against dictionary attacks may be far less than originally hypothesized.[20] They argue that a similar weakness results if users select DAS passwords that are simple by various pattern complexity measures[21]—for example, selecting only a small number of strokes.[22] If the DAS passwords selected by users in practice are consistent with the hypotheses of Thorpe and van Oorschot, then these works may point to ways to strengthen DAS passwords, perhaps by implementing restrictions on DAS passwords similar to those levied on text passwords today.

Authentication

To our knowledge, the only significant user study on the security of graphical passwords for authentication was performed by Davis and the present authors.[23] In that work, we studied the security of two schemes based on image recognition, denoted "Face" and "Story," which are described shortly. This study focused specifically on the impact of user selection of passwords in these schemes, and the security of the passwords that resulted. We recount some of the notable results from this study, and the methodologies used to reach them, as an illustration of some of the challenges that graphical passwords can face. In particular, this study demonstrated that graphical password schemes can be far weaker than textual passwords when users are permitted to choose their passwords.

In the Face scheme, the password is a collection of k faces, each selected from a distinct set of n > 1 faces; for our evaluation we used k = 4 and n = 9. So, while choosing her password, the user is shown four successive 3 × 3 grids containing randomly chosen images (see Figure 9-4(a), for example), and for each, she selects one image from that grid as an element of her password. Images are unique and do not appear more than once for a given user. During the authentication phase, the same sets of images are shown to the user, but with the images permuted randomly.

In the Story scheme, a password is a sequence of k unique images selected by the user to make a "story," from a single set of n > k images, each derived from a distinct category of image types. The images are drawn from categories that depict everyday objects, food, automobiles, animals, children, sports, scenic locations, and male and female models. A sample set of images for the story scheme is shown in Figure 9-4(b).

We chose to study the Face scheme, in particular, because of a depth of psychological literature that revealed factors that could potentially be sources of bias in password selection. For example, the scientific literature abounds with studies that show that people tend to agree about attractiveness even across cultures,[24] and psychologists have argued for decades that the old adage that "beauty is in the eye of the beholder" may be largely false. A natural question is whether general perceptions of beauty (e.g., facial symmetry, youthfulness, averageness)[25], [26] might influence graphical password choices. Similarly, the "race effect" refers to the innate ability of people to better recognize faces from their own race than faces of people from other races.[27], [28], [29] Again, this raises the question as to whether race might influence a user's choice for graphical passwords in the Face scheme.

Figure 9-4. (a, left) In the Face scheme, a user's password is a sequence of k faces, each chosen from a distinct set of n > 1 faces; (b, right) in the Story scheme, a user's password is a sequence of k unique images selected from one set of n images to depict a "story"; in the above examples, n = 9, and images are placed randomly in a 3 × 3 grid

To study both the Story scheme and the Face scheme, we collected user data during the fall semester (roughly the four-month period of late August through early December) of 2003, of graphical password usage by three computer engineering and computer science classes at two universities. Each student used one of the graphical password schemes for access to content including his grades, homework, homework solutions, course reading materials, etc., via standard Java-enabled web browsers.

For the purposes of the experiment, facial images were classified into nonoverlapping categories, namely:

To simplify the analysis, we made the assumption that images in a category are equivalent—that is, the specific images in a category that are available do not significantly influence a user's choice in picking a specific category.

If we simply consider the set of images chosen by men and women using the Face scheme (see Figure 9-5), some differences are apparent immediately: for one, different populations exhibit strong differences in their password choices.

Figure 9-5. Category selection based on gender and race for the Face scheme; the graph shows the distribution of choices from sets of images consisting of typical Asian males, typical Asian females, typical black males, typical black females, typical white males, typical white females, Asian male models, Asian female models, black male models, black female models, white male models, and white female models

Insight into what different groups tend to choose as their passwords in the Face scheme is shown in Tables 9-1 and 9-2, which characterize selections by gender and race, respectively. As can be seen in Table 9-1, both males and females chose females in Face significantly more often than males, and when males chose females, they almost always chose models (roughly 80% of the time).

Moreover, perceptual differences were also observed when we examined image selection across racial categories. In that case, the "race effect" described earlier seemingly influenced the selection of passwords. As depicted in Table 9-2, Asian females and white females chose from within their races roughly 50% of the time; white males chose whites over 60% of the time.

Table 9-1. Gender and attractiveness selection in Face; the results show that "beauty" appeared to play a significant role in the choices of images selected by both genders, albeit more so for males

Population

Female model

Male model

Typical female

Typical male

Female

40.0%

20.0%

28.8%

11.3%

Male

63.2%

10.0%

12.7%

14.0%

Let's Face It

About the Authors

Chapter Ten. Usable Biometrics

Lynne Coventry

BIOMETRICS OFFER A TECHNOLOGICAL SOLUTION TO THE AUTHENTICATION OF INDIVIDUALS. Biometrics confirm that the actual person, rather than merely his or her token or identifier, is present. Thus, biometrics may reduce the effort of a person's trying to identify himself and in doing so potentially reduce the chances of authentication fraud.

The term biometrics is derived from the Greek bio (life) and metric (measure). Biometrics, as a means of identification, can be traced back to both ancient Egypt[1] and China. Pioneering work on modern biometrics technologies dates back to the late 1960s, when researchers began looking at voice patterns, fingerprints, and hand geometry as a means for establishing unique individual identity. By the mid-1980s, the biometrics industry was established and the first systems went live in controlled environments. In recent years, with the heightened security concerns gripping much of the world and the rising amount of fraud in the commercial sector, biometrics technologies have received a great deal of attention. However, the application of biometrics spans all industries.

To date, the growth of biometrics technologies has been driven by a mainly system-centered approach, dealing with the problems of unique digital identifier extraction, template handling, and recognition algorithms. With very few exceptions,[2] , [3] , [4] the usability community has not been involved in the design or evaluation of biometrics, and the traditionally quoted biometrics performance measures are no guarantee of real-world performance. The success of the biometrics approach is strongly influenced by user factors including the size and range of the user population. To facilitate successful biometrics implementations, more research is required into these user factors.

After a brief introduction to biometrics and their uses, this chapter investigates the user factors that will ultimately impact the successful implementation of biometrics security. We primarily focus these ideas within the context of self-service applications such as Automated Teller Machines (ATMs), to highlight the technical and user factors to consider when implementing a biometrics solution. Some technical issues of biometrics are outside the scope of this chapter; these include the handling of templates and images, integration issues, and standards.[5] , [6] , [7] , [8]

Introduction

Biometrics is the comparison of live anatomical, physiological, or behavior characteristics to the stored template of a person. Physiological biometrics include those based on fingerprints, hand and/or finger geometry, and the patterns of retinas, veins, irises, and faces. Behavioral biometrics techniques include those based on voice, signature, and typing behavior.

Table 10-1 provides a brief description of the most well-developed biometrics technologies.

Table 10-1. Common types of biometrics

Biometrics

Description

Fingerprint

Patterns found on the fingertip, including the location and direction of ridge endings and bifurcations

Palm print

A larger-scale version of the fingerprint biometrics

Vein pattern

Vein and capillary patterns on the back of the hand

Hand geometry

Shape of the hand including height and width of bones and joints in the hands and fingers

Where Are Biometrics Used?

Each biometrics technique has its own unique set of advantages and disadvantages. Cost, size, and method of use often dictate applicability to any given situation. For example, a telephone-based system could use only voice as a biometrics.

Early biometrics were cumbersome to use and very expensive, and therefore were used only in very high-security applications. However, as the technology has improved, the number of applications has increased. This section introduces some other areas of application, including physical access control, border control, surveillance, people tracking, and online transaction security.

Physical Access Control

Physical access control is the largest and most commercialized application of biometrics systems, in part because a relatively small user base exists for each system, and in part because these devices could be sold and deployed as standalone systems, allowing for incremental deployments. Physical access security ranges from access to secure locations such as prisons or military facilities, to secure work areas such as bank vaults, to general workplaces, and now to countries via border controls. Nowadays, we are also seeing biometrics used in conjunction with clocks for time cards, to confirm "season passes" at attractions, for access to schools or subsidized meals at school cafeterias, and as an alternative to keys for a car, house, hotel room, or locker.

Biometrics and Public Technology: The ATM Example

As different application areas place different requirements on biometrics technology, we must consider the appropriateness of different technologies for a specific application. This section explores the applicability of the different biometrics technologies for use within an ATM or other self-service environment.

For a number of reasons (to be enumerated in the following sections), I assume here that verification rather than identification will be utilized at an ATM.

ATM Fingerprint Verification

Integration of fingerprint sensors into an ATM for verification is feasible, especially if contact silicon (capacitance or thermal swipe) devices are used (see Figure 10-1). Optical sensors and static capacitance systems are not viable for ATMs because of dirt and latent prints that accumulate with use. Swipe devices, on the other hand, at least have the potential for self-cleaning with use. Unfortunately, the physical positioning of the sensor may be difficult for all users (left-handed/right-handed aside) to get their fingers flat on the reader. Smart cards equipped with fingerprint sensors—where the fingerprint is used to allow use of a private key—could remove many of the technical problems of deploying biometrics in the self-service environment.

Figure 10-1. Fingerprint reader

ATM Face Verification

Another possibility for ATMs is the integration of facial biometrics systems (see Figure 10-2). In many cases, a security camera is already present; this camera could be used for both video recording and biometrics verification.

Because face verification could be a universal biometrics—everybody has a face—most research on this technology to date has focused on the potential negatives. Several complicating factors have been identified:

The system's field of view must cover a wide range of heights, from the tallest standing user to a user in a wheelchair.

Today's technology requires that users find and look at the camera at the right time, facing straight on and with a neutral expression.

Lighting must be uniform and consistent—with good front lighting and little back lighting—posing a significant problem in many locations.

Evaluating Biometrics

Testing biometrics is complicated and requires objective comparisons.[21] That's because biometrics authentication is not a simple yes/no process. It involves complicated statistical analysis of the incoming live signal from the biometrics device. Performance metrics should indicate how well a system performs, but it is difficult to get reliable data on particular systems, and in general, more independent testing is required.

To this end, a standard for biometrics testing should be created. A number of private and public testing laboratories have been set up to promote such a standard. These organizations include the U.S. National Biometric Test Center[22] at San Jose State University, the Biometric Consortium,[23] and programs at the International Computer Security Association,[24] the U.S. National Institutes of Standards and Technology[25] (NIST), and the UK National Physical Laboratory.[26] The International Biometric Group[27] is a private initiative offering independent testing that purports to include a usability perspective.

Performance Metrics

The performance of

Incorporating User Factors into Testing

Biometrics researchers have determined that real-life users are the biggest variable in system performance. Thus, performance measures must be qualified with an understanding of that system's user base. Ashbourn[31] provides an index of user characteristics that can impact predicted biometrics system performance; we reprint this index in Table 10-2.

Table 10-2. Ashbourn's index of user characteristics that can impact predicted biometrics system performance

User characteristic

Description

Acceptance of biometrics concept

If the user is hostile toward the idea of using the biometrics in a given application, his behavior may not be optimal. This will influence behavior if the user is forced to use the biometrics, or the user may completely opt out of the application if given a choice.

Knowledge of technology and computers in general

If the user is technology literate and comfortable using and exploring new technology, he will be better able to optimize his behavior with the system.

Familiarity with biometrics characteristic

If the user if familiar with the characteristic and how it should be used to optimize security, he will be more able to use the system appropriately. For example, with a fingerprint system, the core should be within the image captured. If the user knows the location of the core, he will incorporate a better finger position.

Experience with the specific device being used (and other devices)

Each device has its own way of working and its own user requirements. Users who have used that particular device extensively may feel particularly positive or negative toward it. If a user has been habituated to a certain device, he may use a new device inappropriately.

Environment of use

User stress when using a device can have an influence on the ability to acquire and the quality of an image. Public or private milieus, the presence of a queue, time pressure, and environmental conditions affect overall performance, as will assistance from either a human or an effective interface design.

Transaction criticality

The transaction's degree of criticality will affect user stress levels and, potentially, user performance.

Considerations of age, long-term stability, gender, and ethnicity also need to be taken into account when a biometrics system is evaluated. All claims should be qualified with the user categories on which the biometrics has actually been tested.

Fully understanding the user's role in the performance and acceptance of biometrics requires the utilization of a number of research methods through different stages of development of new biometrics technologies. (The most extensive research has been carried out on iris and fingerprint verification.[32] , [33]) These methods include focus groups to identify consumers' understanding, misconceptions, and barriers to acceptance of biometrics techniques—although conclusions drawn from focus groups may or may not correlate with actual user behavior. Specific studies help to understand how well a technology works with the general, untrained public, and to assess whether it can be adapted to a self-service environment.

Developers must work with an iterative design and evaluation process to create a successful biometrics application. Field trials are imperative to fully test the performance and acceptance of any self-service application, as experience with the actual technology can change people's attitudes in either a positive or a negative direction.

Size of User Base

Within some application areas, including the financial self-service environment, the potential size of the user base can be extraordinarily large. Even small financial institutions can have millions of customers, and with cross-institution relationships, we must consider the possibility that any biometrics system might ultimately have to be applicable to the entire banking population of the planet and all the variations within it! This drives a number of factors associated with biometrics, such as the use of verification (as opposed to recognition), template storage, accessibility, the handling of "outliers," enrollment, and user acceptance.

Designing a Biometrics Solution to Maximize the User Experience

Conclusion

The assumption that biometrics is inherently a usable form of security is flawed. For the system to be secure, input must be controlled and the system must provide appropriate education, training, and lead-through to support the user in providing the required input.

On the surface, there appear to be many valid reasons to adopt biometrics—for example, to replace PINs at the ATM. From a high-level perspective, biometrics appears to be a viable alternative to other forms of security. Field trials and surveys have shown that both system performance and consumer acceptance of biometrics is both promising and improving.[43] , [44] , [45] , [46]. Public exposure to biometrics is increasing; and the cost of biometrics devices is coming down. And yet, there has been no large-scale deployment of consumer biometrics in the ATM environment.

About the Author

Lynne Coventry holds a B.Sc. in Computing Science and Psychology, an M.Sc. in Software Engineering, and a Ph.D. in Human Computer Interaction. She joined NCR in 1995 and is part of a multidisciplinary advanced research group. She is responsible for the HCI research for future self-service technologies. Her work includes the design and evaluation of alternative authentication concepts such as biometrics and graphical authentication techniques.

Chapter Eleven. Identifying Users from Their Typing Patterns

Alen Peacock, Xian Ke, and Matt Wilkerson

AS DEFICIENCIES OF TRADITIONAL PASSWORD-BASED ACCESS SYSTEMS BECOME MORE ACUTE, researchers have turned their focus to keystroke biometrics , an approach that seeks to identify individuals by their typing characteristics. Since 1980, a number of techniques have been proposed for accurately harnessing a user's unique typing pattern for system authentication and other novel uses. But do these systems deliver on their promise to increase system security and simultaneously ease the burden of logging into systems and remembering passwords? And do databases of users' keystroke profiles present additional privacy concerns?

Applications

The first suggested use of keystroke characteristics for identification appeared by 1975,[3] but observations about the uniqueness of an individual's typing characteristics stretch as far back as the end of the 19th century when telegraph operators observed that they could often identify one another by listening to the rhythm of each individual's Morse code keying pattern.[4] For a more complete introduction to research leading up to the invention of keystroke biometrics, see Leggett et al.[5] In this section, we highlight some of the most pertinent and interesting ways in which keystroke patterns can be applied.

The most common method used by researchers utilizes keystroke digraphs —the timings of two successive keyboard events. Figure 11-1 illustrates the various timing metrics that can be extracted from a digraph. Researchers have also examined techniques using three or more successive keystroke sequences, and have proposed measuring keystroke force and other features.

Generally, identification requires two stages:

Enrollment, in which a number of keystroke traces are collected to form a profile

Classification, in which the user provides a typing sample for comparison to the profiles in order to identify the user

Enrollment methods include requiring the user to type long texts, collecting traces transparently as the user works normally, and collecting traces only for a static set of words (such as the login typing pattern).

Classification methods range from statistical comparison to machine learning and pattern recognition techniques. Much of the body of research in keystroke biometrics focuses on creating and evaluating classifiers.

By far the most commonly researched usage scenario, keystroke-enhanced login , collects enrollment data from users while they sign into a system (either transparently, during a transition period, or in a session that collects a number of login samples in one or more sittings). Once enrolled, the system continues to collect keystroke timing information from each user during sign-in, using the initial profile to either accept or reject the user. If either the keystroke classifier rejects the user or the password value does not match the expected value, the user is rejected. The process is completely transparent to the user under such a scheme, as long as the keystroke biometrics classifier achieves accurate classification.

Authentication

Harnessing a user's typing patterns for authentication is highly attractive because it can be layered transparently onto existing systems. Applications involving financial transactions are excellent candidates for utilizing keystroke pattern authentication systems. Gartner Group estimates that online retailers in the U.S. lost .64 billion to fraudulent sales in 2002 and rejected another .82 billion in legitimate sales that looked suspicious.[6] Not only are these applications among the most likely to be targeted by attackers, but also consumers' tolerance for inconvenient security solutions is tempered by laws and regulations that frequently place the burden of financial loss on retailers. Because users would not perceive the difference between a traditional username/password login and one that had been modified to use keystroke biometrics , financial applications could add this type of authentication with little fear of user rejection.

Overview of Previous Research

The first studies on the effectiveness of keystroke characteristics as personal identifiers occurred in 1977[17] and 1980[18] (for a fuller treatment of work prior to 1990, see Joyce and Gupta[19]). Over the years, many different classifiers have been evaluated in an effort to improve recognition capabilities of keystroke biometrics , ranging from statistical analysis to neural networks. It is beyond the scope of this chapter to delve into the details of each approach. In general, each classifier measures the similarity between an input keystroke timing pattern and a reference model of the legitimate user's typing pattern. The model is built from training samples previously provided by each user and maintains varying characteristics depending on the classifier. The time required to generate each model also varies according to the classifier, with neural networks generally taking significantly longer than other approaches.

Table 11-1 compares the various experimental designs and techniques that have been analyzed in key published research. We include as much information as we have available from the relevant papers, though often experimental details are omitted from the primary source.[20]

Table 11-1. Comparison of published research, 1980–2004

Authors/Year

Input Data

Design

Features

Preprocessing

Classifiers

Notes

a Gaines et al.

b D. Umphress and G. Williams, "Identity Verification Through Keyboard Characteristics," International Journal of Man-Machine Studies 23: 3 (1985), 263–273.

c J. Leggett and G. Williams, "Verifying Identity Via Keystroke Characteristics," International Journal of Man-Machine Studies 28: 1 (1988), 67–76.

d Joyce and Gupta.

e S. Bleha, C. Slivinsky, and B. Hussein, "Computer-Access Security Systems Using Keystroke Dynamics," IEEE Transactions on Pattern Analysis and Machine Intelligence 12:12 (1990), 1217–1222.

f Leggett and Williams.

g S. A. Bleha, J. Knopp, and M. S. Obaidat, "Performance of the Perceptron Algorithm for the Classification of Computer Users," Proceedings of the 1992 ACM/SIGAPP Symposium on Applied Computing (ACM Press, 1992), 863–866.

h M. Brown and S. J. Rogers, "User identification Via Keystroke Characteristics of Typed Names Using Neural Networks," International Journal of Man-Machine Studies 39:6 (1993), 999–1014.

i M. S. Obaidat, "A Verification Methodology for Computer Systems Users," Proceedings of the 1995 ACM Symposium on Applied Computing (ACM Press, 1995), 258–262.

j D.-T. Lin, "Computer-Access Authentication with Neural Network Based Keystroke Identity Verification," IEEE International Conference on Neural Networks 1 (June 1997), 174–178.

k W. de Ru and J. Eloff, "Enhanced Password Authentication Through Fuzzy Logic," IEEE Expert 12 (Nov./Dec. 1997), 38–45.

l Song, Venable, and Perrig.

m J. A. Robinson, V. W. Liang, J. A. M. Chambers, and C. L. MacKenzie, "Computer User Verification Using Login String Keystroke Dynamics," IEEE Transactions on Systems, Man, and Cybernetics, Part A 28 (March 1998), 236–241.

n Monrose, Reiter, and Wetzel.

o Monrose and Rubin.

p Peacock.

q Cho, Han, Han, and Kim.

r S. Haider, A. Abbas, and A. K. Zaidi, "A Multi-Technique Approach for User Identification Through Keystroke Dynamics," IEEE International Conference on Systems, Man, and Cybernetics 2 (Oct. 2000), 1336–1341.

s Z. Changshui and S. Yanhua, "AR Model for Keystroker Verification," IEEE International Conference on Systems, Man, and Cybernetics 4 (Oct. 2000), 2887–2890.

t F. W. M. H. Wong, A. S. M. Supian, A. Ismail, L. W. Kin, and O. C. Soon, "Enhanced User Authentication Through Typing Biometrics with Artificial Neural Networks and K-Nearest Neighbor Algorithm," Conference Record of the Thirty-Fifth Asilomar Conference on Signals, Systems and Computers 2, (Nov. 2001), 911–915.

u F. Bergadano, D. Gunetti, and C. Picardi, "User Authentication Through Keystroke Dynamics," ACM Transacations on Information and System Security, 5:4 (2002), 367–397.

v Clarke et al.

w V. Kacholia and S. Pandit, "Biometric Authentication Using Random Distributions (BioART)," Proceedings of the 15th Canadian IT Security Symposium (CITSS), Government of Canada (May 2003).

x E. Yu and S. Cho, "GA-SVM Wrapper Approach for Feature Subset Selection in Keystroke Dynamics Identity Verification," Proceedings of the IEEE International Joint Conference on Neural Networks 3 (July 2003), 2253–2257.

Gaines, Lisowski, Press, and Shapiro a; 1980

Three 300–400 character passages

Seven professional secretaries typed two samples each with a delay of four months between samples

Interkey delays

Used only the 87 digraphs that had at least 10 or more replications per sample and per user; eliminated outliers; took logarithm of values

Two-sample t-test on whether the means of each value were the same assuming that variances were the same

Identified five core digraphs that discriminated perfectly: in, io, no, on, and ul

Umphress and Williamsb; 1985

Fixed 1,400-character reference input, 300-character test input

17 programmers typed samples with a delay of at least one month; errors allowed

Interkey delays

Single low-pass temporal filter to remove outliers

Closeness between test value and corresponding reference value, measured according to a standard deviation threshold and a passing ratio

Evaluating Previous Research

The majority of academic papers published on keystroke biometrics systems since 1980 have presented independent studies, each collecting its own samples from unique sets of individuals. These samples were collected through diverse methods, varying widely in the mechanics of user input, the granularity of measured data, the amount of input required to train the system and authenticate users, the number of test subjects employed, and the diversity of the typing experience of these subjects. Such nonuniformity alone makes comparison among different studies difficult; add to this difficulty the diversity of keystroke pattern classification approaches and the application of these technologies to different domains, and the task becomes more complex.

In this section, we will compare results from the field with commonly used metrics for measuring accuracy, and propose new metrics for measuring the usability of keystroke systems. Unfortunately, much of the literature in this area lacks sufficient reported data to measure all of the preceding features. We therefore restrict results reported in the following sections to the subset of reports reviewed in the earlier section titled "Overview of Previous Research" that do provide the necessary numbers. Notably missing in all these comparisons is the only commercial offering, for which published numbers are scant (although public documents at the BioNet web site[21] do claim that performance is on par with some of the earliest published results).

Classifier Accuracy

Three metrics are commonly used to describe biometrics classifier performance with regard to accuracy:

FRR. False Rejection Rate—the percentage of valid (genuine) user attempts identified as imposters

FAR. False Acceptance Rate—the percentage of imposter access attempts identified as valid users[22]

ERR. Equal Error Rate—the crossover point at which FRR = FAR, given some independent variable(s) that can be adjusted to produce curves for FRR and FAR

Although ERR is a desirable metric in terms of its ability to condense FAR and FRR into one value, the amount of data needed to turn FAR and FRR into curves (usually through the introduction of an independent variable) is usually prohibitive. Few researchers report ERR in their published results. For this reason, we present an alternative approach to combining FAR and FRR: averaging the two values. We will call this value:

AFR. Average False Rate—the average of FAR and FRR. Empirically, AFR closely approximates ERR for those few papers that did report ERR.

In 2000 and 2002, the U.K.'s Biometrics Working Group produced guidelines for "Best Practices in Testing and Reporting Performance of Biometric Devices."[23] Going forward, we hope that researchers in the field of keystroke typing patterns will consider these guidelines when reporting results. The main corpus of results we review herein, however, did not have the advantage of access to these standards, and therefore did not present data that we could use to produce such useful evaluation criteria as Receiving Operator Characteristic (ROC) or Detection Error Trade-off (DET) curves.[24] In this light, AFR can be viewed as a useful stopgap for comparing overall classifier accuracy.

Figure 11-2 graphs the performance of each of the highlighted systems in terms of AFR, FRR, and FAR. Although we make no claim about the validity of a system designed to favor either FAR/FRR over the other (such as password hardening[25]), we feel that in the absence of reported ERR, AFR is a good descriptor of the overall accuracy of a given classifier in terms of discriminating between users.

This figure demonstrates that the very best reported results are able to achieve an AFR of less than 1%, and roughly one-third are capable of AFR near 2%—values generally considered to be acceptable for this type of system. The worst performers have average AFR values between 8% and 27%, and are not likely to provide sufficient accuracy for common usage.

Usability

Privacy and Security Issues

If keystroke dynamics gain widespread acceptance, privacy and security issues must be evaluated carefully. Of concern are databases that maintain the keystroke timing patterns of users. With this information, attackers can subvert authentication systems that rely on keystroke biometrics.

If an attacker is able to obtain a particular user's keystroke timing profile, he may be able to guess which keys the user is typing simply by analyzing the timing of keystrokes. Such an attack may succeed against encrypted keystrokes as they are sent over a network, without ever needing to decrypt the data. For example, if it is known that the user spends 800 milliseconds typing the digraph io but usually spends around 1,400 milliseconds typing yr, an attacker can narrow the search space of plausible keystrokes, with the hope of finding an interpretation of the timings that results in a plausible original text. In 2001, researchers proved that this attack could work when they deciphered encrypted passwords sent through version 2 of the SSH protocol (used for remote access to computer terminals). Their technique allowed them to find the password an average of 50 times faster than brute force methods by utilizing a weakness in the protocol that allowed an attacker to determine precise timings for each keystroke and a database of user keystroke profiles.[31] Even when user-specific keystroke profiles are not available, generic keystroke profiles created from any representative population subset have been found to weaken security. In addition, the attacker may be able to employ this tactic with very low-tech means—for example, by using a tape recorder to listen to the click-clack of a user typing at a keyboard.

Conclusion

About the Authors

Chapter Twelve. The Usability of Security Devices

Ugo Piazzalunga, Paolo Salvaneschi, and Paolo Coffetti

AN INCREASING NUMBER OF HARDWARE DEVICES ARE BECOMING AVAILABLE TO HELP USERS ACHIEVE A higher level of security in authentication systems. However, little research has been performed on the usability of these devices. This chapter takes a first step toward understanding the usability issues associated with security devices. After briefly reviewing security devices, we define an experimental approach suitable for their usability evaluation. We apply this approach to evaluating cryptographic smart card devices, and we report and comment on the results, including the impact of usability on security. We conclude by presenting general recommendations to minimize usability problems while deploying security devices.

Overview of Security Devices

Security devices provide the "something I have" factor for authentication systems.[3] Passwords, an example of the "something I know" factor, lend themselves to a plethora of security attacks. Security devices can offer an alternative or, even better, a complement to the knowledge factor to increase security.

Take, for example, the process of withdrawing cash from an ATM. You need to insert your card (the "something you have") and enter its corresponding PIN ("something you know"). Similarly, a security device lets you securely withdraw precious electronic resources, like logging into your laptop or accessing your company's private networks, from anywhere.

There are a number of ways to categorize security devices. As you do with your ATM card, you need to carry your security device with you to access electronic resources. But, depending on the type of device, you may or may not need to physically insert it into your computer or a separate reader device. One way to distinguish among different security devices is according to whether they need to be physically plugged in, as follows:

Smart cards

Smart cards[4] are among the smallest computers we currently handle in our daily life. Smart cards are essentially plastic credit cards that include an integrated circuit (IC) whose current computational power is similar to the desktop PCs of the 1980s.[5] However, smart cards must be plugged into a reader. Other kinds of security devices do not require a special reader. Universal Serial Bus (USB) tokens, for example, use the USB port built into most computers.

One-time password (OTP) tokens

OTP tokens [6] fall into the category of security devices that do not have to be plugged in. Similar in shape to a small pocket calculator, OTP tokens display authentication data that users type in manually. The authentication data changes each time a user authenticates (or, in some cases, after a designated small time interval regardless of whether an authentication has taken place)—hence the name "one-time password" token. The fact that OTP tokens do not require a hardware reader means that they can be used with more kinds of computers than USB tokens can—for example, an airport Internet kiosk may not expose its USB ports to travellers. On the other hand, the fact that the user must physically type the value displayed on the OTP token can be an annoyance, make OTP tokens slower to use, and introduce the opportunity for transcription errors.

Devices that must be plugged in can be further subcategorized: can they be plugged into an existing port on a typical laptop or desktop computer, or do they require a special reader?

Passive storage versus active storage is an additional way to categorize security devices. Let's go back to our ATM card example: our "secret" authentication information is stored passively on the magnetic strip. On the other hand, OTP tokens and cryptographic smart cards embed a small computer that actively performs the computations needed for authentication.

We can further subcategorize "active" security devices by considering their external cryptographic functionalities. OTP tokens, for example, provide no cryptographic functionality

Usability Testing of Security Devices

Security devices, like other security technologies, are deployed in a physical and social context to allow users to complete their tasks. Thus, it is essential to perform usability studies on the system as a whole—in other words, taking into consideration the specific contexts, users, and tasks users must perform.[10] In the testing described in this chapter, we have tried to capture many of the important usability attributes that will be applicable generally. Do keep in mind, however, that while we are striving for general attributes, you should review your own actual environment to identify additional attributes that might affect your usability, or drop one or more of those we have identified.

Setting Up the Test

Our attributes definition draws on prior research[11] , [12] and the ISO 9126 standard.[13] The ISO 9126 software quality model defines six independent software quality characteristics, each of which is broken down into a set of subcharacteristics. However, usability studies of security devices deal not only with software but also with systems. Therefore, while the ISO standard is an important reference point, we have had to extend and adapt it to accommodate our specific aspects of interest.

The usability attributes we adopted are an extension of the following three ISO 9126 usability subcharacteristics:

Learnability. How difficult/easy is it for users to learn how to use the security devices?

Operability. How difficult is it for users and their organizations to carry out the assigned tasks properly while using the security devices?

Attractiveness. To what extent are the security devices attractive to users?

Operability is a very broad attribute. Therefore, we have broken it down into the following subattributes:[14] , [15]

Mobility. Organizations' procedures and structures are increasingly encompassing remote and mobile employees. Are applications that require security devices easily operable from different computers?

Installability. In order to operate security devices properly, users must be able to install them. Can users do so without too much difficulty?

User friendliness. Can users operate the devices easily? Are the devices prone to user error?

Low operating costs

A Usability Study of Cryptographic Smart Cards

This section describes the aim, scope, context, user selection, task definition, measurement apparatus, processing, and results of our usability study.

Aim and Scope

The aim of this usability study was to compare alternative form factors of cryptographic smart cards —that is, comparing traditional smart cards with USB tokens.

Smart cards are often praised for their usability:[21] they are mobile, can be used in multiple applications, and carry lower administrative costs than systems based on multiple usernames/passwords. On the other hand, smart cards are also criticized for their low market acceptance.[22] , [23] Garfinkel states that "few people use this [smart card] added security option because smart cards and readers are not widely deployed."[24] However, alternative form factors to the familiar plastic smart card are emerging, and proponents of these technologies claim that they overcome the limitations of smart cards.[25]

Context and Roles Definition

The scenario set up for this study compares three form factors: the traditional plastic smart card with a USB smart card reader, and two types of USB tokens (see Figure 12-1). We label these two types of USB tokens base and advanced; the advanced type is identical to the base one, but has an additional feature, as described shortly.

The base type of USB token integrates, in one single object, both the smart card reader and the cryptographic smart card IC. The IC embedded in the tokens used in the usability test is the same one embedded in the smart cards. The advanced USB token adds mass storage to the base type; when connected to the host system, this token will make available as separate resources both the smart card (and its reader) and a removable hard drive for general-purpose storage. The tokens used in our study contain 64 MB of storage and embed the same smart card IC found in the smart card and base token.

The "advanced" USB tokens' additional mass storage resource motivated our decision to include these tokens in our testing. We were interested in discovering whether usability

Figure 12-1. The three form factors deployed in the usability test

could be enhanced by deploying, in a single hardware device, not only cryptographic material but also software and data on how to use the software (e.g., installation software). On the other hand, smart card IC and mass storage components are isolated; therefore, this additional functionality does not introduce security vulnerabilities to the smart card IC. (Figure 12-2 shows a schematic diagram of both types of tokens.)

Figure 12-2. Schematic diagram of base and advanced USB tokens

Except for low-level drivers, all three form factors share the same middleware (e.g., Microsoft CAPI). The software used in the study was Microsoft Outlook Express running on the Windows XP and Windows 2000 operating systems. These were chosen so that users would see no difference at the software level while using any of the three devices.

Furthermore, given the standard interfaces between the application-level software and the devices provided by the middleware, the specific devices used in these experiments could just as well be replaced by any other cryptographic smart card or cryptographic USB token (so long as this replacement provided the same standards-compliant middleware). The outcome of this usability evaluation applies, therefore, to any specific instance of these kinds of devices.

The social context, or "setup," for the user tests draws inspiration from the work of Whitten and Tygar.[26] Each participant was told to imagine that they were responsible for the preparation and launch of an advertising campaign to promote a new product. Tasks for this job position included frequent travel between the different company sites, and most of the material to be delivered for the campaign had to be sent to colleagues through emails. Given the high competition in the market targeted by the new product, a strong level of security protection was demanded on all email communications. To this end, we prepared a cryptographic device with the user's personal digital certificates. The imaginary company provided a technical support team to assist the user, should any trouble arise with the device. However, the subject's manager advised the subject to minimize calls to the support team, as it had limited staff.

This scenario motivates users to actively "protect some secret they consider worth protecting."[27] and includes the use of security while carrying out certain tasks, as opposed to instructing participants "to perform a security task directly."[28] Indeed, security itself is not a user function. The user wants to send a protected email, not use an encryption algorithm. This means that security tools and devices should be integrated seamlessly into the applications.

The test's active participants were the user and the supervisor (experimenter). The supervisor had the following roles:

Drove the briefing phase, giving the user all the required inputs

Collected the measures during the test

Acted as the customer support service during the test execution

Drove the debriefing phase to collect the final questionnaire

User Selection

We selected 10 participants for the user test. All were in their second or third year of undergraduate studies at an engineering college. While all were skilled in the use of email and computers, none had any previous experience with securing email or cryptographic devices.

Task Definition

The user test consisted of the following three phases:

Briefing phase

Before the execution of the user test, the supervisor introduced each participant to the test scenario, described the task to be executed, and explained the role of the supervisor. The supervisor gave the user a brief document describing the context of the study and the tasks to be completed. A set of manuals on the installation and use of the devices was made available for reference, together with the hardware needed for the first form factor to be tested.

Execution phase

During this phase, the participant had to move across three company sites where her work was required. Three workstations in three different university labs simulated this setup. At each site, the user had to execute the following tasks:

Install drivers and software for the device.

Recommendations and Open Research Questions

What lessons have we learned about the usability of cryptographic smart cards? How many of these lessons can be generalized? We believe the following recommendations hold for any security device:

The system as a whole matters

Good behaviors, as well as dangerous ones, arise from the interaction of social, human, and technical components. Every evaluation or design activity must take into account the system as a whole.

Devices that require a reader affect mobility

Conclusion

Acknowledgments

We would like to thank Lorrie Cranor, Chris Long, and Karen Renaud for the many constructive comments and suggestions provided on a draft of this chapter.

About the Authors

Part III. Secure Systems

Chapter Thirteen. Guidelines and Strategies for Secure Interaction Design

Ka-Ping Yee

ALTHOUGH A RELIABLE, USABLE AUTHENTICATION METHOD IS ESSENTIAL, it is far from the only human interface concern. After a user signs in to a system, the system has to carry out the user's wishes correctly in order to be considered secure. The question of secure interaction design, addressed in this and the other chapters in this part of the book, is:

To give you a sense of how important it is to look beyond authentication, consider some of today's most serious security problems. Viruses are a leading contender, with email viruses making up a large part. Spyware is growing into a nightmare for home users and IT staff. Identity theft is becoming widespread, perpetrated in part through "phishing" scams in which forged email messages entice people to give away private information. None of these problems is caused by defeating a login mechanism. They would be better described as failures of computers to behave as their users expect.

This chapter suggests some guidelines for designing and evaluating usable secure software and proposes two strategies for getting security and usability to work in harmony: security by designation and user-assigned identifiers. I'll begin by providing a little background for our discussion, then present the guidelines and strategies, and finally look at real design problems to show how these strategies can be applied in practice.

Introduction

This section introduces the topic of secure interaction design , touching briefly on mental models , the apparent conflict between security and usability, and the overall issues underlying interaction design.

Mental Models

Design Guidelines

Having established this background, we are now ready to look at the main challenge of secure interaction design: minimizing the likelihood of undesired events while accomplishing the user's intended tasks correctly and as easily as possible. Let's dissect that general aim into more specific guidelines for software behavior.

Minimizing the risk of undesired events is a matter of controlling authorization. Limiting the authority of other parties to access valuable resources protects those resources from harm. The authorization aspect of the problem can be broken down into five guidelines:

Match the most comfortable way to do tasks with the least granting of authority.

Grant authority to others in accordance with user actions indicating consent.

Offer the user ways to reduce others' authority to access the user's resources.

Maintain accurate awareness of others' authority as relevant to user decisions.

Maintain accurate awareness of the user's own authority to access resources.

Accomplishing the user's intended tasks correctly depends on good communication between the user and the system. The user should be able to convey his or her desires to the system accurately and naturally. I'll discuss the following additional guidelines concerning the communication aspect of the problem:

These guidelines are built on straightforward logic and are gleaned from the experiences of security software designers. They are not experimentally proven, although the reasoning and examples given here should convince you that violating these guidelines is likely to lead to trouble. I'll present each guideline along with some questions to consider when trying to evaluate and improve designs. Strategies to help designs follow some of these guidelines are provided in the second half of this chapter.

Authorization

1. Match the most comfortable way to do tasks with the least granting of authority.

What are the typical user tasks? What is the user's path of least resistance for each one? What authorities are given to software components and other users when the user follows this path? How can the safest ways of accomplishing a task be made more comfortable, or the most comfortable ways made safer?

In 1975, Jerome Saltzer and Michael Schroeder wrote a landmark paper on computer security[4] proposing eight design principles; their principle of least privilege demands that we grant processes the minimum privilege necessary to perform their tasks. This guideline combines that principle with an acknowledgment of the reality of human preferences: when people are trying to get work done, they tend to choose methods that require less effort, are more familiar, or appear more obvious. Instead of fighting this impulse, use it to promote security. Associate greater risk with greater effort, less conventional operations, or less visible operations so that the user's natural tendency leads to safe operation.

A natural consequence of this guideline is to default to a lack of access and to require actions to grant additional access, instead of granting access and requiring actions to shut it off. (Saltzer and Schroeder called this fail-safe defaults.) Although computer users are often advised to change network parameters or turn off unnecessary services, the easiest and most obvious course of action is not to reconfigure anything. Wendy Mackay has identified many barriers to user customization: customization takes time, it can be hard to figure out, and users don't want to risk breaking their software.[5]

Consider Microsoft Internet Explorer's handling of remote software installation in the context of this guideline. Just before Internet Explorer runs downloaded software, it looks for a publisher's digital signature on the software and displays a confirmation window like the one shown in Figure 13-1. In previous versions of this prompt, the default choice was "Yes". As this guideline would suggest, the default is now "No", the safer choice. However, the prompt offers an option to "Always trust content" from the current source, but no option to never trust content from this source. Users who choose the safer path are assured a continuing series of bothersome prompts.

Figure 13-1. Internet Explorer 6.0 displays a software certificate

Regardless of the default settings, the choice being offered is poor: the user must either give the downloaded program complete access to all the user's resources, or not use the program at all. The prompt asks the user to be sure he or she trusts the distributor before proceeding. But if the user's task requires using the program, the most comfortable path is to click "Yes" without thinking. It will always be easier to just choose "Yes" than to choose "Yes" after researching the program and its origins. Designs that rely on users to assess the soundness of software are unrealistic. The principle of least privilege suggests that a mechanism for running programs with less authority would be better.

2. Grant authority to others in accordance with user actions indicating consent.

When does the system authorize software components or other users to access the user's resources? What user actions trigger these transfers of authority? Does the user consider these actions to indicate consent to such access?

The user's mental model of the system includes a set of expectations about who can do what. To prevent unpleasant surprises, we should ensure that other parties don't gain access that exceeds the user's expectations. When a program or another user is granted access to the user's resources, that granting should be related to some user action. If another party gains access without user action, the user lacks any opportunity to update his or her mental model to include knowledge of the other party's access.

The authorizing action doesn't have to be a security setting or a response to a prompt about security; ideally, it shouldn't feel like a security task at all. It should just be some action that the user associates with the granting of that power. For example, if we ask users whether they expect that double-clicking on a Microsoft Word document would give Word access to its contents, and the vast majority say yes, the double-click alone is sufficient to authorize Word to access the document.

Design Strategies

Two strategies—security by designation and user-assigned identifiers—can be used to implement some of the aforementioned guidelines in a broad range of situations. After describing them, I'll propose ways to solve some everyday security problems using these strategies.

Security by Admonition and Security by Designation

Users often want to make use of things they do not completely trust. For example, it's reasonable for people to want to run programs or visit web pages without having to understand and audit their source code. Instead of trusting the unknown entity, users trust an agent (such as a secure operating system or a secure web browser) to protect their interests by placing constraints on the unknown entity. The agent's challenge is to determine the correct constraints.

Consider two contrasting styles of establishing these security constraints that I'll call security by admonition and security by designation. Security by admonition consists of providing notifications to which the user attends in order to maintain security. With security by designation, the user simultaneously designates an action and conveys the authority to perform the action.

The case of an operating system constraining a potentially dangerous program provides an example to illustrate these concepts. I'll use Venn diagrams to depict the space of the program's possible actions. In each run of the program, its actions form a path through this space, represented by a sequence of black dots. Arrows represent user actions that trigger these program actions. The rectangle delimits the actions that the system allows; it is a simple shape to signify the rigidity of a typical policy specification. The shaded region is the set of actions acceptable to the user; its shape is irregular to signify that the user's desires are probably complex and imprecisely specified. Figure 13-5 displays these notational conventions.

Figure 13-5. This Venn diagram of the space of program actions shows the notational conventions to be used in the next two figures

Security by admonition

Figure 13-6 shows a system that enforces a static, preestablished security policy. Because the policy is only finitely detailed, it can only roughly approximate the user's desires. Because the policy is static, it must accommodate a wide range of situations other than this particular run of the program. Thus, the solid box in Figure 13-6 includes large areas outside the shaded region. On Mac OS, Unix, and Microsoft Windows systems, programs usually run with the user's full authority, creating a vast discrepancy between the sets of allowed and acceptable actions. For example, even though I may start a program intending to edit only one file, the system allows it to modify or delete any of my files, send email in my name, upload my files to the Internet, and so on.

Figure 13-6. With security by admonition, the system has to guess when to warn the user of potential dangers

To prevent the program from taking undesirable actions, the admonition approach would be to show a warning so that the user has an opportunity to intervene before the undesirable action happens. Commercial products like ZoneAlarm (see Chapter Twenty Seven) do just this, stepping in with a prompt when programs access the network. Unfortunately, the computer can't read the user's mind, so it has to guess when to warn and when to proceed. Warning too little places the user at risk; warning too much annoys the user. The larger the discrepancy between acceptable and allowed actions, the more severely we are forced to compromise between security and usability.

Security by designation

Figure 13-7 acknowledges that user expectations change over time and that the security policy should change to match. In this approach, the program starts out with minimal authority (solid box). As before, the user can interact with the program's user interface (solid arrow) to have the program carry out actions using its initial authority. When the user wants to take an action requiring new authority, the user interacts with the system's user interface (dotted arrows) to express both the command and the extension of authority (dotted box) at the same time. Combining the authorization with the designation of intent in a single user action maintains a close, dynamic match between the allowed set and the acceptable set of actions. Users don't have to establish a detailed security policy beforehand and don't have to express their intentions twice.

Figure 13-7. With security by designation, the user simultaneously designates an action and conveys the authority to take that action

User-Assigned Identifiers

Practically every user interaction depends on the correct designation of an object or action. The names by which a user refers to objects and actions come from many different sources: some are assigned by the user, some are assigned by other users, some are allocated by global or local organizations, and some are assigned by programs.

Giving objects the power to choose their own names places the user at a significant disadvantage. Unless another naming mechanism is provided, the user is forced to work with identifiers controlled by outside parties—potentially, to communicate with his or her most trusted software agents using terms defined by an adversary. Each name that enters the user's sphere is a potential avenue of attack if it can be made misleading or confusing.

A good way to avoid the danger of identification problems is to let users assign and use their own local identifiers for objects and actions. Computers are harder to confuse than humans; they can take care of the translation between users' familiar names and the names used by machines or other people.

You already use user-assigned identifiers all the time. For example, each file on your desktop has a name that you can assign. The true, unique identifier for each file is a number that tells the system how to locate the file on the disk. But you never have to deal with that number; you assign a filename, and the operating system takes care of translating between the name and the number. Imagine how difficult it would be to identify files if the desktop displayed only disk addresses instead of filenames, or how inconvenient it would be if files all chose names for themselves that you couldn't change.

Conclusion

Acknowledgments

The 10 guidelines in this chapter are based on an earlier set of principles[23]

About the Author

Ka-Ping Yee is a Ph.D. student at the University of California, Berkeley. His research interests include interaction design, capability-based security, information visualization, and computer-supported argumentation.

http://zesty.ca/

Chapter Fourteen. Fighting Phishing at the User Interface

Robert C. Miller and Min Wu

AS PEOPLE INCREASINGLY RELY ON THE iNTERNET FOR BUSINESS, PERSONAL FINANCE, AND INVESTMENT, Internet fraud becomes a greater and greater threat. Internet fraud takes many forms, from phony items offered for sale on eBay, to scurrilous rumors that manipulate stock prices, to scams that promise great riches if you will help a foreign financial transaction through your own bank account.

One interesting and fast-growing species of Internet fraud is phishing . Phishing attacks use email messages and web sites designed to look as if they come from a known and legitimate organization, in order to deceive users into disclosing personal, financial, or computer account information. The attacker can then use this information for criminal purposes, such as identity theft, larceny, or fraud. Users are tricked into disclosing their information either by providing it through a web form or by downloading and installing hostile software.

A phishing attack succeeds when a user is tricked into forming an inaccurate mental model of an online interaction and thus takes actions that have effects contrary to the user's intentions. Because inferring a user's intentions can be difficult, building an automated system to protect users from phishing attacks is a challenging problem.

Introduction

Phishing attacks are rapidly increasing in frequency; many are good enough to fool users. According to the Anti-Phishing Working Group (APWG) ,[1] reports of phishing attacks increased by 180% in April 2004 alone, and by 4,000% in the six months prior to April. A recent study done by the antispam firm MailFrontier Inc. found that phishing emails fooled users 28% of the time.[2] Estimates of losses resulting from phishing approached million in 2002.[3]

Anatomy of a Phishing Attack

The Anti-Phishing Working Group collects and archives examples of phishing attacks, a valuable service because the web site used in an attack exists only for a short time. One example on APWG is an attack against eBay customers, first reported on March 9, 2004.[4]

The attack begins when the potential victim receives an email (Figure 14-1

Attack Techniques

Phishing attacks use a variety of techniques to make the presentation of an email message or web page deceptively different from its implementation. In this section, we catalog a few of the techniques that have been seen in the wild:

Copying images and page designs

A phishing attack often copies a legitimate page nearly verbatim, including its style, layout, and embedded images. The eBay attack shown in Figure 14-2 is an example of this approach. Because logos and page style are the most prominent indicators of a site's identity—the "face" of a web site—unsuspecting users are likely to fall for this very simple deception.

Similar domain names

Another way that users authenticate web sites is by examining the URL displayed in the address bar. To deceive this indicator, the attacker may register a domain name that bears a superficial similarity to the imitated site's domain. Sometimes a variation in capitalization or use of special characters is effective. Because most browsers display the URL in a sans-serif font, http://paypaI.com has been used to spoof http://paypal.com, and http://barcIays.com

Defenses

As we showed earlier in the example of the eBay attack, we can separate an online interaction into four steps (Figure 14-5):

Message retrieval. An email message or web page arrives at the user's personal computer from the Internet.

Presentation. The message is displayed in the user interface, the user perceives it, and the user forms a mental model.

Action. Guided by the mental model, the user performs an action in the user interface, such as clicking a link or filling in a form.

System operation. The user's action is translated into system operations, such as connecting to a web server and submitting data.

In this section, we survey existing defenses against phishing attacks, classifying them according to which of these four steps they address.

Figure 14-5. Four steps of human-Internet interaction

Message Retrieval

In an ideal world, the best defense against phishing would simply block all phishing communications from being shown to the user, by filtering them at message retrieval time. The essential requirement for this solution is that the computer alone must be able to accurately differentiate phishing messages from legitimate ones. Defenses that filter at message retrieval depend on message properties that are easily understood by a computer.

Identity of the sender

One of these properties is the identity of the sender. Black listing is widely used to block potentially dangerous or unwelcome messages, such as spam. If the sender's IP address is found in a black list, the incoming message can be categorized as spam or even simply rejected without informing the user. A black list may be managed by an individual user, the approach taken by Internet Explorer's Content Advisor (Figure 14-6). Alternatively, it may be managed by an organization or by collaboration among many users. For phishing, the EarthLink Toolbar alerts the user about web pages that are found on a black list of known fraudulent sites.[8]

Figure 14-6. Internet Explorer's Content Advisor

Black listing is unlikely to be an effective defense on today's Internet, because it is so easy to generate new identities such as new email addresses and new domain names. Even new IP addresses are cheap and easy to obtain. The black list must be updated constantly to warn users about dangerous messages from newly created sources. Because phishing sites exist for only a short time, the black list must be updated within hours or minutes in order to be effective at blocking the attack.

The converse of black listing is white listing, allowing users to see messages only from a list of acceptable sources. For example, Secure Browser controls where users may browse on the Internet using a list of permitted URLs.[9] White listing avoids the new-identity problem because newly created sources are initially marked as unacceptable. But defining the white list is a serious problem. Because it is impossible to predict where a user might need to browse, a predefined, fixed white list invariably blocks users from accessing legitimate web sites. On the other hand, a dynamic white list that needs the user's involvement puts a burden on users because, for every site they want to visit, they must first decide whether to put it in the white list. This also creates vulnerability: if a phishing site can convince users to submit sensitive data to it, it may also be able to convince them to put it into a white list.

Textual content of the message

Another property amenable to message filtering is the textual content of the message. This kind of content analysis is used widely in antispam and antivirus solutions. Dangerous messages are detected by searching for well-known patterns, such as spam keywords and virus code signatures. In order to beat content analysis, an attacker can tweak the content to bypass the well-known filtering rules. For example, encryption and compression are added to existing viruses in order to bypass antivirus scans.[10] Random characters are inserted into spam emails to enable them to bypass spam filters. One sophisticated phishing attack used images to display text messages so that they would defeat content analysis.[11]

Spam filtering is one defense that applies at message retrieval time. Because nearly all phishing attacks are currently launched by spam, getting spam under control may reduce the risk of phishing attacks significantly. Unfortunately, the techniques used by many spam filters, which scan for keywords in the message content to distinguish spam from legitimate mail, are insufficient for classifying phishing attacks, because phishing messages are designed expressly to mimic legitimate mail from organizations with which the user already has a relationship. Even if spam filters manage to reduce the spam problem substantially, we can anticipate that phishing will move to other transmission vectors, such as anonymous comments on discussion web sites, or narrowly targeted email attacks rather than broadcast spam.

Presentation

When a message is presented to the user, in either an email client or a web browser, the user interface can provide visual cues to help the user decide whether the message is legitimate.

Current web browsers reflect information about the source and integrity of a web page through a set of visual cues. For example, the address bar at the top of the window displays the URL of the retrieved web page. A lock icon, typically found in the status bar, indicates whether the page was retrieved through an encrypted, authenticated connection. These cues are currently the most widely deployed and most user-accessible defenses against phishing, and security advisories about phishing warn users to pay close attention to them at all times.[12] , [13] ,

Looking Ahead

Phishing attacks are likely to grow more sophisticated in the days ahead, and our defenses against them must continue to improve. Phishing succeeds because of a gap between the user's mental model and the true implementation, so promising technical solutions should try to bridge this gap, either by finding ways to visualize for the user details of implementation that would otherwise be invisible, or by finding ways to see the message from the user's point of view.

About the Authors

Chapter Fifteen. Sanitization and Usability

Simson Garfinkel

DELETION POSES A FUNDAMENTAL QUANDARY TO SECURE USABILITY. On the one hand, users would like to be able to undo their mistakes—to undelete files after they have been deleted accidentally. On the other hand, users would like to know that sensitive data is actually removed from a disk when they delete it—removed so that it cannot be recovered by an adversary.

Anybody who has a paper shredder lives with this quandary. If you get one of those preapproved credit card offers in the mail and you don't need it, you can always just throw it into a recycling bin. If you change your mind and decide that the 0% introductory rate might help you finance a new laptop, you can always pull the offer out of the bin and fill it out. Of course, these preapproved offers can also be used by crooks in the commission of identity theft: if you are really sure that you don't want to take out that new credit card, you're better off shredding the offer and perhaps even the envelope in which it came. The best paper shredders make it easy for you to inspect the chad to make sure that the information is no longer intelligible. Some companies that care about their data security let their employees throw whatever documents they wish into recycling, but the paper is then shredded before it is given to a waste hauler.

The Remembrance of Data Passed Study

In August 1998, I was chief technology officer of a computer security start-up. One of my jobs involved setting up a test bed of modem-equipped computers that would answer incoming phone calls and respond with a variety of different prompts. Instead of purchasing new computers for this somewhat mundane task, I bought 10 used machines at each from a small-town computer store. Most of the computers had been sitting on a shelf for more than a year, and the store's owner didn't even know if they worked. My plan was to mix and match the components until I had five or six operational systems.

When I got the computers back to my house and started to inventory the parts, I discovered that the computer store had neglected to sanitize the hard drives prior to selling me the machines. Intrigued, I inventoried the drives and discovered the following:

One of the larger machines, a '486-class system with a 40-gigabyte hard drive, had been the Novell file server for a small law firm. The computer had considerable client material on it, including contracts, wills, and billing records.

A second computer had been used by an organization that delivered mental health services to community residents under contract to a state agency. The computer included a FileMaker Pro database that had the names, addresses, and diagnoses of several dozen individuals living in the community.

A third machine had belonged to a writer who worked for a national magazine and also wrote novels. This machine contained many unpublished works, works-in-progress, and correspondence.

A fourth machine had letters sent between a woman and her daughter in college. This computer also had a copy of Quicken, which the woman apparently used to manage her finances.

All of this information was visible once the computers were turned on; no special disk recovery software was needed at all. I called the store's owner. Once he got over his shock and embarrassment, he asked me to wipe the systems as a favor. Apparently, he had meant to sanitize the machines before he sold them, but he had forgotten to do so.

Other Anecdotal Information

My experience with data left on disks that were subsequently sold on the secondary market is hardly unique. In recent years, there have been numerous reports of such cases, including:

In April 1997, a woman in Pahrump, Nevada, purchased a used IBM PC and discovered records from 2,000 patients who had prescriptions filled at a Smitty's Supermarkets pharmacy in Tempe, Arizona.[1]

In August 2001, more than 100 computers from the consulting firm Viant containing confidential client data were sold at auction by Dovebid following the closure of Viant's San Francisco office.[2]

In spring 2002, the Pennsylvania State Department of Labor and Industry sold computers containing "thousands of files of information about state employees."[3]

In August 2002, a Purdue student purchased a used Macintosh computer at the equipment exchange and discovered that the computer contained a FileMaker database with names and demographic information of 100 applicants to the Entomology Department.

Also in August 2002, the United States Veterans Administration Medical Center in Indianapolis retired 139 computers. Some of these systems were donated to schools, others were sold on the open market, and at least three ended up in a thrift shop where they were purchased by a journalist. Examination of the computer hard drives revealed sensitive medical information, including the names of veterans with AIDS and mental health problems. Also found were 44 credit card numbers used by the Indianapolis facility.[4]

In June 2004, the UK computer security firm Pointsec purchased 100 hard disks on eBay as part of a project on the "life cycle of a lost laptop." Although all of the hard drives had "supposedly" been "wiped clean" or "reformatted," the company was able to recover data from approximately 70 of the drives. The company also purchased laptops at auction that had been lost at airport terminals in England, Germany, Sweden, and the U.S. and verified that police did not sanitize the laptops prior to selling them. Reportedly, the laptop recovered from Sweden "contained sensitive information from a large food manufacturer. The data recovered included four Microsoft Access databases containing company and customer-related information and 15 Microsoft PowerPoint presentations containing highly sensitive company information."[5]

While these cases are certainly notable, they represent a tiny fraction of the number of hard disks that are being repurposed, recycled, or otherwise resold on the secondary market.

According to the market research firm Dataquest,[6] nearly 150 million disk drives will be retired in 2002—up from 130 million in 2001. Dataquest estimates that 7 disk drives will be retired for every 10 drives that ship in the year 2002; this is up from a 3-for-10 rate of retirement in 1997. Thus, more and more drives are being retired every year!

But the term retired is something of a misnomer. As the experience at the VA Hospital demonstrates, many disk drives that are "retired" by one organization can appear elsewhere. Indeed, mainstream businesses are increasingly turning to used equipment in an effort to cut costs—the editors at CIO Magazine even ran a cover story giving their readers advice on finding the best deals.[7]

These anecdotal reports are interesting both because of their similarity to each other and because of their relative scarcity. Clearly, confidential information has been disclosed through computers sold on the secondary market more than a few times. Why, then, have there been so few reports of unintended disclosure?

In the initial publication detailing this study,[8] Shelat and I proposed three possible hypotheses to answer this question:

Disclosure of so-called "data passed" information, while it occurs from time to time, is nevertheless exceedingly rare.

Confidential information might be disclosed so often on retired systems that such events are simply not newsworthy.

While used equipment is awash with confidential information, nobody is looking for it—or at least, few people who are looking for this data are publicizing the fact.

This chapter argues that the third hypothesis is correct. Based on a combination of the information found on the drives and interviews conducted with some of the original data owners, it seems that most confidential information on these "retired" drives is erased but not overwritten. As a result, I believe that many repurposed drives contain significant amounts of personal or confidential information, but few of the drives' current users are aware of this fact.

Study Methodology

Between January 1999 and January 2003, I purchased 235 used hard drives on the secondary market in an effort to determine what information they contained and what, if any, means were taken to clean the drives before they were discarded. Initially the drives were purchased at used computer stores such as WeirdStuff in Sunnyvale, California, and PC Recycle in Belleview, Washington. The majority of drives were purchased as the result of winning bids on the eBay online auction web site. Most purchases consisted of between 3 and 5 drives; in no case were more than 20 drives at a time from the same vendor.

Modern hard disks store information in individually addressable blocks , with each block being 512 bytes in length. A 50-gigabyte disk thus has approximately 10 million blocks.

On receipt, each drive was cataloged and entered into a database. Each drive was then attached to a computer running the FreeBSD operating system and the contents were copied off, block for block, using the command:

dd if=/dev/ad2 of=NNN.img conv=noerror,sync

where /dev/ad2 is the raw device of the disk, noerror instructs that the dd command should continue copying data even if an error is encountered, and sync

Related Work: Sanitization Standards, Software, and Practices

Confidential information that is stored on a computer system can be protected by numerous mechanisms, including operating system mechanisms, physical isolation of the computer system itself, and even the use of cryptography. The first two of these techniques fail when media is physically discarded: discarded media is, by definition, no longer contained within a protected perimeter. And once a disk or tape drive is removed from the computer on which it was created and is taken elsewhere, other operating systems are free to honor or ignore meta information such as file protection attributes. Only cryptographic measures survive when a disk is discarded. Yet experience has shown that many kinds of confidential information are not stored with encryption. Interestingly, the risk that confidential information could be released inadvertently on discarded media has been recognized since the 1960s.[13]

DoD 5220.22-M

In the 1970s, each of the armed services adopted their own standards for sanitizing discarded media. These were combined into the unified Department of Defense standard 5220.22-M.[14] According to the standard, nonremovable rigid disks may be "cleared" simply by "overwrit[ing] all addressable locations with a single character," which is generally taken to mean writing an ASCII NUL onto all user-addressable locations. Sanitizing requires that the media be degaussed with a Type I or Type II degausser, that a three-pass sanitization process be implemented, or that the disk be physically destroyed—"disintegrate, incinerate, pulverize, shred, or melt."

DoD's three-pass procedure has been the subject of some debate. According to the standard, the procedure is: "Overwrite all locations with a character, its complement, then a random character and verify. THIS METHOD IS NOT APPROVED FOR SANITIZING MEDIA THAT CONTAINS TOP SECRET INFORMATION." [Emphasis in original.] This is normally implemented with the following algorithm:

Seed a cryptographically secure random number generator with a random seed R1.

Write the output of the generator to the entire disk.

Reseed the random number generator with the same seed R1.

Write the complement[15] of the generator's output to the entire disk.

Seed the generator with a new random seed R2.

Write the generator's output to the entire disk.

Reseed the random number generator with the seed R2.

Read the contents of the entire disk, verifying that the contents of each block match the output of the random number generator.

This process is designed to guard against a hard-disk drive that claims to be accepting write commands but that does not actually write data onto the disk.

There has been much debate as to why the standard's authors have taken such great pains to specify that this standard is not approved for sanitizing media that contains "top secret" information. The implication is that organizations such as the National Security Agency have capabilities for recovering information that has been overwritten using this procedure. An alternative explanation is that the NSA does not possess this capability, and that the NSA doesn't want the targets of its collection efforts to employ 5220-22-M to sanitize their media!

Peter Gutmann explored the issue of recovering overwritten information in his 1996 USENIX paper, "Secure Deletion of Data from Magnetic and Solid-State Memory." Gutmann notes that it should be possible to recover information that has been overwritten once or twice—or possibly even more times—depending on the particular recording technology in question. For example, when a 1 is written over a 0, the resulting magnetic field sensed by the read head is going to be different from when a 1 is written over a 1. Gutmann then presents a variety of patterns for overwriting data that are designed to interact with various recording schemes used by different generations of disk drives. In total, 35 different patterns are presented, covering several generations of disk-drive technology.

Surprisingly, a few developers of sanitization tools have adopted all 35 patterns, writing one after the other. Gutmann sneered at this misreading of his paper, and added this postscript to the online version:

It is readily apparent that overwriting provides sufficient sanitization for the vast majority of computer users. Yet overwriting has not been built into operating systems until very recently, and even current operating systems do not implement it in a uniform or even consistent manner. This creates profound usability problems: not only does the computer user need to understand that the deleted files aren't actually deleted, but also the user must obtain and use specially written software to properly sanitize his media if he wishes to discard it without risk.

The remainder of this section discusses a variety of techniques that have been developed to allow end users to overwrite data—either selectively or across an entire drive.

One of the simplest ways to encrypt data on a disk is to use a cryptographic filesystem. Such filesystems can operate on a file-by-file basis, as is the case with Matt Blaze's Cryptographic File System[a] and Microsoft's Encrypted File System, or they can operate on individual blocks, as is the case with PGP:Disk.

Another approach is to use an active hardware device that sits between the computer and the hard disk. Such devices have been made by a variety of manufacturers for both tape drives and disk drives.

The cryptographic approach has also been applied to the problem of email sanitization. The system, developed in the 1990s by a company that was coyly known as Disappearing Ink , turned ordinary HTML email into JavaScript-enabled email that contained a small decryption engine and a block of encrypted data. Each message would have a certain date, D, after which it could not be read. When the engine was run by the receiving mail client, the engine would download the decryption key for day D from the Internet and, if the key was still available, decrypt the message. Disappearing Ink promised that on day D +1, it would delete the key for day D

Moving Forward: A Plan for Clean Computing

Although the need for proper sanitization of magnetic recording media has long been recognized as a serious issue for computer security practitioners, the problem has traditionally been addressed through the use of add-on software or physical destruction of the media itself. Only recently has the question of sanitization been addressed by computer operating system vendors themselves, and in the cases that we have considered, both Microsoft and Apple have addressed it poorly.

Some researchers have told me that operating system developers did not deploy sanitizing file deletion in the 1970s and 1980s so that accidentally deleted files could be recovered using special tools. I have looked and have found no evidence to support this claim. Indeed, had recoverability of accidentally deleted data been a goal, companies like Microsoft and Apple would have distributed such tools themselves—either as part of their operating system offerings or as aftermarket additions. (DOS 5.0 included an UNFORMAT command-line utility that could, in some cases, recover a volume that had been formatted inadvertently. But this looks like an after-the-fact attempt to make use of a DOS 2.0 idiosyncrasy, rather than an intentional design attribute that was put into DOS 2.0 and realized only when DOS 5.0 finally shipped.)

Instead, it seems that the lack of a sanitizing delete-file is a historical accident. The first multiuser computer systems did not need a sanitizing delete because of the ways in which they were operated. The developers of the Compatible Time Sharing System (CTSS)

Acknowledgments

The credit card numbers in the forensics study were found using a program written by Abhi Shelat. David Clark and Ron Rivest provided much support, suggestions, and guidance on the original forensics study, as did Brian Carrier, Peter Gutmann, Rich Mahn, Eric Thompson, and Wietse Venema. Matthew Geiger and Lorrie Cranor provided useful comments on an earlier draft of this chapter.

About the Author

Simson Garfinkel has been working in the field of computer security since 1986. He recently completed his dissertation on patterns and principles for aligning usability and security at MIT and is now a postdoctoral fellow at Harvard University's Center for Research on Computation and Society.

http://www.simson.net/

Chapter Sixteen. Making the Impossible Easy: Usable PKI

Dirk Balfanz, Glenn Durfee, and D. K. Smetters

THE WIDESPREAD PERCEPTION THAT USABILITY AND SECURITY ARE AT ODDS WITH ONE ANOTHER OFTEN leads systems designers to shun powerful security technologies. A quintessential example is provided by public key infrastructure (PKI) technology: despite the high degree of security PKI technology can provide, designers frequently avoid this technology because of its notoriously complex deployment and the incomprehensibility of such an infrastructure to end users.

This chapter explains how by designing usability in from the start, one can make PKI-based systems easy to deploy and use. The resulting systems, however, are not large, general-purpose infrastructures, but PKIs that are small, dedicated, easy to set up, and application specific. We refer to these as instant PKIs (iPKIs). Several case studies illustrate interaction paradigms for building such usable, secure iPKIs.

Public Key Infrastructures

Before the invention of public key cryptography, methods for secure digital communications were all but unavailable to mainstream computer users. The reason for this was the difficult problem of key distribution

Problems with Public Key Infrastructures

PKI technology was originally designed to be deployed globally. The idea was to have a single certificate infrastructure in which all users and devices would participate. Despite all the advantages of such a system, this has proven impractical: the amount of coordination and trust required to establish a global infrastructure is enormous, and no one has been able to do it in the 25 years since a global PKI was originally proposed.

Apart from such political problems, usability—or, rather, a lack of usability—is the main reason for the failure of public key infrastructures. Usability issues hamper PKI deployment even at much smaller scales, such as within a single company or within a supply chain. Here is a list of usability problems that people are confronted with when trying to use a PKI:

Making PKI Usable

Can PKI technology be made usable? The following examples argue strongly that with new approaches to design, it is possible to build "instant" public key infrastructures that are both easy to use and, through the use of public key cryptography, highly secure.

Case Study: "Network-in-a-Box"

Wireless local area networks are becoming increasingly popular in homes and small offices. Unfortunately, such networks have been subject to a number of high-profile security problems, leaving users' machines vulnerable to attackers—not only those on their block, but also those accessing their network from quite a distance using high-gain antennas. Such networks can be secured, using new standards. However, the more secure the network, the more difficult it usually is to set up and configure (see the following sidebar for an example).

When we set up a wireless local area network (WLAN) at the Palo Alto Research Center (PARC), we opted for a PKI-based solution. This promised better security and better ease of use than a password-based system. The idea was to give each of about 200 users an X.509 certificate and to use the 802.1x wireless standard's Extensible Authentication Protocol in TLS mode (802.1x EAP-TLS[a] , [b]) to authenticate to the wireless network.[c] Users requested and retrieved their certificates via a fairly automated, standard web-based interface, and configured their devices via the GUI-based 802.1x configuration software supplied with Microsoft Windows XP. To help users enroll, our administrators produced an elaborate set of instructions.

We studied eight subjects' experiences enrolling in the wireless PKI. Our subjects were sophisticated computer users, typically holding Ph.D.s in computer science. Despite using the GUI-based interface for enrollment and configuration of their machines, the process involved a total of 38 distinct steps. Each of these presented an opportunity for end users to make frustrating mistakes. The average time that it took them to request and retrieve their certificate and then configure their system was 140 minutes.[d]

Almost all of the subjects printed the instructions, and even those determined to understand what they were doing soon began following the instructions mechanically. In the end, many test subjects described enrollment as the most difficult computer task that PARC had ever asked them to do. All subjects had little idea of precisely what they had done to their computers. Several commented that if something were to go wrong, they could not perform even basic troubleshooting. For several subjects, this was the first time that they had ever experienced the inability to administer their own machines. Ironically, while PKI technology may have secured their machines for wireless use, it simultaneously reduced these end users' ability to configure and maintain their own machines.

A recent study investigates how to build a wireless network that is extremely easy to set up and that, at the same time, provides the strongest grade of wireless security currently available. The authors of the study call their system a "Network-in-a-Box" (NiaB).[7]

This high level of security is provided by the 802.1x protocol,[8] which authenticates each wireless client before it can access the network, and then provides them with encryption keys that change rapidly and automatically to protect their data. Networks that use 802.1x ask clients to authenticate themselves using a choice of mechanisms. The most secure of these is a protocol called EAP-TLS,[9] which asks clients and the wireless infrastructure to mutually authenticate each other using digital certificates via the standard TLS[10] key exchange protocol. Deploying such a network means setting up a certification authority and an authentication infrastructure, and issuing certificates to each new client. As the example in the earlier sidebar shows, this is extremely difficult in an enterprise staffed by professional administrators. It is normally inconceivable for an average home user.

What was needed was a small-scale PKI (an instant PKI), one that covered a single wireless network and its clients, and one that could be configured and set up automatically. Users also needed a secure way to indicate which clients should be able to join that network, and a way to automatically have those clients obtain a certificate and be set up to use it.

The NiaB access point (NiaB AP) , shown in Figure 16-1, is the solution to these problems. It provides a complete 802.1x-secured wireless network.[11] Clients who want to enroll in that network run a small enrollment application. In addition to being an access point, the NiaB AP has the authentication services necessary to support 802.1x, a certification authority to issue certificates to wireless clients, and an enrollment component (described later) to add new clients to its network securely. When the NiaB AP is switched on for the first time, it configures itself automatically. The access point component chooses a wireless network name and channel. The CA component generates a root key pair and root certificate. The NiaB AP is configured to automatically use an upstream Internet connection if one is available, and to act as a gateway for its clients—it provides all of the services necessary for its wireless clients to safely access the Internet, such as a DHCP server, a firewall, etc. Even if the NiaB AP does not have an external connection to the Internet, it still provides a useful wireless network to its clients, allowing a group of users to easily configure a secure local wireless network.

Figure 16-1. Network-in-a-Box makes it easy to enroll in a secure wireless network

The process to easily and securely add new clients to this network is called gesture-directed automatic configuration : a user wishing to add his laptop to the NiaB network runs a small enrollment application, which tells him to "point out" the NiaB AP whose network he wishes to join. In the NiaB implementation, he points out the AP using the infrared (IR) port on his laptop (although other mechanisms could be used[12]). For a second or two, the devices exchange a small amount of information over infrared. Then, the user is prompted to separate the devices to continue the automatic configuration of the laptop. After a few more seconds, the user is informed that his laptop is ready to use. These simple steps provide a previously unconfigured laptop with everything needed to get a "network dial tone."

This gesture-based approach to configuration is simple and intuitive for the user and, at the same time, highly secure. When the user's laptop and the NiaB AP communicate over infrared, they are actually exchanging digital fingerprint information with each other that will allow them to securely recognize each other's public keys, and the AP tells the client the name of the wireless network it should join. The infrared channel is an example of a location-limited channel.[13] Location-limited channels are channels that can be used to bootstrap secure communications between devices.

The client enrollment software on the laptop then makes a secure connection over the wireless network to the NiaB AP, authenticating each other by proving they have the private keys corresponding to the public key fingerprints they exchanged over IR. This takes the user's clear intention to have his laptop talk to that particular

About the Authors

Chapter Seventeen. Simple Desktop Security with Chameleon

A. Chris Long and Courtney Moskowitz

CHAMELEON IS A DESKTOP INTERFACE AIMED AT HOME COMPUTER USERS THAT IS DESIGNED TO REDUCE the damage caused by malicious software, or malware —for example, viruses, worms, and Trojan horses. Malware is especially a problem for home computer users, in part because on most home computers all software runs with full access to all parts of the system. For example, an email attachment or a file downloaded from the Web has freedom to do anything to any part of the computer.

Introduction

The Chameleon design philosophy is to put the user, and thus the user interface, first. Frequently, security practitioners design detailed security models and mechanisms, then implement them in software or hardware, then design interfaces to expose the security features to users, or to application programmers who then expose them to users. In contrast, our project began with a very high-level idea of the security model of the system, and then moved straight to the user interface design. Details of the security model and decisions about the implementation are driven by the primary focus of making the interface easy to understand and convenient to use.

In the physical world, we have reasonable security in spite of a lack of fine-grained security mechanisms. For example, we routinely allow only partly-trusted people into homes, such as friends for socializing and repairmen to fix our utilities. We often monitor them only loosely, because that is all that is required to ensure that the plumber is not looking through our CDs instead of fixing the pipes. This level of supervision is easy because CDs and pipes are typically not near one another. In general, the varied locations of different types of objects and activities in the home provide a convenient, albeit coarse, security mechanism.

In information space, no analogous mechanism exists. On typical home computers, all programs run by the user have access to all data, and in some circumstances implement security policies the users do not intend.[1] Users do not necessarily store files for different purposes in separate places. Even when they do, current security mechanisms do not automatically take advantage of this, and it is too much to expect users to set good permissions on every file or directory (e.g., using permission bits under Unix[2] or access control lists under modern versions of Microsoft Windows[3] and other operating systems). However, just as people coarsely sort their possessions in the physical world into separate rooms, they may be willing to coarsely sort their files and applications. This sorting may enable the system to protect data from malware more effectively than it does at present.

In Chameleon, users coarsely sort their files and applications by assigning each file to one specific category, called a role,[4] and installing applications into one or more roles.[5] Files are put into one role or another based on how secure the file should be or on the types of activities you do with the file.

The idea of a role-based system is not new. The Chameleon security model borrows ideas from previous security research, especially compartmented mode workstations,[6] sandboxing,[7] and role-based access control.[8] It is also similar to the WindowBox project.[9] Chameleon

Chameleon User Interface

This section describes the current version of the Chameleon interface and highlights issues we considered in its design that are specific to security interfaces.

Security as a secondary (or tertiary, or lower-priority) task means not only that the security operations must be easy to use, but also that they cannot interfere too much with the primary task or tasks. We designed the Chameleon interface to be easy to use itself, and also to not intrude on the ordinary activities that people want to perform.

To make Chameleon easy to understand and use, we used popular interfaces such as Microsoft Windows and the Macintosh as a starting point. Chameleon uses separate, overlapping windows to display applications and to browse and manage files. The Chameleon design attempts to add or modify this interface style only as much as necessary to support roles. The role features are designed to be simple, so they are easy to understand and convenient to use. Chameleon has evolved over time through two low-fidelity (paper) prototypes and one software prototype, based on lessons learned in user studies. These old versions of the interface are shown in Figure 17-1 and 17-2.

The current implementation is under development; a schematic of how its screen will be laid out is shown in Figure 17-3.

In developing Chameleon, we used a technique called low-fidelity prototyping (or low-fi prototyping for short). In general, prototyping is invaluable in many design activities, and user interface design is no exception. Prototyping allows designers to gather feedback early, consider many alternatives, and keep design centered on the user.

Low-fi prototyping usually involves tools like paper-and-pen media that are unstructured and easy to work with. In contrast, typical high-fidelity prototyping uses animation or software tools such as Macromedia Flash or Director, or Visual Basic.

Low-fi prototyping has several advantages over high-fi prototyping. Because it is created with simple, natural media, it is faster and easier to create the prototypes.[a] Even the most advanced software tools for building animations or graphical user interfaces require much more time than paper, pen, scissors, and tape to create simple interfaces.

Chameleon Interface Development

User interfaces, like many other complex artifacts, are virtually impossible to create perfectly the first time. For this reason, a key element of good user interface design is iteration through the design, implementation, and evaluation phases. The Chameleon interface is currently on its third major version, after a paper prototype and a Visual Basic prototype. The paper prototype was evaluated with two different user studies, and the Visual Basic prototype with one study.

Chameleon presents a new paradigm of application and file organization—roles—so the primary goal with our evaluations has been to determine, at a high level, whether Chameleon is easy enough to understand and convenient enough to operate that people would use it and, if not, how to improve the interface to make it usable. Toward that end, our experiments focus on the subjective impressions of participants, gathered by observing how they use the system and from questionnaires participants filled out after using the system.

This section describes our goals for each iteration of the interface, how we evaluated each one, and what we learned. (An in-depth discussion of the user studies with the paper prototypes is given in our technical report.[11])

Study 1: Paper Prototype (Security in Context)

We began the design of the Chameleon interface with sketches and storyboards on paper. The goal at this early stage was to determine what elements the interface needed for the user to interact with the security features of Chameleon, such as copying or moving files across roles. The sketches and storyboards served as vehicles for brainstorming how people might use Chameleon and how its security features might interact with nonsecurity tasks.

In the next stage, we wanted to find out how people would react to the basic role idea of Chameleon, how easy the security features were to use, and how much impact they had on the realistic tasks. We also wanted to learn user preferences about two issues specific to the Chameleon interface: explicit versus implicit role switching and how to display desktop file icons.

Explicit versus implicit role switching is an interesting issue because it is a tradeoff between security and usability. Explicit role switching means that if an application running in role A is active, and the user wants to use an application running in role B, role B must first be explicitly activated. With implicit role switching, the user can simply click on the application running in role B and automatically change role. Explicit role switching may have a security advantage because it is much less likely that users would accidentally change roles than with implicit switching. On the other hand, implicit role switching is much more convenient.

We recruited 10 people from around our campus to use the paper prototype while we observed them and listened to their comments about what they found confusing, easy, difficult, helpful, etc. Participants also filled out a web-based questionnaire about their experiences using the prototype.

Overall, participants were positive about the Chameleon interface, both verbally during and after the experiment and in the post-experiment questionnaire. For example, one section in the questionnaire asked participants to rate the security features of Chameleon with opposing word pairs on a scale of 1 to 9, as shown in Table 17-1.

Table 17-1. Reactions to security features in the paper prototype user studies, on a scale of 1 to 9 (1 being the most negative, 9 being the most positive)

Word pair

Median rating

Intrusive

Helpful

Study 1

Study 2

Annoying

Pleasant

6.0

8.0

Hostile

Friendly

5.5

6.0

Chameleon Implementation

The current implementation of Chameleon is being written in C and C++ for X Windows on Linux (kernel 2.4.x). It will support existing applications and allow new role-aware applications to be developed.

The X Windows implementation does not put support for the Chameleon security model directly into X Windows or the operating system, but instead layers the Chameleon model on top, using existing security features. A schematic of the desktop for the X Windows implementation is shown in Figure 17-3. Because they deal with multirole windows and objects, the desktop and the taskbar are managed by trusted programs. Applications, however, need not be trusted.

The following section describes how this implementation enforces role separation in three places: the windowing system, the filesystem, and the network.

Window System Partitioning

In the Chameleon security model, applications are not allowed to see what is in windows in other roles, or to send input events (e.g., mouse clicks, key presses) to windows in other roles. However, in X Windows, once an application connects to the server, it is allowed to inspect the contents of, and send input to, all windows. To enforce role separation, Chameleon uses a separate X server for each role (see Figure 17-8).

Applications connect to an X server as usual, but it is actually a virtual X server (Xvfb, from the standard X Windows distribution) that writes its output to a buffer in memory instead of to a display, and does not connect to a mouse or keyboard. Chameleon uses a new program, cproxy, to monitor each role-specific X server for output. cproxy takes the output and copies it to the X server with which the user interacts. cproxy also takes the input from the user (via the desktop X server) and sends it to the appropriate X server, depending on which window has input focus. For example, if a window from the Vault role has focus, input will be sent to the X server for the Vault role, which will forward it to

Figure 17-8. Architecture of Chameleon in X Windows implementation

the appropriate application. If a window moves or resizes on a role-specific X server or on the desktop X server, cproxy mirrors this change on the corresponding display to keep them synchronized. Interactions with the desktop and the taskbar do not need to be forwarded, because they are connected directly to the desktop X server.

In addition to cproxy, Chameleon contains these custom software components:

Conclusion

Acknowledgments

Thanks to the participants in the user studies, and to Lorrie Cranor, Dirk Balfanz, Alen Peacock, and Karen Renaud for their valuable comments. This work was funded in part by Carnegie Mellon CyLab.

About the Authors

Chapter Eighteen. Security Administration Tools and Practices

Eser Kandogan and Eben M. Haber

TODAY, HUNDREDS OF MILLIONS OF USERS DEPEND ON RELIABLE ACCESS TO COMPUTING AND INFORMATION SERVICES for business, educational, and personal activities. The growth of the Internet puts a world of information and services at our fingertips, yet also opens computers to attack from anywhere around the globe. The same networks that permit a tourist to read email from an airport in Singapore also permit a student in Romania to release a computer virus that disables computers and the businesses that depend on them. In addition, as the complexity of computer systems increases, new vulnerabilities are discovered each day. There is a worldwide community of people, usually referred to as hackers or crackers, who work to discover and exploit such vulnerabilities to attack and gain control of systems, sharing their techniques through various underground channels. Computers across the Internet have been subject to worms, denial-of-service attacks, password-sniffing, and other malicious activity, leading to significant inconvenience and loss of productivity for legitimate users. On the other side, vendors and computer system administrators race to discover vulnerabilities and to create, release, and apply patches before those vulnerabilities are exploited. On the front lines of this battle are security administrators, the people responsible for continually monitoring both their own systems and the ever-evolving security landscape in order to detect new attacks and prevent known attacks. Their work is crucial because, simply put, we cannot afford critical data and systems to get into the hands of hackers.

In this chapter, we provide an overview of the tools and work practices of security administration based on our ethnographic field studies at various computing centers. We profile two representative security administrators and detail five case studies to illustrate security work and the challenges faced by security administrators. Based on these findings, we outline some of the opportunities that lie ahead to improve security administration tools.

Introduction

Attacks, Detection, and Prevention

The security administrators we observed face two broad categories of attack: autonomous software such as worms and viruses that spread from machine to machine, and human-directed intrusions in which an attacker tries to compromise machines manually or by using semi-automated tools. While most virus attacks appear to have the goal of disrupting computer operations, most human-directed attacks aim to gain control of machines. These privileges are then used to attack other machines, steal data or computational resources, and occasionally destroy data. Some attackers appear to do little damage, instead concentrating on obtaining access to as many systems as possible to gain recognition from their peers. When discovered and blocked by security administrators, however, attackers sometimes strike back and try to damage the administrators' data and machines.

We saw security administrators using a variety of tools for detecting and preventing attacks. These tools include:

Global intrusion detection tools

These monitor network traffic to analyze and report suspicious patterns—for example, Bro.[5]

Scanning tools

These probe machines remotely for known vulnerabilities in their installed software—for example, Nessus.[6]

File/host integrity tools

These run locally to check for compromised states; such tools include:

Virus detection and repair tools—for example, Symantec AntiVirus .

Security Administrators

Over the last two years, we have conducted a number of ethnographic field studies looking at database management, web hosting, operating system administration, and security administration. We observed activities ranging from database backup and recovery to configuration of geographic load balancers, from patch management to computer security forensics. Most system administrator activities are either directly related to security or have security implications.

In this section, we profile two typical security administrators, Joe and Aaron. In the following section, we describe five security administration case studies from our observations. Finally, based upon our observations and interviews, we discuss the unique aspects of security administration work, and how well this work is supported by available tools.

Profile of a Security Manager—Joe

Security Administration: Cases from the Field

In this section, we describe the specifics of a number of actual security cases encountered by the security administrators profiled in this chapter. These cases fall into two categories: security checkup and attack analysis.

Security Checkup

One of the primary responsibilities of security administrators is to check the health of the systems in their charge. System "checkup" activities are performed routinely to ensure that systems are secure with the latest fixes, that only permitted applications are running with the proper approved releases, and that the system is generally free of viruses and worms. While intrusion detection systems are constantly on alert, notifying the security administrators of any emerging suspicious patterns, administrators also proactively scan their systems through various remote security tools to reduce the risk of an attack by eliminating any known vulnerabilities. This, however, is a very intensive task with high cognitive demands on the admin.

While scanning tools produce a comprehensive list of possible vulnerabilities, it is up to the administrator to examine this list, assess the risks involved, judge whether these risks are applicable in their situation, and take the appropriate actions. To illustrate the complexity of the "checkup" tasks we'll take a look, in the following subsections, at cases where we observed Aaron handling a virus incident, dealing with alerts sent by an intrusion detection system, and going through a report generated by one such security scanner tool.

Case 1: MyDoom

Having learned through various security news groups and web sites of the emergence of a new variant of the MyDoom virus, Aaron did a network scan of systems in his subnet. Much to his dismay, he discovered that one of the systems had suspicious and increasing network traffic activity involving port 1034 (used by MyDoom for network communications). Immediately acting upon these findings, he notified the system's owner of the situation and had the system taken off the network. Now, he wanted to examine the situation closely, assess the damage, verify whether this system had actually been involved in a virus attack, and get the system cleaned before bringing it back online.

First, he wanted to understand whether other systems were affected by this incident. To do that he needed to analyze logs from the network monitoring tools. Network monitoring systems generate several gigabytes of data per day, making it impossible for a human to read and understand the logs directly. Instead, Aaron created an ad hoc command-line analysis tool using a series of commands to get the list of all IP addresses associated with port 1034 activity, as shown here:

./bin/ra -xcz args.out -port 1034 | awk '{print }' - | awk -F. '{print , , , }' | sort -u

He then copied the list into a file called http://mydoom.o to process it further by associating each IP address with its hostname using the following commands:

for a in 'cat mydoom.o'; do echo $a; host $a | awk '{print }' -; done

These results suggested that four other systems were potentially infected by the virus. So Aaron decided to collect more information on the MyDoom virus to understand how the virus works. A Google search on the subject quickly got him the information he needed and confirmed his suspicions that 1034 is the MyDoom port. Based on the information he found, he explained to us that this particular version of the virus was also causing problems on some search engines because it queried them for valid email addresses within a particular domain (and thus kept them extremely busy). Talking to his office mate, he added that this made the virus very easy to spot, as it leaves a clear signature in the web logs, which could be scanned to search for matches to the URL templates used by the virus in querying search engines. He then told Joe about the new machines involved in the attack so that they too would be taken off the network.

Within hours, users began to note that their systems had been disconnected, and one of them sent him an email about the situation. He spoke to that user on the telephone and gave instructions on how to clean up the machine so that it could be put back on the network.

Case 2: Intrusion alert—false alarm

Throughout the day, security administrators receive email alerts from intrusion detection systems. Typically, these alerts take the highest priority in their work schedule, because such alerts might indicate an ongoing attack. While detection is for the most part automated, classification of events as normal behavior or an attack requires human judgment. On normal days, most of the alerts are either false alarms or harmless, but it is up to the administrator to make the call. This requires an intimate knowledge of the environment—that is, system workload patterns, network packet traffic, application use, hardware and software architectures, and so on. This case study examines the use of human judgment by Aaron. During our observations, we observed Aaron stop a reporting task he was working on after receiving one such alert, shown here:

Following alerts in tcpred. These seem to be coming from the known compromised systems. Please take time to investigate. External IP (Once compromised IP's) 123.123.10.10 #once.compromised.host.edu > Jul 27 15:10:14 123.123.10.10 0.1kb > 210.210.10.10/http 711kb 9,9m %12345

The alert specified that network traffic was detected involving the IP address of a once-compromised system. While the system had been repaired, extra attention is given to such systems in case the fix was insufficient and the machine is still compromised. Specifically, the HTTP log indicated that a file of 711 KB in size was transferred from the formerly compromised machine to another internal machine. Noting that the alert was referring to the HTTP log with the ID 12345, Aaron did a search on the HTTP log for this particular ID using:

grep %12345 http.log

which returned:

109090.04476 %12345 start 123.213.10.10 > 210.210.10.10 %12345 GET download/perftool-tar.gz

The HTTP log provided more information for Aaron to investigate. Specifically, it revealed what file was downloaded (http://perftool-tar.gz). This gave him sufficient leads to pursue it further. He first found the hostname of the machine using the host command and used this information to query the owner of this particular machine (using the center's online administrative tools). He then did a Google search for this particular user and found a number of documents related to parallel programming on his web page. At this point, he was reasonably sure this could be a legitimate file; he commented:

Just to make sure, he pointed his web browser to the web server on this particular machine to find out what it was being used for. Realizing that the machine was used as a web server for a research group doing performance analysis of parallel programming systems, he commented:

Relieved, he further commented that this case was fairly straightforward because they knew what application was downloaded and who downloaded it. However, he added:

Conclusion

Acknowledgments

We are grateful to our field study subjects, who remain nameless to preserve their anonymity. They cheerfully answered our questions and weren't bothered by our crowding their offices with extra people and video equipment, all the while continuing to perform a crucial job.

About the Authors

Part IV. Privacy and Anonymity Systems

Chapter Ninteen. Privacy Issues and Human-Computer Interaction

Mark S. Ackerman and Scott D. Mainwaring

PRIVACY CAN BE A KEY ASPECT OF THE USER EXPERIENCE WITH COMPUTERS, ONLINE SYSTEMS, AND new technologies. Knowing what to consider about users and their views of computer systems can only improve privacy mechanisms. Human-Computer Interaction (HCI) is the subfield of computer science that studies how people interact with—and through—computational technologies. This chapter examines what HCI, as a research area, offers to both those designing and those researching privacy mechanisms.

Introduction

Privacy and HCI

This chapter necessarily juggles two somewhat amorphous terms: HCI and privacy. HCI has already been introduced, along with its core concerns of improving ease of use and the overall user experience. Privacy is an even broader term. Unlike HCI, it is a term in everyday language, and so its meanings are rooted in larger cultural practices and understandings. It has technical meanings in, for example, law, ethics, and social theory, but also engenders strong, emotional connotations in common usage and daily experience. Many of these meanings are different and, at times, even contradictory.

For the purposes of this chapter, a simple but useful definition of privacy is:

As such, privacy is about individuals' capabilities in a particular social situation to control what they consider to be personal data. Although fairly simple, this definition immediately raises a number of important points:

Relevant HCI Research Streams

HCI is composed of numerous research streams, as is any scientific area. Thus, we cannot hope to cover the field here. Useful surveys include the Handbook of Human-Computer Interaction [7] and Readings in Human-Computer Interaction: Toward the Year 2000,[8] especially the chapter introductions. See also the annotated bibliography of HCI resources provided by Karat, Brodie, and Karat.[9]

In any case, several research streams within HCI are of immediate interest to the examination of privacy and the design of privacy mechanisms. These include:

Basic design considerations—designing for general usability and the evaluation of usability (usability engineering ).

How people interact with and through systems (Computer-Supported Cooperative Work, or CSCW).

How individuals differ in their capabilities and how that affects the human-computer interface (individual differences and tailorability).

The role of HCI in next-generation architectures (ubiquitous computing, pervasive computing).

Each is covered in the following sections.

Usability Engineering

Over the last 20 years, considerable interest and effort have gone into improving the usability of computers. Advances in the 1980s such as mice and GUIs greatly expanded the market by removing ease-of-use barriers. Subsequent investment in and by the HCI community has yielded a wide variety of usability engineering and testing methods. It is now generally recognized that modern software and hardware cannot ignore usability. Potential users simply will not adopt or use features that are difficult to use, and organizations will not deploy hardware and software that are difficult to manage.

Addressing these usability requirements has become an acknowledged part of most development methodologies. In software engineering, it has been adopted into process models such as the prototyping, iterative, and even spiral models.[10] Generally, these recognize the need to iteratively design, develop, and test against real users in order to create usable systems. An excellent example of this process for a privacy mechanism can be seen in Cranor's implementation of Privacy Bird, which went through five iterations of development and evaluation.[11] Only through successive refinement can software engineers meet users' needs, capabilities, and expectations.

Privacy mechanisms are no exception. In many respects, they can be treated as any other critical platform feature, and addressed with existing usability engineering methods. Karat, Brodie, and Karat[12] provide an excellent overview of these methods and their application for security and privacy. They also point out some key differences between privacy (and security) mechanisms and kinds of functional features with which usability engineering is more typically concerned, caveats that are worth paraphrasing and reflecting upon here:

While valued, privacy is not the users' primary task

We would just add that calling attention to privacy and making it an explicit task at any level can be problematic. For example, Cranor discusses users' difficulties in explicitly articulating their privacy preferences.[13] The goal with privacy, then, is often not so much to measure and refine task performance as it is to refine task invisibility or lightweightness.

Designs must encompass many different types of users

Indeed, we devote a later section of this chapter to a discussion of techniques for dealing with individual differences.

Privacy raises the stakes

Badly designed features can lead not only to user rejection and increased development costs, but also to potential injury (even bodily injury, in the case of stalking via location-tracking technologies).

Systems must respond to the legal and regulatory environment

We note that this places additional demands for specialized expertise on the makeup of usability engineering efforts, beyond their traditional interdisciplinary competencies.

Karat, Brodie, and Karat outline the various phases of system development and the types of methods appropriate to each. Instead of repeating that here, we conclude this section by emphasizing and introducing a number of general approaches from usability engineering and user-centered design of particular use to people seeking to understand a design domain in depth. These may be of particular use for new, "disruptive" technologies that do not yet have substantial deployments in the field.

Context

Privacy is extremely contextual, based in the specifics of by whom, for what, where, why, and when a system is being used. Understanding people's needs and attitudes, and developing the necessary empathy to understand the world from their point of view, is best derived by observing them "in the wild" and asking them open-ended questions. There really is no substitute for getting out into the field, and a number of more or less structured ethnographic methods have been developed. For example, contextual design[14] is a highly structured methodology for pulling out the task requirements and context—that is, going beyond the user interface and considering how users will use the privacy mechanisms in their tasks. Other approaches include discount ethnography[15] and rapid ethnography;[16] all of these seek to balance the valuable open-endedness and freedom of ethnographic investigations with practical requirements of timely return on research investments.

Nuanced control

Control over one's personal data is often very nuanced and unconscious in everyday life. In order to understand what people are doing, it is often necessary to get them talking; observation alone is not enough because it does not provide the subjective understanding of the situation. Nor are discussions of past behavior: people are often not conscious of their actions, and the memory of what they did and why they did it can fade within minutes. For this reason, think-aloud protocols[17] were developed. In this methodology, users continuously describe their actions and reasons aloud. The researcher (or designer) may prompt the user from time to time to keep him talking, but the user provides a steady stream of reasons and subjective judgments. Over longer periods, related methodologies such as experience sampling[18] and diary keeping[19] may be useful.

Low and high fidelity

Systems need not be fully constructed in order to evaluate their usability. Both low-fidelity and high-fidelity prototypes can be used by potential users. An often fruitful method is the Wizard of Oz study . In a Wizard of Oz study, the functionality of the system is simulated by people. For example, if the system requires the parsing of natural-language text, this can be effectively done by a person who simulates the functioning, for example, of a natural-language processing component of the potential system. In this way, designers can evaluate the adequacy of the system without constructing the system itself.

Hybrids

The previous approaches—delving into users' worlds, helping them to articulate and self-reflect, and getting prototypes into their hands—can be combined, elaborated on, and experimented with in almost limitless ways. Some particularly interesting hybrids include experience prototyping, bodystorming, and informance.[20] These design techniques go beyond what is traditionally meant by usability engineering, but show promise for more adequately addressing the real-world nuances of domains like privacy.

None of these usability requirement-gathering and usability evaluation techniques was constructed for privacy per se. However, because of the inherent complexity of privacy mechanisms, the large research stream about usability and user-centered design in HCI is potentially of considerable use.

The next research stream to be discussed considers how privacy mechanisms might be made more usable given the wide range of concerns and preferences that people have about their personal data.

Computer-Supported Cooperative Work

An important stream of HCI research is Computer-Supported Cooperative Work (CSCW ).[21] As already mentioned, HCI began by examining largely single-user applications and systems. Starting in the late 1980s, CSCW started as a counter-effort to consider collaborative computer use. Although this subarea of HCI began in the consideration of cooperative or collaborative work, it quickly grew to include many different forms of coordination and social organization. Much of the early work in CSCW simply ignored the issue of privacy: researchers implicitly assumed that people working jointly on a project had no reason to screen information from each other.

The interest of CSCW researchers in privacy began in the 1990s as a side effect of work on "shared spaces"—remote spaces that were electronically linked through audio and video.[22] An example of such a shared space was a video wall linking two lunchrooms in different cities. Media spaces had privacy problems that today are obvious. In one important study,[23] one of the authors found that she forgot a camera was on and began to change clothes. Other authors have reported similar events. Thus began a more systematic investigation into the issue of privacy within the context of CSCW.

Unlike HCI, which found its history in the literature of cognitive psychology, CSCW's literature came from the field of social interaction—specifically, social psychology and cognitive anthropology. Not only is this background important to understanding current CSCW research, but it also is critical in understanding privacy overall. For that reason, we next survey some of the key social theorists. (We follow this with an overview of the current CSCW literature appropriate to privacy mechanisms.) In many ways, these theorists' views have become almost assumptions within CSCW, and many CSCW studies have borne out their theories. The theorists' views most important to a discussion of privacy include the following:

As those interested in privacy are aware, people have very nuanced views of their interactions with other people and find it problematic when those social interactions are constrained.[24] They handle this nuance with agility and contextually.[25]

Goffman[26] noted that people present a "face" to others. Goffman, fascinated by spies and scam artists, proposed that everyone presents bits and pieces of themselves as socially appropriate to the other and, in fact, may wish to present themselves differently depending on the circumstances. A person may present himself as a loyal employee to his supervisor and a job seeker to another company. People find it very disconcerting when that capability is removed.

Harold Garfinkel, in his examination of how people make sense of their everyday worlds, showed that people find it disconcerting when what they believe to be their everyday "normal" world is disrupted.[27] Some people may even become violently angry when they believe the rules of conduct or "normal" behavior are violated.

Privacy mechanisms, in specific, suffer from these issues. People have extremely nuanced views of other people (and groups, companies, and institutions), and want to safeguard their ability to properly present themselves to those others. At the same time, they will find it very difficult when those modes of presenting themselves change, or when the "rules" about their privacy and safeguards change.

Drawing on these theorists, CSCW research relevant to privacy can be roughly divided into three categories: media space applications, other collaborative applications with privacy concerns, and studies discussing privacy in relation to awareness.

Other applications raised similar privacy concerns. Palen[28] explored the issues with shared calendars and sharing information about users' schedules with co-workers, managers, and employees. For example, Palen noted that some workers used viewing their supervisors' open calendars to determine whether layoffs were likely, a move that might not have been in the company's interest. Other work on shared or public displays has raised concerns about automatically generated views or making information public. Finally, allowing people to view one another's temporal information, such as when people are available for communication, also raises obvious privacy concerns.[29]

These privacy problems have been analyzed in a series of papers that discuss the tradeoffs between awareness and privacy. Awareness is knowing what others are doing or even that they are around. First raised in media space studies[30] and other shared work investigations,[31] awareness is a critical issue in distributed, collaborative applications—one needs to know what other people are doing in the shared space. Hudson and Smith[32] went on to discuss the fundamental tradeoff between awareness and privacy. In their view, awareness requires the release of personal information; this necessitates the disruption of privacy or at least requires one's attention to controlling the release of personal data. They proposed a number of interesting technical solutions to solving the privacy-awareness tradeoff. Their video solution allowed cameras to provide awareness of a presence, but the blurred image did not allow the viewer to see details.[33] Their audio solution allowed one to hear voices in a media space, but not to make out the exact words. Neither solution required a user's attention, and yet particularly egregious problems, such as seeing too much or overhearing private conversations, were eliminated for workplace environments. More recently, however, Neustaedter, Greenberg, and Boyle have found that blurring is not always sufficient to protect privacy in home environments.[34]

Work on privacy continues within CSCW. Recently, Palen and Dourish[35] pointed out that privacy is a dynamic, dialectic process. Based on the work of Altman,

Conclusion

About the Authors

Mark S. Ackerman is an associate professor of electrical engineering and computer science and of information at the University of Michigan. He has published extensively on a number of Human-Computer Interaction topics, including Computer-Supported Cooperative Work, information access, collaborative memory systems, and privacy.

Scott D. Mainwaring is a senior researcher in the People and Practices Research Lab at Intel Research. His research interests include design ethnography; home environments; the human relationship to infrastructure; and the community, trust, and privacy implications of ubiquitous computing.

Chapter Twenty. A User-Centric Privacy Space Framework

Benjamin Brunk

WITH THE MASS POPULARIZATION OF THE INTERNET AT THE END OF THE 20TH CENTURY, hundreds of new software tools, systems, and services specifically designed for security and privacy protection became available. Many hoped that this great effort would help fight back the widely acknowledged technological assault on privacy, especially privacy in cyberspace. But most of these tools were difficult to use and, as a result, fell short of their intended goals.

Looking at the failing of existing tools, I decided to create my own tool or suite of tools that would borrow the best features from many other privacy and security tools that were available, but mine would further include a well-designed and soundly tested user interface. Like other authors in this volume, I believed that people would be motivated to protect their privacy only if they could understand and use the privacy-protecting tools that they were given.

Although I never created that ultimate privacy tool, my analysis of the space in which privacy tools exist and operate became the basis of my dissertation. Presented here, it is a useful framework for evaluating the strengths and weaknesses of many privacy and security tools.

Introduction

Soon after I started work on my comprehensive privacy tool, I discovered that I could not decide which features it should include. Privacy tools were a new category of software tools in 1995: many had very specific and novel features that were critical to some people but useless to others. There was no consensus on which features were important to have. Nobody even knew which of the tools that were available for download even worked!

Trolling on the Internet I found some web sites that had basic descriptions of some programs; sometimes, these sites even included ratings. But I couldn't find any comprehensive catalogs of the tools that were available.

The community of people who were actually developing these tools reflected this state of confusion. Many people were working on many products, but little communication was taking place among them. Without marketing experience, open source and independent developers had difficulty communicating why their products were actually needed. Meanwhile, those companies that had strong marketing frequently promoted their products with explanations that often devolved into impenetrable euphemisms.

I realized that I needed to perform a more thoughtful and systematic analysis. No one had yet examined privacy tools from the end user's perspective. How could this be done?

At the time my study took place, personal security solutions such as ZoneAlarm , PGP, and The Anonymizer were new and growing in popularity among Internet users. I began examining these tools specifically in terms of what privacy benefits they offered to the user. Upon examining the 134 tools, systems, and services that I thought had some relationship to privacy, I ended up with a large list of privacy features. As the work progressed, I began to see patterns emerge and could place tools and features into categories. At some point during the process, I began to refer to my collection of tools as the privacy space. I gave the tools, systems, and services within the privacy space the generic label of solutions.

Time has passed since my initial survey of privacy space solutions. Many solutions have been added to the privacy space and others have disappeared or have become insignificant. Despite the ever-changing nature of the privacy space, the framework I developed remains relevant. However, before I can discuss the Privacy Space Framework and why it is useful to us, first we must place this work into the larger context of privacy, especially online privacy.

Privacy

Noam talks about privacy as the place where the information rights of different parties collide.[1] Everyone needs privacy, but there are no "one-size-fits-all" remedies or equations that can decide how privacy should be balanced against other goods. Privacy is inherently a matter of individual choices and needs, a flux that is bounded by societal factors and personal preferences. To Noam, privacy is fundamentally about the flow of personal information between parties that have different preferences for how that information should be utilized.

Technology has a strong influence on our attitudes toward privacy and on how much (or how little) privacy individuals can attain. This is because the balance that Noam describes is inherently altered by the dropping cost of mass surveillance and data retention technologies. The fact that large amounts of information can be economically collected and used increases the desire of organizations to do so.

Security and Privacy Frameworks

Before I introduce the Privacy Space Framework, let's examine some of the other frameworks that served as a basis for this effort.

Codes of Fair Information Practice

Since the 1970s, the guidelines for Fair Information Practice (FIP) have been used as a basis for talking about privacy and ethical data usage. The first code was developed by the U.S. Department of Health, Education, and Welfare[13] in 1973. That code lays down seven basic principles, listed in Table 20-1.[14] Variations on this code represent the most common type of privacy guidelines in use today. In 1980, for example, a code known as the OECD principles[15] was devised to help standardize information-handling practices in international trade.

Table 20-1. Principles of Fair Information Practice

Principle

Description

Openness

The existence of record-keeping systems and databanks that contain personal data must be publicly known, along with a description of the main purpose and uses of the data.

Individual participation

Individuals should have a right to view all information that is collected about them; they must also be able to correct or remove data that is not timely, accurate, relevant, or complete.

Collection limitation

There should exist limits to the collection of personal data; data should be collected by lawful and fair means and should be collected, where appropriate, with the knowledge or consent of the subject.

Data quality

Personal data should be relevant to the purposes for which it is collected and used; personal data should be accurate, complete, and timely.

Finality

There should be limits to the use and disclosure of personal data. Data should be used only for purposes specified at the time of collection; data should not be otherwise disclosed without the consent of the data subject or other legal authority.

Security

Personal data should be protected by reasonable security safeguards against such risks as loss, unauthorized access, destruction, use, modification, or disclosure.

Accountability

Record keepers should be accountable for complying with fair information practices.

The primary drawback of using any code of fair information practice for studying the privacy space is that none of them are particularly user centered. The codes and categories deal with contractual agreements, and not with the actual solutions that may be implemented on the user's computer.

Fair Information Practice does little to enhance our understanding of the privacy space as viewed by the user, because the features examined in my study did not involve actual agreements between a data provider and a data collector.

The ISTPA Privacy Framework

A more recent privacy framework is the International Security, Trust & Privacy Alliance (ISTPA) Privacy Framework , version 1.1.[16]

Researching the Privacy Space

I developed the Privacy Space Framework to make sense of existing "privacy solutions." This study had two distinct phases: a feature analysis phase and a validation phase.

Feature Analysis

The first phase used grounded techniques to assess a sample of solutions in order to "make replicable and valid inferences from data to their context."[20] To do this, I developed a technique known as feature analysis. This technique borrows heavily from the field of content analysis. A central idea in content analysis is that many observed pieces of data are classified into a set of content categories.[21] Text, words, phrases, or other units are classified into categories. Entries in each category are presumed to have the same or similar meanings.

Feature analysis takes a similar approach. Instead of classifying words, feature analysis classifies software features. A software feature is a capability for completing a certain task that has been designed into a system. As such, a software feature can be named and described in words, and the content of the name and description can be analyzed using conventional content-analytic techniques. A privacy feature is a software feature that offers some kind of privacy-related functionality to the user. It is often found that solutions designed for one purpose are later adapted by someone for other purposes. For our purposes, a privacy feature need not be consciously designed, or designed for protecting privacy; it only matters that the feature in question somehow relate to privacy, even if by accident.

To compile a list of privacy features , I started with a list of 134 privacy solutions. This list was compiled by examining software download web sites, privacy-related web sites, vendors' web sites, and news articles; visiting online and offline software stores; and following up on recommendations of friends and colleagues. The major product of this effort was a raw list of 1,291 privacy features and their descriptions.

The data for phase one was collected in just over a month's time during the spring of 2002. I was able to obtain trial versions or academic licenses for most solutions and install them on my own computer for testing. There were a handful of solutions that I was unable to try myself, and for these I had to rely solely on their documentation for the analysis.

To help the reader achieve a better understanding of the utility of the Privacy Space Framework and how it can be used to classify privacy solutions, here are some examples of solutions and their features.

Example 1: PGP Freeware

PGP Freeware[22] is a program for encrypting and decrypting files and email messages using public key cryptography. The program's name (Pretty Good Privacy) and its marketing leave no ambiguity that this product was designed to help protect personal privacy.

My analysis revealed that version 7.0.3 of PGP Freeware includes 24 features that relate to privacy. Obvious privacy-related features include generating public and private keys, encrypting data, decrypting data, digitally signing and verifying data, and wiping files using a secure algorithm. Some of the less obvious privacy features I noticed relate more to the user interface—the toolbar that allows easy access to the program's features; the lock icon in the Windows tray that provides visual information about the status of the program; the key management interface that allows the user to easily import and export public and private keys; and even controls for what to do with data left on the clipboard after the program exists. I categorized most such features as relating to prevention. PGP also has many awareness-related features (e.g., graphical depictions of key lengths) and also some detection features (e.g., the ability to verify a digital signature and thus ascertain its authenticity). My analysis found no response or recovery features in this tool, however. Overall, I categorized PGP Freeware as a prevention and awareness tool.

Example 2: WebWasher

One of the first tools I evaluated was WebWasher AG's WebWasher,[23] a program that removes advertisements and blocks pop-up windows on many web sites. I was already using WebWasher at the start of the Privacy Space study.

I identified 20 privacy features provided by WebWasher. Most of these relate to its web-filtering capability, such as its ability to filter cookies, web bugs, URL prefixes (e.g., http://search.com?url=http://realurl.com gets changed to http://realurl.com

Privacy as a Process

As we progressed, it became more and more apparent that the most natural way to look at the privacy space was to view privacy as a process using the framework to model that process. In the process model, there is a "building up" of information feeding back to the user. We could say that the more categories a solution's features belong to, the more comprehensive it must be (and assume that a more comprehensive solution is better, as it gives the user more control over his privacy). Hence, the "best" solutions have features to address each stage of the process. Figure 20-3 illustrates this idea, showing examples of features from the study

Conclusion

About the Author

Formerly a systems analyst and programmer at UNC Chapel Hill, Dr. Benjamin Brunk is now a web programmer and user interface specialist for Integrian, Inc., working on specialized digital video systems for public safety, transportation, and government applications. Dr. Brunk is also a part-time adjunct professor at the School of Information and Library Science at UNC.

Chapter Twenty One. Five Pitfalls in the Design for Privacy

Scott Lederer, Jason I. Hong, Anind K. Dey, and James A. Landay

TO PARTICIPATE IN MEANINGFUL PRIVACY PRACTICE IN THE CONTEXT OF TECHNICAL SYSTEMS, people require opportunities to understand the extent of the systems' alignment with relevant practice and to conduct discernible social action through intuitive or sensible engagement with the system. To help designers support these processes, this chapter describes five pitfalls to beware when designing interactive systems—on or off the desktop—with personal privacy implications. These are based on a review of the literature, on analyses of existing privacy-affecting systems, and on our own experiences designing a prototypical user interface for managing privacy in ubiquitous computing (ubicomp).[1]

Introduction

One possible reason why designing privacy-sensitive systems is so difficult is that, by refusing to render its meaning plain and knowable, privacy simply lives up to its name. Instead of exposing an unambiguous public representation for all to see and comprehend, it cloaks itself behind an assortment of meanings, presenting different interpretations to different people. When sociologists look at privacy, they see social nuance that engineers overlook. When cryptologists consider privacy, they see technical mechanisms that everyday people ignore. When the European Union looks at privacy, it sees moral expectations that American policymakers do not. Amid this fog of heterogeneous practices, technologies, and policies that characterize the current state of privacy, designers of interactive systems face increasing market pressure and a persistent moral imperative to design systems that support users' privacy needs: systems that are privacy-sensitive.

Note

We will use the term

Faces: (Mis)Managing Ubicomp Privacy

Our investigation into the pitfalls began after we encountered them firsthand while designing Faces , a software prototype for specifying privacy preferences in ubicomp environments.

Faces Design

Ubicomp envisions computation embedded throughout everyday environments to support arbitrary human activities,[3] but the distribution and concealment of displays and sensors can complicate interaction.[4] This can disadvantage users by leaving them unaware of or unable to influence the disclosure of personal information—such as location and identity—as they go about their activities in augmented environments. To address this, we designed Faces to do the following:

Support the specification of disclosure preferences, such as who can obtain what information when (see Figure 21-1).

Provide feedback about past disclosures in an accessible log, not unlike the financial transaction logs in Quicken and Microsoft Money (see Figure 21-2). Users would employ the feedback in the log to iteratively refine their disclosure preferences over time.

Note

Some might object to the Faces disclosure log by claiming that informing the user about a disagreeable disclosure after the fact is too late to be useful. Whitten and Tygar, for example, claim that a so-called "barn door property" governs privacy disclosures: "once a secret has been left accidentally unprotected, even for a short time, there is no way to be sure that it has not already been read by an attacker."[5]

While this may apply to highly sensitive disclosures, a significant component of privacy maintenance is the regulation of mundane disclosures over time to influence an observer's historical, evolving impressions of oneself. People are remarkably capable of finessing the consequences of the occasional—and inevitable—disagreeable disclosure, and they learn to minimize repeat occurrences. The Faces disclosure log was intended to help users transfer such iterative behavior refinement to the domain of the sensed environment.

Figure 21-1. The Faces GUI for creating and assigning faces; each face holds information precision preferences for disclosures to the associated inquirer when the user is in the associated situation; in this example, the user is choosing a face to handle inquiries from his roommate whenever he is studying

Figure 21-2. Faces maintains a "disclosure log" that tracks the release of potentially private information; this log allows users to ascertain the characteristics of disagreeable disclosures and refine their preferences to prevent similar disclosures in the future

Later we will show that the design of Faces involved some crucial missteps that are also present in other systems. What clued us in to the fundamental nature of these missteps is that we made them despite a substantive requirements gathering effort (details in Lederer et al.[6]

Five Pitfalls to Heed When Designing for Privacy

Our pitfalls encode common problems in interaction design across several systems, constituting a preventative guide to help designers avoid mistakes that may appear obvious in retrospect but that continue to be made nonetheless. We encourage designers to carefully heed the pitfalls throughout the design cycle. Naturally, they will apply in different ways and to different degrees for each system. They should be interpreted within the context of the design task at hand.

The pitfalls fit into a history of analyses and guidelines on developing privacy-sensitive systems. They are, in part, an effort to reconcile Palen and Dourish's theoretical insights about how people practice privacy[15] with Bellotti and Sellen's guidelines for designing feedback and control to support it.[16] In reaching for this middle ground, we have tried to honor the fair information practices—as developed by Westin[17] and more recently adapted to the ubicomp design space by Langheinrich[18]—and to encourage minimum information asymmetry between subjects and observers—as argued by Jiang, Hong, and Landay.[19]

Concerning Understanding

Avoiding our first two pitfalls can help fortify the user's understanding of a system's privacy implications by illuminating the system's potential for information disclosure in the future and the actual disclosures made through it in the present and past.

Pitfall 1: Obscuring potential information flow

To whatever degree is reasonable, systems should make clear the nature and extent of their potential for disclosure. Users will have difficulty appropriating a system into their everyday practices if the scope of its privacy implications is unclear. This scope includes:

The types of information the system conveys

The kinds of observers to which it conveys information

The media through which information is conveyed

The length of retention

The potential for unintentional disclosure

The presence of third-party observers

The collection of meta-information like traffic analysis

Clarifying a system's potential for conveying personal information is vital to users' ability to predict the social consequences of its use.

Among the conveyable information types to elucidate are identifiable personae (e.g., true names, login names, email addresses, credit card numbers, Social Security numbers) and monitorable activities (broadly, any of the user's interpretable actions and/or the contexts in which they are performed, such as locations, purchases, clickstreams, social relations, correspondences, and audio/video records). This dichotomy of personae and activities, although imperfect and coarse, can be useful shorthand for conceptualizing a user's identity space, with personae serving as indices to dynamically intersecting subspaces and activities serving as the contents of those subspaces.[20] People work to maintain consistency of character with respect to a given audience, in effect ensuring that an audience cannot access an identity subspace to which it does not already have an index. This can require considerable effort because boundaries between subspaces are fluid and overlapping. Conveying evidence of activity out of character with the apposite persona can rupture the carefully maintained boundaries between identity subspaces, collapsing one's fragmented identities and creating opportunities for social, bodily, emotional, and financial harm.[21]

Privacy-affecting systems tend to involve disclosure both between people and between a person and an organization. Designs should address the potential involvement of each, clarifying if and how primarily interpersonal disclosures (e.g., chat) involve incidental organizational disclosures (e.g., workplace chat monitoring) and, conversely, if and how primarily organizational disclosures (e.g., workplace cameras) involve secondary interpersonal disclosures (e.g., mediaspaces).

Privacy is a broad term whose unqualified use as a descriptor can mislead users into thinking that a system protects or erodes privacy in ways it does not. Making the scope of a system's privacy implications clear will help users understand its capabilities and limits. This, in turn, provides grounding for comprehending the actual flow of information through the system, addressed in pitfall 2, described in the next section.

Evidence: Falling into the pitfall

An easy way to obscure a system's privacy scope is to present its functionality ambiguously. One example is Microsoft's Windows operating systems, whose Internet control panel offers ordinal degrees of privacy protection (from Low to High, as shown in Figure 21-6). First, the functional meaning of this scale is unclear to average users. Second, despite being a component of the operating system's control panel, this mechanism does not control general privacy for general Internet use through the operating system; its scope is limited only to a particular web browser's cookie management heuristics.

Similarly, http://Anonymizer.com's free anonymizing software can give the impression that all Internet activity is anonymous when the service is active, but in actuality it affects only web browsing, not email, chat, or other services. A for-pay version covers those services.

Another example is found in Beckwith's report of an eldercare facility that uses worn transponder badges to monitor the locations of residents and staff.[22] Many residents perceived the badge only as a call-button (which it was), but not as a persistent location tracker (which it also was). They did not understand the disclosures it was capable of facilitating.

Figure 21-6. The Privacy tab of Internet Explorer's Internet Options control panel offers ordinal degrees of privacy protection (from Low to High), but most users do not understand what the settings actually mean

A recent example of a privacy-affecting system that has given ambiguous impressions of its privacy implications is Google's Gmail email system. Gmail's content-triggered advertisements have inspired public condemnation and legal action over claims of invading users' privacy.[24] Some critics may believe that Google discloses email content to advertisers—which Gmail's architecture prohibits—while some may simply protest the commercial exploitation—automated or not—of the content of personal communications. Despite publishing a conspicuous and concise declaration on Gmail's home page that "no email content or other personally identifiable information is ever provided to advertisers,"[25] the privacy implications of Gmail's use were unclear to many users when it launched. Equally unclear, however, is whether the confusion could have been avoided, since other factors beyond system and interaction design were in play. In particular, Google's idiosyncratic brand prominence and reputation for innovation, catalyzed by Gmail's sudden appearance, ensured an immediate—and immediately critical—market of both sophisticated and naïve users.

Evidence: Avoiding the pitfall

Many web sites that require an email address for creating an account give clear notice on their sign-up forms that they do not share email addresses with third parties or use them for extraneous communication with the user. Clear, concise statements like these help clarify scope and are becoming more common.

http://Tribe.net is a social networking service that carefully makes clear that members' information will be made available only to other members within a certain number of degrees of social separation. Of course, this in no way implies that users' privacy is particularly safeguarded, but it does make explicit the basic scope of potential disclosures, helping the user understand her potential audience.

Pitfall 2: Obscuring actual information flow

Having addressed the user's need to understand a system's potential privacy implications, we move now to instances of actual disclosure. To whatever degree is reasonable, designs should make clear the actual disclosure of information through the system. Users should understand what information is being conveyed to whom. The disclosure should be obvious to the user as it occurs; if this is impractical, notice should be provided within a reasonable timeframe. Feedback should sufficiently inform, but not overwhelm, the user.

By avoiding both this and the prior pitfall, designs can clarify the extent to which users' actions engage the system's range of privacy implications. This can help users understand the consequences of their use of the system thus far, and predict the consequences of future use. In the "Discussion" section, we will elaborate on how avoiding both of these pitfalls can support the user's mental model of his personal information flow.

We will not dwell on this pitfall, for it is perhaps the most obvious of the five. We suggest Bellotti and Sellen (1993) as a guide to exposing actual information disclosure.

Evidence: Falling into the pitfall

Web browser support for cookies is a persistent example of obscuring information flow.[26] Most browsers do not, by default, indicate when a site sets a cookie or what information is disclosed through its use. The prevalence of third-party cookies and web bugs (tiny web page images that facilitate tracking) exacerbates users' ignorance of who is observing their browsing activities.

Another example of concealed information flow is in the KaZaA P2P file-sharing application, which has been shown to facilitate the concealed disclosure of highly sensitive personal information to unknown parties.[27]

Another example is worn locator badges like those described in Harper et al

Discussion

Having described the five pitfalls and provided evidence of systems that fall into and avoid them, we now examine some of the deeper implications they have for design. We begin by elaborating on the influence of our first two pitfalls on the user's mental model of his information trajectories. This leads to the introduction of a new conceptual tool to help the design process. Then we present an analytical argument for why designs that avoid our five pitfalls can support the human processes of understanding and action necessary for personal privacy maintenance. Using our Faces prototype as a case study, we then show how falling into these pitfalls can undermine an otherwise ordinary design process. Finally we discuss some successful systems that have largely avoided the pitfalls.

Mental Models of Information Flow

As we said earlier, avoiding our first two pitfalls—obscuring potential and actual information flow—can clarify the extent to which users' actions engage the system's range of privacy implications. Users can understand the consequences of their use of the system thus far, and they can predict the consequences of future use.

Illuminating disclosure contributes constructively to the user's mental model of the portrayal of her identity and behavior in the context of the system. If she has a reasonable understanding of what observers can learn about her (Pitfall 1) and of what they already know about her (Pitfall 2), she can maintain and exploit this mental model to influence the portrayal of her identity and associated activities over time.

In the context of interactive systems, the personal information a user conveys is often tightly integrated with her interaction with the system. For example, by simply browsing the Web, a user generates a rich clickstream that can be used by observers in ways that directly impact her life. When interaction and disclosure are integrated in such a way, an informed user's mental model of the system's operation and her mental model of her disclosures are interdependent.

This suggests an extension to Norman's canonical elucidation of the role of mental models in the design process. According to Norman, the designer's goal is to design the system image (i.e., those aspects of the implementation with which the user interacts) such that the user's mental model of the system's operation coincides with the designer's mental model of the same.[65]

When we take into account the coupling of interaction and disclosure, we see that the designer's goal has expanded. She now strives to design the system image such that the user's mental models of the system's operation and of the portrayal of his identity and behavior through it are both accurate. As with Norman's original notion, ideally the designer's and the user's models of the system's operation will coincide. But the designer generally cannot have a model of the user's personal information; that depends on the user and the context of use. Indeed, here the designer's task is not to harmonize the user's model of his information flow with her own (she likely has none), but to harmonize the user's information model with the observer's

Conclusion

Acknowledgments

This work was funded by Grant No. IIS-0205644 of the United States National Science Foundation and by a United States Department of Defense NDSEG fellowship. We are grateful for the assistance and insights of Jennifer Mankoff, Chris Beckmann, danah boyd, John Canny, Karen Teng, Jeff Huang, Xiaodong Jiang, the anonymous reviewers of earlier drafts of this work, and the participants of the studies mentioned herein.

About the Authors

Scott Lederer is a Ph.D. student in Human-Computer Interaction in the Computer Science Division at the University of California, Berkeley. His research aims to empower people to make sense of and appropriate the deeply augmented world.

Chapter Twenty Two. Privacy Policies and Privacy Preferences

Lorrie Faith Cranor

MOST PRIVACY ENHANCING TECHNOLOGIES (PETs) research and development has focused on just one category of Benjamin Brunk's Privacy Space Framework :[1] prevention. In Brunk's 2002 review of available privacy technology solutions, prevention tools dominated. In the PETs research literature, we see numerous papers on cryptographic protocols, anonymous communication systems, and private information retrieval techniques, all of which are designed to prevent information from being disclosed. However, prevention tools tend to be blunt instruments that offer their users only the ability to turn protections on or off. They usually do not facilitate fine-grained control over the flow of personal information.

The concept of individual control over personal information is central to most conceptions of privacy. To facilitate user control over their information requires tools that can be tuned to allow or block information exchange at a fine-grained level based on a variety of factors. Such tools might be categorized as awareness tools, as they can convey privacy-related information to users to help them inform their decision making. However, we would ultimately like these tools not only to inform users, but also to take actions on their behalf. Thus, these tools might also be categorized as detection tools, because they can analyze privacy-related information and detect situations where preventive steps should be taken. New tools are being developed that straddle these three categories, offering the ability to inform users about privacy -related issues, detect conflicts with user-specified privacy preferences, and take appropriate preventive actions. In this chapter, I focus on tools that make use of computer-readable privacy policies to allow users to control the use of their personal information on the World Wide Web.

Introduction

The Platform for Privacy Preferences (P3P)

In April 2002, the World Wide Web Consortium (W3C) published the Platform for Privacy Preferences 1.0 (P3P 1.0) Specification , which defines a standard way of encoding web site privacy policies in an XML format, as well as mechanisms for locating and transporting these policies.[10] P3P was designed so that web sites can adopt it easily without the need to change their web server software. Two years after P3P 1.0 was published, 1 in 3 of the top 100 web sites and 1 in 4 of the top 500 web sites had adopted it.[11]

How P3P Works

A privacy policy encoded according to the P3P 1.0 Specification is referred to as a P3P policy. P3P policies include eight major components. Each component is represented as an XML element. Some of these XML elements also have subelements, and some of them are described by XML attributes. For example, the use of collected data is represented by the "purpose" element. The specification defines 11 purpose subelements, each representing a data use. In addition, each of these purpose subelements has a "required" attribute that indicates whether the data may be used for this purpose all the time, on an opt-in basis, or on an opt-out basis.

Figure 22-1 gives an overview of the major P3P policy components. The purpose, data, recipients, retention, and consequence elements are bundled together into a structure called a P3P statement. A P3P policy contains one or more statements. Sites use the statement structure to indicate types of data that are treated in similar ways. For example, a site might have one statement to describe the information it stores in logfiles, and one statement to describe the information it collects from individuals who make purchases at the site. A P3P policy for a site with a relatively simple privacy policy is shown in Figure 22-2. This policy includes only one statement.

Figure 22-1. The major components of a P3P policy

Figure 22-2. Example P3P policy

The P3P policy syntax is extensible, allowing for the addition of new required or optional policy components without necessitating a new version of the P3P Specification.

The P3P 1.0 Specification also includes syntax for a P3P compact policy, an abbreviated version of an XML P3P policy that describes a web site's data practices with respect to cookies. Compact policies consist of combinations of three-letter tokens, many of which can be modified by a compact version of the required attribute. Fifty-two such tokens are specified. P3P compact policies are optional for P3P-enabled web sites; they are used by some P3P user agents to facilitate rapid cookie-blocking decisions without the need to fetch a complete P3P policy and load it into an XML parser. Web sites that have compact policies must also have full P3P policies. However, web sites with full policies are not required to have compact policies. Indeed, sites that don't have cookies have no need for compact policies at all. Because the P3P policy in Figure 22-2 does not mention cookies, there is no corresponding compact policy. A web site that uses cookies to tailor site content each time a user returns to that site (unless the user opts out) might have a compact policy that looks something like this:

CP="NON DSP ADM DEV PSDo OUR PRE NAV UNI STA"

These 10 P3P compact policy tokens have the following meanings:

NON

Privacy Bird Design

I began designing a P3P user agent in parallel with the development of the P3P specification. Over a four-year period, during which time many changes were made to the P3P specification, I worked on four prototype P3P user agents.[16] Experience with these prototypes informed the eventual design of Privacy Bird. The first public Privacy Bird beta was released in February 2002. A second beta was released a year later, following a user study.[17] In 2004, AT&T made the Privacy Bird source code publicly available. My goals in developing Privacy Bird and the earlier prototypes were to provide feedback into the P3P specification development process based on implementation experience, and to demonstrate the capabilities of P3P. I hope that other P3P user agent developers will be able to learn from Privacy Bird.

Privacy Bird is implemented as a browser helper object, which loads whenever IE starts up.[18] Users download Privacy Bird as a 1.4 MB self-extracting file that includes an installation wizard. A P3P user agent built into a web browser would likely perform better and be able to more easily integrate with cookie managers and other browser functionality.[19] Much of the Privacy Bird design could be incorporated into a browser implementation.

Designers of P3P user agents face two major design challenges : designing an interface for capturing user privacy preferences , and designing an interface for communicating with users about web site privacy policies. These challenges and our approaches to addressing them are discussed in the sections that follow.

Capturing User Privacy Preferences

Unless you are a privacy researcher, it is unlikely that you have ever had a discussion with someone about his privacy preferences. If you are interested in building privacy tools, especially tools that help individuals control the collection and use of their personal information, I recommend that you spend some time talking to people about their privacy preferences and the type of control they would like to have.

While working on P3P and Privacy Bird, I spoke with many people about their privacy preferences. In this process, I learned a few important lessons:

Most people have little experience articulating their privacy preferences

Most people have never been asked to do this before.

Privacy preferences are often complex and nuanced

Initially, most people with whom I have discussed privacy preferences tell me that their privacy preferences are pretty simple—for example, "I don't want companies to give my information to anyone else." But as our conversations continue, people usually start to articulate a variety of exceptions to their initial simple rules. "If I order something from them, then they can provide my information to fulfill the order and ship a package to me. And if I tell them about my hobby, then it would be OK if they send me catalogs related to that hobby or let me know about clubs I might be interested in." Some people, eager for a good deal, go further: "I should have the right to control my information, but junk mail doesn't really bother me so much. So if they are willing to give me something for free, I don't mind throwing away their junk mail. But if they are profiting from my information, I should get something too." And when the discussion turns to the sharing of location or presence information with other individuals, privacy preferences tend to get very complex.

Most people are unfamiliar with much of the terminology used by privacy experts

Privacy policies, privacy laws, and privacy principles tend to be full of privacy jargon.

Most people do not understand the privacy-related consequences of their behavior

People tend to assume that if they have nothing to hide, there probably isn't any risk associated with sharing their personal information. Often, they are unaware of the potential for information from multiple sources to be combined, perhaps years after the information was initially collected. It is hard for them to imagine how information might be used against them, both accurately and inaccurately.

For these reasons, it is very difficult for most people to articulate anything close to a set of privacy-related rules that might be applied by an automated agent. It is no wonder that Westin describes privacy control as a process in which individuals are "continually engaged."[20] To develop software that can serve as the user's proxy in this continuous decision-making process is indeed a challenge.

To make matters worse, privacy policies are also complex and nuanced. Some natural-language privacy policies are almost incomprehensible to anyone without a law degree, and even those who have law degrees may find internal inconsistencies in some policies. P3P improves the situation somewhat by forcing P3P adopters to articulate their privacy policies in multiple-choice format. Most P3P elements have a fixed set of choices, and P3P adopters have to decide which choices apply. However, if we look at the combinations of data category, purpose, recipient, and retention, there are more than 5,000 possibilities. If we also factor in the access element, whether opt-in or opt-out is provided, specific data elements collected, or any of the other elements or subelements available in a P3P policy, the number of combinations explodes. When we include free text fields, the number of combinations is infinite. We could simply ask P3P user agent users to tell us their preferences for each field individually. However, often their preferences are dependent on multiple fields. "I don't mind if they share my preferences about food or sports or things like that, but I don't want them to share information like my address or phone number, especially if it might result in more unsolicited marketing." Clearly there are too many combinations to ask users to articulate a preference about each one.

I attempted to design a Privacy Bird configuration interface that would allow users to specify all of their preferences on a single screen. As shown in Figure 22-7, the Privacy Bird configuration screen allows users to select from 12 conditions under which they might wish to receive privacy warnings. These conditions were selected after reviewing privacy survey results (primarily of American Internet users, as we were designing a user agent with this group in mind) to determine the aspects of privacy policies that would likely be of most interest to users.[21] The three areas that appeared repeatedly as most important were the type of data collected, how data would be used, and whether data would be shared (represented by the P3P data, purpose, and recipient elements). Among data uses, telemarketing calls and marketing lists seemed to cause the greatest concern. Among data types, financial data and medical data appeared to be most sensitive. In addition, individuals did not like having their data used to build profiles of their interests or activities. We focused our configuration interface on these areas of concern.

Privacy Bird Evaluation

We performed two studies to evaluate the usefulness and usability of Privacy Bird, investigating how it is used in a controlled laboratory setting as well as how it is used in practice. We conducted a laboratory study that allowed us to make detailed firsthand observations of how first-time users interacted with the Privacy Bird software and to compare Privacy Bird with another P3P user agent. In addition, we were able to observe users performing the same tasks with and without the benefit of a P3P user agent and thus evaluate the effectiveness of the user agent. We also conducted a user survey to gather information about how Privacy Bird is used in practice. This survey provided us with self-reported data from individuals who had been using the software for several months in their own homes or offices.

User Survey

We received informal feedback on our first beta release of Privacy Bird from demo audiences and from some of the approximately 30,000 users who downloaded it. Email from our users focused on requests for new features and ports to other platforms, and stability and compatibility problems. In order to get additional feedback and gain a better understanding of how people were actually using Privacy Bird, we conducted a survey of Privacy Bird users in August 2002.[28] We sent email invitations to complete a 35-question online survey to 2,000 of the email addresses provided by individuals who had downloaded Privacy Bird during the first six months of our beta trial and had given their permission to be contacted for user studies. We received 309 completed surveys.

Beyond the Browser

Privacy Bird, in its current form, is useful mostly as a tool to raise user awareness about web site privacy practices. By integrating it with a cookie manager—as Microsoft and Netscape have done with their P3P user agents—we could help facilitate more meaningful automated cookie management. If enough sites adopt P3P, and P3P user agents become widely used, the increased transparency about web site privacy practices may lead to the adoption of more privacy-friendly policies—as a result of either market forces or new regulations.[31] However, the real potential for use of an automated privacy policy framework may lie in applications that go beyond the web browser.

ADVICE FOR PRIVACY SOFTWARE DEVELOPERS

Avoid the use of privacy jargon. Privacy terms used by experts are unfamiliar to most users. However, all of the P3P user agents we examined used jargon such as access, profiling, third-party cookie, and implicit consent.

Provide persistent privacy-related indicators. The Privacy Bird icon serves as an indicator of web site privacy practices that users could always find in their browser window. Users reported that they liked having the ability to get high-level privacy-related information at a glance.

Provide meaningful summaries of privacy-related information in standardized formats

About the Author

Lorrie Faith Cranor is an associate research professor in the School of Computer Science and Department of Engineering & Public Policy at Carnegie Mellon University, where she directs the CMU Usable Privacy and Security Laboratory (CUPS). She came to CMU in December 2003 after seven years at AT&T Labs-Research. Dr. Cranor's research has focused on a variety of areas where technology and policy issues interact, including online privacy, electronic voting, and spam.

Chapter Twenty Three. Privacy Analysis for the Casual User with Bugnosis

David Martin

BUGNOSIS[1]is a privacy analysis tool for the typical end user

The Audience for Bugnosis

Cookies, Web Bugs, and User Tracking

Bugnosis defines a reasonable notion of web surveillance and alerts users when it sees it so that users can just trust the Bugnosis analysis rather than always trying to remember and imagine what's going on behind the scenes. Now, what does it really mean to say "the Web watches you read"? It's not really that complicated, but it's very easy to get bogged down in details.

When a web browser fetches a web page with an HTTP transaction, the remote web server has an opportunity to log the event. In the event log, the server generally gets only these pieces of information:

IP address of the web browser

URL of the web page that referred the browser to the requested page (if any)

Current time

User agent string, usually indicating the type of browser and type of operating system

Cookie value that the same web server previously sent to that web browser

Type of document desired in response to the query

(Human) language and character coding desired in the response

Web servers are configured to record this type of information by default. For example, a recent distribution of the popular Apache web server automatically records the first four items in the preceding list.[3] You should just assume that each web site you visit creates a record of the web pages that it delivers to you.

Conspicuously missing from the preceding list are the user's real name, email address, national identity number, political affiliations, passwords, embarrassing habits, and so on—that is, the extremely sensitive personal information that an underinformed user might imagine flowing over the wire. The actual information transmitted isn't all that immediately alarming, and there is a case to be made for all of it to be there.

Now, how can a web site use this information to identify a user and not just the IP address of the user's computer? Certainly some users have stable and unique IP addresses—for these folks, it could make sense to associate the user's IP address with the actual user. But a large number of us end up with dynamically assigned addresses or shared addresses because of our network provider's policies, a nearby firewall, or a network address translator.

Instead of using the incoming IP address to recognize the computer's current network address, it's far more reliable to use the cookie facility to recognize the web browser itself, and associate that with the user.[4] In the next section, we'll see how it works.

Tracing Alice Through the Web

Suppose that Alice has just bought a new computer and starts visiting web sites. The web server's algorithm for establishing and recognizing users can be extremely simple:

if the web transaction arrived with a cookie then # that cookie value is this user's ID string else # it's a new user cookie := get_currently_unused_user_ID() return cookie to user's web browser as its new persistent cookie value endif deliver requested page

Having never visited the site before (at least through this browser), Alice will get a new ID. When she returns to the site through the same browser, she'll be recognized by that automatically transmitted ID. The ID string can be used by the server as a key into a database of other information they maintain about her—for instance, any information she types into the site, such as an email address, password, or postal code.

Altogether, this cookie facility only allows a server to remember something it already knows about the user behind a particular web browser: there is no way for an evil web server to use a cookie to snatch files from a user's computer, for instance. And there are plenty of good applications for cookies. Probably the most prominent application—the one envisioned by Lou Montulli, who invented cookies while working at Netscape—was to make it possible to implement web shopping carts:

Visiting multiple sites

If Alice visits multiple web sites, then each of the web servers will independently create a new ID for Alice, as pictured in Figure 23-1.

Figure 23-1. Sample cookies actually assigned by various web sites; in effect, each site has a different name for Alice

Because each web server knows Alice by a different ID, it's difficult to combine all of their records into one giant dossier of activity, even if all the servers wanted to do so. After all, if I've gathered a lot of information about Marion Morrison, and you know everything about the movie star John Wayne, would we even suspect that we could combine our information and so get a better picture of the man? We'd first have to learn that John Wayne was Morrison's stage name. Without that knowledge, we would have to treat them as two different people. This is where third-party cookies become important.

Unique identification with referrers and third-party cookies

Whenever a browser fetches web content over HTTP, it automatically sends the appropriate cookie to the destination server, even when it is getting content that wasn't explicitly requested by the user.[6] This means that when Alice loads http://www.nytimes.com/, her browser sends her http://nytimes.com cookie.[7] That page is delivered to her browser. Now, assuming that the page includes an instruction to fetch an image from http://m3.doubleclick.net/, her browser will go grab it automatically. But along with the request to http://m3.doubleclick.net, her browser will also transmit her http://doubleclick.net cookie along with the URL of the page whose content this transaction is part of:

Host: m3.doubleclick.net Cookie: id=80000021e26e40b Referrer: http://www.nytimes.com/

This excerpt shows that the third-party site http://m3.doubleclick.net receives enough information to recognize Alice by her http://doubleclick.net

The Graphic Identity

If you're building software that you want people to use, then find yourself an artist.

Making It Simple Is Complicated

Internet Explorer's extensibility, particularly its Browser Helper Object facility, made Bugnosis's unobtrusive user interface possible. But things didn't always snap together as easily as we would have liked. In the following sections, we'll look at some of the details of the architecture that required special attention.

Using Browser Helper Objects and the Document Object Model

The design decision with the largest impact was where to put the analysis engine. Fairly obviously, it had to run on the end user's computer (rather than on a remote server) in order to maintain acceptable speeds. We also considered the idea of running an HTTP proxy on the user's computer, which would be able to intercept the browser's HTTP requests involved in rendering its web pages. But there were many problems with the proxy approach:

Some users already rely on a proxy for web access

To support these users, we would have to install our proxy in front of theirs, and then their old proxy would appear to vanish from their Internet Options configuration dialog. How confusing!

The web browser visually indicates the multiple steps in fetching a page

These include resolving the DNS name, contacting the remote host, fetching the page, etc. But when a proxy is installed, it has to perform those tasks itself, and there's no standard way for a proxy to send updates directly to the browser's status line. Less visual feedback overall makes the combination of browser and proxy seem sluggish.

The proxy would have to parse HTML in order to apply the relevant web bug tests

For example, the once test (which excludes images with URLs that are repeated on a page) requires counting how many times the image appears on the page, not just counting how many times it is fetched from the network. These two measures are not identical because of browser caching. Similarly, an image's visible size is determined both by its native size (from the image file itself) and by any HTML width and height attributes used in the <IMG> element that fetches the image.

Proxies cannot intercept HTTPS (encrypted) connections to perform any kind of page analysis

This is the case because proxies don't have the decryption keys. But web bugs are easily deployed on HTTPS pages, and we wanted Bugnosis to be able to detect them. In fact, this would present a valuable lesson about security on the Web: seeing the little lock icon doesn't mean that you're safe from all forms of surveillance.

Some web bugs are created by client-side scripts, usually JavaScript programs

In fact, we've seen web bugs generated by JavaScript code that sense the user's operating system type, screen resolution, window size, and JavaScript version, and that encode that information in a web bug. We'd like Bugnosis to detect these dynamically created web bugs as well. But no one could actually write a proxy that would reliably detect a JavaScript program's intent to create a web bug.

The Bugnosis web page analysis needs to be visible somewhere

If it is provided by a proxy, it would either have to be displayed in a separate window or mixed into the HTML content of the analyzed page. Keeping a separate window open just because Bugnosis is installed is clunkier than we'd like, and the alternative of adding extra HTML content to an arbitrary external page without disturbing its layout is very tricky.

So, instead of the proxy approach, we built the analysis directly into the Internet Explorer (IE) web browser using its Browser Helper Object (BHO) hooks.[11] Naturally, this restricted our audience to IE users. But it gave us direct access to the browser's view of the world, including its toolbars, status line, and sidebars. Most importantly, it provided read/write access to the Document Object Model (DOM) representation of the page.[12] This conferred a number of advantages:

In order to find the images on the page, we don't have to parse any HTML at all: we just walk through the DOM data structures, looking for elements that are images.

To find out an image's size, we query its width and height properties: this tells us precisely how the image is rendered, taking into account both the image's native size and its scaling by HTML's width and height attributes.

Internet Explorer handles the loading of pages, and our code just sees the results. So, we can analyze encrypted web pages just as easily as unencrypted ones.

By hooking into DOM events, our code can notice when images are created as a result of client-side scripting and analyze them as well.

The Event Model

Bugnosis is an event-driven BHO. When IE starts up, it creates our Component Object Model (COM) object called IEMonitor, which then identifies itself as an event sink for the containing WebBrowser object. IE subsequently calls IEMonitor::Invoke() with appropriate parameters when it receives reportable events, such as UI manipulation and element download progress. The events of particular interest to us are:

BeforeNavigate2

Looking Ahead

Bugnosis was a bigger success than we planned for, but it remains a very simple tool. It still needs improvement in two main areas: email web bugs and P3P. Future improvements are likely to take it further afield. But first, let's enumerate its current deficiencies.

Exposing Email Tracking

Bugnosis looks only for surveillance built into web pages. However, because HTML is widely supported by email user agents, the same techniques for tracking users on the Web can be used against email users. This is a far more serious situation. Email addresses correspond to individuals, not to computers or network devices, so they are more personal than IP addresses. When a user views a bugged email, the watcher gets a server hit for the embedded images and can generate a log entry recording which particular email recipient read the message. And if the user forwards it to someone else, the watcher gets another hit when the other user reads it, and so on.

Note

As an example, we recently received a party invitation via the Evite company, which describes itself as "the social planning site."

Acknowledgments

Adil Alsaid did most of the original Bugnosis implementation work as a graduate student at the University of Denver. Many others contributed to various design decisions, including Stephen Keating, Andy Cervantes, and Richard M. Smith. John Boak designed the graphic theme and elements, along with the initial web site. Thanks to the editors and reviewers for valuable feedback on this chapter.

About the Author

David Martin (dm@cs.uml.edu) is an assistant professor in the Department of Computer Science at the University of Massachusetts (Lowell, MA). He became interested in Internet privacy as a graduate student at Boston University. As a faculty member at the University of Denver, he collaborated with the Privacy Foundation, a consumer privacy awareness and advocacy group. He is currently co-chair of the program committee of the Privacy Enhancing Technologies Workshop, which meets annually to bring privacy researchers together to discuss their ideas and projects.

Chapter Twenty Four. Informed Consent by Design

Batya Friedman, Peyina Lin, and Jessica K. Miller

CONSUMER PRIVACY ATTITUDES HAVE SHIFTED SIGNIFICANTLY OVER THE PAST DECADE, from being a concern of a minority in the 1980s, to being a concern of the majority. As of 2001, more than three-quarters of Americans (77%) were "very concerned" about potential misuse of their personal information.[1] Consumers not only want to be informed about how their personal information will be used, but also want the opportunity to choose: 88% of Internet users would like "opt-in" privacy policies that require Internet companies to ask consumers for permission to share their personal information with others "always" (based on a four-tier scale from "never" to "always"). In addition, if consumers could choose not to have their personal information collected, 56% would "always" opt out, and 34% would "sometimes" opt out.[2]

While informed consent may not eliminate all privacy concerns for all groups, it can help to gain users' trust by accurately articulating business practices for personal information and by allowing users autonomous choice.

How, then, might the information systems community design for informed consent? We take up that challenge here. Toward answering this question, we ground our work in the interactional theory and tripartite methodology of Value Sensitive Design.[3] , [4] , [5] Our approach integrates conceptual, technical, and empirical investigations. We consider the impact of information systems on both direct and indirect stakeholders.

Introduction

Changes in consumer attitudes are occurring against the backdrop of two major trends: (1) the evolution of the concept of privacy; and (2) the erosion of historical protections for privacy.

The conception of privacy and, correspondingly, that of informed consent, has evolved through changes in political, legal, economic, social, and technological spheres. In earlier times, privacy (and the security it afforded) existed primarily in relation to physical property. Consequently, protections were given against trespasses of physical property and against battery to a person's body. Liberty meant "freedom from actual restraint."[6] In the late 1800s, advances in photographic technology made it possible to take pictures surreptitiously. Thus, the implicit consent given when "sitting" for a portrait became an inadequate safeguard against the improper capturing of one's portrait for circulation and profit. In reaction to these changes, in a landmark 1890 Harvard Law Review article, "The Right to Privacy,"[7]

A Model of Informed Consent for Information Systems

The model of informed consent for information systems we present here was first developed in 2000 by Friedman, Felten, and Millett[13] in the context of online interactions.[14] This model is based on six components:

Disclosure

Comprehension

Voluntariness

Competence

Agreement

Minimal distraction

The word informed encompasses the first two components: disclosure and comprehension. The word consent encompasses the following three components: voluntariness, competence, and agreement. In addition, the activities of being informed and giving consent should happen with minimal distraction, without diverting users from their primary task or overwhelming them with intolerable nuisance.

Disclosure

Disclosure refers to providing accurate information about the benefits and harms that might reasonably be expected from the action under consideration. What is disclosed should address the important values, needs, and interests of the individual, explicitly state the purpose or reason for undertaking the action, and avoid unnecessary technical detail. The information should also disabuse the individual of any commonly held false beliefs. Moreover, if the action involves collecting information about an individual, then the following should also be made explicit:

What information will be collected?

Who will have access to the information?

How long will the information be archived?

What will the information be used for?

How will the identity of the individual be protected?

Comprehension

Comprehension refers to the individual's accurate interpretation of what is being disclosed. This component raises the question: how do we know when something has been adequately comprehended? While there is no easy answer here, at least two methods seem viable: (1) being able to restate in different words what has been disclosed, and (2) being able to apply what has been disclosed to a set of hypothetical events. Take, for example, a web-based recommendation system, such as an e-commerce site recommending products based on the customer's prior purchases or on purchases of other customers with similar profiles. Based on information disclosed to the customer—about what data is being collected and how it will be used—can the customer answer reasonable questions about the data's use, such as:

Will information about the customer's last three purchases be included in the recommendation system?

Will some other user of the recommendation system be able to determine what the customer has purchased in the past?

Will information about the customer's past purchases be a part of the recommendation system two years from now?

Possibilities and Limitations for Informed Consent: Redesigning Cookie Handling in a Web Browser

In this section, we demonstrate that the proposed model of informed consent can be used to:

Assess how effectively a particular system design supports informed consent

Guide successive design

Identify how the underlying technology may constrain the range of possible solutions to support users' informed consent

Specifically, we examine the role the web browser plays in obtaining informed consent for cookies .

What Are Cookies and How Are They Used?

A cookie is a text file stored on the user's machine that can be used to maintain information about the user, such as an identifier or a log of the user's navigation on the web site. One accepted way companies use cookies is to customize their web site to the user (e.g., Amazon uses cookies to remember what users like to buy and what users put in their shopping baskets). However, cookies can also be abused by surreptitiously collecting information about the user.

When a user wants to retrieve a particular web page from the Internet, the user opens a web browser and enters the web page's address. The browser sends the request for the web page to the appropriate web server. The web server then retrieves the requested web page and sends it back to the user's browser where it is displayed. This process involves a couple of additional steps if the web server wants to set a cookie. When sending the requested web page back to the browser, the server sends the browser a request to store the cookie. Depending on how it has been programmed, and on any cookie-related preferences that have been set, the browser may or may not store the cookie as requested. If the browser stores the cookie on the user's computer, the browser will volunteer the cookie each time the user revisits that web page and possibly any other web pages in that domain, depending on the scope of the cookie.

Web Browser as Gatekeeper to Informed Consent

In the previous description, the browser acts as a gatekeeper by determining which web server requests to fulfill. In addition, with respect to informed consent, the web browser plays at least two other critical gatekeeping roles:

The web browser controls whether the user is notified about a server request and, to a large extent, controls the content of that notification. Thus, the components of disclosure and comprehension largely reside in the web browser.

The web browser controls whether the user has an opportunity to agree to or decline the web server's request (e.g., prompting for user input each time a server requests to place a cookie as opposed to the browser handling the request without user input). Thus, the component of agreement also resides in the browser.

Admittedly, a proactive web site could supplement the functionality provided by the web browser by explicitly addressing disclosure and agreement (e.g., privacy policies). However, relying on all web sites to do this individually would result in ad hoc methods and require users to become familiar with each web site's policies and practices.

Web Browser Development and Progress for Informed Consent: 1995-1999

With a conceptualization for informed consent online in hand, Millett, Friedman, and Felten[20]

Informing Through Interaction Design: What Users Understand About Secure Connections Through Their Web Browsing

The process of informing the user can happen as the user interacts with the system instead of through simple, explicit text disclosure. That is, in addition to the user's existing knowledge about how the system functions, the visual cues during interaction and the text displayed on the interface (web pages, browser, etc.) may lead the user to develop an idea or mental model of how the system functions. An issue of concern arises when there is a mismatch between the disclosed text and the interaction cues—in particular, when the latter heavily influences the user's perception of how the system works. As a result, in a best-case scenario, the user could end up confused but not jeopardize any personal data; in a worst-case scenario, the user could construct inaccurate mental models about the security of the system and make poor decisions on what actions to take or not to take to protect personal information.

With the design strategy of informing through interaction in mind, in this section we describe a study by Friedman, Hurley, Howe, Felten, and Nissenbaum[24] on how users across diverse communities conceptualize web security.

Participants

Seventy-two individuals, 24 each from a rural community in Maine, a suburban professional community in New Jersey, and a high-technology community in California, participated in an extensive (two-hour) semistructured interview concerning users' conceptions, views, and values about web security. Equal numbers of men and women participated from each community. We report here on one section of the interview that focused on users' mental models of web security. Both verbal and nonverbal techniques were used to assess users' understandings.

Users' Conceptions of Secure Connections

Participants were asked to define and portray secure connections in various ways, as we describe in the following subsections.

Definition of a secure connection

Participants were first asked to define a secure connection. Participants' definitions of a secure connection encompassed one of the following concepts:

Transit. Protecting the confidentiality of information while it moves between machines on the Web

Encryption. The specific mechanism of encoding and decoding information

Remote site. Protecting information once it has arrived at its destination on the Web

High-technology participants (83%) provided correct definitions of a secure connection more frequently than rural participants (52%) (p < .05) did. Statistically, there was no difference in responses between the high-technology (83%) and suburban (68%) participants.

Recognition of a connection as secure or not secure

Next, participants were shown four screenshots of a browser connecting to a web site and were asked to recognize a secure connection. For each screenshot, participants were asked to state whether the web connection was secure or not secure, as well as to provide the rationale for their evaluation.

Table 24-1 shows the types of evidence participants used to evaluate a connection. As shown, participants depended primarily upon six types of evidence.

HTTPS protocol—for example, "usually, it says http for nonsecure or standard and https for secure, the s meaning secure".

Icon—for example, "[the site is secure] just because the key is there".

Point in transaction—for example, "it looks like one of the main pages on the site and usually main pages are nonsecured connections".

Type of information—for example, "that at least has the indication of a secure connection; I mean, it's obviously asking for a Social Security number and a password".

Type of web site—for example, "I can't imagine a bank would be online and not have security measures in there".

General distrust—for example, "I'm wary of the computer itself...I basically don't think any of the sites are secure".

Table 24-1. Percentage of types of evidence participants used to evaluate a connection as secure or not secure

Type of evidence a

Correct evaluation

Incorrect evaluation

The Scope of Informed Consent: Questions Motivated by Gmail

In the first two cases, we provided "proof-of-concept" projects for ways in which the information systems community can design for informed consent . In our third case—Google's Gmail web mail (web-based email) system—we examine the scope of informed consent—namely, are there issues concerning privacy and security that informed consent cannot reasonably address? And, if so, how do these issues affect informed consent?

What Is Gmail?

Gmail (http://www.gmail.com) is Google's web mail system, currently in beta testing. Similar to other free email services, such as those provided by Yahoo! Mail or Hotmail, Gmail provides three key additional features:

A larger amount of storage space for one's email than is typically provided (as of December 2004, 1 GB of storage as compared to Yahoo! Mail's 250 MB or Hotmail's 250 MB)

The ability to use Google search technology to search one's email messages

Grouping an email and the replies to it as a conversation

As with most free web mail services, Gmail subscribers see advertising alongside their email. However, unlike other free web mail providers, Gmail determines which advertisements to display based on the content of the subscriber's email message. For example, if a Gmail subscriber receives an email message from a friend asking to borrow the subscriber's bicycle, the subscriber would likely see advertisements related to online bicycle vendors alongside the email message (see Figure 24-4).

Figure 24-4. Viewing an email message in Gmail; advertisements targeted to the content of the body of the message—in this case, bicycles—appear to the right of the email message, under the label Sponsored Links

How Gmail Advertisements Work

Let's look briefly at both the Gmail business model and its technology:

The business model. In a nutshell, Google charges advertisers only when a user clicks on an advertisement. When advertisers submit their advertisements to Google, they negotiate a "rate-per-user-click" and designate keywords they believe to be most relevant to their advertisement.[25]

The technology. The following automated process occurs dynamically each time a Gmail subscriber clicks on an email entry. The Gmail system retrieves the message and scans the text (attachments are not scanned) for keywords (provided earlier by advertisers) in the body of the email message.[26] Based on the results of scanning the message, as well as on how well the advertisement keywords match the email content, the amount advertisers pay per user-click, and the prior click-through rate (i.e., the number of clicks divided by the number of times the advertisement has been displayed), Google computers select and determine the order in which to display advertisements. The selected advertisement is displayed near the subscriber's message; no link is established between the advertisement and the email message. The only information Google relinquishes to its advertisers is the number of times their advertisement was chosen for display and the click-through rate. No personal information about subscribers or the content of email messages is released to advertisers.

Gmail and the Six Components of Informed Consent

In the context of this business model and technical implementation, how well does Gmail meet the criteria for informed consent? To make this assessment, we analyzed Gmail's Terms of Use,[27] Program Policy,[28] and Privacy Policy[29] in relation to Gmail's functionality as described above. Between these documents, Gmail's registration interface, and the user interface, Google addresses each of the different components of informed consent in a reasonably thorough manner.

Disclosure

As of December 2004, the Gmail privacy policy explicitly states:

What information will be collected about subscribers and their activities

For example, "personal information including your first and last name, a user name...and a password," "log information...browser type you used, your Internet Protocol address, and the date and time of day...the unique ID provided by our cookie and the URL of the last site you visited."

Who will have access to the information

For example, "No human reads your email...without your consent," "Google will never sell, rent or share your personal information...with any third parties for marketing purposes without your express permission."

What the information will be used for

For example, subscribers' accounts will be used "internally to deliver the best possible service to you, such as improving the Gmail user interface, preventing fraud within our advertising system, and better targeting related information."

How the identity of the individual will be protected

For example, employees who have access to user information are monitored and educated to protect the user's privacy.[30]

Acknowledgments

This work was funded, in part, by NSF Award IIS-0325035.

About the Authors

Chapter Twenty Five. Social Approaches to End-User Privacy Management

Jeremy Goecks and Elizabeth D. Mynatt

THOUSANDS OF YEARS AGO, THE EARLIEST HUMANS LIVED TOGETHER IN TRIBES. They hunted in bands to maximize their effectiveness, shared stories in the evening to entertain themselves, and migrated together to preserve and enhance the knowledge that a tribe had acquired over the years. Today, Internet and communication technologies make it possible for colleagues to collaborate over large distances, for extended families to share experiences via digital photos, and for multinational businesses to easily aggregate and utilize organizational knowledge.

These two snapshots of human society, taken at its dawn and at its present state, reveal the innate social nature of people and how they leverage this nature to beneficial ends. People work, learn, and play together. Every day, each individual engages in numerous social activities and processes with close friends and family, acquaintances, and strangers alike. Because social activities are woven into everyday life, people have substantial experience and expertise in employing social processes to meet various needs, from hearing about local news to learning new skills to acquiring advice for decision making. It makes sense, then, that users can benefit by leveraging social processes when they are managing their digital privacy.

In this chapter, we discuss how a software system can employ social processes to help end users manage their privacy. End users are users who are not computer experts. They have a general understanding of computers but are unlikely to be familiar with most facets of privacy. We first describe a concrete, end-user privacy management problem— the managing of web browser cookies

Acumen: A Solution Using Social Processes

The Acumen system (see Figure 25-1) captures and makes visible people's cookie management actions. By making this information visible, Acumen enables users to employ social processes to maintain awareness of, learn about, and make decisions about their browser cookies. Acumen enables a group of users to manage their cookies via indirect collaboration through their actions; each individual user benefits by leveraging the group's collective knowledge and experiences.

Acumen is a fully functional system—the first system to make the actions of users within a community visible so that other members of that community can employ social processes when managing their privacy. Nevertheless, Acumen is a prototype, not a production system. Acumen is principally an exploratory system—both in design and in implementation—rather than a model of how to build a privacy management system that leverages social processes. We encourage other designers to draw from Acumen's features according to their users' privacy management needs.

Acumen Overview

Acumen enables users to manage cookies at the web site level, allowing or blocking each web site's cookies. Acumen allows all cookies by default; this default reflects the fact that most Internet users do not actively manage their cookies and thus do not block cookies. Acumen is an independent cookie management system and, as a consequence of architectural constraints, cannot interact with web browsers' cookie management tools. Thus, Acumen does not know which cookies are blocked by a web browser, and, conversely, cookies blocked by Acumen are never seen by the browser.

Acumen captures and makes the following data visible in aggregate:

The number of community members who have "visited" a web site (i.e., requested a file from the site)

The number of community members who allow the site's cookies

The number of community members who block the site's cookies

The "community" here is, of course, the community of Acumen users itself. This data is easy to collect because the collection imposes no additional burdens: users simply manage their cookies through the Acumen interface; Acumen records their actions and the resulting data, and aggregates the data for the benefit of the entire community, allowing users to see how others are managing their cookies.

Figure 25-1. The Acumen system

The data that Acumen provides can be utilized for many purposes: maintaining awareness of cookies, learning about cookies, and offering advice when deciding whether to allow or block a web site's cookies.

Supporting Privacy Management Activities with Social Processes

End-user privacy management is rarely a simple task. Instead, there are often multiple and ongoing activities that users must perform in order to manage their privacy. The most common activities for end users performing privacy management are the following; these activities frequently proceed in the order in which they are listed.

Awareness and motivation. An end user must achieve some level of awareness—or the system must provide awareness—of a potential privacy risk. Such awareness is necessary because it motivates the user to engage the risk.

Learning and education. The user must learn about the privacy risk and its accompanying management activity.

Decision making. Based on the user's awareness of and knowledge about the risk, he must make decisions about how to manage the risk.

In other domains, users maintain awareness, learn, and make decisions better when using social processes than they do individually.[11] , [12] It makes sense that users can also benefit from employing social processes to perform these activities in the context of privacy management . In the following sections, we discuss these activities in detail, how social processes can facilitate them, and how Acumen supports them.

Awareness and Motivation

Motivation is a critical aspect of end-user privacy management. If end users do not perceive a privacy risk, they will not use privacy management tools. Hence, a privacy management system frequently needs to raise awareness for the risk in order to motivate users to manage it.

Often, users must maintain ongoing awareness of a privacy risk. However, ongoing awareness can be overwhelming and distracting to users. Privacy management is rarely a primary activity for end users; rather, it is a secondary activity that is integrated into and performed alongside principal tasks.[13] For example, the primary activity for Internet users is browsing the Internet; managing cookies is a secondary activity, if that.

Because privacy management is a secondary activity at best, awareness of privacy risks and requisite management should be limited so as not to distract the user. Ideally, a privacy management system should filter information and alert users only to potentially important or new concerns and threats.

Raising end users' awareness of a privacy risk—and thereby raising users' motivation to manage the risk—can be accomplished via social means. Knowing that others recognize a risk and are managing it can serve to motivate a user to do the same; at the least, most users will take more notice of a risk if they observe that others are concerned about the risk. While there are users who will disregard the activities of others, most users will find others' activities to be useful information.

Awareness and motivation in Acumen

Acumen offers minimal and context-driven awareness. Acumen's toolbar is integrated into the web browser and displays information about only the web sites that are using cookies on the current web page. Thus, the toolbar provides relevant but limited information that a user can easily glance at and understand.

Acumen's icons make it clear that others are managing their cookies and also serve to alert users to potentially problematic cookies. Recall that there are two colored icons next to each site using cookies on the page: an arrowhead or X that indicates whether cookies are allowed/blocked, and the circle icon that denotes the site's user management data.

Just a quick glance by a user often provides valuable information about the cookies on the page. Problematic cookies stand out immediately because their site's user data icon is colored yellow or red, indicating that others have blocked the site's cookies. In addition, a user can glance at the two icons next to a web site name and determine whether her decision to allow/block the site's cookies matches others' decisions. A user's decision matches that of others if the icons are the same color; if not, the icons have different colors (see Figure 25-2). It is worthwhile to note that color perception and comparison are preattentive processes that are very fast and require little cognitive effort.[14]

Recall that Acumen uses nonlinear, biased thresholds for the color categories:

Deployment, Adoption, and Evaluation

So far, we have discussed how end-user privacy management systems can employ social processes to help users more effectively manage their privacy. In this section, we address how to deploy, foster adoption of, and evaluate such systems.

Deployment and Adoption

Successfully deploying and fostering adoption of a privacy management system that employs social processes is difficult. Such systems are groupware: software systems that support groups of people rather than individuals or whole organizations. A groupware deployment is deemed successful when enough individuals adopt the system so as to sustain its use. Several known obstacles prevent successful deployment and adoption of groupware.[23] For privacy management systems that are also groupware, perhaps the two most significant obstacles are obtaining critical mass and mitigating the disparity between work and benefit for users.

A groupware system must obtain a sizable number of users—a critical mass—if it is to succeed. Without a critical mass of users, the system often cannot provide enough benefits to those that use the system, and eventually use of the system stops. Critical mass differs among various groupware systems. Moreover, for some groupware, it is never advantageous for any single user to use it, and thus adoption must be considered from a group perspective rather than an individual perspective.

A groupware system will fail if it requires that some or all users expend significant effort contributing to the system but obtain little benefit from it. It is important for a groupware system to balance the work required and the benefits received for all users; this is especially true for knowledgeable users who, given sufficient motivations or benefits, contribute more to the system than others.

Deployment and adoption in Acumen

Acumen's critical mass is highly correlated with the degree of coverage that its data can obtain. Acumen is most effective when it can provide data about many of the potential decisions a user may face; in other words, Acumen is most effective when it can provide user cookie management data for many different web sites. In a deployment of Acumen, we found that Acumen achieves significant coverage over the most popular web sites with as few as nine users. We discuss Acumen's coverage in more detail in this section.

Gaming and Anti-gaming

Gaming is the process by which an entity manipulates a system to serve her personal interests while harming other users. A system that uses social processes is susceptible to gaming by an entity, whether the entity is a single user, a group of users, or an organization that is acting through users. Using basic economics[26] as an analysis lens, we can infer the conditions under which gaming will occur; these conditions are:

There is an entity with an incentive to game the system.

The benefit that the entity obtains from gaming the system is greater than the cost incurred by the entity to game the system.

Generalizing Our Approach

So far, we have focused on addressing the privacy management challenges that web site cookies pose. There are, however, numerous other domains where users would benefit from using social processes to manage their privacy. In this section, we generalize our approach and describe its necessary components; in the process of this generalization, we discuss how social processes could be used to address another privacy problem: adware and spyware.

Advertiser-supported software, or adware, is software that displays advertisements while running; revenue from the advertisements goes to the software's developers. Because the ads displayed by adware may use cookies, adware poses the same privacy risks as traditional cookies do. In fact, often adware is simply a web browser add-on that displays unwanted pop-up windows.

Spyware is software that gathers and transmits information about a user to an external party unbeknownst to, and without the consent of, the software's user; often, the external party is a marketing company. Of course, spyware presents privacy risks to users, as it is designed to invade users' privacy.

Adware and spyware are overlapping categories of software. Adware can be spyware; spyware need not be adware; and often spyware is bundled and installed with other software. Many users object to adware and spyware because such software violates their privacy, and hence adware and spyware often go to great lengths to hide how and when they collect information about users. The privacy management problem that users face, then, is distinguishing adware and spyware from other software on their computer and on the Internet, and deciding whether to install or uninstall a piece of software.

Existing tools that detect and remove adware and spyware, such as AdAware [27] and Spyware Eliminator ,[28] are often effective but are also limited in some respects. These tools are not always able to recognize and remove the most current adware and spyware. In addition, using an adware/spyware removal program requires that the user trust the program's categorizations of adware and spyware. However, research has shown that people respond to and make more use of personal, concrete information than they do abstract information.

Conclusion

Leveraging social processes to help end users manage their privacy is a promising approach. End users engage in awareness, learning, and decision-making activities in order to effectively manage their privacy, and social processes offer users intuitive, robust methods to support these activities. This chapter has discussed the principal features of end-user privacy management systems that employ social processes.

About the Authors

Chapter Twenty Six. Anonymity Loves Company: Usability and the Network Effect

Roger Dingledine and Nick Mathewson

OTHER CHAPTERS IN THIS BOOK HAVE TALKED ABOUT HOW USABILITY IMPACTS SECURITY. One class of security software is anonymizing networks —overlay networks on the Internet that provide privacy by letting users transact (for example, fetch a web page or send an email) without revealing their communication partners.

In this chapter, we'll focus on the network effects of usability on privacy and security: usability is a factor as before, but the size of the user base also becomes a factor. As we will see, in anonymizing networks, even if you were smart enough and had enough time to use every system perfectly, you would nevertheless be right to choose your system based in part on its usability for other users.

Usability for Others Impacts Your Security

Usability Is Even More Important for Privacy

Usability affects security in systems that aim to protect data confidentiality. But when the goal is privacy, usability can become even more important. A large category of anonymizing networks, such as Tor, JAP, Mixminion, and Mixmaster, aim to hide not only what is being said, but also who is communicating with whom, which users are using which web sites, and so on. These systems have a broad range of users, including ordinary citizens who want to avoid being profiled for targeted advertisements, corporations that don't want to reveal information to their competitors, and law enforcement and government intelligence agencies that need to do operations on the Internet without being noticed.

Anonymizing networks work by hiding users among users. An eavesdropper might be able to tell that Alice, Bob, and Carol are all using the network, but should not be able to tell which of them is talking to Dave. This property is summarized in the notion of an anonymity set—the total set of people who, as far as the attacker can tell, might include the one engaging in some activity of interest. The larger the set, the more anonymous the participants.[2] When more users join the network, existing users become more secure, even if the new users never talk to the existing ones![3] Thus, "anonymity loves company."[4]

In a data confidentiality system like PGP, Alice and Bob can decide by themselves that they want to get security. As long as they both use the software properly, no third party can intercept the traffic and break their encryption. However, Alice and Bob can't get anonymity by themselves: they need to participate in an infrastructure that coordinates users to provide cover for each other.

No organization can build this infrastructure for its own sole use. If a single corporation or government agency were to build a private network to protect its operations, any connections entering or leaving that network would be obviously linkable to the controlling organization. The members and operations of that agency would be easier, not harder, to distinguish.

Thus, to provide anonymity to any of its users, the network must accept traffic from external users, so the various user groups can blend together.

In practice, existing commercial anonymity solutions (like http://Anonymizer.com) are based on a set of single-hop proxies. In these systems, each user connects to a single proxy, which then relays the user's traffic. Single proxies provide comparatively weak security, because a compromised proxy can trivially observe all of its users' actions, and an eavesdropper needs to watch only a single proxy to perform timing correlation attacks against all its users' traffic. Worse, all users need to trust the proxy company to have good security itself as well as to not reveal user activities.

The solution is distributed trust: an infrastructure made up of many independently controlled proxies that work together to make sure that no transaction's privacy relies on any single proxy. With distributed-trust anonymizing networks like the ones discussed in this chapter, users build tunnels or circuits through a series of servers. They encrypt their traffic in multiple layers of encryption, and each server removes a single layer of encryption. No single server knows the entire path from the user to the user's chosen destination. Therefore, an attacker can't break the user's anonymity by compromising or eavesdropping on any one server.

Despite their increased security, distributed-trust anonymizing networks have their disadvantages. Because traffic needs to be relayed through multiple servers, performance is often (but not always) worse. Also, the software to implement a distributed-trust anonymizing network is significantly more difficult to design and implement.

Beyond these issues of the architecture and ownership of the network, however, there is another catch. For users to keep the same anonymity set, they need to act like each other. If Alice's client acts completely unlike Bob's client, or if Alice's messages leave the system acting completely unlike Bob's, the attacker can use this information. In the worst case, Alice's messages stand out entering and leaving the network, and the attacker can treat Alice and those like her as if they were on a separate network of their own. But even if Alice's messages are recognizable only as they leave the network, an attacker can use this information to break exiting messages into "messages from User1," "messages from User2," and so on, and can now get away with linking messages to their senders as groups, instead of trying to guess from individual messages. Some of this partitioning is inevitable: if Alice speaks Arabic and Bob speaks Bulgarian, we can't force them both to learn English in order to mask each other.

What does this imply for usability? More so than with encryption systems, users of anonymizing networks may need to choose their systems based on how usable others will find them, in order to get the protection of a larger anonymity set.

Case Study: Usability Means Users, Users Mean Security

Let's consider an example. Practical anonymizing networks fall into two broad classes: high latency and low latency.

High-latency networks. Networks like Mixminion and Mixmaster can resist strong attackers who can watch the whole network and control a large part of the network infrastructure. To prevent this "global attacker" from linking senders to recipients by correlating when messages enter and leave the system, high-latency networks

Bootstrapping, Confidence, and Reputability

Another area where human factors are critical in privacy is in bootstrapping

Technical Challenges to Guessing the Number of Users in a Network

Conclusion

About the Authors

Part V. Commercializing Usability: The Vendor Perspective

Chapter Twenty Seven. ZoneAlarm: Creating Usable Security Products for Consumers

Jordy Berson

WHEN ZONEALARM 2.0 DEBUTED IN JANUARY 2000, it was downloaded more than 1 million times in the first 10 weeks. We had tapped into a sorely needed space—an easy-to-use personal firewall—and the 20 or so Zone Labs employees suddenly found themselves strapped to a rocket and struggling to hold on. By 2005, the company had grown to more than 180 employees. The ZoneAlarm product family now has more than 30 million users, making it the most widely used software firewall in the world. We believe our products have been so successful because they are both highly secure and easy to use.

Design Principles

This section describes the basic principles underlying the design of ZoneAlarm:

Know your audience

Think like your audience

Eliminate clutter

Eliminate complexity

Create just enough feedback

Be a customer advocate when usability and competitive pressure collide

Know Your Audience

Our company held a weekly lunch meeting series in which we could hear an architect explain a bit of the technology within our firewall. The first meeting was packed with people anxious to learn, and about half of us were from marketing. As the words began to spill out of the architect's mouth, those of us in marketing quickly realized that the lecture may as well have been in Latin. What the architect had to say may have been brilliant, but it failed for the very simple reason that it didn't reach the audience.

Now imagine getting this wrong on your interface design, considering the thousands of hours it takes to design, build, test, and market a product. If you're not speaking your audience's language, all that work is wasted. But even a minor mismatch with your audience means that your product will be less successful than it could be, that it could lose its edge on the competition. and that the very customers you are trying to protect may make wrong decisions that will leave them exposed. That's why you must know your audience intimately. Skip this step, and you leave opportunity on the table. Even if your product doesn't fail outright, it won't be as good as it could have been.

A mistake I made early in my career was thinking that knowing the audience was simply inherent in my job title. I, the product manager with the arts and humanities major and a background in writing, should naturally know better than the nerdy engineers what my audience wants. Of course, I was wrong. To know the audience, I needed to get out of the office and talk to them. Without this information, your guess is no better than anybody else's. If you're the key decision-maker for your product's design, this concept is powerful

Striving to provide the strongest security that's also easy to use may seem impossible at times. For example, antivirus, standard intrusion detection, and other signature-based solutions are very easy to use but offer little protection against emerging and as-yet-unknown viruses and malware. But combating old as well as brand-new threats often involves an aware user who can react to real-time attacks and understand the difference between a real threat and something benign. You know, that user we spoke about who doesn't want to spend any of her time using your software? As pointed out in Chapter Two, "Security is dependable only if it is actually used as intended." While there is no single way to solve this conundrum, here are a few tips that help us to get closer to achieving good security:

Create interaction points only when you absolutely must. If you interrupt users with pop-ups that have noncritical or less meaningful information, then the user is less likely to pay attention to critical information.

If you cannot answer a question yourself, or you have a tough time explaining the question to others, your users probably cannot answer the question either. Come up with a different question to ask that may get to the same answer.

Set up multiple layers of security so that even if a threat gets past one layer, another will stop it. This way, user errors don't have to be catastrophic.

Create automation if you can do so securely. There may be technical solutions that can lessen or eliminate the need for user interaction. But keep in mind the raison d'être for your product and do not sacrifice security in the process.

Explore alternative interfaces, such as wizards. Often, a carefully designed wizard can take what would be a tough decision and break it down into easier and more intuitive questions. But stay away from wizards that are too long. And if a wizard contains questions that are as tough to answer as the original question the wizard was intended to replace, then the wizard is not worth doing.

When you're analyzing the effectiveness of your product's security, always include the customer in your analysis. If it is likely that the customer will make an error and reduce security, either redesign the experience to minimize the risk or, in extreme cases, consider revamping the technology or leaving it as an advanced option rather than a default setting.

Remember that a false sense of security is worse than no security at all. It is better to leave out a portion of functionality if it's highly prone to failure, than to include it.

and indispensable. Know your audience, and you'll not only be able to create much more successful design, but also earn the respect and credibility of your team so that you're empowered to make good decisions. But even if you're not personally talking to users, make sure that someone on your team is.

Efficient Production for a Fast Market

Creating good usability for a single feature can take months of research, testing, and iterative design. But in demanding market conditions, we're often forced to create designs in much less time. This can weaken design and can even end up costing more time and expense if design flaws are discovered after code has already been written. In such a fast-paced environment, here are a few tips for making the most of your time:

Get as much design time as possible from management

Remind them that cutting time from the beginning of the design process often means doubling or tripling time and expense later in the cycle when unforeseen design problems occur.

Form teams early

Conclusion

Design for the ZoneAlarm product line can be best summed up by two things: identify and study the target audience, and apply that knowledge to the design. Following these principles doesn't guarantee great design, but without them, design fails. There is no question that designing usable security is difficult, and no company I've seen, including Zone Labs, does it flawlessly. By reading this book, you are benefiting from the research and experience of all of us and will be well prepared to join us in this ongoing battle to protect computer users of every kind from Internet-borne threats.

About the Author

Jordy Berson is a senior product manager at Zone Labs and has guided the design of the ZoneAlarm product line since 2000. His focus at Zone Labs is on creating software that "his mom could use" to combat Internet threats. He is a graduate of Colorado State University in technical journalism with a mechanical engineering concentration.

Chapter Twenty Eight. Firefox and the Worry-Free Web

Blake Ross

AS THE PRIMARY BARRIER BETWEEN MILLIONS OF UNSUSPECTING WEB USERS AND THE CON ARTISTS who want their credit card numbers, Firefox developers are acutely aware of the challenges in securing the Web. And so, it seems, are our users. More than 15 million people downloaded Firefox 1.0 in its first two months—a gold rush driven by Firefox's speed and reliability, no doubt, but also driven by the barrage of media warnings against online identity theft, data loss, and continuing security problems with Microsoft's Internet Explorer (IE).

Usability and Security: Bridging the Gap

The Five Golden Rules

It's too easy for developers to throw up their hands and say, "Users are the weakest link." In fact, the blame frequently lies with developers for not understanding how users interact with technology and designing interfaces that facilitate mistakes. When designing interfaces that need to both serve users and protect them, a few questions come up again and again: when should the program turn to the user for input? When it does, how should the program solicit that input? This section will offer answers to these questions as we have fine-tuned them during our work on Firefox.

Identifying "The User"

There are undoubtedly people who will take offense at some of the statements made in the following sections. It's important to realize that "the user," as discussed here, is meant to represent the intended majority of our Firefox audience—the dentists, lawyers, teachers, stay-at-home dads, and other members of society who aren't in the computer profession and don't know anything about security beyond what they read in the news. Although Firefox is becoming increasingly popular in enterprises, this chapter does not apply to corporate customers, who usually have more interest in their own security and frequently have trained IT departments who have security as an explicit responsibility. Firefox offers different solutions for corporations—without harming the Firefox consumer users—often by simply putting advanced options behind a technical wall that's impenetrable by most home users.

1. Enforce the Officer/Citizen Model

Society has constructed a variety of organizations to regulate and protect itself. The police maintain order and enforce the law. Insurance companies protect against unpredictable disasters. Lawyers defend personal rights and freedoms. People want to delegate their security to experts who specialize in the field and are able to guarantee it.

It's easy to forget that the same is true of their online security. The old Mozilla software is a chronicle of indecision within the walls of Netscape. Every option, confirmation window, and question to the user marks another case where two internal camps couldn't agree on the most secure way to proceed and instead deferred to the user's decision. This approach, of course, is absurd. Consider arriving home to an unsolicited package on your doorstop. You call the bomb squad, and after some analysis, the lead officer comes to brief you: "An X-ray of the package was perfectly normal, and the organization that sent it seems to be reputable, but a chemical swab of the exterior yielded some concerning figures. Gee, I dunno. Your call."

The example seems over the top until you consider the questions the average user encounters every day on the Internet. Figure 28-1 illustrates a particularly mystifying one that IE users can encounter while attempting to navigate to certain sites, many of which are in fact trustworthy. The "View Certificate" button opens a window containing what amounts to Greek to a majority of web users; in our real-world analogy, it's the equivalent of the officer providing you with the raw data output of the chemical swab.

Figure 28-1. Internet Explorer 6.0 users can encounter this baffling dialog in day-to-day surfing

Before coding up such confirmation windows, software developers—the group most notorious for encouraging them as a suitable compromise—should ask themselves a question: if I can't figure it out, how are users supposed to? What additional qualifications do users possess that will help them arrive at a better conclusion? What evidence is there to suggest that most users even know what a certificate is in the digital sense?

In developing Firefox, we challenge ourselves daily to do something that has proven difficult in the software industry: make decisions. We want users to know that they can go about their business on the Internet with security in the back of their minds, because it's on the front of our minds. The intent is not to be covert, or to pretend that security isn't a concern; we do want our users to know that we're making decisions on their behalf. But we want them to have confidence in our ability to make such judgments (hence our slogan, "the browser you can trust").

So, how do you discern the bombs from the fruitcakes when there's not enough information to make the call? You need to err on the side of caution, but only if the circumstances justify it. That means that if 99.9% of sites matching the profile outlined in Figure 28-1 are reputable sites that forgot to dot the i's in their certificates, you owe it to your users to just navigate to the site—especially because usability studies indicate that almost all of them will click "Yes" without reading the question.[2]

Conclusion

Our decisions in Firefox—security and otherwise—are guided entirely by one core, humbling belief shared by all members of the team: software is just a means to an end. Don't overwhelm your users with talk of ciphers and encryption and other jabberwocky. Empathize with them, and force yourself to make the tough decisions so that they don't have to. Be human, not corporate, and respond to security fire drills with tenacity and alacrity. Do all of these things, and usability will follow.

About the Author

Blake Ross is an undergraduate student at Stanford University. After working at Netscape for several years, he helped create Firefox (then called Phoenix) in fall 2002. He is also working with Dan Boneh and others in the Stanford cryptography group to develop new tools to protect people against phishing attacks.

Chapter Twenty Nine. Users and Trust: A Microsoft Case Study

Blake Ross

BACK IN THE GOOD OLD DAYS, if you lived in a small town, you wouldn't think twice about leaving your house unlocked while you ran errands, about letting kids play in the streets, or about sharing details of your family's life with other people in the town.

However, as the small town grew, and more new people started to arrive, you might have started to hear about unusual things happening: property disappearing, park benches getting vandalized, strange behavior from the new neighbors.

Over time, you'd learn that maybe it was safer to lock your door, to ask your kids where they would be going, not to lend out your lawnmower. Normally, you would learn this through newspaper articles or stories that friends told you. Sometimes, if you were unlucky, you'd learn through personal experience of having something bad happen to you—something that would never have happened in the good old days.

The Internet has paralleled this move from small town to larger city life. With the advent of the first HTML browsers, the Internet became the World Wide Web, and many new neighbors moved in to what had previously been a relatively trusting small town. The new neighbors brought with them confidence tricks, unwanted mail, viruses, and lots of candy that it really wasn't safe to take.

The major difference between real-life small towns and the Internet is the compressed time scale of the Internet's growth. That growth rate, along with the relative anonymity afforded by the Internet and the extreme ease of creating a presence on the Web, has meant that many regular users of the Internet have not had enough time to build or adjust their perceptions of trust to deal well with the online environment.

Instead, the responsibility for helping users decide whom to trust online has fallen to the infrastructure providers: manufacturers of browsers and email programs, antivirus applications, and spyware scanners.

In the early days of the World Wide Web, fewer people were attempting to exploit the gaps in technological or social trust online. As the technologies matured and the user base grew, such exploits became more lucrative.

To counter this rise in the number of exploits, the infrastructure providers have incorporated technologies and user interface elements aimed at shaping users' behaviors, teaching them whom they can trust, and, where necessary, giving them the cues they need to make trust decisions. However, the code that infrastructure providers produce is much better at dealing with problems that have a logical right and wrong outcome (virus/no virus) than problems that have shades of emotional response, such as social engineering attacks.

Obviously, Microsoft is one of those technology providers. This chapter describes how research into users' trust mechanisms led to changes in user interface design philosophy for Internet Explorer and several other products at Microsoft. The changes represent a first step in respecting the emotional aspect of trust decisions, and in giving users the information they need to make good trust decisions within Microsoft applications.

Users and Trust

Consent Dialogs

The name of this much maligned interface element suggests something that isn't always apparent in the design—dialog boxes are supposed to be a conversation ("dialog") between the computer and the user. The consent dialog has a specific conversation topic: "Do you want this thing to happen?" Frequently, consent dialogs ask trust questions. If the question is well phrased, users should have little difficulty making a decision, completing the dialog, and continuing on their way. However, well-phrased dialogs seem to be difficult to design.

After observing many users working with dialogs in the lab, I would suggest the hierarchy of decision points shown in Figure 29-1 that users follow in order to continue with what they perceive as their real task—the one that the dialog box interrupted.

Figure 29-1. Decision points users make in evaluating a (hypothetical) consent dialog

Windows XP Service Pack 2—A Case Study

In August of 2003, senior executives at Microsoft made it clear that continuing with a reactive patching approach to fixing security exploits was insufficient. Instead, users needed a system that was able to deal with whole classes of exploits and to prevent infection. The technologies required to achieve this required a major rewrite of several components of the operating system, and also required us to make changes to the default behaviors in Windows.

While this last point may not sound like a big deal, this would be the first time in the history of Windows that a service pack release made significant changes to the user experience of the product. By working with the program managers for Service Pack 2, we presented evidence from user studies, surveys, instrumentation reports, and site visits to the Windows executives demonstrating that the release would not be successful without several large-scale changes to the user experience.

This data had been collected primarily from research into Passport, Hailstorm, XP Service Pack 1, and early work on Windows Codename "Longhorn"—the next version of Windows. It was fortuitous that it could be applied so directly to the problems that were addressed by XP Service Pack 2. We also worked to ensure that many of the changes could be controlled through group policy in order to satisfy corporate customers.

Some of the major attack points in Windows are found in the Internet-related features—Internet Explorer (IE), Outlook Express (OE), and the Application Execution Service (which prompts users to save or run downloaded or attached files). Various other system components, such as the Windows Firewall and Automatic Updates, are designed to help protect users but require user interaction in order to work correctly.

The user experience focus for SP2 was on making security and privacy "just work," and where that was not possible, making the security and privacy issues understandable to users so that they can make informed decisions.

To that end, we took several components of Windows and gave them a design overhaul. The following sections show before and after images with some commentary on the reasons for the changes. Most frequently, the reasons are based on the consent dialog redesign described earlier.

ActiveX Dialogs

It was very clear from user testing that the ActiveX dialog box in Windows XP and XP Service Pack 1 (XP SP1) (see Figure 29-3) was not very successful at giving users the information they needed in order to make an informed consent decision.

Figure 29-3. Original ActiveX consent dialog

Instead of just applying the consent dialog redesign to this interface, we considered ways of also increasing user satisfaction. This dialog box typically appears while a page is loading—often before any content has appeared on the first page of a site that a user visits. As such, it is both a frustration because it gets in the way of the user's primary task, and a dilemma because the user cannot judge the value of agreeing to the dialog box (if she even understands what it is asking).

Taking a design element from Outlook Express, where additional information about email format options and blocked content is displayed in a "bar" in the message title area, we developed an in-context but discreet notification called the Information Bar, which animates down from the Internet Explorer Toolbar.

The Information Bar (shown in Figure 29-4

Pop-Up Blocking

Pop-up blocking was new to Windows XP Service Pack 2. Pop-ups are windows that are generated by a web page to show additional content. A legitimate use for pop-ups may be to provide a glossary term, assistance, or a product photo when users click a link on a web page. This would allow the user to stay on the same page, but gain additional information that was not deemed important enough to have screen real estate devoted to it.

The Ideal

Conclusion

Throughout this chapter, I have focused on design solutions for the user behavior that I have observed with multiple participants in usability studies of many products. I have shown examples of those design concepts being turned into practical dialogs within Internet Explorer. The following recommendations can—and should—be applied to any trust interaction on computers:

About the Author

Chris Nodder is a user experience specialist with Nielsen Norman Group. Before joining NN/g, Chris worked as a senior user researcher at Microsoft Corporation. During his seven years at Microsoft, Chris worked on products as diverse as videoconferencing, programming tools for web developers, home networking, online communities, and delivering Internet content over cell phones. In 2004, he was responsible for the user experience for XP Service Pack 2, a major upgrade to Windows XP. He has created personas, reality TV episodes, and even whole rooms ("usertoriums") as ways of getting developers to walk in their customers' shoes.

Chapter Thirty. IBM Lotus Notes/Domino: Embedding Security in Collaborative Applications

Mary Ellen Zurko

THERE ARE A MULTITUDE OF APPROACHES TO MAKING SECURITY USABLE AND USEFUL

Embedding and Simplifying Public Key Security

The Lotus Domino PKI is not a free-standing product: it is integral to the Lotus Notes end-user and administrative experience. Every new user of the Notes desktop client is given, by his administrator, a public/private key pair in a keyfile called his ID file.[1] The user types in his password to decrypt and make use of the ID file. The Lotus Notes client uses a private key to authenticate to the Domino server, providing two-factor authentication (the

The IBM Lotus Notes and Domino architecture supports a suite of products that provide a client/server infrastructure for collaboration applications, as well as specific support for electronic messaging and calendaring and scheduling. Lotus Notes is a rich desktop client to the Domino server. It communicates with the Domino server through a proprietary, public key cryptographic protocol called Notes Remote Procedure Call (NRPC) . Notes client security features are generally accessed through the user security panel (Figure 30-1). A user's password (Figure 30-2) decrypts the file that contains her public/private keys.

The Lotus Domino server provides the foundation for custom collaboration applications, messaging, and web applications. Full enterprise deployment of this server can take advantage of administrative features such as clustering, partitioning, and automatic failover, for performance and scalability. A separate rich client, the Administrative client, provides the server administration features in a user interface experience tuned to the administrator's tasks and skills..

Each Domino application is associated with a design template that is instantiated in a Domino database. Access control lists are set and managed at the database level (see Figure 30-3 and 30-4). Domino Designer is application development software that provides the features that make up a Domino application, including forms, views, pages, framesets, integrated instant messaging, XML, Java, JavaScript, and LotusScript. These Domino applications

Designing Security Displays

Information about security attributes can, in theory, occur almost anywhere in a user interface. This placement makes good sense from a task perspective. For example, users can find all their email configuration options in one place, from delegating access to their mail to changing the text placed automatically at the bottom of each mail message. Sometimes, though, users are looking for a security feature and are not sure what task or context it might be associated with. Or they are trying to get a better sense of the overall security picture of the application to form a useful mental model. This latter task is more likely to arise for evaluators, reviewers, and administrators, people who provide guidance to larger populations, than for individual end users.

In this section, we discuss the usability of two displays that provide useful centralized security information in Lotus Notes.

User Security Panel

User-related security information was brought together in a single security panel in Notes 6, as shown in Figure 30-1. The motivation behind consolidating that information in one place is to make it easier to find. For example, information in only one of the resulting tabs, the Mail Security tab, was gathered from five different places in the previous product version. Consolidation can also make it easier to understand, and can help users learn about security features that they might not have noticed otherwise. The Lotus Notes User Security panel was explicitly designed with an eye toward making difficult security concepts easier to understand and use. In addition, access to security-sensitive information is protected with a password prompt (to ensure that it is not viewed or manipulated by anyone other than the user). Interleaving sensitive security and other types of information meant that a user had to type his password unnecessarily in some circumstances. Therefore, localizing that information made the usability of other paths better. An informative and entertaining article also accompanied the change,[4] using humor to highlight the potential utility of the security information. Informal feedback from users and customers to both the panel and the article has been enthusiastic.

Figure 30-1. User Security panel

The initial design of the User Security panel was performed by a security engineer. This allowed someone knowledgeable about the content of the panel to provide initial organization and explanations. The dialogs were designed to explain topics and information directly to the user. Each was structured to highlight basic information or useful task-specific security procedures.

After the initial panel design by a security engineer, design walkthroughs were held with a usability expert. These walkthroughs found and corrected many visual design problems that impeded usability. They applied color and whitespace to make data items and the relationships between them more understandable. Visual design discussions included whether tabular information should be organized by concept (such as infrastructure and specific applications) or by the likelihood of being used. A mixed model of a single first tab for the most-used items, followed by a conceptual grouping for all of the other tabs, was adopted.

The walkthroughs also clarified basic security concepts useful to many users, because they were understood by the usability expert who could offer specific informed suggestions for improvement around them. Making terminology more understandable generated some robust discussions. Security features targeted at technology, not user artifacts, are often the most difficult to explain. For example, Lotus Domino supports several protocols, and therefore several methods of authentication. Notes authentication, as described earlier, uses a password to unlock a private key; subsequently, public key cryptography authenticates the Notes user to the Domino server. Browser access to the Domino server uses web-standard, password-based mechanisms with password checking against a hashed version stored in the directory. These two passwords can easily be unsynchronized, leaving the user with two different Domino passwords: one for Notes RPC and one for other protocols. Usability experts found the term Internet password, used by Notes developers to refer to one of the passwords, to be precise but not informative to users. They recommended web password

User Control of Active Content Security

Developers, administrators, and users are all involved in making (or avoiding) security decisions. Developers determine many of the decisions available to (or imposed on) administrators and users, and administrators do the same for their users. Users have the most context for making those decisions, but have the least time (and often the least experience) for thinking about security impacts and risks. Conversely, any security-related decision not made available to the user or administrator is one less way they can protect themselves.

All of these considerations inform the design of security features. In addition, security features should be policy neutral. They should ensure that the system is securable, but not impose a particular security policy on it. However, any security-related decisions that are made available to users need to have a default behavior chosen for them. Users may find it difficult to evaluate why the default might not be right for them.

Choice of the default will determine how open or secure the feature is. Defaults can have a large impact on both the usability and the security of a system. Open defaults have traditionally allowed security and usability concerns to remain unaddressed, because the security features were not part of the initial user experience of the system. They have also allowed users to deploy systems in an unsecured state, increasing the chance of successful attacks.

Conclusion

About the Author

Mary Ellen Zurko leads security architecture and strategy for Lotus Software at IBM. She defined the field of User-Centered Security in 1996. She is on the steering committee for the New Security Paradigms Workshop and the International World Wide Web Conference series. She has worked in security since 1986, at The Open Group Research Institute and Digital Equipment Corporation, as well as at IBM.

Chapter Thirty One. Achieving Usable Security in Groove Virtual Office

George Moromisato, Paul Boyd, and Nimisha Asthagiri

WHEN WE DESIGNED GROOVE VIRTUAL OFFICE , we struggled to balance security and usability. Ray Ozzie, whose vision we were executing, set out the following core principles early in the design phase:

Users should not be required to be administrators. Users care about getting their work done. They are not interested in setting up accounts, configuring network topologies, or distributing security keys. Groove Virtual Office should "just work."

Groove Virtual Office Design

In the following sections, we look at the key design issues for Groove Virtual Office.

The Weakest Link

In Secrets and Lies, Bruce Schneier concludes that correct cryptographic algorithms are necessary but not sufficient for creating secure systems.[1] Indeed, in complex systems, the cryptographic algorithms are the last place that anyone attacks. Why exploit a weakness in the random number generator when users will launch any email attachment that promises them a visual thrill? Why set up an elaborate man-in-the-middle attack when users are likely to write their passwords down on sticky notes? A system is only as secure as its weakest link, and in many systems, the weakest link is often the user.

Of course, security professionals have been aware of the limitations of the user ever since Troy accepted a free horse. There are two common user interface techniques for strengthening the user link:

Figure 31-1. Groove Workspace was used in the Iraq War to assess the humanitarian assistance needs of people affected by the fighting; Groove was chosen for its ability to work in austere networking infrastructures

Prevent the user from doing the wrong thing—for example, enforce strong password policies

Teach the user how to do the right thing—for example, teach people how to choose a strong password

Both techniques have their place, but unfortunately, both also have limitations. Jakob Nielsen writes that as passwords become stronger (and thus more difficult to remember), users tend to write their passwords down on sticky notes.[2] Thus, Nielsen argues, enforcing strong passwords weakens security as a direct consequence of being less usable. We would argue, however, that the effect on security is subtler. If strong passwords are enforced, remote attacks (i.e., attacks over the Internet) become harder but insider attacks become easier (because passwords will be on sticky notes).

Teaching the user to do the right thing also has tradeoffs. For most users, computers are tools used to get their job done. Most users are not motivated to learn about security procedures. When we first designed Groove, we decided to encourage people to use strong passwords by prompting for a passphrase in our login and account creation screens. We felt that this would be a subtle reminder that people could and should use a phrase rather than a word for their login credentials. Unfortunately, this minor deviation from user expectation ended up causing more confusion than it was worth. In usability testing, we found that users struggled in the account creation screens. Several users were confused by the word passphrase and were not sure what to enter. Moreover, this unfamiliar term reinforced the perception that Groove Virtual Office was nonstandard and thus more complicated than software people were used to. This would lead users to choose to send sensitive information via email rather than Groove. Because email is often sent in the clear, this left the user worse off from a security perspective. However, the performance of workers is rarely measured by the security measures they take. For this reason, they are likely to choose the medium that allows them to accomplish their task (sending a file) quickly and comfortably. Thus, given a choice between an easy solution that is insecure and a difficult solution that is secure, the user is likely to choose the former. Groove Virtual Office had to be a solution that is both easy to use and secure.

Do the Right Thing

From experiences like these, we concluded that we needed a flexible security model. This was all the more important given the different security needs of our diverse target audience—ranging from office workers who mainly use email and Microsoft Word, to military and intelligence personnel. Each user group requires different levels of security but also has different tolerance levels for security inconveniences. The security model of Groove Virtual Office accommodates all of these groups. Depending on the needs and abilities of the user, and the properties of the networking environment, Groove applies one or both of the two user interface (UI) techniques (i.e., enforcing and teaching) to maximize the security of the system.

The main user groups for Groove are:

Office workers. Most office workers want to think about their work, not their infrastructure. They tend to avoid products that force them to learn new concepts, and they want security to be as invisible as possible. For these users, Groove relies on a centralized server to enforce security policies and to serve as a certification authority. As we describe later in this chapter, a central server allows Groove to provide secure communications without any user intervention or effort.

Administrators' Strengths and Weaknesses

The user interface described in the previous sections is an example of teaching and guiding the user to maintain the security of the system. As much as possible, Groove exposes security information (such as authentication colors) in a simple way that the user can understand. Nevertheless, there are situations in which more controlled mechanisms are needed. Groove also uses the technique of enforcing behaviors when appropriate.

Security and Usability

About the Authors

George Moromisato is chief product designer at Groove Networks, Inc. He has had the great fortune to participate in the design and development of Ray Ozzie's software for the last 14 years.

Paul Boyd manages the Design Team at Groove Networks and is responsible for the user interface design and usability of Groove Virtual Office.

Nimisha Asthagiri has managed the Security Team at Groove Networks and has led the design and implementation of all aspects of security in Groove Virtual Office.

Part VI. The Classics

Chapter Thirty Two

Chapter Thirty Three

Chapter Thirty Four

Chapter Thirty Two. Users Are Not the Enemy

Why Users Compromise Security Mechanisms and How to Take Remedial Measures, by Anne Adams and M. Angela Sasse

CONFIDENTIALITY IS AN IMPORTANT ASPECT OF COMPUTER SECURITY. It depends on authentication mechanisms, such as passwords, to safeguard access to information. Traditionally, authentication procedures are divided into two stages:[1], [2]

Identification (user ID), to identify the user

Authentication, to verify that the user is the legitimate owner of the ID

It is the latter stage that requires a secret password. To date, research on password security has focused on designing technical mechanisms to protect access to systems; the usability of these mechanisms has rarely been investigated. Hitchings[3] and Davis and Price[4] argue that this narrow perspective has produced security mechanisms that are, in practice, less effective than they are generally assumed to be. Because security mechanisms are designed, implemented, applied, and breached by people, human factors should be considered in their design. It seems that, currently, hackers pay more attention to the human link in the security chain than security designers do, for example, by using social engineering techniques to obtain passwords.

The key element in password security is the crackability of a password combination. Davies and Ganesan[5] argue that an adversary's ability to crack passwords is greater than usually believed. System-generated passwords are essentially the optimal security approach; however, user-generated passwords are potentially more memorable and thus are less likely to be disclosed (because users do not have to write them down). The U.S. Federal Information Processing Standards[6] (FIPS) suggest several criteria for assuring different levels of password security. Password composition, for example, relates the size of a character set from which a password has been chosen to its level of security. An alphanumeric password is therefore more secure than one composed of letters alone. Short password lifetime—changing passwords frequently—is suggested as reducing the risk associated with undetected compromised passwords. Finally, password ownership, in particular individual ownership, is recommended to:

Increase individual accountability

Reduce illicit usage

Allow for an establishment of system usage audit trails

Reduce frequent password changes due to group membership fluctuations

There is evidence that many password users do not comply with these suggested rules. DeAlvare[7] found that once a password is chosen, a user is unlikely to change it until it has been shown to be compromised. Users were also found to construct passwords that contained as few characters as possible.

Users Lack Security Knowledge

Parker[13] points out that a major doctrine in password security, adopted from the military, is the need-to-know

Security Needs User-Centered Design

Insufficient communication with users produces a lack of a user-centered

Motivating Users

A technical bias toward security mechanisms has produced a simplistic approach to user authentication: restricting access to data by identification and authentication of a user. This simplistic approach may work well in military environments, but it limits usable solutions to the security problems of modern organizations seeking to encourage work practices such as teamwork and shared responsibility. Such organizations require support for trust and information sharing. The authoritarian approach has also led to security departments' reluctance to communicate with users with regard to work practices. It has been suggested by the U.S. Federal Information Processing Standards[14]

Users and Password Behavior

Insecure work practices and low security motivation have been identified by research on information security as major problems that must be addressed.[16] , [17] , [18] , [19] The research presented here does, however, clearly identify the cause of these user-related problems; in the sidebar "Recommendations," we summarize methods for addressing these problems. There is an implicit assumption that users are not inherently motivated to adopt secure behavior, but that such behavior can be achieved through drills and threats of punishment in case of noncompliance. Knowledge from psychology and human-computer interaction indicates that users' behavior is likely to be more complex than a simple conditioned response. This study demonstrates that users forced to comply with password mechanisms incompatible with work practices may produce responses that circumvent the whole procedure. Insecure work practices and low security motivation among users can be caused

About the Authors

Chapter Thirty Three. Usability and Privacy: A Study of KaZaA P2P File Sharing

Nathaniel S. Good and Aaron Krekelberg

P2P FILE SHARING SYSTEMS SUCH AS GNUTELLA, FREENET, AND KaZaA, while primarily intended for sharing multimedia files, frequently allow other types of information to be shared. This raises serious concerns about the extent to which users may unknowingly be sharing private or personal information.[1]

IIn this chapter, we report on a cognitive walkthrough and a laboratory user study of the KaZaA file sharing user interface. The majority of the users in our study were unable to tell what files they were sharing, and sometimes incorrectly assumed that they were not sharing any files when, in fact, they were sharing all files on their hard drive. An analysis of the KaZaA network suggested that a large number of users appeared to be unwittingly sharing personal and private files, and that some users were indeed taking advantage of this and downloading files containing ostensibly private information.

Introduction

The excitement about P2P systems has been encouraged by recent innovations that foster easier sharing of files, such as downloading simultaneously from multiple sources, and the sharing of many different file types, as well as improvements to the usability of these clients. Of the current P2P systems, KaZaA is by far the most popular and widely used, with more than 120 million downloads worldwide and an average of 3 million users online at any given time. The user interface (UI) for finding files is straightforward: the user types a query into a textbox and, from the results, selects a matching filename to download. If sharing is enabled, the files that the user downloads are then shared automatically with other users on the network. The success of a P2P file sharing network depends on people sharing files with one another, so this feature helps promote file sharing by recycling files in the network.

While facilitating file sharing and searching, the systems do a poor job of preventing users from accidentally sharing personal files. Users attracted to the simplicity of downloading files provided by the P2P network can inadvertently allow access to their private data files, such as email, tax reports, work-related spreadsheets, and private documents. This is especially problematic in a single-machine, multiple-user situation typical of families sharing a single computer. In such a setting, a parent could have a secure VPN connection to a corporation for downloading and working on important confidential files, only to have them inadvertently shared by a teenage son or daughter, without either party's knowledge. This is not simply a theoretical problem but describes a scenario that is possible in the current reality. Our research suggests that people are unintentionally sharing what appear to be personal or confidential files via KaZaA. Queries for files such as Inbox for Outlook Express (.dbx

Usability Guidelines

By looking at the KaZaA network, we surmised that abuses are occurring, and their frequency demonstrates that they are not isolated events.

Based on a list of security guidelines

Results of the Cognitive Walkthrough

Recent versions of the KaZaA application have made some progress in addressing access control issues. Default installation of KaZaA for users using a recent version of KaZaA 1.7.1 is relatively safe: it creates a shared file folder, assigns this as the default download file, and indexes these folders for the My KaZaA library. Various versions of KaZaA had various settings, and had different default configurations. The latest version of KaZaA 1.7.2 (the most recent at this time) and other previous versions of KaZaA offered to search for files to share with users during the initial setup. In the latest version at the time of this writing and in some previous versions, file sharing is enabled. While a default setup where file sharing is disabled, as in version 1.7.1, is relatively safe, user modifications of various settings are not. By adding or changing directories to be shared, there are potential interface issues that can create misunderstandings about what files the system is sharing with other users, regardless of the version of KaZaA that the user is using. There are a number of reasons why a user would change default settings. Three common scenarios are driven by users' desire to save the files being downloaded to a different location, to share more files with other users, or to add files to their shared files folder. In the following sections, we will walk through each of these scenarios and the various ways that KaZaA allows these to be accomplished. We will look at the various safeguards that KaZaA employs to prevent users from sharing private files, or files that they do not want others to see, and describe where they seem likely to fail.

Changing the Download File Directory

In KaZaA, as in most P2P applications, the default shared directory provides the dual purpose of specifying the files that the user decides to share with the network, and the place where these files will be stored. The default shared directory is (confusingly) alternatively referred to as My Media, My Shared Folder, My KaZaA, and the folder for downloaded files in KaZaA.

My Shared Folder refers both to a folder created on the machine by KaZaA and abstractly to all folders that the user shares. For simplicity, we will refer to My Shared Folder as the actual folder that KaZaA creates by default to share files, and all files allocated for sharing as the shared files. The My Media and My KaZaA folders list all the shared files, yet organize them according to media type, such as music, video, documents, etc. For purposes of simplicity, we will refer only to My Media. The folder for downloaded files refers to the folder where download files will be stored, but also to the root folder of all shared files. Although each folder is used in a different context, they are essentially different names for the same thing: a list of files that KaZaA can share with others.

While the My KaZaA and My Media folders are accessible through clearly visible buttons, specifying shared files and the folder for downloaded files are managed through the Options menu, in the Tools tab (Figure 33-3). Additionally, the Options → Tools tab contains a checkbox for users to determine whether they would like to share files with the KaZaA community. Users may type in the name of the directory they would like to download files to, or alternately browse their filesystem and select the folder they would like to use to store downloaded files (Figure 33-4).

If the user has decided to share files with others, then all files in this directory, as well as the directories below it, are recursively shared and added to My Media files. The wording of the download folder is confusing. The word folder is singular, implying one folder, and

Figure 33-3. The Traffic tab in the Options menu; here is where users specify the download and My Shared directory as well as toggle sharing of all files

Figure 33-4. Browsing and selecting interface for the shared download folder; note that the interface says browse for folder, and does not mention that subfolders will be recursively searched for files

A Two-Part User Study

We attempted to devise a test to determine how easy it is for users to misconfigure their system. After several trials, we discovered that our attempts to devise a realistic scenario involving sharing and specifying file locations became contrived and only confused our test participants. Nor could we simulate the length of time over which settings are most likely changed and then forgotten during real use. In addition, we could not speculate on all of the various reasons users would want to change their default settings, although we knew from our data that they were indeed modifying the settings and were not aware of it.

From anecdotal evidence, we know that KaZaA users often work in shared computer settings, so it is quite possible for one user to change all the settings and another to know nothing about it. We were able to contact and interview four online users who had accidentally shared their inboxes and to talk to them about their configurations. These users indicated that they had either changed the settings to make files easier to find (1 of 4) or that they shared a computer with other people who may have changed the settings without their knowledge (3 of 4). For this reason, and given the difficulties with any other realistic trial described earlier, we decided to make a very simple test to simulate this scenario. We would share all files on the hard drive (as if we were a user sharing the same machine) and see how easy it was for participants to determine what, if anything, was being shared.

Our study consisted of 12 participants: 10 of these had used file sharing applications (such as Morpheus, Gnutella, KaZaA, and Napster) before, and two had not. All the users spent more than 10 hours a week on their computers.

Parts of the Study

The two parts of the study are described in the following sections.

KaZaA sharing comprehension questions

We were interested in the participants' conceptions of the types of files that peer-to-peer file sharing applications can share, as well as whether they were able to perform the specified task. We asked the participants to indicate what types of files could be shared over peer-to-peer networks.

Current sharing settings discovery task

The participants were initially presented with the KaZaA interface, which had been preconfigured in the following manner:

Conclusion

Acknowledgments

Thanks to Mary Czerwinski, Paul Dourish, Marti Hearst, and the IDL group at HP Labs for helping with early drafts of this document. Special thanks to Victoria Bellotti and Marti Hearst for help with the final draft, and to the University of Minnesota Office of Information Management for allowing us to use their computers to run our study.

About the Authors

Chapter Thirty Four. Why Johnny Can't Encrypt

A Usability Evaluation of PGP 5.0 Alma Whitten and J. D. Tygar

USER ERRORS CAUSE OR CONTRIBUTE TO MOST COMPUTER SECURITY FAILURES, yet user interfaces for security still tend to be clumsy, confusing, or near nonexistent. Is this simply because of a failure to apply standard user interface design techniques to security? We argue that, on the contrary, effective security requires a different usability standard, and that it will not be achieved through the user interface design techniques appropriate to other types of consumer software.[1]

To test this hypothesis, we performed a case study of a security program that does have a good user interface by general standards: PGP 5.0. Our case study used a cognitive walkthrough analysis together with a laboratory user test to evaluate whether PGP 5.0 can be used successfully by cryptography novices to achieve effective electronic mail security. The analysis found a number of user interface design flaws that may contribute to security failures, and the user test demonstrated that when our test participants were given 90 minutes in which to sign and encrypt a message using PGP 5.0, the majority of them were unable to do so successfully.

We conclude that PGP 5.0 is not usable enough to provide effective security for most computer users, despite its attractive graphical user interface, supporting our hypothesis that user interface design for effective security remains an open problem. We close with a brief description of our continuing work on the development and application of user interface design principles and techniques for security.

Introduction

Security mechanisms are effective only when used correctly. Strong cryptography, provably correct protocols, and bug-free code will not provide security if the people who use the software forget to click on the Encrypt button when they need privacy, give up on a communication protocol because they are too confused about which cryptographic keys they need to use, or accidentally configure their access control mechanisms to make their private data world readable. Problems such as these are already quite serious: at least one researcher, Matt Bishop,[2] has claimed that configuration errors are the probable cause of more than 90% of all computer security failures. Because average citizens are now increasingly encouraged to make use of networked computers for private transactions, the need to make security manageable for even untrained users has become critical.[3] ,

Understanding the Problem

Before describing the user test, we provide a specific definition of usability for security, identify the key properties of security as a problem domain for user interface design, and define a usability standard for PGP.

Defining Usability for Security

Usability necessarily has different meanings in different contexts. For some, efficiency may be a priority; for others, learnability; for still others, flexibility. In a security context, our priorities must be whatever is needed in order for the security to be used effectively. We capture that set of priorities in the following definition:

Definition: Security software is usable if the people who are expected to use it:

Are reliably made aware of the security tasks they need to perform

Are able to figure out how to successfully perform those tasks

Don't make dangerous errors

Are sufficiently comfortable with the interface to continue using it

Evaluation Methods

We chose to evaluate PGP's usability through two methods: an informal cognitive walkthrough[12] in which we reviewed PGP's user interface directly and noted aspects of its design that failed to meet the usability standard described in the preceding section, and a user test[13] performed in a laboratory with test participants selected to be reasonably representative of the general population of email users. The strengths and weaknesses inherent in each of the two methods made them useful in quite different ways, and it was more realistic for us to view them as complementary evaluation strategies[14] than to attempt to use the laboratory test to directly verify the points raised by the cognitive walkthrough.

Cognitive walkthrough

Cognitive Walkthrough

Because this chapter is intended for a security audience and is subject to space limitations, we present the results of our cognitive walkthrough in summary form, focusing on the points that are most relevant to security risks.

Visual Metaphors

The metaphor of keys is built into cryptologic terminology, and PGP's user interface relies heavily on graphical depictions of keys and locks. The PGPtools display, shown in Figure 34-1, offers four buttons to the user, representing four operations: Encrypt, Sign, Encrypt & Sign, and Decrypt/Verify, plus a fifth button for invoking the PGPkeys application. The graphical labels on these buttons indicate the encryption operation with an icon of a sealed envelope that has a metal loop on top to make it look like a closed padlock and, for the decryption operation, an icon of an open envelope with a key inserted at the bottom. Even for a novice user, these appear to be straightforward visual metaphors that help make the use of keys to encrypt and decrypt an intuitive concept.

Figure 34-1. PGPtools display

Still more helpful, however, would be an extension of the metaphor to distinguish between public keys for encryption and private keys for decryption; normal locks use the same key to lock and unlock, and the key metaphor will lead people to expect the same for encryption and decryption if it is not visually clarified in some way. Faulty intuition in this case may lead them to assume that they can always decrypt anything they have encrypted, an assumption that may have upsetting consequences. Different icons for public and private keys, perhaps drawn to indicate that they fit together like puzzle pieces, might be an improvement.

Signatures are another metaphor built into cryptologic terminology, but the icon of the blue quill pen that is used to indicate signing is problematic. People who are not familiar with cryptography probably know that quills are used for signing, and will recognize that the picture indicates the signature operation, but what they also need to understand is that they are using their private keys to generate signatures. The quill pen icon, which has nothing keylike about it, will not help them understand this and may even lead them to think that, along with the key objects that they use to encrypt, they also have quill pen objects that they use to sign. Quill pen icons encountered elsewhere in the program may be taken to be those objects, rather than the signatures that they are actually intended to represent. A better icon design might keep the quill pen to represent signing, but modify it to show a private key as the nib of the pen, and use some entirely different icon for signatures, perhaps something that looks more like a bit of inked handwriting and incorporates a keyhole shape.

Signature verification is not represented visually, which is a shame because it would be easy for people to overlook it altogether. The single button for Decrypt/Verify, labeled with an icon that only evokes decryption, could easily lead people to think that "verify" just means "verify that the decryption occurred correctly." Perhaps an icon that showed a private key unlocking the envelope and a public key unlocking the signature inside could suggest a much more accurate model to the user, while still remaining simple enough to serve as a button label.

Different Key Types

Originally, PGP used the popular RSA algorithm for encryption and signing. PGP 5.0 uses the Diffie-Hellman/DSS algorithms . The RSA and Diffie-Hellman/DSS algorithms use correspondingly different types of keys. The makers of PGP would prefer to see all the users of their software switch to use of Diffie-Hellman/DSS, but have designed PGP 5.0 to be backward compatible and to handle existing RSA keys when necessary. The lack of forward compatibility, however, can be a problem: if a file is encrypted for several recipients, some of whom have RSA keys and some of whom have Diffie-Hellman/DSS keys, the recipients who have RSA keys will not be able to decrypt it unless they have upgraded to PGP 5.0; similarly, those recipients will not be able to verify signatures created with Diffie-Hellman/DSS without a software upgrade.

User Test

This section describes the purpose, design, and results of the user test.

Purpose

Our user test was designed to evaluate whether PGP 5.0 meets the specific usability standard described in the section "A Usability Standard for PGP." We gave our participants a test scenario that was both plausible and appropriately motivating, and then avoided interfering with their attempts to carry out the security tasks that we gave them.

Description

This section outlines the characteristics of the test design and participants.

Test design

Our test scenario was that the participant had volunteered to help with a political campaign and had been given the job of campaign coordinator (the party affiliation and campaign issues were left to the participant's imagination, so as not to offend anyone). The participant's task was to send out campaign plan updates to the other members of the campaign team by email, using PGP for privacy and authentication. Because volunteering for a political campaign presumably implies a personal investment in the campaign's success, we hoped that the participants would be appropriately motivated to protect the secrecy of their messages.

Because PGP does not handle email itself, it was necessary to provide the participants with an email handling program to use. We chose to give them Eudora, as that would allow us to also evaluate the success of the Eudora plug-in that is included with PGP. Because we were not interested in testing the usability of Eudora (aside from the PGP plug-in), we gave the participants a brief Eudora tutorial before starting the test, and intervened with assistance during the test if a participant got stuck on something that had nothing to do with PGP.

After briefing the participants on the test scenario and tutoring them on the use of Eudora, we gave them an initial task description, which provided them with a secret message (a proposed itinerary for the candidate), the names and email addresses of the campaign manager and four other campaign team members, and a request to please send the secret message to the five team members in a signed and encrypted email. In order to complete this task, a participant had to generate a key pair, get the team members' public keys, make their own public key available to the team members, type the (short) secret message into an email, sign the email using their private key, encrypt the email using the five team members' public keys, and send the result. In addition, we designed the test so that one of the team members had an RSA key while the others all had Diffie-Hellman/DSS keys; thus, if a participant encrypted one copy of the message for all five team members (which was the expected interpretation of the task), they would encounter the mixed key types warning message. Participants were told that after accomplishing that initial task, they should wait to receive email from the campaign team members and follow any instructions they gave.

Each of the five campaign team members was represented by a dummy email account and a key pair: these were accessible to the test monitor through a networked laptop. The campaign manager's private key was used to sign each of the team members' public keys, including her own, and all five of the signed public keys were placed on the default key server at MIT so that they could be retrieved by participant requests.

Under certain circumstances, the test monitor posed as a member of the campaign team and sent email to the participant from the appropriate dummy account. These circumstances were:

The participant sent email to that team member asking a question about how to do something. In that case, the test monitor sent the minimally informative reply consistent with the test scenario (i.e., the minimal answer that wouldn't make that team member seem hostile or ignorant beyond the bounds of plausibility).[17]

Conclusion

This section summarizes the conclusions of our case study.

Failure of Standard Interface Design

The results seen in our case study support our hypothesis that the standard model of user interface design, represented here by PGP 5.0, is not sufficient to make computer security usable for people who are not already knowledgeable in that area. Our 12 test participants were generally educated and experienced at using email, yet only one-third of them were able to use PGP 5.0 to correctly sign and encrypt an email message when given 90 minutes in which to do so. Furthermore, one-quarter of them accidentally exposed the secret they were meant to protect in the process, by sending it in email they thought they had encrypted but had not.

In the earlier section "Defining Usability for Security

Related Work

We have found very little published research to date on the problem of usability for security. Of what does exist, the most prominent example is the Adage project ,[21] , [22] which is described as a system designed to handle authorization policies for distributed applications and groups. Usability was a major design goal in Adage, but it is intended for use by professional system administrators who already possess a high level of expertise, and as such it does not address the problems posed in making security effectively usable by a more general population. Work has also been done on the related issue of usability for safety-critical systems,[23] like those that control aircraft or manufacturing plants, but we may hope that unlike the users of personal computer security, users of those systems will be carefully selected and trained.

Ross Anderson discusses the effects of user noncompliance on security,[24]

Acknowledgments

We thank Robert Kraut for helpful advice on the design of our user test. This work was supported in part by the National Science Foundation and the United States Postal Service.

The contents of this publication are solely the responsibility of the authors.