Methods for automated facial expression recognition - identifying faces as happy, sad, angry, etc. - typically rely on the classification of features extracted from images. These features, designed to encode shape and texture information, depend on both (1) the expression an individual is making, and (2) the individual's physical characteristics and lighting conditions of the image. To reduce the effect of (2), a common strategy is to establish a "baseline" for an individual and subtract out this individual's baseline neutral feature. This extra neutral feature information often is not available - in particular for in-the-wild, real-time classification of a previously unseen subject. Thus, in order to implement "neutral subtraction," one must estimate the individual's neutral feature. Existing methods to do this are susceptible to class imbalance at test time (e.g., averaging over all facial features), require a more complex model specific to the individual to be trained, or are restricted to features computed entirely from tracked landmark points (taking advantage of a subset of "stable points" which move little as an individual emotes). We extend neutral subtraction to different computer vision feature spaces as a method to correct for inter-face and lighting variance. We further propose a simple, real-time method which is robust to class imbalance and in principal works over a wide class of feature choices. We test this method on feature extraction techniques that lead to high baseline accuracy without neutral subtraction (97% on the Extended Cohn-Kanade Dataset). We find that on difficult classification tasks our method recovers almost 2/3 of the ~ 8% gain shown by a "cheating" neutral-subtracted feature classifier, which uses examples that have been labeled as neutral, validating with both HOG and SIFT features.

A diversity of user goals and strategies make creation-oriented applications such as word processors or photo-editors difficult to comprehensively test. Evaluating such applications requires testing a large pool of participants to capture the diversity of experience, but traditional usability testing can be prohibitively expensive. To address this problem, this article contributes a new usability evaluation method called backtracking analysis, designed to automate the process of detecting and characterizing usability problems in creation-oriented applications. The key insight is that interaction breakdowns in creation-oriented applications often manifest themselves in backtracking operations that can be automatically logged (e.g., undo and erase operations). Backtracking analysis synchronizes these events to contextual data such as screen capture video, helping the evaluator to characterize specific usability problems. The results from three experiments demonstrate that backtracking events can be effective indicators of usability problems in creation-oriented applications, and can yield a cost-effective alternative to traditional laboratory usability testing.

One approach to reducing the costs of usability testing is to facilitate the automatic detection of critical incidents: serious breakdowns in interaction that stand out during software use. This research evaluates the use of undo and erase events as indicators of critical incidents in Google SketchUp (a 3D-modeling application), measuring an indicator's usefulness by the numbers and types of usability problems discovered. We compared problems identified using undo and erase events to problems identified using the user-reported critical incident technique [Hartson and Castillo 1998]. In a within-subjects experiment with 35 participants, undo and erase episodes together revealed over 90% of the problems rated as severe, several of which would not have been discovered by self-report alone. Moreover, problems found by all three methods were rated as significantly more severe than those identified by only a subset of methods. These results suggest that undo and erase events will serve as useful complements to user-reported critical incidents for low cost usability evaluation of creation-oriented applications like SketchUp.

Using gaze information as a form of input poses challenges based on the nature of eye movements and how we humans use our eyes in conjunction with other motor actions. In this paper, we present three techniques for improving the use of gaze as a form of input. We first present a saccade detection and smoothing algorithm that works on real-time streaming gaze information. We then present a study which explores some of the timing issues of using gaze in conjunction with a trigger (key press or other motor action) and propose a solution for resolving these issues. Finally, we present the concept of Focus Points, which makes it easier for users to focus their gaze when using gaze-based interaction techniques. Though these techniques were developed for improving the performance of gaze-based pointing, their use is applicable in general to using gaze as a practical form of input.

Window management research has aimed to leverage users' tasks to organize the growing number of open windows in a useful manner. This research has largely assumed task classifications to be binary -- either a window is in a task, or not -- and context-independent. We suggest that the continual evolution of tasks can invalidate this approach and instead propose a fuzzy association model in which windows are related to one another by varying degrees. Task groupings are an emergent property of our approach. To support the association model, we introduce the WindowRank algorithm and its use in determining window association. We then describe Taskposé, a prototype window switch visualization embodying these ideas, and report on a week-long user study of the system.

We present a practical technique for pointing and selection using a combination of eye gaze and keyboard triggers. EyePoint uses a two-step progressive refinement process fluidly stitched together in a look-press-look-release action, which makes it possible to compensate for the accuracy limitations of the current state-of-the-art eye gaze trackers. While research in gaze-based pointing has traditionally focused on disabled users, EyePoint makes gaze-based pointing effective and simple enough for even able-bodied users to use for their everyday computing tasks. As the cost of eye gaze tracking devices decreases, it will become possible for such gaze-based techniques to be used as a viable alternative for users who choose not to use a mouse depending on their abilities, tasks and preferences.

This paper describes a communication-minded visualization called progressive multiples that supports both the forensic analysis and presentation of multidimensional event data. We combine ideas from progressive disclosure, which reveals data to the user on demand, and small multiples [21], which allows users to compare many images at once. Sets of events are visualized as timelines. Events are placed in temporal order on the x-axis, and a scalar dimension of the data is mapped to the y-axis. To support forensic analysis, users can pivot from an event in an existing timeline to create a new timeline of related events. The timelines serve as an exploration history, which has two benefits. First, this exploration history allows users to backtrack and explore multiple paths. Second, once a user has concluded an analysis, these timelines serve as the raw visual material for composing a story about the analysis. A narrative that conveys the analytical result can be created for a third party by copying and reordering timelines from the history. Our work is motivated by working with network security administrators and researchers in political communication. We describe a prototype that we are deploying with administrators and the results of a user study where we applied our technique to the visualization of a simulated epidemic.

Shoulder-surfing -- using direct observation techniques, such as looking over someone's shoulder, to get passwords, PINs and other sensitive personal information -- is a problem that has been difficult to overcome. When a user enters information using a keyboard, mouse, touch screen or any traditional input device, a malicious observer may be able to acquire the user's password credentials. We present EyePassword, a system that mitigates the issues of shoulder surfing via a novel approach to user input.

Scrolling is an essential part of our everyday computing experience. Contemporary scrolling techniques rely on the explicit initiation of scrolling by the user. The act of scrolling is tightly coupled with the user?s ability to absorb information via the visual channel. The use of eye gaze information is therefore a natural choice for enhancing scrolling techniques. We present several gaze-enhanced scrolling techniques for manual and automatic scrolling which use gaze information as a primary input or as an augmented input. We also introduce the use off-screen gaze-actuated buttons for document navigation and control.

Cameras are a useful source of input for many interactive applications, but computer vision programming is difficult and requires specialized knowledge that is out of reach for many HCI practitioners. In an effort to learn what makes a useful computer vision design tool, we created Eyepatch, a tool for designing camera-based interactions, and evaluated the Eyepatch prototype through deployment to students in an HCI course. This paper describes the lessons we learned about making computer vision more accessible, while retaining enough power and flexibility to be useful in a wide variety of interaction scenarios.

Multi-user, touch-sensing input devices create opportunities for the use of cooperative gestures -- multi-user gestural interactions for single display groupware. Cooperative gestures are interactions where the system interprets the gestures of more than one user as contributing to a single, combined command. Cooperative gestures can be used to enhance users' sense of teamwork, increase awareness of important system events, facilitate reachability and access control on large, shared displays, or add a unique touch to an entertainment-oriented activity. This paper discusses motivating scenarios for the use of cooperative gesturing and describes some initial experiences with CollabDraw, a system for collaborative art and photo manipulation. We identify design issues relevant to cooperative gesturing interfaces, and present a preliminary design framework. We conclude by identifying directions for future research on cooperative gesturing interaction techniques.

We explore how the placement of control widgets (such as menus) affects collaboration and usability for co-located tabletop groupware applications. We evaluated two design alternatives: a centralized set of controls shared by all users, and separate per-user controls replicated around the borders of the shared tabletop. We conducted this evaluation in the context of TeamTag, a system for collective annotation of digital photos. Our comparison of the two design alternatives found that users preferred replicated over shared controls. We discuss the cause of this preference, and also present data on the impact of these interface design variants on collaboration, as well as the role that orientation, co-touching, and the use of different regions of the table played in shaping users' behavior and preferences.

This paper presents a design case study of SIDES: Shared Interfaces to Develop Effective Social Skills. SIDES is a tool designed to help adolescents with Asperger's Syndrome practice effective group work skills using a four-player cooperative computer game that runs on tabletop technology. We present the design process and evaluation of SIDES conducted over six months with a middle school social group therapy class. Our findings indicate that cooperative tabletop computer games are a motivating and supportive tool for facilitating effective group work among our target population and reveal several design lessons to inform the development of similar systems.

Toolglass [Bier et al. 1993] demonstrated a two-handed command selection technique that combined command selection and direct manipulation. While empirical evaluations showed a speed advantage for ToolGlass, they did not examine the relative importance of two possible factors in its improved performance: (1) the use of two hands and (2) the merging of command selection and direct manipulation.We conducted a study comparing the relative benefits of three command selection techniques that merge command selection and direct manipulation: one two-handed technique, Toolglass, and two one-handed techniques, namely, control menus [Pook et al. 2000] and FlowMenu [Guimbretière and Winograd 2000]. Participants performed sequences of operations that required both selecting a color and designating the endpoints of a line. Our results show that control menus and FlowMenu are significantly faster than Toolglass. Further analysis suggests that the merging of command selection and direct manipulation is the most important factor in the performance of all three techniques.

We have built and tested WorkspaceNavigator, which supports knowledge capture and reuse for teams engaged in unstructured, dispersed, and prolonged collaborative design activity in a dedicated physical workspace. It provides a coherent unified interface for post-facto retrieval of multiple streams of data from the work environment, including overview snapshots of the workspace, screenshots of in-space computers, whiteboard images, and digital photos of physical objects. This paper describes the design of WorkspaceNavigator and identifies key considerations for knowledge capture tools for design workspaces, which differ from those of more structured meeting or classroom environments. Iterative field tests in workspace environments for student teams in two graduate Mechanical Engineering design courses helped to identify features that augment the work of both course participants and design researchers.

We introduce a system that allows four users to each receive sound from a private audio channel while using a shared tabletop display. In order to explore how private audio channels affect a collaborative work environment, we conducted a user study with this system. The results reveal differences in work strategies when groups are presented with individual versus public audio, and suggest that the use of private audio does not impede group communication and may positively impact group dynamics. We discuss the findings, as well as their implications for the design of future audio-based "single display privacyware" systems.

We describe the design of and experience with PointRight, a peer-to-peer pointer and keyboard redirection system that operates in multi-machine, multi-user environments. PointRight employs a geometric model for redirecting input across screens driven by multiple independent machines and operating systems. It was created for interactive workspaces that include large, shared displays and individual laptops, but is a general tool that supports many different configurations and modes of use. Although previous systems have provided for re-routing pointer and keyboard control, in this paper we present a more general and flexible system, along with an analysis of the types of re-binding that must be handled by any pointer redirection system This paper describes the system, the ways in which it has been used, and the lessons that have been learned from its use over the last two years.

We present a design and implementation for displaying and manipulating HTML pages on small handheld devices such as personal digital assistants (PDAs), or cellular phones. We introduce methods for summarizing parts of Web pages and HTML forms. Each Web page is broken into text units that can each be hidden, partially displayed, made fully visible, or summarized. A variety of methods are introduced that summarize the text units. In addition, HTML forms are also summarized by displaying just the text labels that prompt the use for input. We tested the relative performance of the summarization methods by asking human subjects to accomplish single-page information search tasks. We found that the combination of keywords and single-sentence summaries provides significant improvements in access times and number of required pen actions, as compared to other schemes. Our experiments also show that our algorithms can identify the appropriate labels for forms in 95% of the cases, allowing effective form support for small screens.

This paper describes new interaction techniques for direct pen-based interaction on the Interactive Mural, a large (6′x3.5′) high resolution (64 dpi) display. They have been tested in a digital brainstorming tool that has been used by groups of professional product designers. Our "interactive wall" metaphor for interaction has been guided by several goals: to support both free-hand sketching and high-resolution materials, such as images, 3D models and GUI application windows; to present a visual appearance that does not clutter the content with control devices; and to support fluid interaction, which minimizes the amount of attention demanded and interruption due to the mechanics of the interface. We have adapted and extended techniques that were developed for electronic whiteboards and generalized the use of the FlowMenu to execute a wide variety of actions in a single pen stroke, While these techniques were designed for a brainstorming tool, they are very general and can be used in a wide variety of application domains using interactive surfaces.

The development of context-aware applications will require tools that are based on clearly defined models of context and system software architecture. This essay introduces models for each of these, examines the tradeoffs among the different alternatives, and describes a blackboard-based context architecture that is being used in the construction of interactive workspaces.

We have designed and implemented new Web browsing facilities to support effective navigation on Personal Digital Assistants (PDAs) with limited capabilities: low bandwidth, small display, and slow CPU. The implementation supports wireless browsing from 3Com's Palm Pilot. An HTTP proxy fetches web pages on the client's behalf and dynamically generates summary views to be transmitted to the client. These summaries represent both the link structure and contents of a set of web pages, using information about link importance. We discuss the architecture, user interface facilities, and the results of comparative performance evaluations. We measured a 45% gain in browsing speed, and a 42% reduction in required pen movements.

We present a new kind of marking menu that was developed for use with a pen device on display surfaces such as large, high resolution, wall-mounted displays. It integrates capabilities of previously separate mechanisms such as marking menus and Quikwriting, and facilitates the entry of multiple commands. While using this menu, the pen never has to leave the active surface so that consecutive menu selections, data entry (text and parameters) and direct manipulation tasks can be integrated fluidly.

The growing interest in multimodal interface design is inspired in large part by the goals of supporting more transparent, flexible, efficient, and powerfully expressive means of human-computer interaction than in the past. Multimodal interfaces are expected to support a wider range of diverse applications, be usable by a broader spectrum of the average population, and function more reliably under realistic and challenging usage conditions. In this article, we summarize the emerging architectural approaches for interpreting speech and pen-based gestural input in a robust manner-including early and late fusion approaches, and the new hybrid symbolic-statistical approach. We also describe a diverse collection of state-of-the-art multimodal systems that process users' spoken and gestural input. These applications range from map-based and virtual reality systems for engaging in simulations and training, to field medic systems for mobile use in noisy environments, to web-based transactions and standard text-editing applications that will reshape daily computing and have a significant commercial impact. To realize successful multimodal systems of the future, many key research challenges remain to be addressed. Among these challenges are the development of cognitive theories to guide multimodal system design, and the development of effective natural language processing, dialogue processing, and error-handling techniques. In addition, new multimodal systems will be needed that can function more robustly and adaptively, and with support for collaborative multiperson use. Before this new class of systems can proliferate, toolkits also will be needed to promote software development for both simulated and functioning systems.

Article Free Access Share on Hi-cites: dynamically created citations with active highlighting Authors: Michelle Q. Wang Baldonado Xerox Palo Alto Research Center, 3333 Coyote Hill Road, Palo Alto, CA Xerox Palo Alto Research Center, 3333 Coyote Hill Road, Palo Alto, CAView Profile , Terry Winograd Gates 3B, Stanford University, Stanford, CA Gates 3B, Stanford University, Stanford, CAView Profile Authors Info & Claims CHI '98: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsJanuary 1998 Pages 408–415https://doi.org/10.1145/274644.274700Published:01 January 1998Publication History 6citation329DownloadsMetricsTotal Citations6Total Downloads329Last 12 Months10Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Article Free Access Share on SenseMaker: an information-exploration interface supporting the contextual evolution of a user's interests Authors: Michelle Q. Wang Baldonado Gates 3B, Computer Science Department, Stanford University, Stanford, CA Gates 3B, Computer Science Department, Stanford University, Stanford, CAView Profile , Terry Winograd Gates 3B, Computer Science Department, Stanford University, Stanford, CA Gates 3B, Computer Science Department, Stanford University, Stanford, CAView Profile Authors Info & Claims CHI '97: Proceedings of the ACM SIGCHI Conference on Human factors in computing systemsMarch 1997 Pages 11–18https://doi.org/10.1145/258549.258563Published:27 March 1997Publication History 95citation1,436DownloadsMetricsTotal Citations95Total Downloads1,436Last 12 Months57Last 6 weeks6 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

As part of the Stanford Digital Libraries Project, we have prototyped a novel architecture for security and access control in heterogeneous, networked environments. Conceptually, this architecture recasts security issues from an “information access” metaphor into a “relationship management” framewo rk and uniformly applies a contracting model. Architecturally, it introduces a “network-centric” design that generalizes previous models of client- or server-centered control into a third, relationship-based form.

We introduce a framework for access/action control which shifts the emphasis from the participants to their relationships. The framework is based on a communication model in which participants negotiate the mutually agreed-upon boundary conditions of their relationships, and create social reference points by encapsulating them in compact "communication pacts", called "commpacts". Commpacts are designed to provide a language enabling a social mechanism of coordinated expectation. We argue that in networked environments characterized by multiple authorities and "trusted proxies" this model can deal with the complexities of general (user- and content-dependent) distributed access/action control and provides a clear user-conceptual metaphor The framework embeds naturally into the existing legal and institutional infrastructure; it generalizes work in electronic contracting. Commpacts can be seen as a third fundamental type next to access-control lists (ACLs) and capabilities.

People keep pieces of information in diverse collections such as folders, hotlists, e-mail inboxes, newsgroups, and mailing lists. These collections mediate various types of collaborations including communicating, structuring, sharing information, and organizing people. Grassroots is a system that provides a uniform framework to support people's collaborative activities mediated by collections of information. The system seamlessly integrates functionalities currently found in such disparate systems as e-mail, newsgroups, shared hotlists, hierarchical indexes, hypermail, etc. Grassroots co-exists with these systems in that its users benefit from the uniform image provided by Grassroots, but other people can continue using other mechanisms, and Grassroots leverages from them. The current Grassroots prototype is based on an http-proxy implementation, and can be used with any Web browser. In the context of the design of a next-generation version of the Web, Grassroots demonstrates the utility of a uniform notification infrastructure.

This paper describes a system we have implemented that enables people to share structured in-place annotations attached to material in arbitrary documents on the WWW. The basic conceptual decisions are laid out, and a prototypical example of the client-server interaction is given. We then explain the usage perspective, describe our experience with using the system, and discuss other experimental usages of our prototype implementation, such as collaborative filtering, seals of approval, and value-added trails. We show how this is a specific instantiation of a more general “virtual document” architecture in which, with the help of light-weight distributed meta information, viewed documents can incorporate material that is dynamically integrated from multiple distributed sources. Development of that architecture is part of a larger project on Digital Libraries that we are engaged in.

This paper describes ActionWorkflowTM approach to workflow management technology: a design methodology and associated computer software for the support of work in organizations. The approach is based on theories of communicative activity as language faction and has been developed in a series of systems for coordination among users of networked computers. This paper describes the approach, gives an example of its application, and shows the architecture of a workflow management system based on it.

We describe the role of mentors in en experimental murse on human-computer interaction (HCI) taught in the Computer Science Department at Stanford University for the past two years.Students practice design within the course by collaborating in smell groups on 12-week projects, in which they analyze a work environment, design and implement a prototype user interface, and evtthtate the prototype with project clients.As part of our experiment in providing contact with the world of practical design, we invited people from local industry to serve es mentors for the stmdent groups.These unpaid volunteers have been important adjuncts in guiding students as they developed their HCI projects.We dkeuss: the background for the projects; the role of mentors in the learning process; what is required of mentors end what benefits they ge~ how to support the mentorirtg process; end what we have learned horn the experience so far.We consider mentors to be a critical pert of the design project within the course, end we welcome a sharing of experience with other people who may have developed similar courses where mentors played an important role.1.

This paper is the closing address for CHI '90. It addresses the problem of educating computer professionals in the area of human-computer interaction, arguing that standard approaches within computer science need to be augmented and that new models of education can aid us in producing students with broad competence in the design of computer systems for human use.

The goal of this paper is to relate theory to invention and application in the design of systems for organizational communication and management. We propose and illustrate a theory of design, technology, and action that we believe has been missing in the mainstream of work on office systems. At the center of our thinking is a theory of language as social action, which differs from the generally taken-for-granted understandings of what goes on in an organization. This approach has been presented elsewhere, and our aim here is to examine its practical implications and assess its effectiveness in the design of The Coordinator, a workgroup productivity system that is in widespread commercial use on personal computers.

In creating computer-based systems, we work within a perspective that shapes the design questions that will be asked and the kinds of solutions that are sought. This article introduces a perspective based on language as action, and explores its consequences for system design. We describe a communication tool called The Coordinator, which was designed from a language/action perspective; and we suggest how further aspects of coordinated work might be addressed in a similar style. The language/action perspective is illustrated with an example based on studies of nursing work in a hospital ward and contrasted to other currently prominent perspectives.

No abstract available.

In designing computer-based systems, we work within a perspective that shapes the design questions that will be asked and the kinds of solutions that are sought. This paper introduces a perspective based on language as action, and explores its consequences for practical system design. The language/action perspective is contrasted to a number of other currently prominent perspectives, and is illustrated with an extended example based on studies of nursing work in a hospital ward. We show how it leads to particular analyses of that work, which reveal potentials for creating new designs that can make the work (and the workers) more effective.