In-home sensing and inference systems impose privacy risks and social tensions, which can be substantial barriers for the wide adoption of these systems. To understand what might affect people's perceptions and acceptance of in-home sensing and inference systems, we conducted an empirical study with 22 participants from 11 households. The study included in-lab activities, four weeks using sensor proxies in situ, and exit interviews. We report on participants' perceived benefits and concerns of in-home sensing applications and the observed changes of their perceptions throughout the study. We also report on tensions amongst stakeholders around the adoption and use of such systems. We conclude with a discussion on how the ubicomp design space might be sensitized to people's perceived concerns and tensions regarding sensing and inference in the home.

We present HeatWave, a system that uses digital thermal imaging cameras to detect, track, and support user interaction on arbitrary surfaces. Thermal sensing has had limited examination in the HCI research community and is generally under-explored outside of law enforcement and energy auditing applications. We examine the role of thermal imaging as a new sensing solution for enhancing user surface interaction. In particular, we demonstrate how thermal imaging in combination with existing computer vision techniques can make segmentation and detection of routine interaction techniques possible in real-time, and can be used to complement or simplify algorithms for traditional RGB and depth cameras. Example interactions include (1) distinguishing hovering above a surface from touch events, (2) shape-based gestures similar to ink strokes, (3) pressure based gestures, and (4) multi-finger gestures. We close by discussing the practicality of thermal sensing for naturalistic user interaction and opportunities for future work.

As advances in technology accelerate, sensors and recording devices are increasingly being integrated into homes. Although the added benefit of sensing is often clear (e.g., entertainment, security, encouraging sustainable behaviors, etc.), the home is a private and intimate place, with multiple stakeholders who may have competing priorities and tolerances for what is acceptable and useful. In an effort to develop systems that account for the needs and concerns of householders, we conducted an anonymous survey (N = 475) focusing on the activities and habits that people do at home that they would not want to be recorded. In this paper, we discuss those activities and where in the home they are performed, and offer suggestions for the design of UbiComp systems that rely on sensing and recording.

Improvements in cameras, computer vision, and machine learning are enabling real-time object recognition in interactive systems. Reliable recognition of uninstrumented objects opens up exciting new scenarios using the real-world objects that surround us. At the same time, it introduces the need to understand and manage the uncertainty and ambiguities that are inherent to such sensing. This paper examines this problem in the context of LEGO OASIS, a camera and projector-based system that recognizes LEGO toys and augments them with projected digital content. We focus on an interaction language to model the creation and manipulation of relationships between physical objects and their digital capabilities. We use this set of abstractions to examine different notions of recognition errors and explore interactive approaches to overcoming fundamental challenges in interactive object-aware systems.

We introduce Whack Gestures, an inexact and inattentive interaction technique. This approach seeks to provide a simple means to interact with devices with minimal attention from the user -- in particular, without the use of fine motor skills or detailed visual attention (requirements found in nearly all conventional interaction techniques). For mobile devices, this could enable interaction without "getting it out," grasping, or even glancing at the device. This class of techniques is suitable for a small number of simple but common interactions that could be carried out in an extremely lightweight fashion without disrupting other activities. With Whack Gestures, users can interact by striking a device with the open palm or heel of the hand. We briefly discuss the development and use of a preliminary version of this technique and show that implementations with high accuracy and a low false positive rate are feasible.

The greatest contributor of CO2 emissions in the average American household is personal transportation. Because transportation is inherently a mobile activity, mobile devices are well suited to sense and provide feedback about these activities. In this paper, we explore the use of personal ambient displays on mobile phones to give users feedback about sensed and self-reported transportation behaviors. We first present results from a set of formative studies exploring our respondents' existing transportation routines, willingness to engage in and maintain green transportation behavior, and reactions to early mobile phone "green" application design concepts. We then describe the results of a 3-week field study (N=13) of the UbiGreen Transportation Display prototype, a mobile phone application that semi-automatically senses and reveals information about transportation behavior. Our contributions include a working system for semi-automatically tracking transit activity, a visual design capable of engaging users in the goal of increasing green transportation, and the results of our studies, which have implications for the design of future green applications.

We present Bonfire, a self-contained mobile computing system that uses two laptop-mounted laser micro-projectors to project an interactive display space to either side of a laptop keyboard. Coupled with each micro-projector is a camera to enable hand gesture tracking, object recognition, and information transfer within the projected space. Thus, Bonfire is neither a pure laptop system nor a pure tabletop system, but an integration of the two into one new nomadic computing platform. This integration (1) enables observing the periphery and responding appropriately, e.g., to the casual placement of objects within its field of view, (2) enables integration between physical and digital objects via computer vision, (3) provides a horizontal surface in tandem with the usual vertical laptop display, allowing direct pointing and gestures, and (4) enlarges the input/output space to enrich existing applications. We describe Bonfire's architecture, and offer scenarios that highlight Bonfire's advantages. We also include lessons learned and insights for further development and use.

Recent advances in small inexpensive sensors, low-power processing, and activity modeling have enabled applications that use on-body sensing and machine learning to infer people's activities throughout everyday life. To address the growing rate of sedentary lifestyles, we have developed a system, UbiFit Garden, which uses these technologies and a personal, mobile display to encourage physical activity. We conducted a 3-week field trial in which 12 participants used the system and report findings focusing on their experiences with the sensing and activity inference. We discuss key implications for systems that use on-body sensing and activity inference to encourage physical activity.

As statistical machine learning algorithms and techniques continue to mature, many researchers and developers see statistical machine learning not only as a topic of expert study, but also as a tool for software development. Extensive prior work has studied software development, but little prior work has studied software developers applying statistical machine learning. This paper presents interviews of eleven researchers experienced in applying statistical machine learning algorithms and techniques to human-computer interaction problems, as well as a study of ten participants working during a five-hour study to apply statistical machine learning algorithms and techniques to a realistic problem. We distill three related categories of difficulties that arise in applying statistical machine learning as a tool for software development: (1) difficulty pursuing statistical machine learning as an iterative and exploratory process, (2) difficulty understanding relationships between data and the behavior of statistical machine learning algorithms, and (3) difficulty evaluating the performance of statistical machine learning algorithms and techniques in the context of applications. This paper provides important new insight into these difficulties and the need for development tools that better support the application of statistical machine learning.

Users’ ability to accurately recall frequent, habitual activities is fundamental to a number of disciplines, from health sciences to machine learning. However, few, if any, studies exist that have assessed optimal sampling strategies for in situ self-reports. In addition, few technologies exist that facilitate benchmarking self-report accuracy for routine activities. We report on a study investigating the effect of sampling frequency of self-reports of two routine activities (sitting and walking) on recall accuracy and annoyance. We used a novel wearable sensor platform that runs a real time activity inference engine to collect in situ ground truth. Our results suggest that a sampling frequency of five to eight times per day may yield an optimal balance of recall and annoyance. Additionally, requesting self-reports at regular, predetermined times increases accuracy while minimizing perceived annoyance since it allows participants to anticipate these requests. We discuss our results and their implications for future studies.

This paper presents MyExperience, a system for capturing both objective and subjective in situ data on mobile computing activities. MyExperience combines the following two techniques: 1) passive logging of device usage, user context, and environmental sensor readings, and 2) active context-triggered user experience sampling to collect in situ, subjective user feedback. MyExperience currently runs on mobile phones and supports logging of more than 140 event types, including: 1) device usage such as communication, application usage, and media capture, 2) user context such as calendar appointments, and 3) environmental sensing such as Bluetooth and GPS. In addition, user experience sampling can be targeted to moments of interest by triggering off sensor readings. We present several case studies of field deployments on people's personal phones to demonstrate how MyExperience can be used effectively to understand how people use and experience mobile technology.

To evaluate ubiquitous computing technologies, which may be embedded in the environment, embedded in objects, worn, or carried by the user throughout everyday life, it is essential to use methods that accommodate the often unpredictable, real-world environments in which the technologies are used. This article discusses how we have adapted and applied traditional methods from psychology and human-computer interaction, such as Wizard of Oz and Experience Sampling, to be more amenable to the in situ evaluations of ubiquitous computing applications, particularly in the early stages of design. The way that ubiquitous computing technologies can facilitate the in situ collection of self-report data is also discussed. Although the focus is on ubiquitous computing applications and tools for their assessment, it is believed that the in situ evaluation tools that are proposed will be generally useful for field trials of other technology, applications, or formative studies that are concerned with collecting data in situ.

Opportunistic augmentation denotes connecting a personal mobile device to another device to gain a transient advantage for the user. For example, a mobile phone user might borrow a large display and keyboard from a desktop personal computer. This uniquely ubiquitous computing activity requires effective device and service discovery as well as appropriate media usable across two or more devices. In this paper we show how session initiation protocol (SIP), the call signaling protocol for voice over IP, effectively separates discovery from media-rendering selection in opportunistic augmentation. This separation improves system flexibility while allowing users or system administrators to choose the most appropriate discovery technologies for the environment. We also describe two phone-centric discovery mechanisms and demonstrate the practicality of the system by implementation and use in a test environment.

This paper reports on three ethnographic studies of how people coordinate their activities in various work settings. The findings reported here are a derived set of relationships reflecting the nature of involvement of people in their activities. These findings were then tested by six analysts, who were conducting field studies of patterns of complex business activities. They used the derived relationships in the analysis of their data and in the representation of activity patterns. These usage cases revealed confusion between involvement relationships and job roles. Finally, several implications of these studies for designing an activity management prototype are presented.

The role of computers in the modern office has divided our activities between virtual interactions in the realm of the computer and physical interactions with real objects within the traditional office infrastructure. This paper extends previous work that has attempted to bridge this gap, to connect physical objects with virtual representations or computational functionality, via various types of tags. We discuss a variety of scenarios we have implemented using a novel combination of inexpensive, unobtrusive and easy to use RFID tags, tag readers, portable computers and wireless networking. This novel combination demonstrates the utility of invisibly, seamlessly and portably linking physical objects to networked electronic services and actions that are naturally associated with their form.

This paper describes a novel physical icon [3] (“phicon”) based system that can be programmed to issue a range of commands about what the user wishes to do with handdrawn whiteboard content. Through the phicon's UI, a command to process whiteboard context is issued using infrared signaling in combination with image processing and a ceiling-mounted camera system. We leverage camera systems that are already used for capturing whiteboard content [4] by further augmenting these systems to detect the presence and location of IR beacons within an image. An HDLC-based protocol and a built-in IR transmitter are used to send these signals.

A Collaborage is a collaborative collage of physically represented information on a surface that is connected with electronic information, such as a physical In/Out board connected to a people-locator database. The physical surface (board) contains items that are tracked by camera and computer vision technology. Events on the board trigger electronic services. This paper motivates this concept, presents three different applications, describes the system architecture and component technologies, and discusses several design issues.

Article A diary study of work-related reading: design implications for digital reading devices Share on Authors: Annette Adler Xerox Palo Alto Research Center, 3333 Coyote Hill Rd., Palo Alto, CA Xerox Palo Alto Research Center, 3333 Coyote Hill Rd., Palo Alto, CAView Profile , Anuj Gujar Xerox Palo Alto Research Center, 3333 Coyote Hill Rd., Palo Alto, CA Xerox Palo Alto Research Center, 3333 Coyote Hill Rd., Palo Alto, CAView Profile , Beverly L. Harrison Xerox Palo Alto Research Center, 3333 Coyote Hill Rd., Palo Alto, CA Xerox Palo Alto Research Center, 3333 Coyote Hill Rd., Palo Alto, CAView Profile , Kenton O'Hara Xerox Research Centre Europe, (formerly Rank Xerox Research Centre) 61 Regent St., Cambridge, CB2 1AB, UK Xerox Research Centre Europe, (formerly Rank Xerox Research Centre) 61 Regent St., Cambridge, CB2 1AB, UKView Profile , Abigail Sellen Xerox Research Centre Europe, (formerly Rank Xerox Research Centre) 61 Regent St., Cambridge, CB2 1AB, UK Xerox Research Centre Europe, (formerly Rank Xerox Research Centre) 61 Regent St., Cambridge, CB2 1AB, UKView Profile Authors Info & Claims CHI '98: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsJanuary 1998 Pages 241–248https://doi.org/10.1145/274644.274679Published:01 January 1998 147citation2,498DownloadsMetricsTotal Citations147Total Downloads2,498Last 12 Months123Last 6 weeks12 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

This paper reports on the design and use of tactile user interfaces embedded within or wrapped around the devices that they control. We discuss three different interaction prototypes that we built These interfaces were embedded onto two handheld devices of dramatically different form factors. We describe the design and implementation challenges, and user feedback and reactions to these prototypes. Implications for future design in the area of manipulative or haptic user interfaces are highlighted.

Article Free Access Share on An experimental evaluation of transparent menu usage Authors: Beverly L. Harrison Dept. of Industrial Engineering and University of Toronto, Toronto, Ontario, Canada, M5S 3G9 and Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada, M5C 1P1 Dept. of Industrial Engineering and University of Toronto, Toronto, Ontario, Canada, M5S 3G9 and Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada, M5C 1P1View Profile , Kim J. Vicente Dept. of Industrial Engineering, University of Toronto, Toronto, Ontario, Canada, M5S 3G9 Dept. of Industrial Engineering, University of Toronto, Toronto, Ontario, Canada, M5S 3G9View Profile Authors Info & Claims CHI '96: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsApril 1996 Pages 391–398https://doi.org/10.1145/238386.238583Published:13 April 1996Publication History 33citation1,125DownloadsMetricsTotal Citations33Total Downloads1,125Last 12 Months52Last 6 weeks6 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF

Article Transparent layered user interfaces: an evaluation of a display design to enhance focused and divided attention Share on Authors: Beverly L. Harrison Dept. of Industrial Engineering, University of Toronto, Toronto, Ontario, Canada, M5S 1A4 and Alias Research Ltd., 110 Richmond St. East, Toronto, Ontario, Canada, M5C 1P1 Dept. of Industrial Engineering, University of Toronto, Toronto, Ontario, Canada, M5S 1A4 and Alias Research Ltd., 110 Richmond St. East, Toronto, Ontario, Canada, M5C 1P1View Profile , Hiroshi Ishii NTT Human Interface Lab, 1-2356 Take, Yokosuka-Shi, Kanagawa, 238-03 Japan NTT Human Interface Lab, 1-2356 Take, Yokosuka-Shi, Kanagawa, 238-03 JapanView Profile , Kim J. Vicente Dept. of Industrial Engineering, University of Toronto, Toronto, Ontario, Canada, M5S 1A4 Dept. of Industrial Engineering, University of Toronto, Toronto, Ontario, Canada, M5S 1A4View Profile , William A. S. Buxton Alias Research Ltd., 110 Richmond St. East, Toronto, Ontario, Canada, M5C 1P1 and Dept. of Computer Science, University of Toronto, Toronto, Ontario, Canada, M5S 1A4 Alias Research Ltd., 110 Richmond St. East, Toronto, Ontario, Canada, M5C 1P1 and Dept. of Computer Science, University of Toronto, Toronto, Ontario, Canada, M5S 1A4View Profile Authors Info & Claims CHI '95: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsMay 1995 Pages 317–324https://doi.org/10.1145/223904.223945Online:01 May 1995Publication History 83citation2,496DownloadsMetricsTotal Citations83Total Downloads2,496Last 12 Months100Last 6 weeks9 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

The central research issue addressed by this paper is how we can design computer interfaces that better support human attention and better maintain the fluency of work.To accomplish this we propose to use semi-transparent user interface objects.This paper reports on an experimental evaluation which provides both valuable insights into design parameters and suggests a systematic evaluation methodology.For this study, we used a variablytransparent tool palette superimposed over different background content, combining text, wire-frame or line art images, and solid images.The experiment explores the issue of focused attention and interference, by varying both visual distinctiveness and levels of transparency.