In this work, we advance research efforts in combining the casual sketching approach of whiteboards with the machine's computing power. We present SketchInsight, a system that applies the familiar and collaborative features of a whiteboard interface to the accurate data exploration capabilities of interactive visualizations. SketchInsight enables data analysis with more fluid interaction, allowing people to visually explore their data by drawing simple charts and directly manipulating them. In addition, we report results from a qualitative study conducted to evaluate user experience in exploring data with SketchInsight, expanding our understanding on how people use a pen- and touch-enabled digital whiteboard for data exploration. We also discuss the challenges in building a working system that supports data analytic capabilities with pen and touch interaction and freeform annotation.

Most location-aware mobile applications only make use of the user's current location, but there is an opportunity for them to infer the user's future locations. We present Trajectory-Aware Search (TAS), a mobile local search application that predicts the user's destination in real-time based on location data from the current trip and shows search results near the predicted location. TAS demonstrates the feasibility of destination prediction in an interactive mobile application. Our user study of TAS shows using predicted destinations to help select search results positively augments the local search experience.

We present AffectAura, an emotional prosthetic that allows users to reflect on their emotional states over long periods of time. We designed a multimodal sensor set-up for continuous logging of audio, visual, physiological and contextual data, a classification scheme for predicting user affective state and an interface for user reflection. The system continuously predicts a user's valence, arousal and engage-ment, and correlates this with information on events, communications and data interactions. We evaluate the interface through a user study consisting of six users and over 240 hours of data, and demonstrate the utility of such a reflection tool. We show that users could reason forward and backward in time about their emotional experiences using the interface, and found this useful.

The awareness that hospital patients have of the people and events surrounding their care has a dramatic impact on satisfaction and clinical outcomes. However, patients are often under-informed about even basic aspects of their care. In this work, we hypothesize that mobile devices - which are increasingly available to patients - can be used as real-time information conduits to improve patient awareness and consequently improve patient care. To better understand the unique affordances that mobile devices offer in the hospital setting, we provided twenty-five patients with mobile phones that presented a dynamic, interactive report on their progress, care plan, and care team throughout their emergency department stay. Through interviews with these patients, their visitors, and hospital staff, we explore the benefits and challenges of using the mobile phone as an information display, finding overall that this is a promising approach to improving patient awareness. Furthermore, we demonstrate that only a small number of technology challenges remain before such a system could be deployed without researcher intervention.

Computers today make it easy for people to scatter copies and versions of digital items across their file systems, but do little to help people manage the resulting mess. In this paper, we introduce the concept of a copy-aware computing ecosystem, inspired by a vision of computing when systems track and surface copy relationships between files. Based on two deployments of a copy-aware software prototype and in-depth interviews with individuals in collaborative relationships, we present our findings on the origins of copies and the barriers to eliminating them, but offer a promising solution based on the set of files that together represent a user's conceptual view of a document - the versionset. We show that the versionset is viable to infer, and we draw upon user activity logs and feedback on personalized views of versionsets to distill guidelines for the factors that define a versionset. We conclude by enumerating the many PIM user experiences that could be transformed as a result.

In this paper, we explore the use of a projection-based handheld device to facilitate in-clinic doctor-patient communication. We present the user-centered design process used to understand the workflow of medical professionals and to identify challenges they currently face in communicating information to patients. Based on the lessons learned, we developed AnatOnMe, a prototype projection-based hand-held system for enhancing information exchange in the current practice of one medical sub-specialty, physical therapy. We then present the results of a controlled experiment to understand the desirability and learning tradeoffs of using AnatOnMe to teach medical concepts on three potential projection surfaces - wall, model, and patient body. Finally, we present results of two expert reviews of the system.

People often search for local information (e.g., a restaurant, store, gas station, or attraction) from their mobile device. We show, via a survey of 929 mobile searchers at a large software company, that local searches tend to be highly contextual, influenced by geographic features, temporal aspects, and the searcher's social context. While location was reported to be very important, respondents looked for information about places close to their current location only 40% of the time. Instead, they were often in transit (68% of our searchers) and wanted information related to their destination (27% of searchers), en route to their destination (12%), or near their destination (12%). Additionally, 63% of our participants' mobile local searches took place within a social context and were discussed with someone else. We discuss these findings to present a picture of how location, time, and social context impact mobile local searches.

The impact of interruptions on workflow and productivity has been extensively studied in the PC domain, but while fragmented user attention is recognized as an inherent aspect of mobile phone usage, little formal evidence exists of its effect on mobile productivity. Using a survey and a screenshot-based diary study we investigated the types of barriers people face when performing tasks on their mobile phones, the ways they follow up with such suspended tasks, and how frustrating the experience of task disruption is for mobile users. From 386 situated samples provided by 12 iPhone and 12 Pocket PC users, we distill a classification of barriers to the completion of mobile tasks. Our data suggest that moving to a PC to complete a phone task is common, yet not inherently problematic, depending on the task. Finally, we relate our findings to prior design guidelines for desktop workflow, and discuss how the guidelines can be extended to mitigate disruptions to mobile taskflow.

We introduce activity-based navigation, which uses human activities derived from sensor data to help people navigate, in particular to retrace a "trail" previously taken by that person or another person. Such trails may include step counts, walking up/down stairs or taking elevators, compass directions, and photos taken along a user's path, in addition to absolute positioning (GPS and maps) when available. To explore the user experience of activity-based navigation, we built Greenfield, a mobile device interface for finding a car. We conducted a ten participant user study comparing users' ability to find cars across three different presentations of activity-based information as well as verbal instructions. Our results show that activity-based navigation can be used for car finding and suggest its promise more generally for supporting navigation tasks. We present lessons for future activity-based navigation interfaces, and motivate further work in this space, particularly in the area of robust activity inference.

Tag clouds have proliferated over the web over the last decade. They provide a visual summary of a collection of texts by visually depicting the tag frequency by font size. In use, tag clouds can evolve as the associated data source changes over time. Interesting discussions around tag clouds often include a series of tag clouds and consider how they evolve over time. However, since tag clouds do not explicitly represent trends or support comparisons, the cognitive demands placed on the person for perceiving trends in multiple tag clouds are high. In this paper, we introduce SparkClouds, which integrate sparklines into a tag cloud to convey trends between multiple tag clouds. We present results from a controlled study that compares SparkClouds with two traditional trend visualizations—multiple line graphs and stacked bar charts—as well as Parallel Tag Clouds. Results show that SparkClouds ability to show trends compares favourably to the alternative visualizations.

Mobile phones are becoming increasingly personalized in terms of the data they store and the types of services they provide. At the same time, field studies have reported that there are a variety of situations in which it is natural for people to share their phones with others. However, most mobile phones support a binary security model that offers all-or-nothing access to the phone. We interviewed 12 smartphone users to explore how security and data privacy concerns affected their willingness to share their mobile phones. The diversity of guest user categorizations and associated security constraints expressed by the participants suggests the need for a security model richer than today's binary model.

Supporting one-handed thumb operation of touchscreen-based mobile devices presents a challenging tradeoff between visual expressivity and ease of interaction. ThumbSpace and Shift---two new application-independent, software-based interaction techniques---address this tradeoff in significantly different ways. ThumbSpace addresses distant objects while Shift addresses small object occlusion. We present two extensive, comparative user studies. The first compares ThumbSpace and Shift to peripheral hardware (directional pad and scrollwheel) and direct touchscreen input for selecting objects while standing and walking. The data favored the Shift design overall, but suggested ThumbSpace is promising for distant objects. Our second study examines the benefits and learnability of combining Shift and ThumbSpace on a device with a larger screen (3.5"). We found their combined use offered users better overall speed and accuracy in hitting small targets (3.6 mm2) than using either method alone.

In this paper we describe a novel approach for searching large data sets from a mobile phone. Existing interfaces for mobile search require keyword text entry and are not suited for browsing. Our alternative uses a hybrid model to de-emphasize tedious keyword entry in favor of iterative data filtering. We propose navigation and selection of hierarchical metadata (facet navigation), with incremental text entry to further narrow the results. We conducted a formative evaluation to understand the relative advantages of keyword entry versus facet navigation for both browse and search tasks on the phone. We found keyword entry to be more powerful when the name of the search target is known, while facet navigation is otherwise more effective and strongly preferred.

This paper describes a two-phase study conducted to determine optimal target sizes for one-handed thumb use of mobile handheld devices equipped with a touch-sensitive screen. Similar studies have provided recommendations for target sizes when using a mobile device with two hands plus a stylus, and interacting with a desktop-sized display with an index finger, but never for thumbs when holding a small device in a single hand. The first phase explored the required target size for single-target (discrete) pointing tasks, such as activating buttons, radio buttons or checkboxes. The second phase investigated optimal sizes for widgets used for tasks that involve a sequence of taps (serial), such as text entry. Since holding a device in one hand constrains thumb movement, we varied target positions to determine if performance depended on screen location. The results showed that while speed generally improved as targets grew, there were no significant differences in error rate between target sizes =9.6 mm in discrete tasks and targets =7.7 mm in serial tasks. Along with subjective ratings and the findings on hit response variability, we found that target size of 9.2 mm for discrete tasks and targets of 9.6 mm for serial tasks should be sufficiently large for one-handed thumb use on touchscreen-based handhelds without degrading performance and preference.

We present two interfaces to support one-handed thumb use for PDAs and cell phones. Both use Scalable User Interface (ScUI) techniques to support multiple devices with different resolutions and aspect ratios. The designs use variations of zooming interface techniques to provide multiple views of application data: AppLens uses tabular fisheye to access nine applications, while LaunchTile uses pure zoom to access thirty-six applications. We introduce two sets of thumb gestures, each representing different philosophies for one-handed interaction. We conducted two studies to evaluate our designs. In the first study, we explored whether users could learn and execute the AppLens gesture set with minimal training. Participants performed more accurately and efficiently using gestures for directional navigation than using gestures for object interaction. In the second study, we gathered user reactions to each interface, as well as comparative preferences. With minimal exposure to each design, most users favored AppLens's tabular fisheye interface.