In this paper, we present Smart Compose, a novel system for generating interactive, real-time suggestions in Gmail that assists users in writing mails by reducing repetitive typing. In the design and deployment of such a large-scale and complicated system, we faced several challenges including model selection, performance evaluation, serving and other practical issues. At the core of Smart Compose is a large-scale neural language model. We leveraged state-of-the-art machine learning techniques for language model training which enabled high-quality suggestion prediction, and constructed novel serving infrastructure for high-throughput and real-time inference. Experimental results show the effectiveness of our proposed system design and deployment approach. This system is currently being served in Gmail.

People often complete tasks and to-dos not only for themselves but also for others in their household. In this work, we examine how household members share and accomplish errands both individually and together. We conducted a three-week diary study with eight households to understand the types of errands that family members and roommates share with each other. We explore their motivations for offering and requesting help to complete their errands and the variety of methods for doing so. Our findings reveal when participants sometimes face challenges completing their errands, and how household members request and receive help. We learned that the cooperative performance of errands is typically dependent on household members' location, availability, and capability. Using these findings, we discuss design opportunities for cooperative errands sharing systems that can assist households.

A rich cognitive map of a space can enhance the individual's experience within the space. However, cognitive maps develop gradually through repeated experience; and because of this, on-demand mobile search services (e.g., Google Maps, Yelp) are often used to compensate for missing knowledge. In this work, we developed and evaluated a context-aware place discovery application called Opportunities Exist to assist in the acquisition of spatial knowledge and meaning. The application differs from traditional search in that places are discovered using an activity (e.g., drink coffee, sit in the sun) and the discovery process runs continuously, maintaining a history of places the user can perform her activities as she goes about her day. We conducted a 4-week deployment in two North American cities. The results show that users were able to discover new places to perform their activities in familiar spaces and learned to associate new activities with familiar places. In addition, participants leveraged the application to perform activities opportunistically, and used continuous place discovery as an opportunistic reminder of routines they wanted to break out of or resume.

Users often engage in tasks that span multiple personal devices. Although many current solutions exist to provide ubiquitous access to one's data, users continue to struggle with cross-device tasks. These solutions often require them to plan ahead for their information needs. In this paper, we present Myngle, a device-agnostic system that lets users quickly find the information they are looking for from previously visited web pages without having to plan ahead. Myngle provides a unified web history from multiple personal devices, and allows users to filter their history based on high-level categories influenced by common mobile information need categories (e.g., address, phone number). We evaluated Myngle with 32 users and found that our category-based method of filtering eases the burden of continuing cross-device tasks.

People interact with numerous personal devices on a daily basis. Sharing content among these devices is often done depending on the device capabilities and context of use; following turn-by-turn directions is more appropriate when mobile. Although several solutions exist to share content among one's devices, these solutions rely on the user planning ahead for the data he may need on another device. In this paper, we describe a system that addresses the unplanned activities, by automatically extracting addresses and points of interest that users view in their web browser and making those readily available through an in-car interface.

Battery interfaces provide important feedback about how much time users can continue using their mobile devices. Based on this information, they may develop mental models of the types of activities, tasks, and applications they can use before needing to recharge. Many of today's battery interfaces tend to report energy in coarse granularities or are highly inaccurate. As a result, users may find it difficult to depend on the estimates given. We conducted a survey with 104 participants to understand how users interact with various mobile battery interfaces. Based on the survey results, we designed and prototyped a task-centered battery interface on a mobile device that shows more accurate information about how long individual and combinations of tasks with several applications can be performed. Our pilot study of eight users demonstrated that fine-grained information separated by tasks can help users be more effective with and increase their understanding of their device's battery usage.

All in-text\treferences\tunderlined\tin\tblue\tare\tlinked\tto\tpublications\ton\tResearchGate, letting you\taccess\tand\tread\tthem\timmediately.

Increasingly, smartphones are being used to access all manner of information: email messages, Facebook status updates, tweets, RSS feeds, photographs and more. Approaches to dealing with this multi-faceted information stream developed on the desktop, such as switching between multiple applications or multiple browser windows, are unwieldy and scale poorly for mobile devices. In this paper, we propose the combination of the universal inbox and a system called 'Lenses' for extracting information of interest as part of a solution to this problem. These mechanisms allow the user to easily specify ways to sort, filter and manage their universal inbox in an intuitive way. We culminate with a discussion of implications for mobile phone interface design.

SMS messages provide an easy and simple method to com-municate with others. These short messages are useful, but can sometimes feel restricted due to the limitations of textual communication. The ability to express subtle nuances and contexts around the message could help add enjoyment and amplify the emotions being expressed by the mobile user. We present SensorComix, a new way of creating comics us-ing SMS messages combined with gestures on mobile phones. Comics are automatically generated from users ’ SMS mes-sages, and augmented with visual icons based on the per-formed gestures. We demonstrate that gestures mapped to comics can help influence the expressiveness of messages sent by mobile users.

Being mobile influences not only the types of information people seek but also the ways they attempt to access it. Mobile contexts present challenges of changing location and social context, restricted time for information access, and the need to share attentional resources among concurrent activities. Understanding mobile information needs and associated interaction challenges is fundamental to improving designs for mobile phones and related devices. We conducted a two-week diary study to better understand mobile information needs and how they are addressed. Our study revealed that depending on the time and resources available, as well as the situational context, people use diverse and, at times, ingenious ways to obtain needed information. We summarize key findings and discuss design implications for mobile technology.

Mobile phones have the potential to be useful agents for their owners by detecting and reporting situations that are of interest. Several challenges emerge in the case of detecting and reporting nice to know situations. Being alerted of these events may not be of critical importance but may be useful if the user is not busy. For detection, the precision of sensing must be high enough to minimize annoying false notifications, despite the constraints imposed by the inaccuracy of commodity sensors and the limited battery power available on mobile phones. For reporting, the notifications cannot be too obtrusive to the user or those in the vicinity. Peripheral cues are appropriate for conveying information like proximity, but have been studied primarily in settings like offices where sensors and cueing mechanisms can be controlled.

Location-based computing (LBC) is becoming increasing important in both industry and academia. A key challenge is the pervasive deployment of LBC technologies; to be effective they must run on a wide variety of client platforms, including laptops, PDAs, and mobile phones, so that location data can be acquired anywhere and accessed by any application. Moreover, as a nascent area, LBC is experiencing rapid innovation in sensing technologies, the positioning algorithms themselves, and the applications they support. Lastly, as a newcomer, LBC must integrate with existing communications and application technologies, including web browsers and location data interchange standard.This paper describes our experience in developing the Place Lab architecture, a widely used first-generation open source toolkit for client-side location sensing. Using a layered, pattern-based architecture, it supports modular development in any dimension of LBC, enabling the field to move forward more rapidly as these innovations are shared with the community as pluggable components. Our experience shows the benefits of domain-specific abstractions, and how we overcame high-level language constraints to support a wide array of platforms in this emerging space. We also describe our experience in re-engineering parts of the architecture based on the needs of the user community, including insights on software licensing issues.

Programming for end users is difficult due to lack of incentive and accessible low barrier tools. We propose to address both of these issues through a community-oriented approach to programming. With millions of users already using instant messaging, it is natural to build a programming environment that leverages the existing community amongst users. Thus we provide an automated agent that users can interact with to learn programming concepts. Through this approach a diverse population will have incentive and no extra required tools to experiment with programming concepts in a communal environment.

This paper describes the visual analysis tool WebQuilt, a web usability logging and visualization system that helps web design teams record and analyze usability tests. The logging portion of WebQuilt unobtrusively gathers clickstream data as users complete specified tasks. This data is then aggregated and presented as an interactive graph, where nodes of the graph are images of the web pages visited, and arrows are the transitions between pages. To aid analysis of the gathered usability test data, the WebQuilt visualization provides filtering capabilities and semantic zooming, allowing the designer to understand the test results at the gestalt view of the entire graph, and then drill down to sub-paths and single pages. The visualization highlights important usability issues, such as pages where users spent a lot of time, pages where users get off track during the task, navigation patterns, and exit pages, all within the context of a specific task. WebQuilt is designed to conduct remote usability testing on a variety of Internet-enabled devices and provide a way to identify potential usability problems when the tester cannot be present to observe and record user actions.