Computer vision and pattern recognition are increasingly being employed by smartphone and tablet applications targeted at lay-users. An open design challenge is to make such systems intelligible without requiring users to become technical experts. This paper reports a lab study examining the role of visual feedback. Our findings indicate that the stage of processing from which feedback is derived plays an important role in users' ability to develop coherent and correct understandings of a system's operation. Participants in our study showed a tendency to misunderstand the meaning being conveyed by the feedback, relating it to processing outcomes and higher level concepts, when in reality the feedback represented low level features. Drawing on the experimental results and the qualitative data collected, we discuss the challenges of designing interactions around pattern matching algorithms.

Personal informatics systems are becoming increasingly prevalent as their price, form, and ease of use improves. Though these systems offer great potential value to users, many systems are hampered by issues that limit their ability to foster engagement, and people often abandon use of these systems without garnering meaningful outcomes. While continued use of these systems is not necessary for all people, there is an opportunity to better support people working towards achievement-based goals.

This paper presents findings from a field deployment that explored a design approach we call eco-coaching: giving personalized suggestions for specific actions that would reduce wasted energy. We studied ThermoCoach, which performs eco-coaching for thermostat scheduling. It senses and models occupancy patterns in a home, and provides occupants alternative suggestions for configuring their thermostat. Our study shows that eco-coaching accomplished four things. First, it made it easier for users to implement an effective thermostat schedule. Second, it supported user agency in negotiating energy savings and comfort goals. Third, it facilitated learning different scheduling strategies as well as weighing different options. Finally, it challenged users' beliefs about how well they were doing. These outcomes, in turn, were successful in getting users to employ and experiment with more efficient setback strategies. Going forward, we propose ways that eco-coaching systems could better support users in customizing and assessing the systems' recommendations.

For decades, researchers have investigated ways to infer human thermal comfort. Studies have usually required cumbersome sensors and human observers, making them inappropriate for use in naturalistic settings such as the home. Emerging wearable and smart home sensing devices offer the opportunity to develop new models of thermal comfort based on data collected in-situ. To explore this opportunity, we deployed a sensing system in seven homes and collected self-report data from 11 participants for four weeks. Our system captures many factors employed in previous thermal comfort research, as well as new factors (e.g., activity level, sweat level). Machine learning-based models derived from the collected data show improvement over previous techniques, however significant prediction errors remain. In analyzing these errors we identify six problems that pose challenges for inferring comfort in the wild. Based on our findings, we suggest techniques to improve future in-situ thermal comfort modeling efforts.

Personal tracking technologies allow users to monitor and reflect on their physical activities and fitness. However, users are uncertain about how accurately their devices track their data. In order to better understand this challenge, we analyzed 600 product reviews and conducted 24 interviews with tracking device users. In this paper, we describe what methods users used to assess accuracy of their tracking devices and identify seven problems they encountered. We found that differences in users' expectations, physical characteristics, types of activities and lifestyle led them to have different perceptions of the accuracy of their devices. With the absence of sound mental models and unclear understanding of the concepts of accuracy and experimental controls, users designed faulty tests and came to incorrect conclusions. We propose design recommendations to better support end-users' efforts to assess and improve the accuracy of their tracking devices as required to suit their individual characteristics and purposes.

The smart home is here. One area where smart home devices promise to deliver great benefits is in the control of home heating, ventilation, and cooling (HVAC) systems. In this paper, we seek to inform the design of future heating and cooling systems by investigating users' experiences with the Nest Learning Thermostat, a commercially available smart home device. We conducted a qualitative study where we compared people's interactions with conventional thermostats with interactions with the Nest. A key finding was that the Nest impacted users' pattern of HVAC control, but only for a while, and caused new problems in unrealized energy savings. In leveraging these findings, we create a set of design implications for Eco-Interaction, the design of features and human-system interactions with the goal of saving energy.

Everyday systems and devices in the home are becoming smarter. In order to better understand the challenges of deploying an intelligent system in the home, we studied the experience of living with an advanced thermostat, the Nest. The Nest utilizes machine learning, sensing, and networking technology, as well as eco-feedback features. We conducted interviews with 23 participants, ten of whom also participated in a three-week diary study. Our findings show that while the Nest was well-received overall, the intelligent features of the Nest were not perceived to be as useful or intuitive as expected, in particular due to the system's inability to understand the intent behind sensed behavior and users' difficulty in understanding how the Nest works. A number of participants developed workarounds for the shortcomings they encountered. Based on our observations, we propose three avenues for future development of interactive intelligent technologies for the home: exception flagging, incidental intelligibility, and constrained engagement.

In order to better understand the opportunities and challenges of an intelligent system in the home, we studied the lived experience of a thermostat, the Nest. The Nest utilizes machine learning, sensing, and networking technology, as well as eco-feedback features. To date, we have conducted six interviews and one diary study. Our findings show that improved interfaces through web and mobile applications changed the interactions between users and their home system. Intelligibility and accuracy of the machine learning and sensing technology influenced the way participants perceive and adapt to the system. The convenient control over the system combined with limitations of the technology may have prevented the desired energy savings. These findings assert that thoughtful, continuous involvement from users is critical to the desired system performance and the success of interventions to promote sustainable choices. We suggest that an intelligent system in the home requires improved intelligibility and a better way in which users can provide deliberate input to the system.

Much of the information designed to help people navigate the built environment is conveyed through visual channels, which means it is not accessible to people with visual impairments. Due to this limitation, travelers with visual impairments often have difficulty navigating and discovering locations in unfamiliar environments, which reduces their sense of independence with respect to traveling by foot. In this paper, we examine how mobile location-based computing systems can be used to increase the feeling of independence in travelers with visual impairments. A set of formative interviews with people with visual impairments showed that increasing one's general spatial awareness is the key to greater independence. This insight guided the design of Talking Points 3 (TP3), a mobile location-aware system for people with visual impairments that seeks to increase the legibility of the environment for its users in order to facilitate navigating to desired locations, exploration, serendipitous discovery, and improvisation. We conducted studies with eight legally blind participants in three campus buildings in order to explore how and to what extent TP3 helps promote spatial awareness for its users. The results shed light on how TP3 helped users find destinations in unfamiliar environments, but also allowed them to discover new points of interest, improvise solutions to problems encountered, develop personalized strategies for navigating, and, in general, enjoy a greater sense of independence.