Recommender systems are ubiquitous and influence the information we consume daily by helping us navigate vast catalogs of information like music databases. However, their linear approach of surfacing content in ranked lists limits their ability to help us grow and understand our personal preferences. In this paper, we study how we can better support users in exploring a novel space, specifically focusing on music genres. Informed by interviews with expert music listeners, we developed TastePaths: an interactive web tool that helps users explore an overview of the genre-space via a graph of connected artists. We conducted a comparative user study with 16 participants where each of them used a personalized version of TastePaths (built with a set of artists the user listens to frequently) and a non-personalized one (based on a set of the most popular artists in a genre). We find that participants employed various strategies to explore the space. Overall, they greatly preferred the personalized version as it helped anchor their exploration and provided recommendations that were more compatible with their personal taste. In addition to that, TastePaths helped participants specify and articulate their interest in the genre and gave them a better understanding of the system’s organization of music. Based on our findings, we discuss opportunities and challenges for incorporating more control and expressive feedback in recommendation systems to help users explore spaces beyond their immediate interests and improve these systems’ underlying algorithms.

Voice assistants offer users access to an increasing variety of personalized functionalities. Researchers and engineers who build these experiences rely on various signals from users to create the machine learning models powering them. One type of signal is explicit feedback. While collecting explicit user feedback in situ via voice assistants would help improve and inspect the underlying models, from a user perspective it can be disruptive to the overall experience, and the user might not feel compelled to respond. However, careful design can help alleviate the friction in the experience. In this paper, we explore the opportunities and the design space for voice assistant explicit feedback elicitation. First, we present four usage categories of explicit feedback in situ for model evaluation and improvement, derived from interviews with machine learning practitioners. Then, using realistic scenarios generated for each category, we conducted an online study to evaluate multiple voice assistant designs. Our results reveal that when the voice assistant is introduced as a learner or a collaborator, users were more willing to respond to its request for feedback and felt less disruptive. In addition, giving users instructions on how to initiate feedback themselves can reduce the perceived disruptiveness compared to asking users for feedback directly. Based on our findings, we discuss the implications and potential future directions for designing voice assistants to elicit user feedback for personalized voice experiences.

It can be difficult for user researchers to explore how people might interact with interactive systems in everyday contexts; time and space limitations make it hard to be present everywhere that technology is used. Digital music services are one domain where designing for context is important given the myriad places people listen to music. One novel method to help design researchers embed themselves in everyday contexts is through remote-controlled speech agents. This paper describes a practitioner-centered case study of music service interaction researchers using a remote-controlled speech agent, called DJ Bot, to explore people's music interaction in the car and the home. DJ Bot allowed the team to conduct remote user research and contextual inquiry and to quickly explore new interactions. However, challenges using a remote speech-agent arose when adapting DJ Bot from the constrained environment of the car to the unconstrained home environment.

Music streaming services collect listener data to support personalization and discovery of their extensive catalogs. Yet this data is typically used in ways that are not immediately apparent to listeners. We conducted design workshops with ten Spotify listeners to imagine future voice assistant (VA) interactions leveraging logged music data. We provided participants with detailed personal music listening data, such as play-counts and temporal patterns, which grounded their design ideas in their current behaviors. In the interactions participants designed, VAs did not simply speak their data out loud; instead, participants envisioned how data could implicitly support introspection, behavior change, and exploration. We present reflections on how VAs could evolve from voice-activated remote controls to intelligent music coaches and how personal data can be leveraged as a design resource.

With the growing adoption of smart home technologies, inhabitants are faced with the challenge of making sense of the data that their homes can collect to configure automated behaviors that benefit their routines. Current commercial smart home interfaces usually provide information on individual devices instead of a more comprehensive overview of a home's behavior. To reduce the complexity of smart home data and integrate it better into inhabitants' lives, we turned to the familiar metaphor of a calendar and developed our smart home interface Casalendar. In order to investigate the concept and evaluate our goals to facilitate the understanding of smart home data, we created a prototype that we installed in two commercial smart homes for a month. The results we present in this paper are based on our analysis of user data from questionnaires, semi-structured interviews, participant-driven audio and screenshot feedback as well as logged interactions with our system. Our findings exposed advantages and disadvantages of this metaphor, emerging usage patterns, privacy concerns and challenges for information visualization. We further report on implications for design and open challenges we revealed through this work.

While researchers have long investigated end-user programming using a trigger-action (if-then) model, the website IFTTT is among the first instances of this paradigm being used on a large scale. To understand what IFTTT users are creating, we scraped the 224,590 programs shared publicly on IFTTT as of September 2015 and are releasing this dataset to spur future research. We characterize aspects of these programs and the IFTTT ecosystem over time. We find a large number of users are crafting a diverse set of end-user programs---over 100,000 different users have shared programs. These programs represent a very broad array of connections that appear to fill gaps in functionality, yet users often duplicate others' programs.

Interacting with smart homes and Internet of Things devices is still far from being a seamless experience as there are often many different interfaces involved. Due to the improvements of speech recognition and synthesis, voice-based agents are becoming more common to give users a unified interface to different individual systems. These agents often exhibit human-like personality traits, such as responding in a humorous way or showing caring behavior in reminders. We are exploring this approach in the context of smart homes and home automation. Should a smart home have a proactive or passive personality? Should it try to socialize with inhabitants? What personality traits do people consider desirable or undesirable? To learn more about this design space, we created two variants of a usage scenario of a domestic routine in a smart home to demonstrate different personality trait combinations. Forty-one participants experienced the scenario and provided feedback about the designs. In this paper, we report findings about participants' preferences, how they responded to the proactive and social behavior our prototype demonstrated and implications for the design of agent-based interfaces in the home.

We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.

A considerable amount of research has been carried out towards making long-standing smart home visions technically feasible. The technologically augmented homes made possible by this work are starting to become reality, but thus far living in and interacting with such homes has introduced significant complexity while offering limited benefit. As these technologies are increasingly adopted, the knowledge we gain from their use suggests a need to revisit the opportunities and challenges they pose. Synthesizing a broad body of research on smart homes with observations of industry and experiences from our own empirical work, we provide a discussion of ongoing and emerging challenges, namely challenges for meaningful technologies, complex domestic spaces, and human-home collaboration. Within each of these three challenges we discuss our visions for future smart homes and identify promising directions for the field.