As artificial agents proliferate, there will be more and more situations in which they must communicate their capabilities to humans, including what they can “see.” Artificial agents have existed for decades in the form of computer-controlled agents in videogames. We analyze videogames in order to not only inspire the design of better agents, but to stop agent designers from replicating research that has already been theorized, designed, and tested in-depth. We present a qualitative thematic analysis of sight cues in videogames and develop a framework to support human-agent interaction design. The framework identifies the different locations and stimulus types – both visualizations and sonifications – available to designers and the types of information they can convey as sight cues. Insights from several other cue properties are also presented. We close with suggestions for implementing such cues with existing technologies to improve the safety, privacy, and efficiency of human-agent interactions.

People often hold inaccurate mental models of robots. When such misconceptions regard a robot’s perceptual capabilities, they can lead to issues with safety, privacy, and interaction efficiency. This work is the first attempt to model users’ beliefs about a robot’s perceptual capabilities and make plans to improve their accuracy—i.e., to perform belief repair. We designed a new domain called the Robot Olympics, implemented it as a web-based game platform for collecting data about users’ beliefs, and developed an approach to estimating and influencing users’ beliefs about a virtual robot in that domain. We then conducted a study that collected user behavior and belief data from 240 online participants who played the game. Results revealed shortcomings in modeling the participant’s interpretations of the robot’s actions, as well as the decision making process behind their own actions. The insights from this work provide recommendations for designing further studies and improving user models to support belief repair in human-robot interaction.

As artificial agents proliferate into society, there will be more and more situations in which they need to communicate their capabilities to humans, including what they can “see.” Humans do this with each other by using mental models of human capabilities coupled with social cues that enable situation awareness. In order to design better agents, we analyze video games, which have been communicating to humans about agent visual perception for decades. We present preliminary findings from a qualitative thematic analysis of sight cues in video games. We focus on three cue properties that warrant further study: whether the cue specifies the perceiver, whether the stimulus’ primary purpose is to be a sight cue, and the communication of non-binary sighting information. We close with an additional call for future work: on the effects of using multiple sight cues in combination.

Telepresence robots offer presence, embodiment, and mobility to remote users, making them promising options for homebound K-12 students. It is difficult, however, for robot operators to know how well they are being heard in remote and noisy classroom environments. One solution is to estimate the operator’s speech intelligibility to their listeners in order to provide feedback about it to the operator. This work contributes the first evaluation of a speech intelligibility feedback system for homebound K-12 students attending class remotely. In our four long-term, in-the-wild deployments we found that students speak at different volumes instead of adjusting the robot’s volume, and that detailed audio calibration and network latency feedback are needed. We also contribute the first findings about the types and frequencies of multimodal comprehension cues given to homebound students by listeners in the classroom. By annotating and categorizing over 700 cues, we found that the most common cue modalities were conversation turn timing and verbal content. Conversation turn timing cues occurred more frequently overall, whereas verbal content cues contained more information and might be the most frequent modality for negative cues. Our work provides recommendations for telepresence systems that could intervene to ensure that remote users are being heard.

Most people do not have direct access to knowledge about the inner workings of robots. Instead, they must develop mental models of the robot, a process that is not well understood. This article presents findings from a long-term, in-the-wild, qualitative, hypothesis-generating study of the mental model formation process. The focus was on how (qualitatively) users form mental models of the robot—specifically its perceptual capabilities, rules of behavior, and communication with other humans. Participants of diverse ages had multiple interactions with the robot over six weeks in a non-laboratory setting. The robot’s rules of behavior were changed every two weeks. A novel, non-anthropomorphic robot was created for the study with a realistic use case: storing people’s shoes during a yoga class. This article reports findings from a case study analysis of 28 interviews conducted over six weeks with six participants. These findings are organized into six topics: (1) variability in the rate at which mental models are updated to be more predictive, (2) types of reasoning and hypothesizing about the robot, (3) borrowing from existing mental models and use of imagination, (4) attributing sensing capabilities where there are no visible sensors, (5) judgments about whether the robot is autonomous or teleoperated, and (6) experimenting with the robot. Specific suggestions for future research are given throughout, culminating in a set of study design recommendations. This work demonstrates the fruitfulness of long-term, in-the-wild studies of human-robot interaction, of which mental model formation is a foundational aspect.

Privacy-sensitive robotics is an emerging area of HRI research. Judgments about privacy would seem to be context-dependent, but none of the promising work on contextual "frames" has focused on privacy concerns. This work studies the impact of contextual "frames" on local users' privacy judgments in a home telepresence setting. Our methodology consists of using an online questionnaire to collect responses to animated videos of a telepresence robot after framing people with an introductory paragraph. The results of four studies indicate a large effect of manipulating the robot operator's identity between a stranger and a close confidante. It also appears that this framing effect persists throughout several videos. These findings serve to caution HRI researchers that a change in frame could cause their results to fail to replicate or generalize. We also recommend that robots be designed to encourage or discourage certain frames.

The advent of robotics technology raises new privacy concerns, but preliminary research in this area has not included members of the general public in the conversation. This study used three focus groups to see what types of privacy concerns would be mentioned by typical users and to see what other topics were deemed important by the participants. The conversations were based around three concrete scenarios involving telepresence robots: an in-home tele-maid, a boss attending a meeting via telepresence, and a medical robo-ceptionist. Codings of our transcripts yielded privacy-relevant concerns not yet categorized by the literature such as hacking, theft, embarrassment, and marketing. These findings will give privacy-sensitive robotics researchers (1) a more complete list of privacy categories to measure, (2) new research questions to pursue, and (3) privacy-enhancing design suggestions to test.

We present a study that examines the efficiency and usability of three different interfaces for specifying which objects should be kept private (i.e., not visible) in an office environment. Our study context is a robot “janitor” system that has the ability to blur out specified objects from its video feed. One interface is a traditional point-and-click GUI on a computer monitor, while the other two operate in the real, physical space: users either place markers on the objects to indicate privacy or use a wand tool to point at them. This late-breaking report presents qualitative feedback from users for improving the interfaces.

We present a study that examines the efficiency and usability of three different interfaces for specifying which objects should be kept private (i.e., not visible) in an office environment. Our study context is a robot “janitor” system that has the ability to blur out specified objects from its video feed. One interface is a traditional point-and-click GUI on a computer monitor, while the other two operate in the real, physical space: users either place markers on the objects to indicate privacy or use a wand tool to point at them. We compare the interfaces using both self-report (e.g., surveys) and behavioral measures. Our results showed that (1) the graphical interface performed better both in terms of time and usability, and (2) using persistent markers increased the participants' ability to recall what they tagged. Choosing the right interface appears to depend on the application scenario. We also summarize feedback from the participants for improving interfaces that specify visual privacy preferences.