Human-Robot Interaction (HRI) researchers use diverse methodologies for empirical, hypothesis-driven research, including laboratory experiments, longitudinal field deployments, and online experiments. Within these, field deployments are typically seen as suffering from lessened ecological control, and online experiments are seen as suffering from lessened ecological validity. Yet HRI researchers have largely ignored other threats to validity that uniquely emerge at the center of this spectrum.

Theatre-based design methods have become recognized as highly effective for robot interaction design. Yet there are many domains in which it would be inappropriate or ineffective for designers to role-play stakeholders, such as when working with vulnerable populations. In such cases, researchers typically engage in participatory methods, so those populations can directly contribute to the design process. We see a key design gap created by this tension: How might community members be effectively involved in theatre-based design methods? In this work, we bring together academics and practitioners across HRI, Theatre, Drama Therapy, and Applied Improvisation to address this challenge, and present Improvisational Participatory Storming (IPS) --- a novel Theatre-based Participatory Design Method that is uniquely well suited for Human-Robot Interaction. In presenting IPS, we make seven key contributions. Specifically, we identify (1) a concrete three-section structure for IPS workshops; (2) a novel reuse of Tabletop Role-Playing Game safety tools to mitigate risks in IPS activities; (3) key design objectives to be met through IPS; (4) context-specific constraints that inform which theatre-based design activities to use to meet those objectives; (5) seven key roles in which participants may participate in IPS activities; (6) key dimensions of IPS activities; and (7) three ways that IPS activities can be sequenced to scaffold participation.

Effective communication is critical to the success of many types of human-robot interaction. A key capability for enabling effective communication is accurate modeling of the cognitive status that entities hold (e.g., modeling what objects interlocutors are currently thinking about, or are generally aware of). However, existing models of cognitive status estimation are not well suited for situated and embodied interactions, as they do not account for nonverbal cues, which are a key way in which humans moderate cognitive status. To address this gap, we make three primary contributions. First, we introduce the BOWTIE corpus of dialogues from a multi-modal open-world referential task, annotated with cognitive status, gesture type, linguistic roles, and grammatical roles of entities across utterances. Second, we introduce RECS, the first LSTM-based model of cognitive status, which we train on the BOWTIE corpus. Third, we present empirical evidence for the success of RECS. These contributions stand to accelerate the future use of cognitively informed algorithms for robot language understanding and generation.

The comparatively recent advent of Large Language Models (LLMs) has resulted in a wide array of new capabilities and components relevant to Human–Robot Interaction (HRI) researchers. LLMs are being applied to vision, manipulation, planning, reasoning, learning, and HRI problems, frequently as “Scarecrows,” in which LLMs serve as black box modules integrated into robot architectures for the purpose of quickly enabling full-pipeline solutions. However, despite this explosion of applications, general questions remain about the best ways to incorporate LLMs into robot architectures, appropriate safety and guardrail considerations, and, critically, how to report properly on HRI research that involves LLMs. In this article, we explore the question of reporting guidelines for HRI researchers who utilize Scarecrows in robot architectures. We identify five key stakeholder groups in the HRI research process, discuss what information each group needs from HRI researchers, and identify appropriate mechanisms for conveying that information from HRI researchers to stakeholders either directly or indirectly. We contribute a set of suggested guidelines regarding what information should be included when researchers disseminate information about HRI research that uses LLMs.

Survivors of domestic abuse face significant challenges securing recovery resources, such as housing, mental health care, and social connections. Accessing these resources requires survivors to disclose their status as survivors of domestic abuse; a process that can be traumatic and emotionally burdensome. In this work, we consider how social robots might help domestic abuse survivors to address challenges surrounding self-disclosure. To do so, we conducted a three-session co-design workshop involving 8 participants from a domestic abuse shelter. Our results provide three key contributions: (1) new insights into the benefits and barriers to self-disclosure; (2) new insights into the ways that social robots can help survivors to better achieve those benefits and overcome those barriers before, during, and after disclosure, including key design recommendations associated with each of these phases; and (3) observations into how a trauma-informed computing perspective may enable more effective design work in Human-Robot Interaction.

Theater-based design methods are seeing increased use in social robotics, as embodied roleplay is an ideal method for designing embodied interactions. Yet theater-based design methods are often cast as simply one possible tool; there has been little consideration of the importance of specific improvisational skills for theater-based design; and there has been little consideration of how to train students in theater-based design methods. We argue that improvisation is not just one possible tool of social robot design, but is instead central to social robotics. Leveraging recent theoretical work on Applied Improvisation, we show how improvisational skills represent (1) a set of key capabilities needed for any socially interactive robot, (2) a set of learning objectives for training engineers in social robot design, and (3) a set of methodologies for training those engineers to engage in theater-based design methods. Accordingly, we argue for a reconceptualization of Social Robotics as an Applied Improvisation project; we present, as a speculative pedagogical artifact, a sample syllabus for an envisioned Applied Improvisation driven Social Robotics course that might give students the technical and improvisational skills necessary to be effective robot designers; and we present a case study in which Applied Improvisation methods were simultaneously used (a) by instructors, to rapidly scaffold engineering students' improvisational skills and (b) by those students, to engage in more effective human-robot interaction design.

In this paper we explore if human mental models of objects, even when flawed, can be integrated with a collaborative robot's decision making framework to allow it to make smarter choices under partial observability for different object-related tasks such as assembly and troubleshooting. We demonstrate how (1) these informative causal models can be extracted from humans through crowdsourcing, (2) object assembly and troubleshooting can be formulated as Partially Observable Markov Decision Processes (POMDPs) and (3) our extracted causal models can be incorporated into those models in the form of approximate priors. Finally, (4) we use systematic experimentation in simulation to demonstrate the success of this approach, with 2 X average improvement in reward observed for object assembly tasks, and 1.4 X average improvement in reward observed for troubleshooting tasks.

For human operators to effectively task teams of robots, it is critical that they maintain situational awareness about the status of those robots. However, maintaining this situational awareness becomes particularly difficult when there are dynamic changes not only in the members of the robot team, but also in the capabilities of those robots. Prior work has shown that situational awareness can be supported through interfaces that effectively visualize task-relevant information. As such, in this work, we introduce a Capability-Level System for Tracking Robots (CLSTR), a new visualization for supporting operators to maintain an appropriate level of situational awareness over the capabilities of a dynamic robot team. In evaluating CLSTR through an online human-subject study ($\mathbf{n} \boldsymbol{=} \mathbf{1 2 3}$), we found that a combination of different visual elements within an interface like the use of icons to summarize robot capabilities and animations to indicate team changes can help operators maintain awareness over robot teams.

Maintaining Situational Awareness (SA) is critical in space exploration contexts, yet made particularly difficult due to the presence of communication latency. In order to increase human SA without inducing cognitive overload, researchers have proposed Performative Autonomy (PA), in which robots intentionally interact at a lower level of autonomy than they are capable of. While researchers have demonstrated positive impacts of PA on team performance even under high latency, previous work on PA has not examined how the benefits of PA might be mediated by latency. In this work, we thus evaluate the impact of latency and PA on trust, SA, and human perceptions of robot intelligence and autonomy. Our results suggest that lower performed autonomy leads to increased cognitive load, especially when robot communication happens frequently and latency is present. In addition, we observe no effect of the PA strategies used within our experimental paradigm on SA, and instead find evidence that operating under high latency leads to negative perceptions of robots regardless of choice of PA strategy.

Robots that use gestures in conjunction with speech can achieve more effective and natural communication with human teammates, however, not all robots have capable and dexterous arms. Augmented Reality technology has effectively enabled deictic gestures for morphologically limited robots in prior work, however, the design space of AR-facilitated iconic gestures remains under-explored. Moreover, existing work largely focuses on closed-world context, where all referents are known a priori. In this work, we present a human-subject study situated in an open-world context, and compare the task performance and subjective perception associated with three different iconic gesture designs (anthropomorphic, non-anthropomorphic, deictic-iconic) against previously studied abstract gesture design. Our quantitative and qualitative results demonstrate that deictic iconic gestures (in which a robot hand is shown pointing to a visualization of a target referent) outperforms all other gestures on all metrics – but that non-anthropomorphic iconic gestures (where a visualization of a target referent appears on its own) is overall most preferred by users. These results represent a significant step forward to enabling effective human-robot interactions in realistic large-scale open-world environments.

As public space robots navigate our streets, they are likely to witness various human behavior, including verbal or physical violence. In this paper we investigate whether people believe delivery robots should intervene when they witness violence, and their perceptions of the effectiveness of different conflict de-escalation strategies. We consider multiple types of violence (verbal, physical), sources of violence (civilian, police), and robot designs (wheeled, humanoid), and analyze their relationship with participants’ perceptions. Our analysis is based on two experiments using online questionnaires, investigating the decision to intervene (N=80) and intervention mode (N=100). We show that participants agreed more with human than robot intervention, though they often perceived robots as more effective, and preferred certain strategies, such as filming. Overall, the paper shows the need to investigate whether and when robot intervention in human-human conflict is socially acceptable, to consider police-led violence as a special case of robot de-escalation, and to involve communities that are common victims of violence in the design of public space robots with safety and security capabilities.

Due to their unique persuasive power, language-capable robots must be able to both adhere to and communicate human moral norms. These requirements are complicated by the possibility that people may blame humans and robots differently for violating those norms. These complications raise particular challenges for robots giving moral advice to decision makers, as advisors and deciders may be blamed differently for endorsing the same moral action. In this work, we thus explore how people morally evaluate human and robot advisors to human and robot deciders. In Experiment 1 (n = 555), we examine human blame judgments of robot and human moral advisors and find clear evidence for an advice as decision hypothesis: advisors are blamed similarly to how they would be blamed for making the decisions they advised. In Experiment 2 (n = 1326), we examine blame judgments of a robot or human decider following the advice of a robot or human moral advisor. We replicate the results from Experiment 1 and also find clear evidence for a differential dismissal hypothesis: moral deciders are penalized for ignoring moral advice, especially when a robot ignores human advice. Our results raise novel questions about people's perception of moral advice, especially when it involves robots, and present challenges for the design of morally competent robots.

As robots are deployed into large-scale human environments, they will need to engage in task-oriented dialogues about objects and locations beyond those that can currently be seen. In these contexts, speakers use a wide range of referring gestures beyond those used in the small-scale interaction contexts that HRI research typically investigates. In this work, we thus seek to understand how robots can better generate gestures to accompany their referring language in large-scale interaction contexts. In service of this goal, we present the results of two human-subject studies: (1) a human-human study exploring how human gestures change in large-scale interaction contexts, and to identify human-like gestures suitable to such contexts yet readily implemented on robot hardware; and (2) a human-robot study conducted in a tightly controlled Virtual Reality environment, to evaluate robots' use of those identified gestures. Our results show that robot use of Precise Deictic and Abstract Pointing gestures afford different types of benefits when used to refer to visible vs. non-visible referents, leading us to formulate three concrete design guidelines. These results highlight both the opportunities for robot use of more humanlike gestures in large-scale interaction contexts, as well as the need for future work exploring their use as part of multi-modal communication.

For social robots to succeed in human environments, they must respond in effective yet appropriate ways when humans violate social and moral norms, e.g., when humans give them unethical commands. Humans expect robots to be competent and proportional in their norm violation responses, and there are a wide range of strategies robots could use to tune the politeness of their utterances to achieve effective, yet appropriate responses. Yet it is not obvious whether all such strategies are suitable for robots to use. In this work, we assess a robot's use of human-like Face Theoretic linguistic politeness strategies. Our results show that while people expect robots to modulate the politeness of their responses, they do not expect them to strictly mimic human linguistic behaviors. Specifically, linguistic politeness strategies that use direct, formal language are perceived as more effective and more appropriate than strategies that use indirect, informal language.

Research has shown that gendered robot designs prompt users to carry their gender biases into human-robot interactions. Yet avoiding gendered designs in human-robot interaction may be infeasible, as humans readily gender robots based on factors like name, voice, and pronouns. One solution to this challenge could be to use an intentionally agender robot design. Yet it is unclear whether trans, non-binary, or otherwise gender nonconforming people would view this as a positive and inclusive step, or as appropriative or otherwise problematic. In fact, little is known about trans and nonbinary perspectives on human-robot interaction, which have not been previously studied. In this work, we thus present the first study of trans and non-binary perspectives on robot design, with a particular focus on perceptions of robot gender and agender robot design. Our results suggest that trans and non-binary users readily accept robots depicted as agender, and view this as a positive design strategy that could help normalize non-cisgender identities. Yet our results also highlight key risks posed by this design strategy, including risks of backlash, caricature, and dehumanization, and show how those risks are shaped by political and economic factors.

In this work, we present Robots for Social Justice (R4SJ): a framework for an equitable engineering practice of Human-Robot Interaction, grounded in the Engineering for Social Justice (E4SJ) framework for Engineering Education and intended to complement existing frameworks for guiding equitable HRI research. To understand the new insights this framework could provide to the field of HRI, we analyze the past decade of papers published at the ACM/IEEE International Conference on Human-Robot Interaction, and examine how well current HRI research aligns with the principles espoused in the E4SJ framework. Based on the gaps identified through this analysis, we make five concrete recommendations, and highlight key questions that can guide the introspection for engineers, designers, and researchers. We believe these considerations are a necessary step not only to ensure that our engineering education efforts encourage students to engage in equitable and societally beneficial engineering practices (the purpose of E4SJ), but also to ensure that the technical advances we present at conferences like HRI promise true advances to society, and not just to fellow researchers and engineers.

Wizard-of-Oz (WoZ) is one of the most widely used experimental methodologies across the field of Human-Robot Interaction (HRI), making WoZ teleoperation interfaces a critical tool for HRI research. Yet current WoZ teleoperation interfaces are overwhelmingly tailored towards a narrow set of HRI interaction paradigms. In this work, we conducted a set of interviews with HRI researchers to better understand the diversity of teleoperation needs across the HRI community. Our analysis highlighted (1) human challenges, with respect to wizards’ expertise, the need for quick responses, and research participants’ unpredictability; (2) robot challenges, with respect to robot malfunctions, delays, and robot-driven complexity, and (3) interaction challenges, with respect to researchers’ varying control requirements and the need for precise experimental control. Moreover, our results revealed unexpected parallels between the experiences of HRI researchers and real-world teleoperators, which open up fundamentally new possibilities for future work in robot control interfaces and encourage radically different perspectives on what types of interfaces are even needed to best facilitate WoZ experimentation. Leveraging these insights, we recommend that WoZ interfaces (1) be designed with extensibility and customization in mind, (2) ease interaction management by accounting for unpredictability and multi-robot interactions, and (3) consider WoZ teleoperators beyond the context of experimentation.

To critically analyze and adapt to the risks and benefits of social robotics, future user communities will require technology and AI literacy: the ability to use new robotic technologies, understand their strengths and limitations, and critically evaluate the implications of their use. Research shows that collaborative, creative, and informal learning experiences can support AI literacy among non-technologists. Therefore, we designed Degrees of Freedom, a multiplayer interactive storytelling game that supports technology literacy about social robots. Degrees of Freedom supports technology literacy competencies by encouraging players to explore how values are encoded in robot designs, compelling players to consider the risks and limitations of robots, and encouraging them to make connections to their own lives and values. We present both the design of Degrees of Freedom and the results of game playtesting. Our results show that the narrative, collaborative nature of the game supported players in critical thinking about the role robots can or should have in their communities.

Recent work on Socially Assistive Robotics in Therapy has revealed a dual-cycle model, with the vast majority of prior work on Socially Assistive Robotics narrowly focused on the human-robot interaction, termed the "inner cycle". In contrast, little attention has been paid to the activities performed before and after the interaction, termed the "outer cycle", in which authoring and evaluation also take place. Authoring and evaluation are activities that are key sources of invisible labor for Therapists who serve as Care Wizards (i.e., SAR teleoperators). In this work, we consider the outer cycle needs of Care Wizards in another key Socially Assistive Robotics domain, Special Education, with a careful eye toward the barriers to entry and invisible labor that may manifest in this domain, and how those barriers and invisible labor might be subverted and mitigated. Our interviews with six Care Wizards who teleoperate robots in Special Education contexts reveal new insights surrounding these stakeholders’ needs. Our key insights are that (1) support systems are necessary for SAR adoption; (2) currently invisible Care Wizard labor may be indirectly compensated; and (3) training must be personalized to specific Care Wizards.

As robots become increasingly used in space exploration, it is important to ensure that space robots are developed with the appropriate level of autonomy. Semiautonomous robots operating in space contexts face unique challenges, as these robots often operate in situations that may be safety-critical, environments that are not fully known, and with communication delay to operators on Earth. Due to these challenges, there exist both advantages and risks to developing systems with high levels of autonomy to operate in space contexts. Therefore, we aim to investigate perspectives on the trade-offs of increased autonomy for space robotic systems and the human factors considerations that should be evaluated when designing these systems. We conducted qualitative interviews with five professionals in the space robotics industry to explore these perspectives. Our findings demonstrate that decisions regarding the level of autonomy of space robots are shaped not only by technical considerations, but also by operators’ willingness to accept new technology, financial considerations, and even human operators’ sense of control. Based on these results, we present design recommendations for roboticists and human factors engineers in the space robotics domain.

Robots deployed into real-world task-based environments may need to provide assistance, troubleshooting, and on-the-fly instruction for human users. While previous work has considered how robots can provide this assistance while co-located with human teammates, it is unclear how robots might best support users once they are no longer co-located. We propose the use of Augmented Reality as a medium for conveying long-distance task guidance from humans’ existing robot teammates, through Augmented Reality facilitated Robotic Guidance (ARRoG). Moreover, because there are multiple ways that a robot might project its identity through an Augmented Reality Head Mounted Display, we identify two candidate designs inspired by existing interaction patterns in the human-robot interaction (HRI) literature (re-embodiment-based and telepresence-based identity projection designs), present the results of a design workshop to explore how these designs might be most effectively implemented, and the results of a human-subject study intended to validate these designs.

We investigate the utility of functional near-infrared spectroscopy (fNIRS) for workload-based adaptive automation through the lens of multiple resource theory. We focus on the criteria of unobtrusiveness, responsiveness, load sensitivity (low vs high load), and load diagnosticity (differentiating types of load). We report a large meta-review, in which we conclude that only a few studies were suitable for evaluating sensitivity and diagnosticity in complex real-world tasks. While these reveal that the fNIRS signal is adequately sensitive to gradations of load level changes (sensitivity), the diagnosticity of fNIRS to different sources of cognitive load remained uncertain. We manipulated mental load of a complex shape sorting task via working memory load (WM) and visual perceptual load (VL), while a secondary auditory task was present throughout. We measured the effect of these manipulations at the group-level using conventional secondary and eyetracking workload measures, as well as hemodynamic response in specific functional regions in the brain, including regions involved in multi-tasking (MT), VL, WM, and auditory load (AL). Our findings revealed that fNIRS is both sensitive and diagnostic to load in complex tasks, with greater sensitivity revealed by deoxyhemoglobin than oxyhemoglobin and the brain regions associated with diagnosticity align with neuroscience literature on perceptual load, WM, and goal-directed multitasking.

The proliferation of Large Language Models (LLMs) presents both a critical design challenge and a remarkable opportunity for the field of Human–Robot Interaction (HRI). While the direct deployment of LLMs on interactive robots may be unsuitable for reasons of ethics, safety, and control, LLMs might nevertheless provide a promising baseline technique for many elements of HRI. Specifically, in this article, we argue for the use of LLMs asScarecrows: “brainless,” straw-man black-box modules integrated into robot architectures for the purpose of quickly enabling full-pipeline solutions, much like the use of “Wizard of Oz” (WoZ) and other human-in-the-loop approaches. We explicitly acknowledge that these Scarecrows, rather than providing a satisfying or scientifically complete solution, incorporate a form of the wisdom of the crowd and, in at least some cases, will ultimately need to be replaced or supplemented by a robust and theoretically motivated solution. We provide examples of how Scarecrows could be used in language-capable robot architectures as useful placeholders and suggest initial reporting guidelines for authors, mirroring existing guidelines for the use and reporting of WoZ techniques.

In this paper, we examine the risks posed by roboticists’ collaboration with law enforcement agencies in the U.S. Using Trust frameworks from AI Ethics, we argue that collaborations with law enforcement present not only risks of technology misuse, but also risks of legitimizing bad actors, and of exacerbating our field’s challenges of representation. We discuss evidence of bad dispositions justifying these risks, grounded in the behavior, origins, and incentivization of American policing, and suggest courses of action for American roboticists seeking to pursue research projects that currently require collaboration with law enforcement agencies, closing with a call for abolitionist robotics.

Augmented Reality (AR) technologies present an exciting new medium for human-robot interactions, enabling new opportunities for both implicit and explicit human-robot communication. For example, these technologies enable physically-limited robots to execute non-verbal interaction patterns such as deictic gestures despite lacking the physical morphology necessary to do so. However, a wealth of HRI research has demonstrated real benefits to physical embodiment (compared to, e.g., virtual robots on screens), suggesting AR augmentation of virtual robot parts could face challenges. In this work, we present empirical evidence comparing the use of virtual (AR) and physical arms to perform deictic gestures that identify virtual or physical referents. Our subjective and objective results demonstrate the success of mixed reality deictic gestures in overcoming these potential limitations, and their successful use regardless of differences in physicality between gesture and referent. These results help to motivate the further deployment of mixed reality robotic systems and provide nuanced insight into the role of mixed-reality technologies in HRI contexts.

Interactive intelligent systems are increasingly being deployed in safety critical contexts like Space Exploration. For humans to safely and successfully complete collaborative tasks with robots in these contexts, they must maintain Situational Awareness of their task context without being cognitively overloaded -- regardless of whether they are co-located with robots or interacting with them from a distance of thousands or millions of miles. In this paper, we present a novel autonomy design strategy we term Performative Autonomy, in which robots behave as if they have a lower level of autonomy than they are truly capable of (i.e., asking for advice they do not believe they truly need), for the sole purpose of maintaining interactants' Situational Awareness. In our first experiment (n=264), we begin by demonstrating that Performative Autonomy can increase Situational Awareness (SA) without overly increasing workload, and that this is true across tasks with different baseline levels of Mental Workload. In our second experiment (n=318), we consider cases where robots do not believe they need advice, but in fact have faulty perception or decision making capabilities. In this experiment, we only observed benefits to Performative Autonomy for specific types of questions, and only when there was significant cognitive load imposed by a secondary task; yet we observed uniform benefit on task performance for asking these types of questions regardless of task-imposed Mental workload. Our results from these two studies (total n=582) thus provide strong support for using this autonomy design strategy in future safety-critical missions as humanity explores the Moon, Mars, and beyond.

Deployed social robots are increasingly relying on wakeword-based interaction, where interactions are human-initiated by a wakeword like "Hey Jibo". While wakewords help to increase speech recognition accuracy and ensure privacy, there is concern that wakeword-driven interaction could encourage impolite behavior because wakeword-driven speech is typically phrased as commands. To address these concerns, companies have sought to use wakeword design to encourage interactant politeness, through wakewords like "Name?, please". But while this solution is intended to encourage people to use more "polite words", researchers have found that these wakeword designs actually decrease interactant politeness in text-based communication, and that other wakeword designs could better encourage politeness by priming users to use Indirect Speech Acts. Yet there has been no previous research to directly compare these wakewords designs in in-person, voice-based human-robot interaction experiments, and previous in-person HRI studies could not effectively study carryover of wakeword-driven politeness and impoliteness into human-human interactions. In this work, we conceptually reproduced these previous studies (n=69) to assess how the wakewords "Hey "Name"", "Excuse me "Name?", and "Name?, please" impact robot-directed and human-directed politeness. Our results demonstrate the ways that different types of linguistic priming interact in nuanced ways to induce different types of robot-directed and human-directed politeness.

Language-capable robots must be able to efficiently and naturally communicate about objects in the environment.A key part of communication is Referring Form Selection (RFS): the process of selecting a form like it, that, or the N to use when referring to an object.Recent cognitive status-informed computational RFS models have been evaluated in terms of goodness-of-fit to human data.But it is as yet unclear whether these models actually select referring forms that are any more natural than baseline alternatives, regardless of goodness-offit.Through a human subject study designed to assess this question, we show that even though cognitive status-informed referring selection models achieve good fit to human data, they do not (yet) produce concrete benefits in terms of naturality.On the other hand, our results show that human utterances also had high variability in perceived naturality, demonstrating the challenges of evaluating RFS naturality.

Human interactions with robot groups are more complex than interactions with individual robots. This is especially true for groups of robots that do not have humanlike 1-1 associations between bodies and identities, such as when multiple robots share a single identity. This is further complicated by the lack of direct observability of the relationship between body and identity, which may be inferred by users on the basis of various robot group identity performance strategies. Previous research on Deconstructed Trustee Theory has argued that this complexity is critical, as different perceived bodyidentity configurations may lead users to build and develop trust in distinct ways. In this paper, we thus investigate (n=94) the ways that different robot group identity performance strategies might influence the distribution of trust amongst robot group members, as well as the impact of these strategies on perceptions of robot group entitativity.

Some labor is overlooked or devalued, while necessary within the context of paid employment. This is “invisible labor”. Invisible labor is often performed by minoritized groups and is typically invisible to those in power. Novel technologies can introduce new sociotechnical labor paradigms that reduce labor visibility. In this paper, we consider how invisible labor might manifest for teleoperated Socially Assistive Robots (SARs). By combining an analysis of the labor context of teleoperated SAR use with insights from interviews with SAR teleoperators, we demonstrate how invisible labor manifests in the practical deployment of teleoperated SARs. Finally, we provide recommendations for developers and policymakers to remedy this labor invisibility.

With the deployment of robots in public realms, researchers are seeing more and more cases of abusive disinhibition towards robots. Because robots embody gendered identities, poor navigation of antisocial dynamics may reinforce or exacerbate gender-based violence. Robots deployed in social settings must recognize and respond to abuse in a way that minimizes ethical risk. This will require designers to first understand the risk posed by abuse of robots, and how humans perceive robot-directed abuse. To that end, we conducted an exploratory study of reactions to a physically abusive interaction between a human perpetrator and a victimized agent. Given extensions of gendered biases to robotic agents, as well as associations between an agent’s human likeness and the experiential capacity attributed to it, we quasi-manipulated the victim’s humanness (via use of a human actor vs. NAO robot) and gendering (via inclusion of stereotypically masculine vs. feminine cues in their presentation) across four video-recorded reproductions of the interaction. Analysis of data from 417 participants, each of whom watched one of the four videos, indicates that the intensity of emotional distress felt by an observer is associated with their gender identification, previous experience with victimization, hostile sexism, and support for social stratification, as well as the victim’s gendering.

The current work explored to what extent a robot could persuade people to participate in charitable giving by offering moral advice grounded in different ethical theories. In a laboratory, participants, who are students at a university, first performed a task to acquire lottery tickets and then received from a robot information about a charity event organized for students at their university. The robot also offered them moral advice of which the underlying framework was grounded in either deontological or Confucian role ethics to encourage donating their lottery tickets to the event. We found advice grounded in Confucian role ethics to be more effective in inducing donations than advice grounded in deontological ethics. We also found that the more strongly participants felt close to other students at their university, the less donations they would make after receiving advice grounded in deontological ethics. These findings suggest the benefits of framing moral messages of robots based upon theories of Confucian role ethics in promoting prosocial behavior. We discuss potential explanations for the negative relationship between participants’ sense of closeness with other students and their donation behavior when the robot’s advice focuses on theories of deontological ethics.

In this paper, we explore how robots can properly explain failures during navigation tasks with privacy concerns. We present an integrated robotics approach to generate visual failure explanations, by combining a language-capable cognitive architecture (for recognizing intent behind commands), an object- and location-based context recognition system (for identifying the locations of people and classifying the context in which those people are situated) and an infeasibility proof-based motion planner (for explaining planning failures on the basis of contextually mediated privacy concerns). The behavior of this integrated system is validated using a series of experiments in a simulated medical environment.

Wilderness search and rescue teams face challenges in hazardous environments. While robots show promise for these teams, their success depends on their ability to account for sociotechnical considerations, including human factors, as well as the organizational, economic, and emotional realities of search missions. We investigate these considerations through interviews with wilderness search team members who handle search dogs. These interviews reveal underexplored perspectives on awareness and uncertainty, the value of training experiences, team dynamics, and financial feasibility. Our findings motivate design recommendations for semiautonomous systems in the wilderness, yet also raise key questions regarding the role that robots can and should play in this domain.

Social robots will inevitably confront social or moral norm violations. While researchers have identified preliminary strategies for when and how robots should respond in such situations, it is not well understood how humans will make sense of these robot behaviors. We used qualitative, narrative-based methods inspired by design fiction to better understand how humans appraise these interactions. Our narrative survey invited participants to share their assumptions and expectations, analyze scenarios, and make suggestions. Our results highlight key situational and psychological factors that characterize norm-sensitive robot interactions, and suggest clear insights for the development of socially competent robot teammates.

Robots deployed in space exploration contexts need to efficiently communicate with both co-located and remote teammates to perform tasks and resolve points of uncertainty. In recent work, researchers have proposed Performative Autonomy, an autonomy design strategy for enabling language-capable robots in these contexts to enhance interactants’ Situation Awareness. However, it is not yet clear how the efficacy of this autonomy design strategy might be impacted by the extreme latency that characterizes interplanetary communication. In this work, we thus present the results of the first study exploring the impact of interaction latency on the effectiveness of Performative Autonomy. Our results suggest that while Performative Autonomy exacerbates the increased task performance times required under high latency, this autonomy design strategy can be used without increasing cognitive load, even under substantial communication latency. Moreover, our results suggest that robots performing lower levels of autonomy were viewed as better teammates, and that this autonomy design strategy helped provide resilience to degradation to such perceptions that would otherwise be caused by increasing levels of latency. Overall, these results motivate further work within the new Performative Autonomy paradigm for both remote and proximal human-robot interactions, in both space-oriented and traditional, terrestrial, human-robot interaction domains.

Socially assistive robots play an effective role in children's therapy and education. Robots engage children and provide interaction that is free of the potential judgment of human peers and adults. Research in socially assistive robots for children generally focuses on therapeutic and educational outcomes for those children, informed by a vision of autonomous robots. This perspective ignores therapists and educators, who operate these robots in practice. Through nine interviews with individuals who have used robots to deliver socially assistive services to neurodivergent children, we (1) define a dual-cycle model of therapy that helps capture the domain expert view of therapy, (2) identify six core themes of teleoperator needs and patterns across these themes, (3) provide high-level guidelines and detailed recommendations for designing teleoperated socially assistive robot systems, and (4) outline a vision of robot-assisted therapy informed by these guidelines and recommendations that centers teleoperators of socially assistive robots in practice.

Mixed Reality provides a powerful medium for transparent and effective human-robot communication, especially for robots with significant physical limitations (e.g., those without arms). To enhance nonverbal capabilities for armless robots, this article presents two studies that explore two different categories of mixed reality deictic gestures for armless robots: a virtual arrow positioned over a target referent (a non-ego-sensitive allocentric gesture) and a virtual arm positioned over the gesturing robot (an ego-sensitive allocentric gesture). In Study 1, we explore the tradeoffs between these two types of gestures with respect to both objective performance and subjective social perceptions. Our results show fundamentally different task-oriented versus social benefits, with non-ego-sensitive allocentric gestures enabling faster reaction time and higher accuracy, but ego-sensitive gestures enabling higher perceived social presence, anthropomorphism, and likability. In Study 2, we refine our design recommendations by showing that in fact these different gestures should not be viewed as mutually exclusive alternatives, and that by using them together, robots can achieve both task-oriented and social benefits.

Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI) has been gaining considerable attention in HRI research in recent years. However, the HRI community lacks a set of shared terminology and framework for characterizing aspects of mixed reality interfaces, presenting serious problems for future research. Therefore, it is important to have a common set of terms and concepts that can be used to precisely describe and organize the diverse array of work being done within the field. In this article, we present a novel taxonomic framework for different types of VAM-HRI interfaces, composed of four main categories of virtual design elements (VDEs). We present and justify our taxonomy and explain how its elements have been developed over the past 30 years as well as the current directions VAM-HRI is headed in the coming decade.

Because robots are perceived as moral agents, they must behave in accordance with human systems of morality. This responsibility is especially acute for language-capable robots because moral communication is a method for building moral ecosystems. Language capable robots must not only make sure that what they say adheres to moral norms; they must also actively engage in moral communication to regulate and encourage human compliance with those norms. In this work, we describe four experiments (total N =316) across which we systematically evaluate two different moral communication strategies that robots could use to influence human behavior: a norm-based strategy grounded in deontological ethics, and a role-based strategy grounded in role ethics. Specifically, we assess the effectiveness of robots that use these two strategies to encourage human compliance with norms grounded in expectations of behavior associated with certain social roles. Our results suggest two major findings, demonstrating the importance of moral reflection and moral practice for effective moral communication: First, opportunities for reflection on ethical principles may increase the efficacy of robots’ role-based moral language; and second, following robots’ moral language with opportunities for moral practice may facilitate role-based moral cultivation.

Research has shown that the relationship between robot mind, body, and identity is flexible and can be performed in a variety of ways. Our research explores how identity performance strategies used among robot groups may be presented through group identity observables (design cues), and how those strategies impact human-robot interactions. Specifically, we ask how group identity observables lead observers to develop different mental models of robot groups, and different perceptions of trust and group dynamics constructs.

Research has shown how the connections between robots' minds, bodies, and identities can be configured and performed in a variety of ways. In this work, we consider group identity observables: the set of design cues that robot groups use to perform different identity configurations. We explore how group identity observables lead observers to develop different mental models of robot groups. Specifically, we make four key contributions: (1) we define, conceptualize, and taxonomize group identity observables; (2) we use Grounded Theory-informed analysis of qualitative data to produce a taxonomy of users' mental models invoked by variation in those observables; (3) we empirically demonstrate (n=166) how variations in observables lead to different mental models; and (4) we further demonstrate how variations in those observables, and the mental models they evoke, influence key group dynamics constructs like entitativity.

Socially Assistive Robots are effective at supporting autistic children in a variety of different therapies. Therapists can control the robots' motions and verbalizations to engage children and deliver therapeutic interventions based on their needs. We present teleoperation capabilities to support therapists in customizing therapy to their clients' needs. Specifically, we introduce a documentation sidebar that aims to prime therapists using their clients' documented needs, and a session summary report that helps therapists reflect on the session with the child. We present preliminary designs for these capabilities and describe future work to build upon them.

Socially Assistive Robots (SARs) show significant promise in a number of domains: providing support for the elderly, assisting in education, and aiding in therapy. Perhaps unsurprisingly, SAR research has traditionally focused on providing evidence for this potential. In this paper, we argue that this focus has led to a lack of critical reflection on the appropriate level of autonomy (LoA) for SARs, which has in turn led to blind spots in the research literature. Through an analysis of the past five years of HRI literature, we demonstrate that SAR researchers are overwhelmingly developing and envisioning autonomous robots. Critically, researchers do not include a rationale for their choice in LoA, making it difficult to determine their motivation for fully autonomous robots. We argue that defaulting to research fully autonomous robots is potentially short-sighted, as applying LoA selection guidelines to many SAR domains would seem to warrant levels of autonomy that are closer to teleoperation. We moreover argue that this is an especially critical oversight as teleoperated robots warrant different evaluation metrics than do autonomous robots since teleoperated robots introduce an additional user, the teleoperator. Taken together, this suggests a mismatch between LoA selection guidelines and the vision of SAR autonomy found in the literature. Based on this mismatch, we argue that the next five years of SAR research should be characterized by a shift in focus towards teleoperation and teleoperators.

In many domains, robots must be able to commu-nicate to humans through natural language. One of the core capabilities needed for task-based natural language communication is the ability to refer to objects, people, and locations. Existing work on robot referring expression generation has focused nearly exclusively on generation of definite descriptions to visible objects. But humans use many other linguistic forms to refer (e.g., pronouns) and commonly refer to objects that cannot be seen at time of reference. Critically, existing corpora used for modeling robot referring expression generation are insufficient for modeling this wider array of referring phenomena. To address this research gap, we present a novel interaction task in which an instructor teaches a learner in a series of construction tasks that require repeated reference to a mixture of present and non-present objects. We further explain how this task could be used in principled data collection efforts.

For mobile robots, mobile manipulators, and autonomous vehicles to safely navigate around populous places such as streets and warehouses, human observers must be able to understand their navigation intent. One way to enable such understanding is by visualizing this intent through projections onto the surrounding environment. But despite the demonstrated effectiveness of such projections, no open codebase with an integrated hardware setup exists. In this work, we detail the empirical evidence for the effectiveness of such directional projections, and share a robot-agnostic implementation of such projections, coded in C++ using the widely-used Robot Operating System (ROS) and rviz. Additionally, we demonstrate a hardware configuration for deploying this software, using a Fetch robot, and briefly summarize a full-scale user study that motivates this configuration. The code, configuration files (roslaunch and rviz files), and documentation are freely available on GitHub at https://github.com/umhan35/arrow_projection.

Robots are increasingly present in ethically fraught childhood spaces. In such contexts, HRI researchers should leverage mixed-methods approaches. This is especially true in domains where robots are teleoperated by adult experts-such as therapy. A mixed-methods approach can help researchers build a thorough qualitative understanding of adult experts' needs, incorporate stakeholder perspectives through participatory design, and motivate experimental evaluations with this insight. Through such a user-centered, mixed-methods approach, robotics researchers can ultimately improve the experience of both adults and children in these spaces.

Children are stakeholders of robotic technologies who deserve to have their voices heard in the design process just as much as adult stakeholders. This is especially true for robotic technologies explicitly designed for child-robot interaction, in areas like education, healthcare, and therapy. Researchers face the challenge of cultivating children's critical awareness on the design of robots and accompanying ethical concerns, as the types of exercises typically used to engage with adult stakeholders can be ineffective with children. This requires developmentally appropriate methods for understanding children's perspectives that also address the imbalanced power dynamics between children and adults-such that children feel comfortable sharing their ideas. In this work, we demonstrate that participatory design research techniques already accepted in the Human Robot Interaction (HRI) community can fulfill this purpose. Specifically, through the design and analysis of two co-design workshops with children of different ages at a school in Denver, Colorado, we demonstrate that co-design workshops can be used to effectively understand how children make sense of robotic technologies and to facilitate children's critical reflection on the ethical dilemmas surrounding their own relationships with robots.

Language-capable robots require moral competence, including representations and algorithms for moral reasoning and moral communication. We argue for an ethical pluralist approach to moral competence that leverages and combines disparate ethical frameworks, and specifically argue for an approach to moral competence that is grounded not only in Deontological norms (as is typical in the HRI literature) but also in Confucian relational roles. To this end, we introduce the first computational approach that centers relational roles in moral reasoning and communication, and demonstrate the ability of this approach to generate both context-oriented and role-oriented explanations for robots' rejections of norm-violating commands, which we justify through our pluralist lens. Moreover, we provide the first investigation of how computationally generated role-based expla-nations are perceived by humans, and empirically demonstrate (N=120) that the effectiveness (in terms of of trust, understanding confidence, and perceived intelligence) of explanations grounded in different moral frameworks is dependent on nuanced mental modeling of human interlocutors.

This article presents a cross-cultural replication of recent work on productively violating gender norms; specifically demonstrating that breaking norms can boost robot credibility while avoiding harmful stereotypes. In this work we demonstrate via a 3 (country) x 3 (robot behaviour) between-subject experiment that these findings replicate cross-culturally across the US, Sweden, and Japan, finding evidence that breaking gender norms boosts robot credibility regardless of gender or cultural context, and regardless of pretest gender biases. Our findings further motivate a call for feminist robots that subvert the existing gender norms of robot design.

For enhanced performance and privacy, companies deploying voice-activated technologies such as virtual assistants and robots are increasingly tending toward designs in which technologies only begin attending to speech once a specified wakeword is heard. Due to concerns that interactions with such technologies could lead users, especially children, to develop impolite habits, some companies have begun to develop use modes in which interactants are required to use ostensibly polite wakewords such as “ Please”. In this paper, we argue that these “please-centering” wakewords are likely to backfire and actually discourage polite interactions due to the particular types of lexical and syntactic priming induced by those wakewords. We then present the results of a human-subject experiment (n=90) that validates those claims.

Robots that use natural language in collaborative tasks must refer to objects in their environment. Recent work has shown the utility of the linguistic theory of the Givenness Hierarchy (GH) in generating appropriate referring forms. But before referring expression generation, collaborative robots must determine the content and structure of a sequence of utterances, a task known as document planning in the natural language generation community. This problem presents additional challenges for robots in situated contexts, where described objects change both physically and in the minds of their interlocutors. In this work, we consider how robots can “think ahead” about the objects they must refer to and how to refer to them, sequencing object references to form a coherent, easy to follow chain. Specifically, we leverage GH to enable robots to plan their utterances in a way that keeps objects at a high cognitive status, which enables use of concise, anaphoric referring forms. We encode these linguistic insights as a mixed integer program within a planning context, formulating constraints to concisely and efficiently capture GH-theoretic cognitive properties. We demonstrate that this GH-informed planner generates sequences of utterances with high intersentential coherence, which we argue should enable substantially more efficient and natural human-robot dialogue.

Socially Assistive Robots (SARs) have shown promise, but there are still practical challenges to their widespread adoption. Recent research has demonstrated the advantages of teleoperated systems in this space and called for better guidelines for teleoperation interfaces. We ran group usability tests with therapists with no experience with robots to learn more about the challenges they face. We found that robot-novice therapists understand how robots can be effective in therapy. However, learning to use a robot interface can be challenging for new users. These challenges include the unfamiliar metaphors used for robot connection and the need to create, acquire, or share robot interaction content. We also identify users’ needs that are perhaps non-obvious in a research context, such as privacy of client health information and professional boundaries with client families when using electronic tools. We make several recommendations based on analysis of our group usability tests: (1) developing dedicated interfaces for content authoring that account for caregiver technical expertise, (2) implementing content organization and sharing tools, (3) using connection metaphors that non-technical users may be more familiar with such as phone calls or web URLs, (4) considering user privacy in connection methods chosen, especially within telehealth. Most importantly, we encourage further research in SAR teleoperation that focuses on caregivers as teleoperators.

The design of current natural language-oriented robot architectures enables certain architectural components to circumvent moral reasoning capabilities. One example of this is reflexive generation of clarification requests as soon as referential ambiguity is detected in a human utterance. As shown in previous research, this can lead robots to (1) miscommunicate their moral dispositions and (2) weaken human perception or application of moral norms within their current context. We present a solution to these problems by performing moral reasoning on each potential disambiguation of an ambiguous human utterance and responding accordingly, rather than immediately and naively requesting clarification. We implement our solution in the Distributed Integrated Cognition Affect and Reflection robot architecture, which, to our knowledge, is the only current robot architecture with both moral reasoning and clarification request generation capabilities. We then evaluate our method with a human subjects experiment, the results of which indicate that our approach successfully ameliorates the two identified concerns.

The Model AI Assignments session seeks to gather and disseminate the best assignment designs of the Artificial Intelligence (AI) Education community. Recognizing that assignments form the core of student learning experience, we here present abstracts of six AI assignments from the 2021 session that are easily adoptable, playfully engaging, and flexible for a variety of instructor needs. Assignment specifications and supporting resources may be found at http://modelai.gettysburg.edu.

Therapist-operated robots can play a uniquely impactful role in helping children with autism practice and acquire social skills. While extensive research within Human-Robot Interaction has focused on teleoperation interfaces for robots in general, little work has been done on teleoperation interface design for robots in the context of therapy for children with autism. Moreover, while clinical research has shown the positive impact robots can have on children with autism, much of that research has been performed in a controlled environment, with little understanding of the way these robots are used in practice. We analyze archival data of therapists teleoperating robots as part of their regular therapy sessions, to (1) determine common themes and difficulties in therapists’ use of teleoperation interfaces, and (2) provide design recommendations to improve therapists’ overall experience. We believe that following these recommendations will help maximize the effectiveness of therapy for children with autism when using Socially Assistive Robotics and the scale at which robots can be deployed in this domain.

Mixed Reality visualizations provide a powerful new approach for enabling gestural capabilities on non-humanoid robots. This paper explores two different categories of mixed-reality deictic gestures for armless robots: a virtual arrow positioned over a target referent (a non-ego-sensitive allocentric gesture) and a virtual arm positioned over the gesturing robot (an ego-sensitive allocentric gesture). Specifically, we present the results of a within-subjects Mixed Reality HRI experiment (N=23) exploring the trade-offs between these two types of gestures with respect to both objective performance and subjective social perceptions. Our results show a clear trade-off between performance and social perception, with non-ego-sensitive allocentric gestures enabling faster reaction time and higher accuracy, but ego-sensitive gestures enabling higher perceived social presence, anthropomorphism, and likability.

This paper introduces and justifies (through an n=210 online human-subject study) Deconstructed Trustee Theory, a theory of human-robot trust that factors the representation of trustee into robot body and robot identity in order to differentially model perceived trustworthiness of robot body and identity. This theory predicts (a) that different levels of trustworthiness can be attributed to a robot body and a robot identity, (b) that divergence between levels of perceived trustworthiness of body and identity may be effected by communication policies that reveal the potential for phenomena such as re-embodiment, co-embodiment, and agent migration in multi-robot systems, and (c) that perceived trustworthiness of body and identity may further diverge and be refined through moral cognitive processes triggered on observation of blameworthy actions.

Acceptance of social robots in human-robot collaborative environments depends on the robots’ sensitivity to human moral and social norms. Robot behavior that violates norms may decrease trust and lead human interactants to blame the robot and view it negatively. Hence, for long-term acceptance, social robots need to detect possible norm violations in their action plans and refuse to perform such plans. This paper integrates the Distributed, Integrated, Affect, Reflection, Cognition (DIARC) robot architecture (implemented in the Agent Development Environment (ADE)) with a novel place recognition module and a norm-aware task planner to achieve context-sensitive moral reasoning. This will allow the robot to reject inappropriate commands and comply with context-sensitive norms. In a validation scenario, our results show that the robot would not comply with a human command to violate a privacy norm in a private context.

The COVID-19 pandemic will have a profound and long-lasting impact on the entire scientific endeavor. Scientists already are adapting research programs to adapt to changes in what is prioritized—and what is possible; educators are changing the way that the next generation of researchers are trained, and flagship conferences in many fields are being cancelled, postponed, and fundamentally transformed. These broad-reaching changes are particularly impactful to human-oriented domains such as human-robot interaction (HRI). Because in-person human-subject experiments can take a year or more to conduct, the research we will see published in the field in the immediate future may appear to be “business as usual,” with accounts of laboratory studies with large numbers of in-person participants. The research currently being performed, however, is of course a different story entirely. Studies that were under way when the current crisis began will be truncated, resulting either in work that cannot be published or in work whose true impact is difficult to accurately assess. Yet HRI research performed in the coming years will be changed in fundamentally different ways; the inability to perform—or expect future performance of—in-person human subjects research, especially research involving tactile or multiparty interaction, will change both the dominant methodological techniques employed by HRI researchers and the very research questions that the field chooses to—and is able to—address. These challenges demand that HRI researchers identify precisely how the field can maintain research quality and impact while the ability to conduct human-subject studies is severely impaired for an undetermined amount of time. A natural inclination may be simply to wait the crisis out in the hope of a speedy return to normalcy; however, in this article, we argue that the community can also take this opportunity to reevaluate and refocus how research in this field is conducted and how students are mentored in ways that will yield benefits for years to come after the current crisis has ended.

For robots to successfully operate as members of human-robot teams, it is crucial for robots to correctly understand the intentions of their human teammates. This task is particularly difficult due to human sociocultural norms: for reasons of social courtesy (e.g., politeness), people rarely express their intentions directly, instead typically employing polite utterance forms such as Indirect Speech Acts (ISAs). It is thus critical for robots to be capable of inferring the intentions behind their teammates' utterances based on both their interaction context (including, e.g., social roles) and their knowledge of the sociocultural norms that are applicable within that context. This work builds off of previous research on understanding and generation of ISAs using Dempster-Shafer Theoretic Uncertain Logic, by showing how other recent work in Dempster-Shafer Theoretic rule learning can be used to learn appropriate uncertainty intervals for robots' representations of sociocultural politeness norms.

We propose an experimental ethics-based curricular module for an undergraduate course on Robot Ethics. The proposed module aims to teach students how human subjects research methods can be used to investigate potential ethical concerns arising in human-robot interaction, by engaging those students in real experimental ethics research. In this paper we describe the proposed curricular module, describe our implementation of that module within a Robot Ethics course offered at a medium-sized engineering university, and statistically evaluate the effectiveness of the proposed curricular module in achieving desired learning objectives. While our results do not provide clear evidence of a quantifiable benefit to undergraduate achievement of the described learning objectives, we note that the module did provide additional learning opportunities for graduate students in the course, as they helped to supervise, analyze, and write up the results of this undergraduate-performed research experiment.

A key capability of morally competent robots is to reject or question potentially immoral human commands. However, robot rejections of inappropriate commands must be phrased with great care and tact. Previous research has shown that failure to calibrate the "face threat" in a robot's command rejection to the severity of the norm violation in the command can lead humans to perceive the robot as inappropriately harsh and can needlessly decrease robot likeability. However, it is well-established that gender plays a significant role in determining linguistic politeness norms and that people have a powerful natural tendency to gender robots. Yet, the effect of robotic gender presentation on these noncompliance interactions is not well understood. We present an experiment that explores the effects of robot and human gender on perceptions of robots in noncompliance interactions, and find evidence of a complicated interplay between these gendered factors. Our results suggest that (1) it may be more favorable for a male robot to reject commands than for a female robot to do so, (2) it may be more favorable to reject commands given by a male human than by a female human, and (3) that robots may be perceived more favorably when their gender matches that of human interactants and observers.

There is a significant body of research seeking to enable moral decision making and ensure moral conduct in robots. One aspect of moral conduct is rejecting immoral human commands. For social robots, which are expected to follow and maintain human moral and sociocultural norms, it is especially important not only to engage in moral decision making, but also to properly communicate moral reasoning. We thus argue that it is critical for robots to carefully phrase command rejections. Specifically, the degree of politeness-theoretic face threat in a command rejection should be proportional to the severity of the norm violation motivating that rejection. We present a human subjects experiment showing some of the consequences of miscalibrated responses, including perceptions of the robot as inappropriately polite, direct, or harsh, and reduced robot likeability. This experiment intends to motivate and inform the design of algorithms to tactfully tune pragmatic aspects of command rejections autonomously.

In previous work, researchers have repeatedly demonstrated that robots' use of deictic gestures enables effective and natural human-robot interaction. However, new technologies such as augmented reality head mounted displays enable environments in which mixed-reality becomes possible, and in such environments, physical gestures become but one category among many different types of mixed reality deictic gestures. In this paper, we present the first experimental exploration of the effectiveness of mixed reality deictic gestures beyond physical gestures. Specifically, we investigate human perception of videos simulating the display of allocentric gestures, in which robots circle their targets in users' fields of view. Our results suggest that this is an effective communication strategy, both in terms of objective accuracy and subjective perception, especially when paired with complex natural language references.

There has recently been an explosion of work in the human-robot interaction (HRI) community on the use of mixed, augmented, and virtual reality. We present a novel conceptual framework to characterize and cluster work in this new area and identify gaps for future research. We begin by introducing the Plane of Interaction: a framework for characterizing interactive technologies in a 2D space informed by the Model-View-Controller design pattern. We then describe how Interactive Design Elements that contribute to the interactivity of a technology can be characterized within this space and present a taxonomy of mixed-reality interactive design elements. We then discuss how these elements may be rendered onto both reality- and virtuality-based environments using a variety of hardware devices and introduce the Reality-Virtuality Interaction Cube: a three-dimensional continuum representing the design space of interactive technologies formed by combining the Plane of Interaction with the Reality-Virtuality Continuum. Finally, we demonstrate the feasibility and utility of this framework by clustering and analyzing the set of papers presented at the 2018 VAM-HRI workshop.

The 2ndInternational Workshop on Virtual, Augmented, and Mixed Reality for Human-Robot Interactions (VAM-HRI) will bring together HRI, Robotics, and Mixed Reality researchers to identify challenges in mixed reality interactions between humans and robots. Topics relevant to the workshop include development of robots that can interact with humans in mixed reality, use of virtual reality for developing interactive robots, the design of new augmented reality interfaces that mediate communication between humans and robots, comparisons of the capabilities and perceptions of robots and virtual agents, and best design practices. VAM-HRI was held for the first time at HRI 2018, where it served as the first workshop of its kind at an academic AI or Robotics conference, and served as a timely call to arms to the academic community in response to the growing promise of this emerging field. VAM-HRI 2019 will follow on the success of VAM-HRI 2018, and present new opportunities for expanding this nascent research community.

The HRI community is working to develop interactive robots for a wide variety of pro-social tasks and ideals. As such we naturally focus on the positive side of HRI including how robots and humans may collaborate and the benefits of doing so. This workshop, in contrast, will focus on the dark side of HRI with the goal of identifying, understanding and guarding against the potential negative consequences of interactive robots. The primary objective of the workshop is to articulate and discuss the most pertinent ethical issues facing the HRI community and to develop a set of common community guidelines.

Previous research in moral psychology and human-robot interaction has shown that technology shapes human morality, and research in human-robot interaction has shown that humans naturally perceive robots as moral agents. Accordingly, we propose that language-capable autonomous robots are uniquely positioned among technologies to significantly impact human morality. We therefore argue that it is imperative that language-capable robots behave according to human moral norms and communicate in such a way that their intention to adhere to those norms is clear. Unfortunately, the design of current natural language oriented robot architectures enables certain architectural components to circumvent or preempt those architectures' moral reasoning capabilities. In this paper, we show how this may occur, using clarification request generation in current dialog systems as a motivating example. Furthermore, we present experimental evidence that the types of behavior exhibited by current approaches to clarification request generation can cause robots to (1) miscommunicate their moral intentions and (2) weaken humans' perceptions of moral norms within the current context. This work strengthens previous preliminary findings, and does so within an experimental paradigm that provides increased external and ecological validity over earlier approaches.

Naturally interacting robots must be able to understand natural human speech. As such, recent work has sought to allow robots to infer the intentions behind commonly used non-literal utterances such as indirect speech acts (ISAs). However, it is still unclear to what extent ISAs will actually be used in task-based human-robot dialogue, and to what extent robots could function without the ability to understand ISAs. In this paper, we present the results of a Wizard-of-Oz experiment that examined human ISA use in scenarios that did or did not have conventionalized social norms, and analyzed both ISA use and perceptions of robots when robots were or were not capable of understanding ISAs. Our results suggest that (1) ISAs are commonly used in task-based human-robot dialogues, even when robots show themselves unable to understand ISAs; (2) ISA use is more common in contexts with conventionalized social norms; and (3) a robot's inability to understand ISAs harms both the robot's task performance and human perception of the robot.

For situated agents to effectively engage in natural-language interactions with humans, they must be able to refer to entities such as people, locations, and objects. While classic referring expression generation (REG) algorithms like the Incremental Algorithm (IA) assume perfect, complete, and accessible knowledge of all referents, this is not always possible. In this work, we show how a previously presented consultant framework (which facilitates reference resolution when knowledge is uncertain, heterogeneous and distributed) can be used to extend the IA to produce DIST-PIA, a domain-independent algorithm for REG under uncertain, heterogeneous, and distributed knowledge. We also present a novel framework that can be used to evaluate such REG algorithms without conflating the performance of the algorithm with the performance of classifiers it employs.

As the linguistic capabilities of interactive robots advance, it becomes increasingly important to understand how humans will instruct robots through natural language. What is more, with the increased use of teleoperated humanoid robots, it is important to recognize whether any differences between instructions given to humans and to robots are due to the physical embodiment or to the perceived autonomy of the instructee. In this paper, we present the results of a human-subject experiment in which participants interacted in a collaborative, task-based setting with both a human and a suit-based, teleoperated humanoid robot said to be either autonomous or teleoperated. Our results suggest that humans will use politeness strategies equally with human, autonomous robotic, and teleoperated robotic teammates, reinforcing recent findings that autonomous robots must comprehend and appropriately respond to human utterances that follow such strategies. Our results also suggest variations in how different teammates were perceived. Specifically, our results suggest that human-teleoperated robots were perceived as less intelligent than human teammates; a finding with serious implications for human-robot team dynamics.

An important open problem for enabling truly taskable robots is the lack of task-general natural language mechanisms within cognitive robot architectures that enable robots to understand typical forms of human directives and generate appropriate responses. In this paper, we first provide experimental evidence that humans tend to phrase their directives to robots indirectly, especially in socially conventionalized contexts. We then introduce pragmatic and dialogue-based mechanisms to infer intended meanings from such indirect speech acts and demonstrate that these mechanisms can handle all indirect speech acts found in our experiment as well as other common forms of requests.

As natural language capable robots and other agents become more commonplace, the ability for these agents to understand truly natural human speech is becoming increasingly important. What is more, these agents must be able to understand truly natural human speech in realistic scenarios, in which an agent may not have full certainty in its knowledge of its environment, and in which an agent may not have full knowledge of the entities contained in its environment. As such, I am interested in developing architectural mechanisms which will allow robots to understand natural language in uncertain and open-worlds. My work towards this goal has primarily focused on two problems: (1) reference resolution, and (2) pragmatic reasoning.

We present a domain-independent approach to reference resolution that allows a robotic or virtual agent to resolve references to entities (e.g., objects and locations) found in open worlds when the information needed to resolve such references is distributed among multiple heterogeneous knowledge bases in its architecture. An agent using this approach can combine information from multiple sources without the computational bottleneck associated with centralized knowledge bases. The proposed approach also facilitates “lazy constraint evaluation”, i.e., verifying properties of the referent through different modalities only when the information is needed. After specifying the interfaces by which a reference resolution algorithm can request information from distributed knowledge bases, we present an algorithm for performing open-world reference resolution within that framework, analyze the algorithm’s performance, and demonstrate its behavior on a simulated robot.

A robot participating in natural dialogue with a human interlocutor may need to discuss, reason about, or initiate actions concerning dialogue-referenced entities. To do so, the robot must first identify or create new representations for those entities, a capability known as reference resolution. We previously presented algorithms for resolving references occurring in definite noun phrases. In this paper we present GH-POWER: an algorithm for resolving references occurring in a wider array of linguistic forms, by making novel extensions to the Givenness Hierarchy, and evaluate GH-POWER on natural task-based human-human and human-robot dialogues.

The ultimate goal of human natural language interaction is to communicate intentions. However, these intentions are often not directly derivable from the semantics of an utterance (e.g., when linguistic modulations are employed to convey polite-ness, respect, and social standing). Robotic architectures withsimple command-based natural language capabilities are thus not equipped to handle more liberal, yet natural uses of linguistic communicative exchanges. In this paper, we propose novel mechanisms for inferring in-tentions from utterances and generating clarification requests that will allow robots to cope with a much wider range of task-based natural language interactions. We demonstrate the potential of these inference algorithms for natural human-robot interactions by running them as part of an integrated cognitive robotic architecture on a mobile robot in a dialogue-based instruction task.

The problem of uniquely identifying an entity described in natural language, known as reference resolution, has become recognized as a critical problem for the field of robotics, as it is necessary in order for robots to be able to discuss, reason about, or perform actions involving any people, locations, or objects in their environments. However, most existing algorithms for reference resolution are domain-specific and limited to environments assumed to be known a priori. In this paper we present an algorithm for reference resolution which is both domain independent and designed to operate in an open world. We call this algorithm POWER: Probabilistic Open-World Entity Resolution. We then present the results of an empirical study demonstrating the success of POWER both in properly identifying the referents of referential expressions and in properly modifying the world model based on such expressions.

As future human-robot teams are envisioned for a variety of application domains, researchers have begun to investigate how humans and robots can communicate effectively and naturally in the context of human-robot team tasks. While a growing body of work is focused on human-robot communication and human perceptions thereof, there is currently little work on human perceptions of robot-robot communication. Understanding how robots should communicate information to each other in the presence of human teammates is an important open question for human-robot teaming. In this paper, we present two human-robot interaction (HRI) experiments investigating the human perception of verbal and silent robot-robot communication as part of a human-robot team task. The results suggest that silent communication of task-dependent, human-understandable information among robots is perceived as creepy by cooperative, co-located human teammates. Hence, we propose that, absent specific evidence to the contrary, robots in cooperative human-robot team settings need to be sensitive to human expectations about overt communication, and we encourage future work to investigate possible ways to modulate such expectations.

Being able to quickly and naturally teach robots new knowledge is critical for many future open-world human-robot interaction scenarios. In this paper we present a novel approach to using natural language context for one-shot learning of visual objects, where the robot is immediately able to recognize the described object. We describe the architectural components and demonstrate the proposed approach on a robotic platform in a proof-of-concept evaluation.

Recent research indicates that other factors in addition to appearance may contribute to the “Uncanny Valley” effect, and it is possible that “uncanny actions” such as “robot telepathy” - the nonverbal exchange of information among multiple robots - could be one such factor. We thus specifically examine whether humans are negatively affected by displays of nonverbal robot-robot communication through a disaster relief scenario in which one robot must relay information from a human participant to another robot in order to successfully complete a task. Our results showed no significant difference between the verbal and nonverbal communication strategies, thus suggesting that “telepathic information transmission” is acceptable. However, we also found several unexplained robot-specific effects, prompting future follow-up studies to determine their causes and the extent to which these effects might impact human perception and acceptance of robot communication strategies.

While much research exists on resolving spatial natural language references to known locations, little work deals with handling references to unknown locations. In this paper we introduce and evaluate algorithms integrated into a cognitive architecture which allow an agent to learn about its environ-ment while resolving references to both known and unknown locations. We also describe how multiple components in the architecture jointly facilitate these capabilities.

This project represents a first step towards bridging the gap between HCI and cognition research. Using functional near-infrared spectroscopy (fNIRS), we introduce tech-niques to non-invasively measure a range of cognitive workload states that have implications to HCI research, most directly usability testing. We present a set of usability experiments that illustrates how fNIRS brain measurement provides information about the cognitive demands placed on computer users by different interface designs.