To learn the complex norms and behaviors of society, social robots will need human teachers. But teachers must trust their learners so they will continue teaching them. The present study experimentally assigned different teaching methods (instruction, evaluation, or free choice between them) to human teachers of virtual robots. Human trust formation (and recovery from initial trust loss) was robust over these methods as long as robots markedly improved over the course of their training. Teaching methods elicited different initial expectations in teachers, but in the end, robots’ improving performance made all teachers converge at high levels of trust.

Building and maintaining trust is critically important for continued human-robot teaching and the prospect of robots learning social skills from natural environments. Whereas previous work often explored strategies to reduce system errors, mitigate trust loss, or enhance learning by interactive teaching, few studies have investigated the possible benefits of fully engaged, interactive teaching on human trust. Motivated by a pair of discrepant previous investigations, the present studies for the first time directly tested the causal impact of interactivity on the loss and recovery of trust in a human-robot social skills training context. Building on a previously developed experimental paradigm, we randomly assigned participants to one of two modes of interaction: interactive teacher vs. supervisor of an experimentally controlled virtual robot. The robot was engaged in learning norm-appropriate behavior in a healthcare setting and improved from mistake-prone to near-flawless performance. Participants indicated their changing trust during the 15-trial training session and how much they attributed the robot's improvement to their own training contributions. Interactive teachers were more resilient to initial trust loss, showed increased trust in the robot's performance on additional tasks, and attributed more of the robot's improvement to themselves than did supervisors, even when the robots were slow learners.

Due to their unique persuasive power, language-capable robots must be able to both adhere to and communicate human moral norms. These requirements are complicated by the possibility that people may blame humans and robots differently for violating those norms. These complications raise particular challenges for robots giving moral advice to decision makers, as advisors and deciders may be blamed differently for endorsing the same moral action. In this work, we thus explore how people morally evaluate human and robot advisors to human and robot deciders. In Experiment 1 (n = 555), we examine human blame judgments of robot and human moral advisors and find clear evidence for an advice as decision hypothesis: advisors are blamed similarly to how they would be blamed for making the decisions they advised. In Experiment 2 (n = 1326), we examine blame judgments of a robot or human decider following the advice of a robot or human moral advisor. We replicate the results from Experiment 1 and also find clear evidence for a differential dismissal hypothesis: moral deciders are penalized for ignoring moral advice, especially when a robot ignores human advice. Our results raise novel questions about people's perception of moral advice, especially when it involves robots, and present challenges for the design of morally competent robots.

Human-robot trust research often measures people's trust in robots in individual scenarios. However, humans may update their trust dynamically as they continuously interact with a robot. In a well-powered study (n = 220), we investigate the trust updating process across a 15-trial interaction. In a novel paradigm, participants act in the role of teacher to a simulated robot on a smartphone-based platform, and we assess trust at multiple levels (momentary trust feelings, perceptions of trustworthiness, and intended reliance). Results reveal that people are highly sensitive to the robot's learning progress trial by trial: they take into account both previous-task performance, current-task difficulty, and cumulative learning across training. More integrative perceptions of robot trustworthiness steadily grow as people gather more evidence from observing robot performance, especially of faster-learning robots. Intended reliance on the robot in novel tasks increased only for faster-learning robots.

Robots deployed in social communities must act according to the communities’ social and moral norms. To acquire the large number of nuanced norms, robots can rely on human teaching. While humans tend to naturally use more than one teaching method when training a novice, current human-in-the-loop teaching frameworks have typically relied on single teaching methods (e.g., instruction or reward). To gain insight into how humans would teach robots to master social and moral norms, we present a novel paradigm in which participants interactively teach a simulated robot to behave appropriately in a healthcare setting, choosing to either instruct the robot or evaluate its proposed actions. We demonstrate that 89.5% of human teachers naturally use both teaching methods. Importantly, they adapt their teaching method as they observe the robot’s task performance, responding dynamically to the task’s difficulty, the robot’s most recent action, and the accumulated evidence of the robot’s learning progress.

Norms are indispensable for human communities, and so they will be for robot-human communities. We analyze some of the requirements for a robot to represent norms and conform its actions to them. These requirements include both cognitive and social properties that human norms instantiate. We examine which of these properties can be implemented in a robot’s architecture and review some previous computational approaches. We then introduce an approach using behavior trees, argue for its promise to implement properties of norms, and discuss unsolved challenges.

The topics of fairness and explainability have dominated recent discussions of ethical AI. However, these are only two criteria that would make artificial agents anywhere close to ethical. I frame the question of ethical AI, and especially ethical social robots, as the question of what would make them worthy of human trust and actually eliciting human trust. Relying on a recent investigation of the multi-dimensionality of human trust, I lay out five criteria of trustworthiness-being competent, reliable, transparent, benevolent, and having ethical integrity. I will argue that an essential ingredient of such trustworthiness is norm competence-the ability to represent, comply with, and learn relevant social-moral norms (including fairness as one among many). I discuss the challenges to implementing norm competence and the critical role that justification, not just explanation, will play in providing evidence for such competence.

Robots deployed in social settings must act appropriately-that is, in compliance with social and moral norms. However, efforts of teaching norms to robots have typically relied on single teaching methods (e.g., instruction, reward). By contrast, humans may naturally use more than one teaching method when training a novice. To test this claim in the domain of human-robot teaching, we present a novel paradigm in which participants interactively teach a simulated robot to behave appropriately in a healthcare setting, choosing to either instruct the robot or evaluate its proposed actions. We demonstrate that 89% of human teachers naturally adopt mixed teaching strategies. We further identify some of the factors that influence people's choices. Results reveal that human teachers dynamically update their impression of the robot from early to late in the teaching session, and they choose their teaching strategy based on the robot's specific actions and their accumulated perceptions of the robot's learning progress.

As robots become more prevalent in society, they will need to learn to act appropriately under diverse human teaching styles. We present a human-centered approach for teaching robots reward functions by using a mixture of teaching strategies when communicating action appropriateness and goal success. Our method incorporates two teaching strategies for learning: explicit action instruction and evaluative, scalar-based feedback. We demonstrate that a robot instantiating our method can learn from humans who use both kinds of strategies to train the robot in a complex navigation task that includes norm-like constraints.

People are increasingly interacting with artificial agents in social settings, and as these agents become more sophisticated, people will have to teach them social norms. Two prominent teaching methods include instructing the learner how to act, and giving evaluative feedback on the learner’s actions. Our empirical findings indicate that people naturally adopt both methods when teaching norms to a simulated robot, and they use the methods selectively as a function of the robot’s perceived expertise and learning progress. In our algorithmic work, we conceptualize a set of context-specific norms as a reward function and integrate learning from the two teaching methods under a single likelihood-based algorithm, which estimates a reward function that induces policies maximally likely to satisfy the teacher’s intended norms. We compare robot learning under various teacher models and demonstrate that a robot responsive to both teaching methods can learn to reach its goal and minimize norm violations in a navigation task for a grid world. We improve the robot’s learning speed and performance by enabling teachers to give feedback at an abstract level (which rooms are acceptable to navigate) rather than at a low level (how to navigate any particular room).

As human-robot interaction (HRI) researchers, like all scientists, we must demonstrate the reproducibility of findings-especially across robots. We present a three-study replication effort that illustrates the challenges and opportunities for replication science in HRI.

Previous work has shown that people provide different moral judgments of robots and humans in the case of moral dilemmas. In particular, robots are blamed more when they fail to intervene in a situation in which they can save multiple lives but must sacrifice one person's life. Previous studies were all conducted with U.S. participants; the present two experiments provide a careful comparison of moral judgments among Japanese and U.S. participants. The experiments assess multiple ways in which cross-cultural differences in moral evaluations may emerge: in the willingness to treat robots as moral agents; the norms that are imposed on robots' behaviors; and the degree of blame that accrues to them when they violate the imposed norms. Even though Japanese and U.S. participants differ to some extent in their treatment of robots as moral agents and in the particular norms they impose on them, the two cultures show parallel patterns of greater blame for robots who fail to intervene in moral dilemmas.

introduction Open AccessIntroduction to the Special Issue on Explainable Robotic Systems Share on Authors: Maartje M. A. De Graaf Utrecht University Utrecht UniversityView Profile , Anca Dragan University of California, Berkeley University of California, BerkeleyView Profile , Bertram F. Malle Brown University Brown UniversityView Profile , Tom Ziemke Linkoping University Linkoping UniversityView Profile Authors Info & Claims ACM Transactions on Human-Robot InteractionVolume 10Issue 3July 2021 Article No.: 22pp 1–4https://doi.org/10.1145/3461597Online:11 July 2021Publication History 0citation320DownloadsMetricsTotal Citations0Total Downloads320Last 12 Months320Last 6 weeks80 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF

Human behavior is powerfully guided by social and moral norms. Robots that enter human societies must therefore behave in norm-conforming ways as well. However, there is currently no cognitive, let alone computational model available of how humans represent, activate, and learn norms. We offer first steps toward such a model and apply it to the design of a norm-competent social robot. We propose a general methodology for such a design, from empirical identification of relevant norms to computational implementations of norm learning to thorough and iterative evaluation of the robot's norm compliance by means of community feedback.

Human behavior is frequently guided by social and moral norms, and no human community can exist without norms. Robots that enter human societies must therefore behave in norm-conforming ways as well. However, currently there is no solid cognitive or computational model available of how human norms are represented, activated, and learned. We provide a conceptual and psychological analysis of key properties of human norms and identify the demands these properties put on any artificial agent that incorporates norms-demands on the format of norm representations, their structured organization, and their learning algorithms.

It has long been assumed that when people observe robots they intuitively ascribe mind and intentionality to them, just as they do to humans. However, much of this evidence relies on experimenter-provided questions or self-reported judgments. We propose a new way of investigating people's mental state ascriptions to robots by carefully studying explanations of robot behavior. Since people's explanations of human behavior are deeply grounded in assumptions of mind and intentional agency, explanations of robot behavior can reveal whether such assumptions similarly apply to robots. We designed stimulus behaviors that were representative of a variety of robots in diverse contexts and ensured that people saw the behaviors as equally intentional, desirable, and surprising across both human and robot agents. We provided 121 participants with verbal descriptions of these behaviors and asked them to explain in their own words why the agent (human or robot) had performed them. To systematically analyze the verbal data, we used a theoretically grounded classification method to identify core explanation types. We found that people use the same conceptual toolbox of behavior explanations for both human and robot agents, robustly indicating inferences of intentionality and mind. But people applied specific explanatory tools at somewhat different rates and in somewhat different ways for robots, revealing specific expectations people hold when explaining robot behaviors.

In addition to the aggregated analyses across ten intentional behaviors, reported in the main paper, we also broke the behaviors down by surprise, desirability, and currentness (see Table 1).

Human-robot trust is crucial to successful human-robot interaction. We conducted a study with 798 participants distributed across 32 conditions using four dimensions of human-robot trust (reliable, capable, ethical, sincere) identified by the Multi-Dimensional-Measure of Trust (MDMT). We tested whether these dimensions can differentially capture gains and losses in human-robot trust across robot roles and contexts. Using a 4 scenario × 4 trust dimension × 2 change direction between-subjects design, we found the behavior change manipulation effective for each of the four subscales. However, the pattern of results best supported a two-dimensional conception of trust, with reliable-capable and ethical-sincere as the major constituents.

Anthropomorphic robots, or robots with human-like appearance features such as eyes, hands, or faces, have drawn considerable attention in recent years. To date, what makes a robot appear human-like has been driven by designers» and researchers» intuitions, because a systematic understanding of the range, variety, and relationships among constituent features of anthropomorphic robots is lacking. To fill this gap, we introduce the ABOT (Anthropomorphic roBOT) Database---a collection of 200 images of real-world robots with one or more human-like appearance features (http://www.abotdatabase.info). Harnessing this database, Study 1 uncovered four distinct appearance dimensions (i.e., bundles of features) that characterize a wide spectrum of anthropomorphic robots and Study 2 identified the dimensions and specific features that were most predictive of robots» perceived human-likeness. With data from both studies, we then created an online estimation tool to help researchers predict how human-like a new robot will be perceived given the presence of various appearance features. The present research sheds new light on what makes a robot look human, and makes publicly accessible a powerful new tool for future research on robots» human-likeness.

In three studies we found further evidence for a previously discovered Human-Robot (HR) asymmetry in moral judgments: that people blame robots more for inaction than action in a moral dilemma but blame humans more for action than inaction in the identical dilemma (where inaction allows four persons to die and action sacrifices one to save the four). Importantly, we found that people's representation of the “robot” making these moral decisions appears to be one of a mechanical robot. For when we manipulated the pictorial display of a verbally described robot, people showed the HR asymmetry only when making judgments about a mechanical-looking robot, not a humanoid robot. This is the first demonstration that robot appearance affects people's moral judgments about robots.

Trust serves as a powerful social capacity that can influence the course of a relationship, either spurring a willingness or refusal of one agent to interact with another. As we attempt to build increasingly complex and useful social robots, we must consider what factors will engender such trust and thus benefit human-robot interaction. In this paper we describe a line of inquiry that is investigating how a person's perceived involvement in helping a robot recover from failure affects the person's trust in the robot and in its future actions. We posit that a person's active involvement with a robot, compared with passive observation, will lead to greater trust in the robot.

Visual perspective taking plays a fundamental role in both human-human interaction and human-robot interaction (HRI). In three experiments, we took a novel approach to the topic of visual perspective taking in HRI, examining whether, and under what conditions, people spontaneously take a robot's visual perspective. Using two different robot models, we found that specific behaviors performed by a robot-namely, object-directed gaze and goal-directed reaching-led many human viewers to take the robot's visual perspective, though slightly fewer than when the same behaviors were performed by a person. However, we found no difference in people's perspective-taking tendency toward robots that differed in their human-likeness. Also, reaching became an especially effective perspective-taking trigger when it was displayed in a video rather than in a photograph. Taken together, these findings suggest that certain nonverbal behaviors in robots are sufficient to trigger the mechanism of mental state attribution-visual perspective taking in particular-in human observers. Therefore, people's spontaneous perspective-taking tendencies should be taken into account when designing intuitive and effective human-centered robots.

Robots will eventually perform norm-regulated roles in society (e.g. caregiving), but how will people apply moral norms and judgments to robots? By answering such questions, researchers can inform engineering decisions while also probing the scope of moral cognition. In previous work, we compared people's moral judgments about human and robot agents' behavior in moral dilemmas. We found that robots, compared with humans, were more commonly expected to sacrifice one person for the good of many, and they were blamed more than humans when they refrained from that decision. Thus, people seem to have somewhat different normative expectations of robots than of humans. In the current project we analyzed in detail the justifications people provide for three types of moral judgments (permissibility, wrongness, and blame) of robot and human agents. We found that people's moral judgments of both agents relied on the same conceptual and justificatory foundation: consequences and prohibitions undergirded wrongness judgments; attributions of mental agency undergirded blame judgments. For researchers, this means that people extend moral cognition to nonhuman agents. For designers, this means that robots with credible cognitive capacities will be considered moral agents but perhaps regulated by different moral norms.

Moral norms play an essential role in regulating human interaction. With the growing sophistication and proliferation of robots, it is important to understand how ordinary people apply moral norms to robot agents and make moral judgments about their behavior. We report the first comparison of people's moral judgments (of permissibility, wrongness, and blame) about human and robot agents. Two online experiments (total N = 316) found that robots, compared with human agents, were more strongly expected to take an action that sacrifices one person for the good of many (a "utilitarian" choice), and they were blamed more than their human counterparts when they did not make that choice. Though the utilitarian sacrifice was generally seen as permissible for human agents, they were blamed more for choosing this option than for doing nothing. These results provide a first step toward a new field of Moral HRI, which is well placed to help guide the design of social robots.

We propose that moral competence consists of five distinct but related elements: (1) having a system of norms; (2) mastering a moral vocabulary; (3) exhibiting moral cognition and affect; (4) exhibiting moral decision making and action; and (5) engaging in moral communication. We identify some of the likely triggers that may convince people to (justifiably) ascribe each of these elements of moral competence to robots. We suggest that humans will treat robots as moral agents (who have some rights, obligations, and are targets of blame) if they perceive them to have at least elements (1) and (2) and one or more of elements (3)-(5).

Autonomous social robots embedded in human societies have to be sensitive to human social interactions and thus to moral norms and principles guiding these interactions. Actions that violate norms can lead to the violator being blamed. Robots thus need to be able to anticipate possible norm violations and attempt to prevent them while they execute actions. If norm violations cannot be prevented (e.g., in a moral dilemma situation in which every action leads to a norm violation), then the robot needs to be able to justify the action to address any potential blame. In this paper, we present a first attempt at an action execution system for social robots that can (a) detect (some) norm violations, (b) consult an ethical reasoner for guidance on what to do in moral dilemma situations, and (c) it can keep track of execution traces and any resulting states that might have violated norms in order to produce justifications.