To collaborate with humans, robots must infer goals that are often ambiguous, difficult to articulate, or not drawn from a fixed set. Prior approaches restrict inference to a predefined goal set, rely only on observed actions, or depend exclusively on explicit instructions, making them brittle in real-world interactions. We present BALI (Bidirectional Action–Language Inference) for goal prediction, a method that integrates natural language preferences with observed human actions in a receding-horizon planning tree. BALI combines language and action cues from the human, asks clarifying questions only when the expected information gain from the answer outweighs the cost of interruption, and selects supportive actions that align with inferred goals. We evaluate the approach in collaborative cooking tasks, where goals may be novel to the robot and unbounded. Compared to baselines, BALI yields more stable goal predictions and significantly fewer mistakes.

Human-Robot Interaction (HRI) continues to rely on commercial social robot platforms to support academic research. Yet again and again, these systems prove short-lived, inaccessible, or misaligned with research needs. We argue that this is not an industry problem – the goals, needs, and constraints of industry are inherently distinct. Instead, this is a fundamental structural problem in HRI research, and one that must be solved from within. In short, HRI researchers must build their own products. In this paper, we trace the recent problems of industry-supplied robots and frame a new type of HRI research artifact in response: Deployable Research Products (DRPs), which bridge the gap between lab prototypes and commercial products. Drawing on mental models from business and innovation theory, we outline the mindset shifts that HRI must embody to move towards DRPs. We conclude with three emerging examples of this alternative path in the HRI community. These projects differ in scope and approach but share a common thread: to ensure the longevity of our science, we cannot outsource what we value most.

The fields of human-robot interaction (HRI) and robotics at large have developed around a stable set of assumptions about what robots are and how they should behave. These assumptions arise from the constitutive traits of robots, which together shape social expectations. Over time, these expectations have hardened into tacit rules that quietly govern research and design: robots should always engage, help, be productive, remain polite, never lie, never err, and never model harm. While these prevailing norms have merit, they also constrain the field's imagination of the interactions robots can meaningfully support. We propose rule-breaking as a generative design strategy and illustrate how deliberate violations—robots that interrupt, refuse, mislead, or err—can produce interactions that are more ethical, effective, and socially intelligent. In doing so, we argue for a more reflexive and imaginative HRI that learns as much from breaking the rules as from following them.

Robots are often designed to help, but help is not always helpful. In everyday situations, it is a socially delicate act: the right offer of help at the wrong moment can be intrusive, unnecessary, or even undermining. In this article, we challenge the prevailing assumption that robots should always offer help, prompting an essential discussion of how robots can discern when to offer help. We introduce a theoretical framework that enables robots to assess the appropriateness of offering help by considering factors such as the relative skill levels of the robot and human user, as well as the social value and cost of assistance. To validate this framework, we conducted a large-scale online study in which participants rated the appropriateness of robot assistance across diverse task scenarios. Their responses supported our core predictions and highlighted additional contextual factors. Building on these results, we discuss potential extensions of the simplified model for real-world settings, including uncertainty management, perception of ability, autonomy preferences, and social presence. We present these directions as opportunities for future research.

The rapid and nearly pervasive impact of artificial intelligence on fields as diverse as medicine, law, banking, and the arts has made many students who would never enroll in a computer science class become interested in understanding elements of artificial intelligence. Fueled by questions about how this technology would change their own fields, these students are not seeking to become experts in building AI systems but instead are searching for a sufficient understanding to be safe, effective, and informed users. In this paper, we describe a first-of-its-kind course offering, "Artificial Intelligence for Future Presidents" designed and taught during the spring of 2024. We share rationale on the design and structure of the course, consider how best to convey complex technical information to students without the background in programming or mathematics, and consider methods for supporting an understanding of the limits of this technology.

Mistakes, failures, and transgressions committed by a robot are inevitable as robots become more involved in our society. When a wrong behavior occurs, it is important to understand what factors might affect how the robot is perceived by people. In this paper, we investigated how the type of transgressor (human or robot) and type of backstory depicting the transgressor's mental capabilities (default, physio-emotional, socio-emotional, or cognitive) shaped participants' perceptions of the transgressor's morality. We performed an online, between-subjects study in which participants (N =720) were first intro-duced to the transgressor and its backstory, and then viewed a video of a real-life robot or human pushing down a human. Although participants attributed similarly high intent to both the robot and the human, the human was generally perceived to have higher morality than the robot. However, the backstory that was told about the transgressors' capabilities affected their perceived morality. We found that robots with emotional backstories (i.e., physio-emotional or socio-emotional) had higher perceived moral knowledge, emotional knowledge, and desire than other robots. We also found that humans with cognitive backstories were perceived with less emotional and moral knowledge than other humans. Our findings have consequences for robot ethics and robot design for HRI.

Atypical gaze behavior is a diagnostic hallmark of Autism Spectrum Disorder (ASD), playing a substantial role in the social and communicative challenges that individuals with ASD face. This study explores the impacts of a month-long, in-home intervention designed to promote triadic interactions between a social robot, a child with ASD, and their caregiver. Our results indicate that the intervention successfully promoted appropriate gaze behavior, encouraging children with ASD to follow the robot's gaze, resulting in more frequent and prolonged instances of spontaneous eye contact and joint attention with their caregivers. Additionally, we observed specific timelines for behavioral variability and novelty effects among users. Furthermore, diagnostic measures for ASD emerged as strong predictors of gaze patterns for both caregivers and children. These results deepen our understanding of ASD gaze patterns and highlight the potential for clinical relevance of robot-assisted interventions.

As robot deployments become more commonplace, people are likely to take on the role of supervising robots (i.e., correcting their mistakes) rather than directly teaching them. Prior works on Learning from Corrections (LfC) have relied on three key assumptions to interpret human feedback: (1) people correct the robot only when there is significant task objective divergence; (2) people can accurately predict if a correction is necessary; and (3) people trade off precision and physical effort when giving corrections. In this work, we study how two key factors (robot competency and motion legibility) affect how people provide correction feedback and their implications on these existing assumptions. We conduct a user study$(N=60)$under an LfC setting where participants supervise and correct a robot performing pick-and-place tasks. We find that people are more sensitive to suboptimal behavior by a highly competent robot compared to an incompetent robot when the motions are legible$(p=0.0015)$and predictable$(p=0.0055)$. In addition, people also tend to withhold necessary corrections$(p < 0.0001)$when supervising an incompetent robot and are more prone to offering unnecessary ones$(p=0.0171)$when supervising a highly competent robot. We also find that physical effort positively correlates with correction precision, providing empirical evidence to support this common assumption. We also find that this correlation is significantly weaker for an incompetent robot with legible motions than an incompetent robot with predictable motions$(p=0.0075)$. Our findings offer insights for accounting for competency and legibility when designing robot interaction behaviors and learning task objectives from corrections.

In this work, we introduce and formalize the Zero-Knowledge Task Planning (ZKTP) problem, i.e., formulating a sequence of actions to achieve some goal without task-specific knowledge. Additionally, we present a first investigation and approach for ZKTP that leverages a large language model (LLM) to decompose natural language instructions into subtasks and generate behavior trees (BTs) for execution. If errors arise during task execution, the approach also uses an LLM to adjust the BTs on-the-fly in a refinement loop. Experimental validation in the AI2-THOR simulator demonstrate our approach’s effectiveness in improving overall task performance compared to alternative approaches that leverage task-specific knowledge. Our work demonstrates the potential of LLMs to effectively address several aspects of the ZKTP problem, providing a robust framework for automated behavior generation with no task-specific setup.

Robots are increasingly present in our everyday environments, offering services and products. But can they influence our choices through movement alone? This paper investigates whether a robot manipulator can nudge user decisions using only its arm movements, without speech, facial expressions, or physical contact. We first identified plausible nudging motions through a bodystorming session, then designed and implemented three composite nudges (positive, neutral, and negative) using a UR5 robot arm. In a video-based online study (N=35), participants more often chose positively nudged items and avoided negatively nudged ones. A small in-person study (N=9) confirmed the effect. These results demonstrate that movement-only nudges can influence decision-making and highlight the potential of subtle physical behaviors for nonverbal persuasion.

Many schools have built de-escalation and sensory rooms to support students who experience heightened emotional states, sensory overload, or difficulty self-regulating in traditional classroom settings. Yet, effective implementation remains challenging due to diverse student needs and resource constraints. Hence, we developed RESET (Robot-Enhanced Social-Emotional Therapy), a robot for facilitating students’ self-regulation in their school’s existing de-escalation space. We present our co-design process, iterative development, and final system components. Following a fully autonomous, month-long deployment in an elementary school, we assessed the robot’s usability and impacts. Results indicate RESET integrated well into the school environment, promoting more efficient deescalation, smoother transitions back to classroom learning, and lasting impacts beyond its deployment period.

In the past two decades, the field of social robotics has undergone significant growth, witnessing a surge in long-term human–robot interaction (HRI) studies. This review paper provides an in-depth analysis of 120 long-term HRI studies conducted between 2003 and 2023, spanning 7 major domains including education, entertainment, and physical and mental health. We define “long-term” as studies deploying social robots with the same users for more than three sessions across 3 consecutive days, aiming to employ a comprehensive approach and identify trends in this dynamic field. Our analysis explores various aspects of these studies, from participant demographics to the characteristics of the HRI and engagement measures. The findings reveal promising trends, such as diverse age group representation, a strong focus on real-world contexts, and autonomous robot operation. We also identify gaps, notably the limited representation of studies involving teenagers and those studying workplace settings. By presenting this overview, we aim to empower the HRI community to address challenges, refine methodologies, and foster innovation in the domain of long-term HRI.

This article discusses the design, development, and evaluation of Ommie , a novel socially assistive robot that supports deep breathing practices for the purposes of anxiety reduction. Research has shown that practicing deep breathing (breathing while extending one’s inhales, holds, and exhales) has a strong capacity to calm the autonomic nervous system and reduce anxiety. The robot’s primary function is to guide users through a series of deep breaths by way of haptic interactions and audio cues. We utilized a user-centered design approach and present our design methodology in addition to core decisions across robot morphology, tactility, and interactivity. As reported in prior work, the final robot prototype was tested with a two-cohort usability study (n = 43) at a local university wellness center, including participants with anxiety and those with varying levels of experience with deep breathing. Interacting with Ommie resulted in a significant reduction in STAI-6 anxiety measures across all participants, who also found the robot intuitive, approachable, and engaging. Participants also reported feelings of focus and companionship when using the robot, often elicited by the haptic interaction. This article describes how our design process and design goals contributed to these results showing Ommie’s capacity for supporting those with anxiety. Our work also serves as an example of how researchers can design robots for behavioral practices for mental health.

Recent work in Human-Robot Interaction (HRI) has shown that robots can leverage implicit communicative signals from users to understand how they are being perceived during interactions. For example, these signals can be gaze patterns, facial expressions, or body motions that reflect internal human states. To facilitate future research in this direction, we contribute the \textttREACT database, a collection of two datasets of human-robot interactions that display users' natural reactions to robots during a collaborative game and a photography scenario. Further, we analyze the datasets to show that interaction history is an important factor that can influence human reactions to robots. As a result, we believe that future models for interpreting implicit feedback in HRI should explicitly account for this history. \textttREACT opens up doors to this possibility in the future.

Stress detection in real-world settings presents significant challenges due to the complexity of human emotional expression influenced by biological, psychological, and social factors. While traditional methods like EEG, ECG, and EDA sensors provide direct measures of physiological responses, they are unsuitable for everyday environments due to their intrusive nature. Therefore, using non-contact, commonly available sensors like cameras and microphones to detect stress would be helpful. In this work, we use stress indicators from four key affective modalities extracted from audio-visual data: facial expressions, vocal prosody, textual sentiment, and physical fidgeting. To achieve this, we first labeled 353 video clips featuring individuals in monologue scenarios discussing personal experiences, indicating whether or not the individual is stressed based on our four modalities. Then, to effectively integrate signals from the four modalities, we extract stress signals from our audio-visual data using unimodal classifiers. Finally, to explore how the different modalities would interact to predict if a person is stressed, we compare the performance of three multimodal fusion methods: intermediate fusion, voting-based late fusion, and learning-based late fusion. Results indicate that combining multiple modes of information can effectively leverage the strengths of different modalities and achieve an F1 score of 0.85 for binary stress detection. Moreover, an ablation study shows that the more modalities are integrated, the higher the F1 score for detecting stress across all fusion techniques, demonstrating that our selected modalities possess complementary stress indicators.

In this work, we introduce a strategy that frames the sequential action selection problem for robots in terms of resolving blocking conditions, i.e., situations that impede progress on an action en route to a goal. This strategy allows a robot to make one-at-a-time decisions that take in pertinent contextual information and swiftly adapt and react to current situations. We present a first instantiation of this strategy that combines a state-transition graph and a zero-shot Large Language Model (LLM). The state-transition graph tracks which previously attempted actions are currently blocked and which candidate actions may resolve existing blocking conditions. This information from the state-transition graph is used to automatically generate a prompt for the LLM, which then uses the given context and set of possible actions to select a single action to try next. This selection process is iterative, with each chosen and executed action further refining the state-transition graph, continuing until the agent either fulfills the goal or encounters a termination condition. We demonstrate the effectiveness of our approach by comparing it to various LLM and traditional task-planning methods in a testbed of simulation experiments. We discuss the implications of our work based on our results.

To enable sophisticated interactions between humans and robots in a shared environment, robots must infer the intentions and strategies of their human counterparts. This inference can provide a competitive edge to the robot or enhance human-robot collaboration by reducing the necessity for explicit communication about task decisions. In this work, we identify specific states within the shared environment, which we refer to as Critical Decision Points, where the actions of a human would be especially indicative of their high-level strategy. A robot can significantly reduce uncertainty regarding the human’s strategy by observing actions at these points. To demonstrate the practical value of Critical Decision Points, we propose a Receding Horizon Planning (RHP) approach for the robot to influence the movement of a human opponent in a competitive game of hide-and-seek in a partially observable setting. The human plays as the hider and the robot plays as the seeker. We show that the seeker can influence the hider to move towards Critical Decision Points, and this can facilitate a more accurate estimation of the hider’s strategy. In turn, this helps the seeker catch the hider faster than estimating the hider’s strategy whenever the hider is visible or when the seeker only optimizes for minimizing its distance to the hider.

Gossip is a human behavior that has been shown to strengthen bonds, trust, and the feeling of inclusion between the gossiper and the person with whom they share the gossip. As humans engage more with social robots, fostering bonds between them is critical for meaningful interactions. In this paper, we investigated how gossiping can affect the perception of group inclusion and trust between a human and a robot. In this between-subjects user study (N = 38), we compared the effects of a robot that gossips to the participant in either a positive or negative way about the experimenter during an interaction. We found that participants in the positive condition reported a significant increase in group inclusion with the robot, while participants in the negative condition did not. We also found that participants’ moral trust in the negative condition significantly decreased. Our results suggested that positive gossip can be beneficial to human-robot team cohesion.

As robots are increasingly integrated into various aspects of everyday life, it becomes essential to develop intelligent systems capable of providing assistance while maintaining social appropriateness. In this paper, we challenge the prevailing assumption that robots should always offer help, prompting an essential discussion of when robots should offer help. We present a systematic way of considering socially appropriate assistance in human-robot interaction and introduce a theoretical framework that enables robots to discern whether or not to offer help to a human user. We examine the factors that influence the social appropriateness of help, including the relative skill levels between the robot and user and measures for assessing the social value and cost of help. Through a series of illustrative examples, we demonstrate the feasibility of our framework in providing socially appropriate assistance.

A wide range of studies in Human-Robot Interaction (HRI) has shown that robots can influence the social behavior of humans. This phenomenon is commonly explained by the Media Equation. Fundamental to this theory is the idea that when faced with technology (like robots), people perceive it as a social agent with thoughts and intentions similar to those of humans. This perception guides the interaction with the technology and its predicted impact. However, HRI studies have also reported examples in which the Media Equation has been violated, that is when people treat the influence of robots differently from the influence of humans. To address this gap, we propose a model of Robot Social Influence (RoSI) with two contributing factors. The first factor is a robot’s violation of a person’s expectations, whether the robot exceeds expectations or fails to meet expectations. The second factor is a person’s social belonging with the robot, whether the person belongs to the same group as the robot or a different group. These factors are primary predictors of robots’ social influence and commonly mediate the influence of other factors. We review HRI literature and show how RoSI can explain robots’ social influence in concrete HRI scenarios.

One important aspect of effective human--robot collaborations is the ability for robots to adapt quickly to the needs of humans. While techniques like deep reinforcement learning have demonstrated success as sophisticated tools for learning robot policies, the fluency of human-robot collaborations is often limited by these policies' inability to integrate changes to a user's preferences for the task. To address these shortcomings, we propose a novel approach that can modify learned policies at execution time via symbolic if-this-then-that rules corresponding to a modular and superimposable set of low-level constraints on the robot's policy. These rules, which we call Transparent Matrix Overlays, function not only as succinct and explainable descriptions of the robot's current strategy but also as an interface by which a human collaborator can easily alter a robot's policy via verbal commands. We demonstrate the efficacy of this approach on a series of proof-of-concept cooking tasks performed in simulation and on a physical robot.

Social robots in the home will need to solve audio identification problems to better interact with their users. This article focuses on the classification between (a) natural conversation that includes at least one co-located user and (b) media that is playing from electronic sources and does not require a social response, such as television shows. This classification can help social robots detect a user’s social presence using sound. Social robots that are able to solve this problem can apply this information to assist them in making decisions, such as determining when and how to appropriately engage human users. We compiled a dataset from a variety of acoustic environments that contained either natural or media audio, including audio that we recorded in our own homes. Using this dataset, we performed an experimental evaluation on a range of traditional machine learning classifiers and assessed the classifiers’ abilities to generalize to new recordings, acoustic conditions, and environments. We conclude that a C-Support Vector Classification (SVC) algorithm outperformed other classifiers. Finally, we present a classification pipeline that in-home robots can utilize, and we discuss the timing and size of the trained classifiers as well as privacy and ethics considerations.

A growing population of adults with Autism Spec-trum Disorders (ASD) chronically struggles to find and maintain employment. Previous work reveals that one barrier to employment for adults with ASD is dealing with workplace interruptions. In this paper, we present our design and evaluations of an in-home autonomous robot system that aims to improve users' tolerance to interruptions. The Interruptions Skills Training and Assessment Robot (ISTAR) allows adults with ASD to practice handling interruptions to improve their employability. ISTAR is evaluated by surveys of employers and adults with ASD, and a week-long study in the homes of adults with ASD. Results show that users enjoy training with ISTAR, improve their ability to handle various work-relevant interruptions, and view the system as a valuable tool for improving their employment prospects.

In peer tutoring, the learner is taught by a colleague rather than by a traditional tutor. This strategy has been shown to be effective in human tutoring, where students have higher learning gains when taught by a peer instead of a traditional tutor. Similar results have been shown in child-robot interactions studies, where a peer robot was more effective than a tutor robot at teaching children. In this work, we compare skill increase and perception of a peer robot to a tutor robot when teaching adults. We designed a system in which a robot provides personalized help to adults in electronic circuit construction. We compare the number of learned skills and preferences of a peer robot to a tutor robot. Participants in both conditions improved their circuit skills after interacting with the robot. There were no significant differences in number of skills learned between conditions. However, participants with low prior domain knowledge learned significantly more with a peer robot than a tutor robot. Furthermore, the peer robot was perceived as friendlier, more social, smarter, and more respectful than the tutor robot, regardless of initial skill level.

In this paper, we introduce Ommie, a novel robot that supports deep breathing practices for the purposes of anxiety reduction. The robot’s primary function is to guide users through a series of extended inhales, exhales, and holds by way of haptic interactions and audio cues. We present core design decisions during development, such as robot morphology and tactility, as well as the results of a usability study in collaboration with a local wellness center. Interacting with Ommie resulted in a significant reduction in STAI-6 anxiety measures, and participants found the robot intuitive, approachable, and engaging. Participants also reported feelings of focus and companionship when using the robot, often elicited by the haptic interaction. These results show promise in the robot’s capacity for supporting mental health.

Robot-to-Human handovers are common exercises in many robotics application domains. The requirements of handovers may vary across these different domains. In this paper, we first devised a taxonomy to organize the diverse and sometimes contradictory requirements. Among these, task- oriented handovers were not well-studied but important because the purpose of the handovers in human–robot collaboration (HRC) is not merely to pass an object from a robot to a human receiver, but to enable the human receiver to use it in a subsequent tool-use task. A successful task-oriented handover should incorporate task-related information – orienting the tool such that the human can grasp it in a way that is suitable for the task. We identified multiple difficulty levels of task-oriented handovers, and implemented a system to generate task-oriented handovers with novel tools on a physical robot. Unlike previous studies on task-oriented handovers, we trained the robot with tool-use demonstrations rather than handover demonstrations, since task-oriented handovers are dependent on the tool usages in the subsequent task. We demonstrated that our method can adapt to all difficulty levels of task-oriented handovers, including tasks that matched the typical usage of the tool (level I), tasks that required an improvised and unusual usage of the tool (level II), and tasks where the handover was adapted to the pose of a manipulandum (level III). We evaluated the generated handovers with online surveys. Participants rated our handovers to appear more comfortable for the human receiver and more appropriate for subsequent tasks when compared with typical handovers from prior work.

Regular exercise provides many mental and physical health benefits. However, when exercises are done incorrectly, it can lead to injuries. Because the COVID-19 pandemic made it challenging to exercise in communal spaces, the growth of virtual fitness programs was accelerated, putting people at risk of sustaining exercise-related injuries as they received little to no feedback on their exercising techniques. Co-located robots could be one potential enhancement to virtual training programs as they can cause higher learning gains, more compliance, and more enjoyment than non-co-located robots. In this study, we compare the effects of a physically present robot by having a person exercise either with a robot (robot condition) or a video of a robot displayed on a tablet (tablet condition). Participants (N=25) had an exercise system in their homes for two weeks. Participants who exercised with the co-located robot made fewer mistakes than those who exercised with the video-displayed robot. Furthermore, participants in the robot condition reported a higher fitness increase and more motivation to exercise than participants in the tablet condition.

In this paper, we argue in favor of creating robots that both teach and learn. We propose a methodology for building robots that can learn a skill from an expert, perform the skill independently or collaboratively with the expert, and then teach the same skill to a novice. This requires combining insights from learning from demonstration, human-robot collaboration, and intelligent tutoring systems to develop knowledge representations that can be shared across all three components. As a case study for our methodology, we developed a glockenspiel-playing robot. The robot begins as a novice, learns how to play musical harmonies from an expert, collaborates with the expert to complete harmonies, and then teaches the harmonies to novice users. This methodology allows for new evaluation metrics that provide a thorough understanding of how well the robot has learned and enables a robot to act as an efficient facilitator for teaching across temporal and geographic separation.

The practice of social distancing during the COVID-19 pandemic resulted in billions of people quarantined in their homes. In response, we designed and deployed VectorConnect, a robot teleoperation system intended to help combat the effects of social distancing in children during the pandemic. VectorConnect uses the off-the-shelf Vector robot to allow its users to engage in physical play while being geographically separated. We distributed the system to hundreds of users in a matter of weeks. This paper details the development and deployment of the system, our accomplishments, and the obstacles encountered throughout this process. Also, it provides recommendations to best facilitate similar deployments in the future. We hope that this case study about Human-Robot Interaction practice serves as an inspiration to innovate in times of global crises.

Creating an accurate model of a user's skills is necessary for intelligent tutoring systems. Without an accurate model, sample problems or tasks must be selected haphazardly by the tutor. Once an accurate model has been trained, the tutor can selectively focus on training essential or deficient skills. Prior work offers mechanisms for optimizing the training of a single skill or for multiple skills when individual tasks involve testing only a single skill at a time, but not for multiple skills when individual tasks can contain evidence for multiple skills. In this paper, we present a system that estimates user skill models for multiple skills by selecting tasks which maximize the information gain across the entire skill model. We compare our system's policy against several baselines and an optimal policy in both simulated and real tasks. Our system outperforms baselines and performs almost on par with the optimal policy.

Studies have shown that people conform their answers to match those of group members even when they believe the group’s answer to be wrong [2]. In this experiment, we test whether people conform to groups of robots and whether the robots cause informational conformity (believing the group to be correct), normative conformity (feeling peer pressure), or both. We conducted an experiment in which participants (N = 63) played a subjective game with three robots. We measured humans’ conformity to robots by how many times participants changed their preliminary answers to match the group of robots’ in their final answer. Participants in conditions that were given more information about the robots’ answers conformed significantly more than those who were given less, indicating that informational conformity is present. Participants in conditions where they were aware they were a minority in their answers conformed more than those who were unaware they were a minority. Additionally, they also report feeling more pressure to change their answers from the robots, and the amount of pressure they reported was correlated to the frequency they conformed, indicating normative conformity. Therefore, we conclude that robots can cause both informational and normative conformity in people.

Inspired by the benefits of human prosocial behavior, we explore whether prosocial behavior can be extended to a Human-Robot Interaction (HRI) context. More specifically, we study whether robots can induce prosocial behavior in humans through a 1x2 between-subjects user study (N=30) in which a confederate abused a robot. Through this study, we investigated whether the emotional reactions of a group of bystander robots could motivate a human to intervene in response to robot abuse. Our results show that participants were more likely to prosocially intervene when the bystander robots expressed sadness in response to the abuse as opposed to when they ignored these events, despite participants reporting similar perception of robot mistreatment and levels of empathy for the abused robot. Our findings demonstrate possible effects of group social influence through emotional cues by robots in human-robot interaction. They reveal a need for further research regarding human prosocial behavior within HRI.

Team member inclusion is vital in collaborative teams. In this work, we explore two strategies to increase the inclusion of human team members in a human-robot team: 1) giving a person in the group a specialized role (the 'robot liaison') and 2) having the robot verbally support human team members. In a human subjects experiment (N = 26 teams, 78 participants), groups of three participants completed two rounds of a collaborative task. In round one, two participants (ingroup) completed a task with a robot in one room, and one participant (outgroup) completed the same task with a robot in a different room. In round two, all three participants and one robot completed a second task in the same room, where one participant was designated as the robot liaison. During round two, the robot verbally supported each participant 6 times on average. Results show that participants with the robot liaison role had a lower perceived group inclusion than the other group members. Additionally, when outgroup members were the robot liaison, the group was less likely to incorporate their ideas into the group's final decision. In response to the robot's supportive utterances, outgroup members, and not ingroup members, showed an increase in the proportion of time they spent talking to the group. Our results suggest that specialized roles may hinder human team member inclusion, whereas supportive robot utterances show promise in encouraging contributions from individuals who feel excluded.

While abstract knowledge like cause-and-effect relations enables robots to problem-solve in new environments, acquiring such knowledge remains out of reach for many traditional machine learning techniques. In this work, we introduce a method for a robot to learn an explicit model of cause-and-effect by constructing a structural causal model through a mix of observation and self-supervised experimentation, allowing a robot to reason from causes to effects and from effects to causes. We demonstrate our method on tool affordance learning tasks, where a humanoid robot must leverage its prior learning to utilize novel tools effectively. Our results suggest that after minimal training examples, our system can preferentially choose new tools based on the context, and can use these tools for goal-directed object manipulation.

Autonomous robots are increasingly placed in contexts that require them to interact with groups of people rather than just a single individual. Interactions with groups of people introduce nuanced challenges for robots, since robots? actions influence both individual group members and complex group dynamics. We review the unique roles robots can play in groups, finding that small changes in their nonverbal behavior and personality impacts group behavior and, by extension, influences ongoing interpersonal interactions.

Selecting appropriate tutoring help actions that account for both a student’s content mastery and engagement level is essential for effective human tutors, indicating the critical need for these skills in autonomous tutors. In this work, we formulate the robot-student tutoring help action selection problem as the Assistive Tutor partially observable Markov decision process (AT-POMDP). We designed the AT-POMDP and derived its parameters based on data from a prior robot-student tutoring study. The policy that results from solving the AT-POMDP allows a robot tutor to decide upon the optimal tutoring help action to give a student, while maintaining a belief of the student’s mastery of the material and engagement with the task. This approach is validated through a between-subjects field study, which involved 4th grade students (n=28) interacting with a social robot solving long division problems over five sessions. Students who received help from a robot using the AT-POMDP policy demonstrated significantly greater learning gains than students who received help from a robot with a fixed help action selection policy. Our results demonstrate that this robust computational framework can be used effectively to deliver diverse and personalized tutoring support over time for students.

The ability for autonomous agents to learn and conform to human norms is crucial for their safety and effectiveness in social environments. While recent work has led to frameworks for the representation and inference of simple social rules, research into norm learning remains at an exploratory stage. Here, we present a robotic system capable of representing, learning, and inferring ownership relations and norms. Ownership is represented as a graph of probabilistic relations between objects and their owners, along with a database of predicate-based norms that constrain the actions permissible on owned objects. To learn these norms and relations, our system integrates (i) a novel incremental norm learning algorithm capable of both one-shot learning and induction from specific examples, (ii) Bayesian inference of ownership relations in response to apparent rule violations, and (iii) perceptbased prediction of an object’s likely owners. Through a series of simulated and real-world experiments, we demonstrate the competence and flexibility of the system in performing object manipulation tasks that require a variety of norms to be followed, laying the groundwork for future research into the acquisition and application of social norms.

The Robots for Learning workshop series aims at advancing the research topics related to the use of social robots in educational contexts. This year's half-day workshop follows on previous events in Human-Robot Interaction conferences focusing on efforts to design, develop and test new robotics systems that help learners. This 5th edition of the workshop will be dealing in particular on the potential use of robots for adaptive learning. Since the past few years, inclusive education have been a key policy in a number of countries, aiming to provide equal changes and common ground to all. In this workshop, we aim to discuss strategies to design robotics system able to adapt to the learners' abilities, to provide assistance and to demonstrate long-term learning effects.

We conducted a study on whether dogs (Canis familiaris) perceive robots as agents, using the classic pointing paradigm in animal cognition research. While few studies to date have explored the pointing paradigm with robots, an initial study did not suggest dogs understood non-humanoid robot pointing. In this study, we tested 20 dogs with the humanoid robot Nao. Our results did not suggest that dogs understand humanoid robot pointing. We are currently working on revising the design and will conduct more experiments.

When a robot breaks a person's trust by making a mistake or failing, continued interaction will depend heavily on how the robot repairs the trust that was broken. Prior work in psychology has demonstrated that both the trust violation framing and the trust repair strategy influence how effectively trust can be restored. We investigate trust repair between a human and a robot in the context of a competitive game, where a robot tries to restore a human's trust after a broken promise, using either a competence or integrity trust violation framing and either an apology or denial trust repair strategy. Results from a 2×2 between-subjects study ( n=82) show that participants interacting with a robot employing the integrity trust violation framing and the denial trust repair strategy are significantly more likely to exhibit behavioral retaliation toward the robot. In the Dyadic Trust Scale survey, an interaction between trust violation framing and trust repair strategy was observed. Our results demonstrate the importance of considering both trust violation framing and trust repair strategy choice when designing robots to repair trust. We also discuss the influence of human-to-robot promises and ethical considerations when framing and repairing trust between a human and robot.

Current robotic terrain mapping techniques require expensive sensor suites to construct an environmental representation. In this work, we present a cube-shaped robot that can roll through unstructured terrain and construct a detailed topographic map of the surface that it traverses in real time with low computational and monetary expense. Our approach devolves many of the complexities of locomotion and mapping to passive mechanical features. Namely, rolling movement is achieved by sequentially inflating latex bladders that are located on four sides of the robot to destabilize and tip it. Sensing is achieved via arrays of fine plastic pins that passively conform to the geometry of underlying terrain, retracting into the cube. We developed a topography by shade algorithm to process images of the displaced pins to reconstruct terrain contours and elevation. We experimentally validated the efficacy of the proposed robot through object mapping and terrain locomotion tasks.

Personalized learning environments have the potential to improve learning outcomes for children in a variety of educational domains, as they can tailor instruction based on the unique learning needs of individuals. Robot tutoring systems can further engage users by leveraging their potential for embodied social interaction and take into account crucial aspects of a learner, such as a student’s motivation in learning. In this article, we demonstrate that motivation in young learners corresponds to observable behaviors when interacting with a robot tutoring system, which, in turn, impact learning outcomes. We first detail a user study involving children interacting one on one with a robot tutoring system over multiple sessions. Based on empirical data, we show that academic motivation stemming from one’s own values or goals as assessed by the Academic Self-Regulation Questionnaire (SRQ-A) correlates to observed suboptimal help-seeking behavior during the initial tutoring session. We then show how an interactive robot that responds intelligently to these observed behaviors in subsequent tutoring sessions can positively impact both student behavior and learning outcomes over time. These results provide empirical evidence for the link between internal motivation, observable behavior, and learning outcomes in the context of robot--child tutoring. We also identified an additional suboptimal behavioral feature within our tutoring environment and demonstrated its relationship to internal factors of motivation, suggesting further opportunities to design robot intervention to enhance learning. We provide insights on the design of robot tutoring systems aimed to deliver effective behavioral intervention during learning interactions for children and present a discussion on the broader challenges currently faced by robot--child tutoring systems.

Children with insufficient exposure to language during critical developmental periods in infancy are at risk for cognitive, language, and social deficits [55]. This is especially difficult for deaf infants, as more than 90% are born to hearing parents with little sign language experience [48]. We created an integrated multi-agent system involving a robot and virtual human designed to augment language exposure for 6-12 month old infants. Human-machine design for infants is challenging, as most screen-based media are unlikely to support learning in [33]. While presently, robots are incapable of the dexterity and expressiveness required for signing, even if it existed, developmental questions remain about the capacity for language from artificial agents to engage infants. Here we engineered the robot and avatar to provide visual language to effect socially contingent human conversational exchange. We demonstrate the successful engagement of our technology through case studies of deaf and hearing infants.

Thinking aloud, while requiring extra mental effort, is a metacognitive technique that helps students navigate through complex problem-solving tasks. Social robots, bearing embodied immediacy that fosters engaging and compliant interactions, are a unique platform to deliver problem-solving support such as thinking aloud to young learners. In this work, we explore the effects of a robot platform and the think-aloud strategy on learning outcomes in the context of a one-on-one tutoring interaction. Results from a 2x2 between-subjects study (n=52) indicate that both the robot platform and use of the think-aloud strategy promoted learning gains for children. In particular, the robot platform effectively enhanced immediate learning gains, measured right after the tutoring session, while the think-aloud strategy improved persistent gains as measured approximately one week after the interaction. Moreover, our results show that a social robot strengthened students» engagement and compliance with the think-aloud support while they performed cognitively demanding tasks. Our work indicates that robots can support metacognitive strategy use to effectively enhance learning and contributes to the growing body of research demonstrating the value of social robots in novel educational settings.

Asch's [2] conformity experiment has shown that people are prone to adjusting their view to match those of group members even when they believe the answer of the group to be wrong. Previous studies have attempted to replicate Asch's experiment with a group of robots but have failed to observe conformity [7, 25]. One explanation can be made using Hodges and Geyers work [17], in which they propose that people consider distinct criteria (truth, trust, and social solidarity) when deciding whether to conform to others. In order to study how trust and truth affect conformity, we propose an experiment in which participants play a game with three robots, in which there are no objective answers. We measured how many times participants changed their preliminary answers to match the group of robots' in their final answer. We conducted a between-subjects study (N = 30) in which there were two conditions: one in which participants saw the group of robots' preliminary answer before deciding their final answer, and a control condition in which they did not know the robots' preliminary answer. Participants in the experimental condition conformed significantly more (29%) than participants in the control condition (6%). Therefore we have shown that groups of robots can cause people to conform to them. Additionally trust plays a role in conformity: initially, participants conformed to robots at a similar rate to Asch's participants, however, many participants stop conforming later in the game when trust is lost due to the robots choosing an incorrect answer.

Successful teams are characterized by high levels of trust between team members, allowing the team to learn from mistakes, take risks, and entertain diverse ideas. We investigated a robot's potential to shape trust within a team through the robot's expressions of vulnerability. We conducted a between-subjects experiment (N = 35 teams, 105 participants) comparing the behavior of three human teammates collaborating with either a social robot making vulnerable statements or with a social robot making neutral statements. We found that, in a group with a robot making vulnerable statements, participants responded more to the robot's comments and directed more of their gaze to the robot, displaying a higher level of engagement with the robot. Additionally, we discovered that during times of tension, human teammates in a group with a robot making vulnerable statements were more likely to explain their failure to the group, console team members who had made mistakes, and laugh together, all actions that reduce the amount of tension experienced by the team. These results suggest that a robot's vulnerable behavior can have "ripple effects" on their human team members' expressions of trust-related behavior.

Research in human teamwork shows that a key element of fluid and fluent interactions is the interpretation of implicit verbal and non-verbal cues in context. This poses an issue to robotic platforms, however, as they have historically worked best when controlled through explicit commands that have employed structured, unequivocal representations of the external world and their human partners. In this work, we present a framework for effectively grounding situated and naturalistic speech to action selection during human-robot collaborative activities. This is accomplished by maintaining and incrementally updating separate “speech” and “context” models that jointly classify a collaborator's utterance. We evaluate the efficacy of the system on a collaborative construction task with an autonomous robot and human participants. We first demonstrate that our system is capable of acquiring and deploying new task representations from limited and naturalistic data sets, and without any prior domain knowledge of language or the task itself. Finally, we show that our system is capable of significantly improving performance on an unfamiliar task after a one-shot exposure.

Human-Robot Collaboration (HRC) aims to develop robots that provide assistance to human workers while performing physical tasks. Such assistance comes in the form of supportive behaviors that are different from the actions part of the task, and that are meant to help a human worker more effectively accomplish the task. Learning how to provide useful behaviors that are tailored to a human peer represents a difficult challenge. This is due to the need of large amounts of training data in the form of real world observations that include information about such preferences. This data needs to encode not only the structure and progression of the task, but also the different workers' preferences with respect to when and what assistance the robot should provide. Our work separates the challenge of learning a model of the task (which requires a large amount of training data) from that of learning supportive behavior preferences for the interaction (which has obvious restrictions for the number of user-provided demonstrations to which we have access). We first learn a hidden Markov model (HMM) from a training set consisting of observed human workers performing the considered task in simulation. We then use this model to predict, while observing the human peer, what supportive behaviors a robot should offer throughout the task. Building upon the hidden state representation, our system is able to learn the supportive behaviors based on as few as five user-annotated demonstrations, learning a personalized supportive behavior model. We evaluate our system on a user study with 14 participants, and show results on par with human-level prediction for the task.

In this paper, we present a model set for designing human-robot collaboration (HRC) experiments. It targets a common scenario in HRC, which is the collaborative assembly of furniture, and it consists of a combination of standard components and custom designs. With this work, we aim at reducing the amount of work required to set up and reproduce HRC experiments, and we provide a unified framework to facilitate the comparison and integration of contributions to the field. The model set is designed to be modular, extendable, and easy to distribute. Importantly, it covers the majority of relevant research in HRC, and it allows tuning of a number of experimental variables that are particularly valuable to the field. Additionally, we provide a set of software libraries for perception, control and interaction, with the goal of encouraging other researchers to proactively contribute to our work.

The benefits of personalized social robots must be evaluated in real-world educational contexts over periods of time longer than a single session to understand their full potential to impact learning outcomes. In this work, we describe a personalization system designed for longer-term personalization that orders curriculum based on an adaptive Hidden Markov Model (HMM) that evaluates students’ skill proficiencies. We present a study investigating the effectiveness of this system in a five-session interaction with a robot tutor, taking place over the course of 2 weeks. Our system is evaluated in the context of native Spanish-speaking first-graders interacting with a social robot tutor while completing an English Language Learning educational task. Participants either received lessons: (1) ordered by our adaptive HMM personalization system which selects a lesson based on a skill that the individual participant needs more practice with (“personalized condition”) or (2) ordered randomly from among the lessons the participant had not yet seen (“non-personalized condition”). We found that participants who received personalized lessons from the robot tutor outperformed participants who received non-personalized lessons on a post-test by 2.0 standard deviations on average, corresponding to a mean learning gain in the 98th percentile.

A common practice in education to accommodate the short attention spans of children during learning is to provide them with non-task breaks for cognitive rest. Holding great promise to promote learning, robots can provide these breaks at times personalized to individual children. In this work, we investigate personalized timing strategies for providing breaks to young learners during a robot tutoring interaction. We build an autonomous robot tutoring system that monitors student performance and provides break activities based on a personalized schedule according to performance. We conduct a field study to explore the effects of different strategies for providing breaks during tutoring. By comparing a fixed timing strategy with a reward strategy (break timing personalized to performance gains) and a refocus strategy (break timing personalized to performance drops), we show that the personalized strategies promote learning gains for children more effectively than the fixed strategy. Our results also reveal immediate benefits in enhancing efficiency and accuracy in completing educational problems after personalized breaks, showing the restorative effects of the breaks when administered at the right time.

Collaborative robots represent a clear added value to manufacturing, as they promise to increase productivity and improve working conditions of such environments. Although modern robotic systems have become safe and reliable enough to operate close to human workers on a day-to-day basis, the workload is still skewed in favor of a limited contribution from the robot's side, and a significant cognitive load is allotted to the human. We believe the transition from robots as recipients of human instruction to robots as capable collaborators hinges around the implementation of transparent systems, where mental models about the task are shared between peers, and the human partner is freed from the responsibility of taking care of both actors. In this work, we implement a transparent task planner able to be deployed in realistic, near-future applications. The proposed framework is capable of basic reasoning capabilities for what concerns role assignment and task allocation, and it interfaces with the human partner at the level of abstraction he is most comfortable with. The system is readily available to non-expert users, and programmable with high-level commands in an intuitive interface. Our results demonstrate an overall improvement in terms of completion time, as well as a reduced cognitive load for the human partner.

This article reviews the state of the art in social eye gaze for human-robot interaction (HRI). It establishes three categories of gaze research in HRI, defined by differences in goals and methods: a human-centered approach, which focuses on people's responses to gaze; a design-centered approach, which addresses the features of robot gaze behavior and appearance that improve interaction; and a technology-centered approach, which is concentrated on the computational tools for implementing social eye gaze in robots. This paper begins with background information about gaze research in HRI and ends with a set of open questions.

Far infrared thermography, which can be used to detect thermal radiation emitted by humans, has been used to detect physical disease, physiological changes relating to emotion, and polygraph testing, but has not been used for eye tracking. However, because the surface temperature of the cornea is colder than the limbus, it is theoretically possible to track corneal movements through thermal imaging. To explore the feasibility of thermal eye tracking, we invited 10 adults and tracked their corneal movements with passive thermal imaging at 60 Hz. We combined shape models of eyes with intensity threshold to segment the cornea from other parts of the eye in thermal images. We used an animation sequence as a calibration target for 5 point calibration/validation 5 times. Our results were compared to simultaneously collected data using an SR EyeLink eye tracker at 500 Hz, demonstrating the feasibility of eye tracking with thermal images. Blinking and breathing frequencies, which reflect the psychophysical status of the participants, were also robustly detected during thermal eye tracking.

Nonverbal behaviors increase task efficiency and improve collaboration between people and robots. In this paper, we introduce a model for generating nonverbal behavior and investigate whether the usefulness of nonverbal behaviors changes based on task difficulty. First, we detail a robot behavior model that accounts for top-down and bottom-up features of the scene when deciding when and how to perform deictic references (looking or pointing). Then, we analyze how a robot's deictic nonverbal behavior affects people's performance on a memorization task under differing difficulty levels. We manipulate difficulty in two ways: by adding steps to memorize, and by introducing an interruption. We find that when the task is easy, the robot's nonverbal behavior has little influence over recall and task completion. However, when the task is challenging— because the memorization load is high or because the task is interrupted—a robot's nonverbal behaviors mitigate the negative effects of these challenges, leading to higher recall accuracy and lower completion times. In short, nonverbal behavior may be even more valuable for difficult collaborations than for easy ones.

Robots are often employed to proactively engage children with Autism Spectrum Disorder (ASD) in well-defined physical or social activities to promote specific educational or therapeutic outcomes. However, much can also be learned by leveraging a robot's unique ability to objectively deliver stimuli in a consistent, repeatable way and record child-robot interactions that may be indicative of developmental ability and autism severity in this population. In this study, we elicited affective responses with an emotion-simulating robot and recorded child-robot interactions and child-other interactions during robot emotion states. This research makes two key contributions. First, we analyzed child-robot interactions and affective responses to an emotion-simulating robot to explore differences between the responses of typically developing children and children with Autism Spectrum Disorder (ASD). Next, we characterized play and affective responsivity and its connection to severity of autism symptoms using the Autism Diagnostic Observation Schedule (ADOS) calibrated severity scores. This preliminary work delivers a novel and robust robot-enabled technique for (1) differentiating child-robot interactions of a group of very young children with ASD (n=12) from a group of typically developing children (n=15) and, (2) characterizing within-group differences in play and affective response that may be associated with symptoms of autism severity.

Current state-of-the-art robotic systems deployed in industry work in isolation from humans and do not allow for collaboration. Developing a robot that can work side-by-side with a human presents the advantage of allowing both the robot and the human worker to focus on the task each is best suited for, while assisting one another as needed. For the robot to provide assistive behavior to a human co-worker, it needs to learn what actions it should perform at each time step depending upon the state of the task. Such assistive actions are not intended to simply contribute to the completion of a particular task by instructing the robot to work on subtasks in isolation from the human worker; rather they are meant to help the worker complete the task more efficiently. As such, employing standard policy search or task and motion planning techniques is not sufficient to discover the supportive types of actions my system seeks to offer based on accurate estimations of the current task state. To this end, my research focuses on investigating policy search within hierarchical tasks that allow for two main abilities, namely helping the human co-worker more effectively complete a task and taking communicative actions that reduce state estimation uncertainty by asking the worker direct questions. The policy dictates what action the robot should take at each time step, based on inputs from a motion capture system providing observations about the configuration of the person's hands relative to the objects needed for accomplishing the task, as well as the person's answers to any questions posed by the robot.

In intelligent tutoring systems, one fundamental problem that limits learning gains is the unproductive use of on-demand help features, namely overuse or aversion, resulting in students misusing the system rather than engaging in active learning. Social robots as tutoring agents have the potential to mitigate those behaviors by actively shaping productive help-seeking behaviors. We hypothesize that effectual help-seeking behavior is a critical contributor to learning gains in a robot-child tutoring interaction. We conduct a between-subjects study where children interacted with a social robot solving fractions problems over multiple sessions (29 children; 4 sessions per child) in one of two groups. Results showed that participants in our experimental group, who received adaptive shaping strategies from the robot targeting suboptimal help requests, reduced their suboptimal behaviors over time significantly more than a control group, as well as improved their scores from pretest to posttest significantly more than a control group.

As robotic technology becomes more robust and interactive, robots are increasingly stepping into the role of a collaborator to humans in various contexts. In addition to performing collaborative tasks accurately and efficiently, robots should also contribute socially by improving team effectiveness and cohesion. This work is a first step toward developing a high-level reasoning model of the motivations and strategies held by each individual of the team. With this model, social robots will be able to promote more efficient and enjoyable collaboration by suggesting improvements to specific actions, aligning diverging strategies, and encouraging actions that promote higher team cohesiveness.

This paper presents a model that uses a robot's verbal and nonverbal behaviors to successfully communicate object references to a human partner. This model, which is informed by computer vision, human-robot interaction, and cognitive psychology, simulates how low-level and high-level features of the scene might draw a user's attention. It then selects the most appropriate robot behavior that maximizes the likelihood that a user will understand the correct object reference while minimizing the cost of the behavior. We present a general computational framework for this model, then describe a specific implementation in a human-robot collaboration. Finally, we analyze the model's performance in two human evaluations—one video-based (75 participants) and one in person (20 participants)—and demonstrate that the system predicts the correct behaviors to perform successful object references.

Collaboration between humans and robots requires solutions to an array of challenging problems, including multi-agent planning, state estimation, and goal inference. There already exist feasible solutions for many of these challenges, but they depend upon having rich task models. In this work we detail a novel type of Hierarchical Task Network we call a Clique/Chain HTN (CC-HTN), alongside an algorithm for autonomously constructing them from topological properties derived from graphical task representations. As the presented method relies on the structure of the task itself, our work imposes no particular type of symbolic insight into motor primitives or environmental representation, making it applicable to a wide variety of use cases critical to human-robot interaction. We present evaluations within a multi-resolution goal inference task and a transfer learning application showing the utility of our approach.

In their expanding role as tutors, home and healthcare assistants, robots must effectively interact with individuals of varying ability and temperament. Indeed, deploying robots in long-term social engagements will almost certainly require robots to reliably detect and adapt to changes in the demeanor of social partners to promote trust and more productive collaboration. However, the recognition of emotional state typically relies on the interpretation of very subtle cues, often varying from one person to the next. In addition, while facial expressions, body posture and features of speech have been used to detect affective changes, the robustness of these measures is often hindered by cultural and age differences. Recently, infrared thermography has shown promise in detecting guilt, fear and stress, indicating that it may be a viable sensing modality for improved human-robot interaction. In this study, we evaluated the efficacy of using a far infrared (FIR) camera for detecting robot-elicited affective response compared to video-elicited affective response by tracking thermal changes in five areas of the face. Further, we analyzed localized changes in the face to assess whether thermal and electrodermal responses to emotions elicited by traditional video techniques and by robots are similar. Finally, we performed principal component analysis to reduce the dimensionality of data and evaluated the performance using machine learning techniques for classifying thermal data by emotion state, resulting in a thermal classifier with a performance accuracy of 77.5%.

As research on robotic tutors increases, it becomes more relevant to understand whether and how robots will be able to keep students engaged over time. In this paper, we propose an algorithm to monitor engagement in small groups of children and trigger disengagement repair interventions when necessary. We implemented this algorithm in a scenario where two robot actors play out interactive narratives around emotional words and conducted a field study where 72 children interacted with the robots three times in one of the following conditions: control (no disengagement repair), targeted (interventions addressing the child with the highest disengagement level) and general (interventions addressing the whole group). Surprisingly, children in the control condition had higher narrative recall than in the two experimental conditions, but no significant differences were found in the emotional interpretation of the narratives. When comparing the two different types of disengagement repair strategies, participants who received targeted interventions had higher story recall and emotional understanding, and their valence after disengagement repair interventions increased over time.

Despite the growing body of research in human-robot collaboration, there has been little focus on how social robots can support human-to-human teaming. In this paper, we investigate whether a social robot can improve human-human collaboration. We conducted a between-subjects study where pairs of children play a collaborative game with a social robot. During pauses in the game, the robot either (1) asks the children questions to better focus the participants on the task they are working on, (2) asks the children questions that are targeted at developing and reinforcing the relationship between the participants, or (3) doesn't ask any questions. Our results show that participants who were asked task-focused questions had higher performance scores in the collaborative game than the other groups, however, had a lower perception of their performance than the participants who were asked relationally-focused questions. We did not find any differences between the groups in interpersonal cohesiveness. Our findings suggest that social robots can be used to improve performance measures and perception of performance in groups of children.

Mimicry, the automatic imitation of gestures, postures, mannerisms, and other motor movements, has been shown to be a critical component of human interaction but needs further exploration in human-robot interaction. Understanding mimicry is important for building better robots, learning about human categorization of robots in social ingroups/outgroups, and understanding social contagion in human-robot interaction. We investigate the extent to which humans will mimic a robot during the task of describing paintings by comparing the time participants put their hands on their hips before and after observing a robot cue that behavior. We observed no significant difference in participants' hands on hips time before and after the robot's cue. However, we did find that some participants performed the behavior more after the robot's cue while others performed it less. Furthermore, the direction of this change was a function of whether or not a participant performed the specified behavior prior to the robot's cue. This was similarly observed both for frequency of behavior performance and for a second behavior (hands behind back). As such, this study informs future research on human-robot mimicry, particularly on the importance of prior behavior during a human-robot interaction. In doing so, this study provides a baseline for further understanding and exploring mimicry in human-robot interaction as well as evidence for a social component in human-robot mimicry.

This research employs an expressive robot to elicit affective response in young children and explore correlations between autonomously-detected play, affective response and developmental ability. In this study, we introduce a new, affective interface that combines sound, color, movement and context to simulate the expression of emotions. Our approach exploits social contingencies to emphasize the importance of situational cues in the proper interpretation of affective state. We studied a group of young children at various ages and stages of cognitive development, to: (1) evaluate the efficacy of using captured motion data to autonomously detect physical patterns of play while interacting with a robot, (2) examine relationships between physical play patterns and observed affective response and, (3) explore associations between developmental ability and play or affective response. This pilot study demonstrates that aggregate patterns of physical interaction with a robot are distinguishable through autonomous data collection. Further, statistical analyses demonstrates that developmental ability may be directly related to how a child interacts with and responds to an affective robot.

While interactive technologies frequently are designed to be enjoyable, there are particular reasons to prioritize this for technologies intended to support autism interventions. Most broadly, enjoyment of activities or materials used in interventions has been associated with heightened improvements in the behaviors targeted by the interventions. In the largest group study to date of school-aged children with high-functioning autism (N=24), we present evidence of more positive affect elicited with a robot than with an adult, during matched triadic interactions designed to facilitate social and conversational interaction with a clinician. Robot-mediated increases in positive affect were found to be associated with production of spoken language directed to the clinician during robot interaction. We further found that robot-mediated increases in positive affect were associated with greater autism severity, particularly in the social affect domain, and with lower nonverbal IQ. Our findings suggest that robots may have a unique advantage in interventions for children with autism spectrum disorders by eliciting more positive affect, and that we should explore robot support for interventions with lower-functioning, affected individuals.

Robot assistive technology is becoming increasingly prevalent. Despite the growing body of research in this area, the role of type of interaction (i.e., small groups versus individual interactions) on effectiveness of interventions is still unclear. In this paper, we explore a new direction for socially assistive robotics, where multiple robotic characters interact with children in an interactive storytelling scenario. We conducted a between-subjects repeated interaction study where a single child or a group of three children interacted with the robots in an interactive narrative scenario. Results show that although the individual condition increased participant's story recall abilities compared to the group condition, the emotional interpretation of the story content seemed more dependent on the difficulty level rather than the study condition. Our findings suggest that, despite the type of interaction, interactive narratives with multiple robots are a promising approach to foster children's development of social-related skills.

Changes in type of interaction (e.g., individual vs. group interactions) can potentially impact data-driven models developed for social robots. In this paper, we provide a first investigation in the effects of changing group size in data-driven models for HRI, by analyzing how a model trained on data collected from participants interacting individually performs in test data collected from group interactions, and vice-versa. Another model combining data from both individual and group interactions is also investigated. We perform these experiments in the context of predicting disengagement behaviors in children interacting with two social robots. Our results show that a model trained with group data generalizes better to individual participants than the other way around. The mixed model seems a good compromise, but it does not achieve the performance levels of the models trained for a specific type of interaction.

Short et al. found that in a game between a human participant and a humanoid robot, the participant will perceive the robot as being more agentic and as having more intentionality if it cheats than if it plays without cheating. However, in that design, the robot that actively cheated also generated more motion than the other conditions. In this paper, we investigate whether the additional movement of the cheating gesture is responsible for the increased agency and intentionality or whether the act of cheating itself triggers this response. In a between-participant design with 83 participants, we disambiguate between these causes by testing (1) the cases of the robot cheating to win, (2) cheating to lose, (3) cheating to tie from a winning position, and (4) cheating to tie from a losing position. Despite the fact that the robot changes its gesture to cheat in all four conditions, we find that participants are more likely to report the gesture change when the robot cheated to win from a losing position, compared with the other conditions. Participants in that same condition are also far more likely to protest in the form of an utterance following the cheat and report that the robot is less fair and honest. It is therefore the adversarial cheat itself that causes the effect and not the change in gesture, providing evidence for a cheating detector that can be triggered by robots.

As social robots become more widespread in educational environments, their ability to understand group dynamics and engage multiple children in social interactions is crucial. Social dominance is a highly influential factor in social interactions, expressed through both verbal and nonverbal behaviors. In this paper, we present a method for determining whether a participant is high or low in social dominance in a group interaction with children and robots. We investigated the correlation between many verbal and nonverbal behavioral features with social dominance levels collected through teacher surveys. We additionally implemented Logistic Regression and Support Vector Machines models with classification accuracies of 81% and 89%, respectively, showing that using a small subset of nonverbal behavioral features, these models can successfully classify children's social dominance level. Our approach for classifying social dominance is novel not only for its application to children, but also for achieving high classification accuracies using a reduced set of nonverbal features that, in future work, can be automatically extracted with current sensing technology.

In this work, we present an algorithm for improving collaborator performance on sequential manipulation tasks. Our agent-decoupled, optimization-based, task and motion planning approach merges considerations derived from both symbolic and geometric planning domains. This results in the generation of supportive behaviors enabling a teammate to reduce cognitive and kinematic burdens during task completion. We describe our algorithm alongside representative use cases, with an evaluation based on solving complex circuit building problems. We conclude with a discussion of applications and extensions to human-robot teaming scenarios.

Fixation identification algorithms facilitate data comprehension and provide analytical convenience in eye-tracking analysis. However, current fixation algorithms for eye-tracking analysis are heavily dependent on parameter choices, leading to instabilities in results and incompleteness in reporting. This work examines the nature of human scanning patterns during complex scene viewing. We show that standard implementations of the commonly used distance-dispersion algorithm for fixation identification are functionally equivalent to greedy spatiotemporal tiling. We show that modeling the number of fixations as a function of tiling size leads to a measure of fractal dimensionality through box counting. We apply this technique to examine scale-free gaze behaviors in toddlers and adults looking at images of faces and blocks, as well as large number of adults looking at movies or static images. The distributional aspects of the number of fixations may suggest a fractal structure to gaze patterns in free scanning and imply that the incompleteness of standard algorithms may be due to the scale-free behaviors of the underlying scanning distributions. We discuss the nature of this hypothesis, its limitations, and offer directions for future work.

As assistive robots become popular in factories and homes, there is greater need for natural, multi-channel communication during collaborative manipulation tasks. Non-verbal communication such as eye gaze can provide information without overloading more taxing channels like speech. However, certain collaborative tasks may draw attention away from these subtle communication modalities. For instance, robot-to-human handovers are primarily manual tasks, and human attention is therefore drawn to robot hands rather than to robot faces during handovers. In this paper, we show that a simple manipulation of a robot's handover behavior can significantly increase both awareness of the robot's eye gaze and compliance with that gaze. When eye gaze communication occurs during the robot's release of an object, delaying object release until the gaze is finished draws attention back to the robot's head, which increases conscious perception of the robot's communication. Furthermore, the handover delay increases peoples' compliance with the robot's communication over a non-delayed handover, even when compliance results in counterintuitive behavior.

In education research, there is a widely-cited result called "Bloom's two sigma" that characterizes the differences in learning outcomes between students who receive one-on-one tutoring and those who receive traditional classroom instruction. Tutored students scored in the 95th percentile, or two sigmas above the mean, on average, compared to students who received traditional classroom instruction. In human-robot interaction research, however, there is relatively little work exploring the potential benefits of personalizing a robot's actions to an individual's strengths and weaknesses. In this study, participants solved grid-based logic puzzles with the help of a personalized or non-personalized robot tutor. Participants' puzzle solving times were compared between two non-personalized control conditions and two personalized conditions (n=80). Although the robot's personalizations were less sophisticated than what a human tutor can do, we still witnessed a "one-sigma" improvement (68th percentile) in post-tests between treatment and control groups. We present these results as evidence that even relatively simple personalizations can yield significant benefits in educational or assistive human-robot interactions.

Socially assistive robotics (SAR) aims to develop robots that help people through interactions that are inherently social, such as tutoring and coaching. For these interactions to be effective, socially assistive robots must be able to recognize and use nonverbal social cues like eye gaze and gesture. In this paper, we present a preliminary model for nonverbal robot behavior in a tutoring application. Using empirical data from teachers and students in human-human tutoring interactions, the model can be both predictive (recognizing the context of new nonverbal behaviors) and generative (creating new robot nonverbal behaviors based on a desired context) using the same underlying data representation.

Effective robot collaborators that work with humans require an understanding of the underlying constraint network of any joint task to be performed. Discovering this network allows an agent to more effectively plan around co-worker actions or unexpected changes in its environment. To maximize the practicality of collaborative robots in real-world scenarios, humans should not be assumed to have an abundance of either time, patience, or prior insight into the underlying structure of a task when relied upon to provide the training required to impart proficiency and understanding. This work introduces and experimentally validates two demonstration-based active learning strategies that a robot can utilize to accelerate context-free task comprehension. These strategies are derived from the action-space graph, a dual representation of a Semi-Markov Decision Process graph that acts as a constraint network and informs query generation.We present a pilot study showcasing the effectiveness of these active learning algorithms across three representative classes of task structure. Our results show an increased effectiveness of active learning when utilizing feature-based query strategies, especially in multi-instructor scenarios, achieving better task comprehension from a relatively small quantity of training demonstrations. We further validate our results by creating virtual instructors from a model of our pilot study participants, and applying it to a set of 12 more complex, real world food preparation tasks with similar results.

Robots that engage in social behaviors benefit greatly from possessing tools that allow them to manipulate the course of an interaction. Using a non-anthropomorphic social robot and a simple counting game, we examine the effects that empathy-generating robot dialogue has on participant performance across three conditions. In the self-directed condition, the robot petitions the participant to reduce his or her performance so that the robot can avoid punishment. In the externally-directed condition, the robot petitions on behalf of its programmer so that its programmer can avoid punishment. The control condition does not involve any petitions for empathy. We find that externally-directed petitions from the robot show a higher likelihood of motivating the participant to sacrifice his or her own performance to help, at the expense of incurring negative social effects. We also find that experiencing these emotional dialogue events can have complex and difficult to predict effects, driving some participants to antipathy, leaving some unaffected, and manipulating others into feeling empathy towards the robot.

This paper describes an extended (6-session) interaction between an ethnically and geographically diverse group of 26 first-grade children and the DragonBot robot in the context of learning about healthy food choices. We find that children demonstrate a high level of enjoyment when interacting with the robot, and a statistically significant increase in engagement with the system over the duration of the interaction. We also find evidence of relationship-building between the child and robot, and encouraging trends towards child learning. These results are promising for the use of socially assistive robotic technologies for long-term one-on-one educational interventions for younger children.

The ability to use a mirror as an instrument for spatial reasoning enables an agent to make meaningful inferences about the positions of objects in space based on the appearance of their reflections in mirrors. The model presented in this paper enables a robot to infer the perspective from which objects reflected in a mirror appear to be observed, allowing the robot to use this perspective as a virtual camera. Prior work by our group presented an architecture through which a robot learns the spatial relationship between its body and visual sense, mimicking an early form of self-knowledge in which infants learn about their bodies and senses through their interactions with each other. In this work, this self-knowledge is utilized in order to determine the mirror's perspective. Witnessing the position of its end-effector in a mirror in several distinct poses, the robot determines a perspective that is consistent with these observations. The system is evaluated by measuring how well the robot's predictions of its end-effector's position in 3D, relative to the robot's egocentric coordinate system, and in 2D, as projected onto it's cameras, match measurements of a marker tracked by its stereo vision system. Reconstructions of the 3D position end-effector, as computed from the perspective of the mirror, are found to agree with the forward kinematic model within a mean of 31.55mm. When observed directly by the robot's cameras, reconstructions agree within 5.12mm. Predictions of the 2D position of the end-effector in the visual field agree with visual measurements within a mean of 18.47 pixels, when observed in the mirror, or 5.66 pixels, when observed directly by the robot's cameras.

While there is a rich history of studies involving robots and individuals with autism spectrum disorders (ASD), few of these studies have made substantial impact in the clinical research community. In this paper we first examine how differences in approach, study design, evaluation, and publication practices have hindered uptake of these research results. Based on ten years of collaboration, we suggest a set of design principles that satisfy the needs (both academic and cultural) of both the robotics and clinical autism research communities. Using these principles, we present a study that demonstrates a quantitatively measured improvement in human-human social interaction for children with ASD, effected by interaction with a robot.

The intertwining of people and robots is at the very heart of human-robot interaction. The design of effective human-robot interactions depends not only upon an elegant and thorough engineering design for the robot's behavior, but also a clear and insightful understanding of how the human user is likely to behave toward the robot. This presents a challenge and an opportunity for engaging robotics systems research with the cognitive, behavioral, and social sciences. Without a deeper understanding of human user contexts, real user needs, and an ability to evaluate a robotic system's performance in terms of user needs, robotics runs the risk of inventing technologies for the sake of the technologies themselves. Without a thorough understanding of the state-of-the-art in robotics, user research runs the risk of generating superficial or irrelevant recommendations for robotic system design.

In this work we identify low-level aspects of robot motion that can be exploited to create impressions of agency and lifelikeness. In two experiments, participants view split-screen videos of multiple robots set to music and rate the robots on their dance ability, lifelikeness, and entertainment value. The first experiment tests the impact of the correspondence (or lack thereof) of the robot's motion to the underlying rhythm of the music, and the effect of matching changes in the robot's movement to changes in the music, such as a phrase of vocals or drumming. This motivates a second experiment which more deeply explores the relationships of asynchrony and changes in motion repertoire to participants' perceptions of the lifelikeness of the robot's motion. Findings indicate that perceptions of the lifelikeness of the robot and the quality of the dance can be manipulated by simple changes, such as variation in the repertoire of motions, coordination of changes in behavior with events in the music, and the addition of flaws to the robot's synchrony with the music.

Does the emotional content of a robot's speech affect how people teach it? In this experiment, participants were asked to demonstrate several "dances" for a robot to learn. Participants moved their bodies in response to instructions displayed on a screen behind the robot. Meanwhile, the robot faced the participant and appeared to emulate the participant's movements. After each demonstration, the robot received an accuracy score and the participant chose whether or not to demonstrate that dance again. Regardless of the participant's input, however, the robot's dancing and the scores it received were arranged in advance and constant across all participants. The only variation between groups in this study was what the robot said in response to its scores. Participants saw one of three conditions: appropriate emotional responses, often-inappropriate emotional responses, or apathetic responses. Participants that taught the robot with appropriate emotional responses demonstrated the dances, on average, significantly more frequently and significantly more accurately than participants in the other two conditions.

This paper presents preliminary research investigating whether preschool children (ages four to six years old) would be as comfortable sharing a secret they had been told not to share, with a humanoid robot as they would an adult, to explore the possible future use of robots to gather sensitive information from children that may have experienced maltreatment. The children in this research played the game “follow-the-leader” with an adult and a humanoid robot. As part of this research, the lead investigator shared a unique secret with each child. During a break in the “follow-the-leader” game with the adult and the robot, the children were prompted with five questions to determine if they would share the secret they were told by the investigator. The qualitative results from the study indicate that the children were as likely to share the secret with the robot as the adult with a similar amount of prompting effort. Additionally, the children interacted with the robot using similar social conventions (e.g., greeting, turn-taking, etc) as observed in their interactions with the adult.

Using a humanoid robot and a simple children's game, we examine the degree to which variations in behavior result in attributions of mental state and intentionality. Participants play the well-known children's game "rock-paper-scissors" against a robot that either plays fairly, or that cheats in one of two ways. In the "verbal cheat" condition, the robot announces the wrong outcome on several rounds which it loses, declaring itself the winner. In the "action cheat"' condition, the robot changes its gesture after seeing its opponent's play. We find that participants display a greater level of social engagement and make greater attributions of mental state when playing against the robot in the conditions in which it cheats.

No abstract available.

We examine affective vocalizations provided by human teachers to robotic learners. In unscripted one-on-one interactions, participants provided vocal input to a robotic dinosaur as the robot selected toy buildings to knock down. We find that (1) people vary their vocal input depending on the learner's performance history, (2) people do not wait until a robotic learner completes an action before they provide input and (3) people naively and spontaneously use intensely affective prosody. Our findings suggest modifications may be needed to traditional machine learning models to better fit observed human tendencies. Our observations of human behavior contradict the popular assumptions made by machine learning algorithms (in particular, reinforcement learning) that the reward function is stationary and path-independent for social learning interactions.

The Wizard of Oz experiment method has a long tradition of acceptance and use within the field of human-robot interaction. The community has traditionally downplayed the importance of interaction evaluations run with the inverse model: the human simulated to evaluate robot behavior, or Oz of Wizard. We argue that such studies play an important role in the field of human-robot interaction. We differentiate between methodologically rigorous human modeling and placeholder simulations using simplified human models. Guidelines are proposed for when Oz of Wizard results should be considered acceptable. This paper also describes a framework for describing the various permutations of Wizard and Oz states.

In this paper we evaluate several of the most popular algorithms for segmenting fixations from saccades by testing these algorithms on the scanning patterns of toddlers. We show that by changing the parameters of these algorithms we change the reported fixation durations in a systematic fashion. However, we also show how choices in analysis can lead to very different interpretations of the same eye-tracking data. Methods for reconciling the disparate results of different algorithms as well as suggestions for the use of fixation identification algorithms in analysis, are presented.

This study explores how a robotpsilas physical or virtual presence affects unconscious human perception of the robot as a social partner. Subjects collaborated on simple book-moving tasks with either a physically present humanoid robot or a video-displayed robot. Each task examined a single aspect of interaction: greetings, cooperation, trust, and personal space. Subjects readily greeted and cooperated with the robot in both conditions. However, subjects were more likely to fulfill an unusual instruction and to afford greater personal space to the robot in the physical condition than in the video-displayed condition. The same tendencies occurred when the virtual robot was supplemented by disambiguating 3-D information.

The difference between self-organizing maps based phoneme classifiers that emerge for different input languages is studied. For each such language a self-organizing map is trained on mel-frequency cepstral coefficient (MFCC) converted auditory input to form a phoneme classifier. Unsupervised learning is used as the training method. The emerging classes are then compared to the classes found in the International Phonetic Alphabet. Particular class differences across languages and speakers are discussed. We show that SOMs adapt to speakers and languages, even when only given a small training data-set. Additionally, we show that some neurons in SOMs react only to input in one of the two trained languages and that some neurons can be used as word boundary classifiers.

We present a method of grounded word learning that can learn the meanings of first and second person pronouns. The model selectively associates new words with agents in the environment by using already understood words to establish context. The method uses chi-square tests to find significant associations between the new words and attributes of the relevant agents. We show that this model can learn from a transcript of a parent-child interaction that "I" refers to the person who is speaking. With the additional information that questions about wants refer to the person being asked about them, the system learns that "you" refers to the person being addressed. We show that an incorrect assumption about the subject of "want" questions can lead to pronoun reversal, a linguistic error most commonly found in autistic and congenitally blind children. Finally, we present results from a physical implementation on a robot that runs in real time.

This paper examines human perceptions of humanoid robot behavior, specifically how perception is affected by variations in head tracking behavior under constant gestural behavior. Subjects were invited to the lab to "play with Nico," an upper-torso humanoid robot. The follow-up survey asked subjects to rate and write about the experience. A coding scheme originally created to gauge human intentionality was applied to written responses to measure the level of intentionality that subjects perceived in the robot. Subjects were presented with one of four variations of head movement: a motionless head, a smooth tracking head, a tracking head without smoothed movements, and an avoidance behavior, while a pre-scripted wave and beckon sequence was carried out in all cases. Surprisingly, subjects rated the interaction as most enjoyable and Nico as possessing more intentionality when avoidance and unsmooth tracking were used. These data suggest that naïve users of robots may prefer caricatured and exaggerated behaviors to more natural ones. Also, correlations between ratings across modes suggest that simple features of robot behavior reliably evoke notable changes in many perception scales.

Previous research has suggested that developmental learning can make the learning of advanced sensorimotor and cognitive skills possible. In this paper, we demonstrate that developmental learning based on skill progression is also more efficient than traditional divide-and-conquer methods. Using a model based on the skills of reaching and pointing to visual targets, we demonstrate an implementation for a humanoid robot that is more efficient at learning joint attention skills than other published methods. This efficiency results from (1) a structured set of learning tasks that progresses from low-dimensional to high-dimensional problems and (2) a greater exploitation of the learning environment that does not follow from the completely task-based decomposition that divide-and-conquer provides.

Computational models of development aim to describe the mechanisms that underlie the acquisition of new skills or the emergence of new capabilities. The strength of a model is judged by both its ability to explain the phenomena in question as well as its ability to generate new hypotheses, generalize to new situations, and provide a unifying conceptual framework. Although often constructed using traditional engineering methodologies, evaluating the performance of a computational model of development in terms of traditional perspectives is a flawed approach. This paper addresses the fundamental issues that confound quantitative analysis of computational models of developmental systems. In particular we focus on the following recommendations: 1) don't equate the success of a developmental model with its peak performance at some task; 2) don't employ purely subjective or vague measures of model fitness; and 3) don't hide or reject variation as found in the computational model. Along the way, we discuss the aspects of computational models of development that lead to the requirements for specialized methods of analysis.

Music performance is an important, well-structured setting for evaluating a robot's ability to detect, understand and respond appropriately to complex human activity. Social tasks such as cooperative performance require participants to detect, interpret and attune to the actions of their partners quickly and accurately. The synthesis of multiple sensory perceptions may be a fruitful approach to this problem. In order to evaluate this approach, we programmed a humanoid robot, Nico, to play a drum in concert with human drummers and at the direction of a human conductor. Our results show that sensory integration can enable precise synchronization in social tasks even when perceptual data is imperfect, misleading and subject to extensive processing delay. By integrating several streams of information - visual, auditory, and proprioceptive - Nico can attune to a tempo that is set by a human conductor, in concert with human performers. Nico continuously evaluates its perceptions of its own actions and those of the humans around it, dealing with unforeseen changes in tempo and affect in real time

By learning a range of possible times over which the effect of an action can take place, a robot can reason more effectively about causal and contingent relationships in the world. An algorithm is presented for learning the interval of possible times during which a response to an action can take place. The algorithm was implemented on a physical robot for the domains of visual self-recognition and auditory social-partner recognition. The environment model assumes that natural environments generate Poisson distributions of random events at all scales. A linear-time algorithm called Poisson threshold learning can generate a threshold T that provides an arbitrarily small rate of background events λ (T), if such a threshold exists for the specified error rate.

We present a method for allowing a humanoid robot to recognize its own motion in its visual field, thus enabling it to distinguish itself from other agents in the vicinity. Our approach consists of learning a characteristic time window between the initiation of motor movement and the perception of arm motions. The method has been implemented and evaluated on an infant humanoid platform. Our results demonstrate the effectiveness of using the delayed temporal contingency in the action-perception loop as a basis for simple self-other discrimination. We conclude by suggesting potential applications in social robotics and in generating forward models of motion.

Robots designed to learn from and interact with humans require an intuitive method for humans to communicate with them. Normal human speech is very difficult to process, requiring many kinds of complex analysis for robots to interpret it. An intermediate method for communication is recognition of prosody, the affective content of speech. Using prosody recognition, a human interacting with a robot can reward or punish its actions by scolding or praising it. In this project, prosody recognition of male voices is performed by feature-based analysis of sound files containing short utterances, which were recorded from subjects who were directed to emulate infant-directed speech, which generally contains exaggerated prosody (Breazeal, C and Aryanada, L, 2000). The features used are extracted from the energy and pitch contours in the preprocessing stage. The classifier discriminates amongst four affective classes of speech and neutral utterances. The four classes are prohibition, attentional bids, approval, and soothing, while the neutral utterances are speech, which carries none of the above affective intents. Discrimination is performed using a multistage k-nearest neighbor classifier. The five-way single-stage classifier operates at 62.5 accuracy on the entire male speech data set, while the female single-stage classifier classifies 66.7 percent correctly. Chi-square analysis resulted in a p of less than or equal to 0.001 for each. The data seem to indicate that while female voice data may be somewhat easier to classify than male, fundamental differences that make male utterances unsuitable for classification do not exist.

Human social dynamics rely upon the ability to correctly attribute beliefs, goals, and percepts to other people. The set of abilities that allow an individual to infer these hidden mental states based on observed actions and behavior has been called a "theory of mind". Drawing from the models of Baron-Cohen (1995) and Leslie (1994), a novel architecture called embodied theory of mind was developed to link high-level cognitive skills to the low-level perceptual abilities of a humanoid robot. The implemented system determines visual saliency based on inherent object attributes, high-level task constraints, and the attentional states of others. Objects of interest are tracked in real-time to produce motion trajectories which are analyzed by a set of naive physical laws designed to discriminate animate from inanimate movement. Animate objects can be the source of attentional states (detected by finding faces and head orientation) as well as intentional states (determined by motion trajectories between objects). Individual components are evaluated by comparisons to human performance on similar tasks, and the complete system is evaluated in the context of a basic social learning mechanism that allows the robot to mimic observed movements.

Ballard (1991) described the implications of having a visual system that could actively position the camera coordinates in response to physical stimuli. In humanoid robotic systems, or in any animate vision system that interacts with people, social dynamics provide additional levels of constraint and additional opportunities for processing economy. In this paper, we describe an integrated visual-motor system that was implemented on a humanoid robot to negotiate the robot's physical constraints, the perceptual needs of the robot's behavioral and motivational systems, and the social implications of the motor acts.

In order to interact socially with a human, a robot must convey intentionality, that is, the human must believe that the robot has beliefs, desires, and intentions. We have constructed a robot which exploits natural human social tendencies to convey intentionality through motor actions and facial expressions. We present results on the integration of perception, attention, motivation, behavior, and motor systems which allow the robot to engage in infant-like interactions with a human caregiver.