Large language models (LLMs) are increasingly used to simulate human behavior in social settings such as legal mediation, negotiation, and dispute resolution. However, it remains unclear whether these simulations reproduce the personality–behavior patterns observed in humans. Human personality, for instance, shapes how individuals navigate social interactions, including strategic choices and behaviors in emotionally charged interactions. This raises the question: Can LLMs, when prompted with personality traits, reproduce personality-driven differences in human conflict behavior? To explore this, we introduce an evaluation framework that enables direct comparison of human-human and LLM-LLM behaviors in dispute resolution dialogues with respect to Big Five Inventory (BFI) personality traits. This framework provides a set of interpretable metrics related to strategic behavior and conflict outcomes. We additionally contribute a novel dataset creation methodology for LLM dispute resolution dialogues with matched scenarios and personality traits with respect to human conversations. Finally, we demonstrate the use of our evaluation framework with three contemporary closed-source LLMs and show significant divergences in how personality manifests in conflict across different LLMs compared to human data, challenging the assumption that personality-prompted agents can serve as reliable behavioral proxies in socially impactful applications. Our work highlights the need for psychological grounding and validation in AI simulations before real-world use.

Both haptic signals and simple, non-anthropomorphic robots can convey complex emotions and enhance remote communication. In this study, we integrated a zoomorphic socially expressive Blossom robot and a haptic sleeve to create a novel multimodal telepresence platform for remote social interaction. Through a within-subject user study with 16 participants, we explored the individual and combined effects of socially expressive robots and mediated social touch on affective communication and social presence during a semi-collaborative LEGO assembly task. Across all participants, the robot and wearable device significantly impacted how participants perceived expressions of gratitude, calming, attention-grabbing, and sadness, evaluated through self-reported valence and arousal. The robot and wearable device in our setting did not show a significant effect on social presence. The observations from this exploratory study can inform the design of multimodal telepresence systems and interactions using non-anthropomorphic robots and mediated touch.

Artificial intelligence (AI) has demonstrated significant potential in supporting pediatric speech therapy.However, no systematic review has examined the quality and expanding landscape of commercially available AI-enabled speech therapy mobile apps for children.We conducted a systematic review and analysis of 21 identified commercially available AI-enabled apps designed for speech-sound practice with children.Using 15 evaluation criteria consolidated and extended from prior HCI design guidelines for human-AI and child-AI interaction, our content and risk analysis revealed that existing apps have not yet leveraged the full potential of existing AI tools, and often fail to rigorously adhere to the best recommended practice.Incorporating feedback from the speech therapy community, we categorized our findings into four areas of ethical concern.To address these concerns, we propose actionable recommendations for app designers, families, and speech therapists to guide more ethical development and safe use of AI-enabled applications for pediatric speech therapy.

As the global population ages, there is increasing need for accessible technologies that promote cognitive health and detect early signs of cognitive decline. This research demonstrates the potential for in-residence monitoring and assessment of cognitive health using large language model (LLM)-powered socially assistive robots (SARs). We conducted a 5-week within-subjects study involving 22 older adults in retirement homes to investigate the feasibility of large language model (LLM)-powered socially assistive robots (SARs) for promoting and assessing cognitive health. We designed tasks that involved verbal dialogue based on clinically validated cognitive tools. Our findings reveal improved task performance after three robot-administered sessions, with significantly more detailed picture descriptions, fewer word repetitions in semantic fluency, and reduced need for hints. We found that older adults were more socially engaged in robot-administered tasks compared to those administered by a human, and they accepted and were willing to engage with socially assistive robots (SARs) in this context, which had not been tested before.

People have a variety of preferences for how robots behave. To understand and reason about these preferences, robots aim to learn a reward function that describes how aligned robot behaviors are with a user's preferences. Good representations of a robot's behavior can significantly reduce the time and effort required for a user to teach the robot their preferences. Specifying these representations-what “features“ of the robot's behavior matter to users-remains a difficult problem; Features learned from raw data lack semantic meaning and features learned from user data require users to engage in tedious labeling processes. Our key insight is that users tasked with customizing a robot are intrinsically motivated to produce labels through exploratory search; they explore behaviors that they find interesting and ignore behaviors that are irrelevant. To harness this novel data source of exploratory actions, we propose contrastive learning from exploratory actions (CLEA) to learn trajectory features that are aligned with features that users care about. We learned CLEA features from exploratory actions users performed in an open-ended signal design activity$(N=25)$with a Kuri robot, and evaluated CLEA features through a second user study with a different set of users$(N=42)$. CLEA features outperformed self-supervised features when eliciting user preferences over four metrics: completeness, simplicity, minimality, and explainability.

Hair care robots can help address labor shortages in elderly care while enabling those with limited mobility to maintain their hair-related identity. We present MOE-Hair, a soft robot system that performs three hair-care tasks: head patting, finger combing, and hair grasping. The system features a tendon-driven soft robot end-effector (MOE) with a wrist-mounted RGBD camera, leveraging both mechanical compliance for safety and visual force sensing through deformation. In testing with a force-sensorized mannequin head, MOE achieved comparable hair-grasping effectiveness while applying significantly less force than rigid grippers. Our novel force estimation method combines visual deformation data and tendon tensions from actuators to infer applied forces, reducing sensing errors by up to 60.1% and 20.3% compared to actuator current load-only and depth image-only baselines, respectively. A user study with 12 participants demonstrated statistically significant preferences for MOE-Hair over a baseline system in terms of comfort, effectiveness, and appropriate force application. These results demonstrate the unique advantages of soft robots in contact-rich hair-care tasks, while highlighting the importance of precise force control despite the inherent compliance of the system. Videos, data, and code are available at moehair.github.io.

Many researchers are working to address the worldwide mental health crisis by developing therapeutic technologies that increase the accessibility of care, including leveraging large language model (LLM) capabilities in chatbots and socially assistive robots (SARs) used for therapeutic applications. Yet, the effects of these technologies over time remain unexplored. In this study, we use a factorial design to assess the impact of embodiment and time spent engaging in therapeutic exercises on participant disclosures. We assessed transcripts gathered from a two-week study in which 26 university student participants completed daily interactive Cognitive Behavioral Therapy (CBT) exercises in their residences using either an LLM-powered SAR or a disembodied chatbot. We evaluated the levels of active engagement and high intimacy of their disclosures (opinions, judgments, and emotions) during each session and over time. Our findings show significant interactions between time and embodiment for both outcome measures: participant engagement and intimacy increased over time in the physical robot condition, while both measures decreased in the chatbot condition.

Generative models can serve as surrogates for some real data sources by creating synthetic training datasets, but in doing so they may transfer biases to downstream tasks. We focus on protecting quality and diversity when generating synthetic training datasets. We propose quality-diversity generative sampling (QDGS), a framework for sampling data uniformly across a user-defined measure space, despite the data coming from a biased generator. QDGS is a model-agnostic framework that uses prompt guidance to optimize a quality objective across measures of diversity for synthetically generated data, without fine-tuning the generative model. Using balanced synthetic datasets generated by QDGS, we first debias classifiers trained on color-biased shape datasets as a proof-of-concept. By applying QDGS to facial data synthesis, we prompt for desired semantic concepts, such as skin tone and age, to create an intersectional dataset with a combined blend of visual features. Leveraging this balanced data for training classifiers improves fairness while maintaining accuracy on facial recognition benchmarks. Code available at: https://github.com/Cylumn/qd-generative-sampling.

As artificial intelligence (AI) is playing an increasingly important role in our society and global economy, AI education and literacy have become necessary components in college and K-12 education to prepare students for an AI-powered society. However, current AI curricula have not yet been made accessible and engaging enough for students and schools from all socio-economic backgrounds with different educational goals. In this work, we developed an open-source learning module for college and high school students, which allows students to build their own robot companion from the ground up. This open platform can be used to provide hands-on experience and introductory knowledge about various aspects of AI, including robotics, machine learning (ML), software engineering, and mechanical engineering. Because of the social and personal nature of a socially assistive robot companion, this module also puts a special emphasis on human-centered AI, enabling students to develop a better understanding of human-AI interaction and AI ethics through hands-on learning activities. With open-source documentation, assembling manuals and affordable materials, students from different socio-economic backgrounds can personalize their learning experience based on their individual educational goals. To evaluate the student-perceived quality of our module, we conducted a usability testing workshop with 15 college students recruited from a minority-serving institution. Our results indicate that our AI module is effective, easy-to-follow, and engaging, and it increases student interest in studying AI/ML and robotics in the future. We hope that this work will contribute toward accessible and engaging AI education in human-AI interaction for college and high school students.

Mothers of infants have specific demands in fos-tering emotional bonds with their children, characterized by dynamics that are different from adult-adult interactions, notably requiring heightened maternal emotional regulation. In this study, we analyzed maternal emotional state by modeling mater-nal emotion regulation reflected in smiles. The dataset comprises$\mathrm{N}=94$videos of approximately$3 \pm 1$-minutes, capturing free play interactions between 6 and 12-month-old infants and their mothers. Corresponding demographic details of self-reported ma-ternal mental health provide variables for determining mothers' relations to emotions measured during free play. In this work, we employ diverse methodological approaches to explore the temporal evolution of maternal smiles. Our findings reveal a correlation between the temporal dynamics of mothers' smiles and their emotional state. Furthermore, we identify specific smile features that correlate with maternal emotional state, thereby enabling informed inferences with existing literature on general smile analysis. This study offers insights into emotional labor, defined as the management of one's own emotions for the benefit of others, and emotion regulation entailed in mother-infant interactions.

College students with ADHD respond positively to simple socially assistive robots (SARs) that monitor attention and provide non-verbal feedback, but studies have been done only in brief in-lab sessions. We present an initial design and evaluation of an in-dorm SAR study companion for college students with ADHD. This work represents the introductory stages of an ongoing user-centered, participatory design process. In a three-week within-subjects user study, university students (N=11) with self-reported symptoms of adult ADHD had a SAR study companion in their dorm room for two weeks and a computer-based system for one week. Toward developing SARs for long-term, in-dorm use, we focus on 1) evaluating the usability and desire for SAR study companions by college students with ADHD, and 2) collecting participant feedback about the SAR design and functionality. Participants responded positively to the robot; after one week of regular use, 91% (10 of 11) chose to continue using the robot voluntarily in the second week.

Children diagnosed with anxiety disorders are taught a range of strategies to navigate situations of heightened anxiety. Techniques such as deep breathing and repetition of mantras are commonly employed, as they are known to be calming and reduce elevated heart rates. Although these strategies are often effective, their successful application relies on prior training of the children for successful use when faced with challenging situations. This paper investigates a pocket-sized companion robot designed to offer a relaxation technique requiring no prior training, with a focus on immediate impact on the user’s heart rate. The robot utilizes a tactile game to divert the user’s attention, thereby promoting relaxation. We conducted two studies with children who were not diagnosed with anxiety: a 14-day pilot study with two children (age 8) and a main study with 18 children (ages 7-8). Both studies employed a within-subjects design and focused on measuring heart rate during tactile interaction with the robot and during non-use. Interacting with the robot was found to significantly lower the study participants’ heart rate (p<0.01) compared to the nonuse condition, indicating a consistent calming effect across all participants. These results suggest that tactile companion robots have the potential to enhance the therapeutic value of relaxation techniques.

Recent works have demonstrated the effectiveness of machine learning (ML) techniques in detecting anxiety and stress using physiological signals, but it is unclear whether ML models are learning physiological features specific to stress. To address this ambiguity, we evaluated the generalizability of physiological features that have been shown to be correlated with anxiety and stress to high-arousal emotions. Specifically, we examine features extracted from electrocardiogram (ECG) and electrodermal (EDA) signals from the following three datasets: Anxiety Phases Dataset (APD), Wearable Stress and Affect Detection (WESAD), and the Continuously Annotated Signals of Emotion (CASE) dataset. We aim to understand whether these features are specific to anxiety or general to other high-arousal emotions through a statistical regression analysis, in addition to a within-corpus, cross-corpus, and leave-one-corpus-out cross-validation across instances of stress and arousal. We used the following classifiers: Support Vector Machines, LightGBM, Random Forest, XGBoost, and an ensemble of the aforementioned models. We found that models trained on an arousal dataset perform relatively well on a previously unseen stress dataset, and vice versa. Our experimental results suggest that the evaluated models may be identifying emotional arousal instead of stress. This work is the first cross-corpus evaluation across stress and arousal from ECG and EDA signals, contributing new findings about the generalizability of stress detection.

In our multicultural world, affect-aware AI systems that support humans need the ability to perceive affect across variations in emotion expression patterns across cultures. These systems must perform well in cultural contexts without annotated affect datasets available for training models. A standard assumption in affective computing is that affect recognition models trained and used within the same culture (intracultural) will perform better than models trained on one culture and used on different cultures (intercultural). We test this assumption and present the first systematic study of intercultural affect recognition models using videos of real-world dyadic interactions from six cultures. We develop an attention-based feature selection approach under temporal causal discovery to identify behavioral cues that can be leveraged in intercultural affect recognition models. Across all six cultures, our findings demonstrate that intercultural affect recognition models were as effective or more effective than intracultural models. We identify and contribute useful behavioral features for intercultural affect recognition; facial features from the visual modality were more useful than the audio modality in this study's context. Our paper presents a proof-of-concept and motivation for the future development of intercultural affect recognition systems, especially those deployed in low-resource situations without annotated data.

Mindfulness-based therapies have been shown to be effective in improving mental health, and technology-based methods have the potential to expand the accessibility of these therapies. To enable real-time personalized content generation for mindfulness practice in these methods, high-quality computer-synthesized text-to-speech (TTS) voices are needed to provide verbal guidance and respond to user performance and preferences. However, the user-perceived quality of state-of-the-art TTS voices has not yet been evaluated for administering mindfulness meditation, which requires emotional expressiveness. In addition, work has not yet been done to study the effect of physical embodiment and personalization on the user-perceived quality of TTS voices for mindfulness. To that end, we designed a two-phase human subject study. In Phase 1, an online Mechanical Turk between-subject study (N=471) evaluated 3 (feminine, masculine, child-like) state-of-the-art TTS voices with 2 (feminine, masculine) human therapists' voices in 3 different physical embodiment settings (no agent, conversational agent, socially assistive robot) with remote participants. Building on findings from Phase 1, in Phase 2, an in-person within-subject study (N=94), we used a novel framework we developed for personalizing TTS voices based on user preferences, and evaluated user-perceived quality compared to best-rated non-personalized voices from Phase 1. We found that the best-rated human voice was perceived better than all TTS voices; the emotional expressiveness and naturalness of TTS voices were poorly rated, while users were satisfied with the clarity of TTS voices. Surprisingly, by allowing users to fine-tune TTS voice features, the user-personalized TTS voices could perform almost as well as human voices, suggesting user personalization could be a simple and very effective tool to improve user-perceived quality of TTS voice.

Virtual, augmented, and mixed reality for human-robot interaction (VAM-HRI) is a new and rapidly growing field of research. The field of socially assistive robot (SAR) has made impactful advances in educational settings, but has not yet benefited from VAM-HRI advances. We developed MoveToCode - an open-source, embodied (i.e., kinesthetic) learning visual programming language that aims to increase student (ages 8-12) curiosity during programming. MoveToCode uses an augmented reality (AR) autonomous robot tutor named Kuri that models the students’ kinesthetic curiosity and acts to promote their curiosity in programming. MoveToCode design was informed by pilot studies and tested in Los Angeles elementary classrooms $(n =21)$. Results from main study validated our design decisions compared to the pilot study which was conducted in a real elementary school classroom environment $(n =15)$, showing an improvement in perceived robot helpfulness (median $+ \Delta1.25$ out of 5) and number of completed exercises (median $+ \Delta1$, maximum of 11). While no significant changes were found in pre/post student curiosity or intention to program later in life, students wrote more open-ended questions post-study on topics related to robots, programming, research, and if they would like to do the activity again. This work demonstrates the potential of using VAM-HRI in a kinesthetic context for SAR tutors, and highlights the existing conventions and new design considerations for creating AR applications for SAR.

The physical design of a robot suggests expectations of that robot’s functionality for human users and collaborators. When those expectations align with the robot’s true capabilities, users are more likely to adopt the technologies for their intended use. However, the relationship between expectations and socially interactive robot design is not well understood. This article applies the concept of design metaphors to robot design and contributes the Metaphors for Understanding Functional and Social Anticipated Affordances dataset of 165 extant robots and the expectations users place on them. We used Mechanical Turk to crowd-source user expectation over three user studies. The first study ( N = 382) associated crowd-sourced design metaphors to different robot embodiments. The second study ( N = 803) assessed initial social expectations of robot embodiments. The final study ( N = 805) addressed the degree of abstraction of the design metaphors and the functional expectations projected on robot embodiments. We performed analyses to gain insights into how design metaphors can be used to understand social and functional expectations of robots and how these data can be visualized to be useful for study designers and robot designers. Together, these results can serve to guide robot designers toward aligning user expectations with true robot capabilities, facilitating positive human–robot interaction.

Communicating empathy is important for building relationships in numerous contexts. Consequently, the failure of robots to be perceived as empathetic by human users could be detrimental to developing effective human-robot interaction. Work on computational models of empathy has been growing rapidly, reflecting the importance of this ability for machines. Despite growing recent work, there remain unanswered questions about how users perceive different forms of empathetic expression by robots and how attitudes towards robots may mediate perceptions of robot empathy. Do people really believe that robots can feel or understand emotions? This work studied the difference in viewers' perceptions of cognitive and affective empathetic statements made by a robot in response to hu-man disclosure. In a within-subjects study, participants (n=111) watched videos in which a human disclosed negative emotions around COVID-19, and a robot responded with either affective or cognitive empathetic responses. Using an adapted version of the Robot's Perceived Empathy (RoPE) scale, participants rated their perceptions of the robot's empathy in both cases. We found that participants perceived the robot that made affective empathetic statements as being more empathetic that the robot that made cognitive empathetic statements; we also found that participants with more negative attitudes toward robots were more likely to rate the cognitive condition as more empathetic than the affective condition. These results inform HRI in general and future work into developing robots that will be perceived as empathetic and could personalize empathetic responses to each user.

Behavior trees (BTs) are hierarchical agent control architectures popular for robot task-level planning that can be autonomously learned from robot demonstrations via decision tree (DT) intermediaries, making them accessible to non-expert users. Conversion algorithms from DTs to BTs, such as the BT-Espresso algorithm, focus on replicating DT logic in a BT format but do not exploit the strengths of the BT architecture. We introduce the Representation Exploitation of BT-Espresso (RE:BT-Espresso) algorithm, which builds on BT-Espresso and improves the learned BT's interpretability and expressivity. RE:BT-Espresso improves interpretability by removing logical redundancies in the generated BTs and improves expressivity by exploiting desired BT structures, such as adding Inverter nodes, Repeater sequences, and Parallel Selector Action nodes that gives the user a choice of actions for state spaces that did not resolve to a concise action in the DT. The RE:BT-Espresso algorithm was evaluated against BT-Espresso using demonstration data synthesized by BTs. When compared to the synthesized BTs using graph edit distance (GED), RE:BT-Espresso outscored BT-Espresso on 54 subtrees, tied on 178, and lost on 2. Further, the proposed reduction strategies reduced the number of nodes in a generated tree by a median of 7.82%. The results validate improved interpretability and expressivity of learned RE:BT-Espresso task-level BT policies from robot demonstration.

Study reproducibility and generalizability of results to broadly inclusive populations is crucial in any research. Previous meta-analyses in HRI have focused on the consistency of reported information from papers in various categories. However, members of the HRI community have noted that much of the information needed for reproducible and generalizable studies is not found in published papers. We address this issue by surveying the reported study metadata over the main proceedings of the 2021 IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) and the past three years (2019 through 2021) of the main proceedings of the International Conference on Human-Robot Interaction (HRI) and alt.HRI. Based on the analysis results, we propose a set of recommendations for the HRI community that follow the longer-standing reporting guidelines from human-computer interaction (HCI), psychology, and other fields most related to HRI. Finally, we examine two key areas for user study reproducibility: recruitment details and participant compensation. We find a lack of reporting of both of these study metadata categories: of the 414 studies across both conferences and all years, 258 studies failed to report recruitment method and 255 studies failed to report compensation. This work provides guidance about specific types of needed reporting improvements for the field of HRI.

This paper describes a between-subjects Amazon Mechanical Turk study (n = 220) that investigated how a robot’s affective narrative influences its ability to elicit empathy in human observers. We first conducted a pilot study to develop and validate the robot’s affective narratives. Then, in the full study, the robot used one of three different affective narrative strategies (funny, sad, neutral) while becoming less functional at its shopping task over the course of the interaction. As the functionality of the robot degraded, participants were repeatedly asked if they were willing to help the robot. The results showed that conveying a sad narrative significantly influenced the participants’ willingness to help the robot throughout the interaction and determined whether participants felt empathetic toward the robot throughout the interaction. Furthermore, a higher amount of past experience with robots also increased the participants’ willingness to help the robot. This work suggests that affective narratives can be useful in short-term interactions that benefit from emotional connections between humans and robots.

From RViz to augmented reality (AR), a wide variety of robot signal visualizations exist for conveying robot capabilities. Many of the visualizations designed for AR, however, have not isolated multiple salient Virtual Design Elements (VDEs) for a given signal and comparatively evaluated combinations of those VDEs. To address this, we identify multiple VDEs for AR signaling of the following core robot capabilities: navigation, light detection and ranging (LiDAR), camera, face detection, audio localization, and natural language processing. We evaluated each signal's VDE combinations with an Amazon Mechanical Turk study (n=150) where participants watched 4 videos for each signal (consisting of 2 independent VDE choices) and rated the clarity and visual appeal of each signal. The results define a set of the most clear and visually appealing signal visualization designs and inform about interaction effects among VDEs. The resulting VDEs offer design insights and a baseline for continued research into AR robot capability signalling.

People often hold inaccurate mental models of robots. When such misconceptions regard a robot’s perceptual capabilities, they can lead to issues with safety, privacy, and interaction efficiency. This work is the first attempt to model users’ beliefs about a robot’s perceptual capabilities and make plans to improve their accuracy—i.e., to perform belief repair. We designed a new domain called the Robot Olympics, implemented it as a web-based game platform for collecting data about users’ beliefs, and developed an approach to estimating and influencing users’ beliefs about a virtual robot in that domain. We then conducted a study that collected user behavior and belief data from 240 online participants who played the game. Results revealed shortcomings in modeling the participant’s interpretations of the robot’s actions, as well as the decision making process behind their own actions. The insights from this work provide recommendations for designing further studies and improving user models to support belief repair in human-robot interaction.

Affect-aware socially assistive robotics (SAR) has shown great potential for augmenting interventions for children with autism spectrum disorders (ASD). However, current SAR cannot yet perceive the unique and diverse set of atypical cognitive-affective behaviors from children with ASD in an automatic and personalized fashion in long-term (multi-session) real-world interactions. To bridge this gap, this work designed and validated personalized models of arousal and valence for children with ASD using a multi-session in-home dataset of SAR interventions. By training machine learning (ML) algorithms with supervised domain adaptation (s-DA), the personalized models were able to tradeoff between the limited individual data and the more abundant less personal data pooled from other study participants. We evaluated the effects of personalization on a long-term multimodal dataset consisting of four children with ASD with a total of 19 sessions, and derived inter-rater reliability (IR) scores for binary arousal (IR = 83%) and valence (IR = 81%) labels between human annotators. Our results show that personalized Gradient Boosted Decision Trees (XGBoost) models with s-DA outperformed two non-personalized individualized and generic model baselines not only on the weighted average of all sessions, but also statistically ( p < .05) across individual sessions. This work paves the way for the development of personalized autonomous SAR systems tailored toward individuals with atypical cognitive-affective and socio-emotional needs.

Empathy is the ability to share someone else’s feelings or experiences; it influences how people interact and relate. Socially assistive robots (SAR) are a promising means of conveying and eliciting empathy toward facilitating human-robot interaction. This work examines factors that influence the amount of empathy elicited by a SAR storyteller and users’ perceptions of that robot. We conducted an empirical mixed-design study (N=46) using an autonomous SAR storyteller that told three stories, each with a different human or robot target of empathy. The robot storyteller used the first-person narrative voice (1PNV) with half of the participants and the third-person narrative voice (3PNV) with the other half. We found that the SAR storyteller elicited significantly more empathy when the story target of empathy matched the SAR narrator, i.e., was also a robot. Additionally, the 1PNV robot elicited significantly more empathy and was perceived as more human-like, easy to interact with, and trustworthy than the 3PNV robot. Finally, participants who empathized more with the robot displayed facial expressions consistent with the emotional story content. These insights inform the design of SAR storytellers capable of eliciting empathy toward creating compelling and effective human-robot interactions.

Hardware platforms for socially interactive robotics can be limited by cost or lack of functionality. This article presents the overall system—design, hardware, and software—for Quori, a novel, affordable, socially interactive humanoid robot platform for facilitating noncontact human–robot interaction (HRI) research. The design of the system is motivated by feedback sampled from the HRI research community. The overall design maintains a balance of affordability and functionality. Initial Quori testing and a six-month deployment are presented. Ten Quori platforms have been awarded to a diverse group of researchers from across the United States to facilitate HRI research to build a community database from a common platform.

Virtual and robotic agents capable of perceiving human empathy have the potential to participate in engaging and meaningful human-machine interactions that support human well-being. Prior research in computational empathy has focused on designing empathic agents that use verbal and nonverbal behaviors to simulate empathy and attempt to elicit empathic responses from humans. The challenge of developing agents with the ability to automatically perceive elicited empathy in humans remains largely unexplored. Our paper presents the first approach to modeling user empathy elicited during interactions with a robotic agent. We collected a new dataset from the novel interaction context of participants listening to a robot storyteller (46 participants, 6.9 hours of video). After each storytelling interaction, participants answered a questionnaire that assessed their level of elicited empathy during the interaction with the robot. We conducted experiments with 8 classical machine learning models and 2 deep learning models (long short-term memory networks and temporal convolutional networks) to detect empathy by leveraging patterns in participants’ visual behaviors while they were listening to the robot storyteller. Our highest-performing approach, based on XGBoost, achieved an accuracy of 69% and AUC of 72% when detecting empathy in videos. We contribute insights regarding modeling approaches and visual features for automated empathy detection. Our research informs and motivates future development of empathy perception models that can be leveraged by virtual and robotic agents during human-machine interactions.

Epidemic diseases bring many challenges to universities. In the case of airborne contagious diseases like COVID-19, health agencies’ guidelines recommend that people maintain a physical distance of about 2 meters from each other. Enforcing such physical distancing on a university campus means that it will potentially take longer for students to get into and out of classrooms and buildings on campus. We use real course registration data from a large US university to study wait times students would encounter to enter and exit campus buildings while keeping the recommended 2 meter physical distance, and show that peak wait times can be longer than 20 minutes. We propose LBCS, a load-balanced course scheduling algorithm that intelligently reduces the peak wait time while ensuring that conflicting classes are scheduled at different times. Through simulations we show that LBCS can reduce the peak wait time by a factor of 3×, better than naive alternatives such as shifting some classes to the weekend or randomly perturbing class start times.

Automated systems that detect the social behavior of deception can enhance human well-being across medical, social work, and legal domains. Labeled datasets to train supervised deception detection models can rarely be collected for real-world, high -stakes contexts. To address this challenge, we propose the first unsupervised approach for detecting realworld, high-stakes deception in videos without requiring labels. This paper presents our novel approach for affect-aware unsupervised Deep Belief Networks (DBN) to learn discriminative representations of deceptive and truthful behavior. Drawing on psychology theories that link affect and deception, we experimented with unimodal and multimodal DBN-based approaches trained on facial valence, facial arousal, audio, and visual features. In addition to using facial affect as a feature on which DBN models are trained, we also introduce a DBN training procedure that uses facial affect as an aligner of audio-visual representations. We conducted classification experiments with unsupervised Gaussian Mixture Model clustering to evaluate our approaches. Our best unsupervised approach (trained on facial valence and visual features) achieved an AVC of 80%, outperforming human ability and performing comparably to fully-supervised models. Our results motivate future work on unsupervised, affect-aware computational approaches for detecting deception and other social behaviors in the wild.

With their ability to embody users in physically distant spaces, telepresence robots have gained popularity in environments including hospitals, schools, and offices. However, with platforms lacking in individuation and social presence, users often personalize telepresence robots with clothing and accessories to increase their recognizability and sense of embodiment. Toward understanding personalization preferences, as well as perceptions of personalized platforms, we conducted a series of five studies that investigate patterns in personalization of a telepresence robot and evaluate the impacts of common personalizations along five dimensions (robot uniqueness, humanness, pleasantness/unpleasantness, and people's willingness to interact with it). Finding a strong preference for the use of clothing and headwear in Studies 1-2 (N=52), we systematically manipulated a robot's appearance using these items and evaluated the qualitative and quantitative impacts on observer perceptions in Studies 3-4 (N=160). Observing that personalization increased perceptions of uniqueness and humanness, but also decreased positive responding, we then investigated the associations between personalization preferences and perceptions via a fifth study (N=100). Across the five studies, tensions emerged between operators' interest in using wigs and interlocutors' dislike of wigs. This result highlights a need to consider both operator and interlocutor perspectives when personalizing telepresence robots.

Automated systems that detect deception in high-stakes situations can enhance societal well-being across medical, social work, and legal domains. Existing models for detecting high-stakes deception in videos have been supervised, but labeled datasets to train models can rarely be collected for most real-world applications. To address this problem, we propose the first multimodal unsupervised transfer learning approach that detects real-world, high-stakes deception in videos with-out using high-stakes labels. Our subspace-alignment (SA) approach adapts audio-visual representations of deception in lab-controlled low-stakes scenarios to detect deception in real-world, high-stakes situations. Our best unsupervised SA models outperform models without SA, outperform human ability, and perform comparably to a number of existing supervised models. Our research demonstrates the potential for introducing subspace-based transfer learning to model high-stakes deception and other social behaviors in real-world contexts with a scarcity of labeled behavioral data.

Observing how infants and mothers coordinate their behaviors can highlight meaningful patterns in early communication and infant development. While dyads often differ in the modalities they use to communicate, especially in the first year of life, it remains unclear how to capture coordination across multiple types of behaviors using existing computational models of interpersonal synchrony. This paper explores Dynamic Mode Decomposition with control (DMDc) as a method of integrating multiple signals from each communicating partner into a model of multimodal behavioral coordination. We used an existing video dataset to track the head pose, arm pose, and vocal fundamental frequency of infants and mothers during the Face-to-Face Still-Face (FFSF) procedure, a validated 3-stage interaction paradigm. For each recorded interaction, we fit both unimodal and multimodal DMDc models to the extracted pose data. The resulting dynamic characteristics of the models were analyzed to evaluate trends in individual behaviors and dyadic processes across infant age and stages of the interactions. Results demonstrate that observed trends in interaction dynamics across stages of the FFSF protocol were stronger and more significant when models incorporated both head and arm pose data, rather than a single behavior modality. Model output showed significant trends across age, identifying changes in infant movement and in the relationship between infant and mother behaviors. Models that included mothers’ audio data demonstrated similar results to those evaluated with pose data, confirming that DMDc can leverage different sets of behavioral signals from each interacting partner. Taken together, our results demonstrate the potential of DMDc toward integrating multiple behavioral signals into the measurement of multimodal interpersonal coordination.

Telepresence robots offer presence, embodiment, and mobility to remote users, making them promising options for homebound K-12 students. It is difficult, however, for robot operators to know how well they are being heard in remote and noisy classroom environments. One solution is to estimate the operator’s speech intelligibility to their listeners in order to provide feedback about it to the operator. This work contributes the first evaluation of a speech intelligibility feedback system for homebound K-12 students attending class remotely. In our four long-term, in-the-wild deployments we found that students speak at different volumes instead of adjusting the robot’s volume, and that detailed audio calibration and network latency feedback are needed. We also contribute the first findings about the types and frequencies of multimodal comprehension cues given to homebound students by listeners in the classroom. By annotating and categorizing over 700 cues, we found that the most common cue modalities were conversation turn timing and verbal content. Conversation turn timing cues occurred more frequently overall, whereas verbal content cues contained more information and might be the most frequent modality for negative cues. Our work provides recommendations for telepresence systems that could intervene to ensure that remote users are being heard.

The research and development of socially interactive robots is a complex challenge because of the wide variety of capabilities needed for effective social human-robot interactions (HRI). Many of these capabilities, including perception, dialog, and control, have state of the art methods and solutions, but combining those into a comprehensive and seamless interaction is still an open challenge. We describe HARMONI, a multi-modal, open-source tool for rapid social HRI development and deployment. HARMONI is centered around a ROS package for interaction development, including decision management and node orchestration. HARMONI systematically integrates with disparate functionalities needed to conduct a meaningful social human-robot interaction such as external cloud services, AI models, and modules for sensing, planning, and acting on a variety of platforms. HARMONI was applied to the QT robot platform and usability tests were conducted to evaluate the ease and speed of development and deployment. This paper describes the architecture and design of HARMONI and reports the results of a pilot study with novice users.

This work introduces an approach for automatic hair combing by a lightweight robot. For people living with limited mobility, dexterity, or chronic fatigue, combing hair is often a difficult task that negatively impacts personal routines. We propose a modular system for enabling general robot manipulators to assist with a hair-combing task. The system consists of three main components. The first component is the segmentation module, which segments the location of hair in space. The second component is the path planning module that proposes automatically-generated paths through hair based on user input. The final component creates a trajectory for the robot to execute. We quantitatively evaluate the effectiveness of the paths planned by the system with 48 users and qualitatively evaluate the system with 30 users watching videos of the robot performing a hair-combing task in the physical world. The system is shown to effectively comb different hairstyles.

In this work we address the Next Speaker Prediction sub challenge of the ACM '21 MultiMediate Grand Challenge. This challenge poses the problem of turn taking prediction in physically situated multiparty interaction. Solving this problem is essential for enabling fluent real-time multiparty human-machine interaction. This problem is made more difficult by the need for a robust solution that can perform effectively across a wide variety of settings and contexts. Prior work has shown that current state-of-the-art methods rely on machine learning approaches that do not generalize well to new settings and feature distributions. To address this problem, we propose the use of group-level focus of visual attention as additional information. We show that a simple combination of group-level focus of visual attention features and publicly available audio-video synchronizer models is competitive with state-of-the-art methods fine-tuned for the challenge dataset.

Learning movement control is a fundamental process integral to infant development. However, it is still unclear how infants learn to control leg movement. This work explores the potential of using socially assistive robots to provide real-time adaptive reinforcement learning for infants. Ten 6 to 8-month old typically-developing infants participated in a study where a robot provided reinforcement when the infant’s right leg acceleration fell within the range of 9 to 20 m/s2. If infants increased the proportion of leg accelerations in this band, they were categorized as "performers". Six of the ten participating infants were categorized as performers; the performer subgroup increased the magnitude of acceleration, proportion of target acceleration for right leg, and ratio of right/left leg acceleration peaks within the target acceleration band and their right legs increased movement intensity from the baseline to the contingency session. The results showed infants specifically adjusted their right leg acceleration in response to a robot- provided reward. Further study is needed to understand how to improve human-robot interaction policies for personalized interventions for young infants.

Children and adults with cerebral palsy (CP) can have involuntary upper limb movements as a consequence of the symptoms that characterize their motor disability, leading to difficulties in communicating with caretakers and peers. We describe how a socially assistive robot may help individuals with CP to practice non-verbal communicative gestures using an active orthosis in a one-on-one number-guessing game. We performed a user study and data collection with participants with CP; we found that participants preferred an embodied robot over a screen-based agent, and we used the participant data to train personalized models of participant engagement dynamics that can be used to select personalized robot actions. Our work highlights the benefit of personalized models in the engagement of users with CP with a socially assistive robot and offers design insights for future work in this area.

Telepresence robots act as the remote embodiments of human operators, enabling people to stay connected to friends, family, and coworkers over lengthy physical separations. However, the factors affecting how humans can best make use of such systems are not yet well understood. This paper explores the effects of personalization and relationship closeness on telepresence via two studies. Study 1 was a between-participants experiment that investigated telepresence robot personalization. 32 pairs of friends (N = 64) participated in the study's team-building-style activities and answered questions about robot operator presence. The results unexpectedly indicated that relationship closeness influenced the interaction experience more than any other considered predictor variable. To study closeness more rigorously as the central manipulation, we conducted Study 2, a between-participants experiment with 24 pairs (N = 48) and a similar procedure. Robot operators who reported a closer relationship with their teammate felt more present in this investigation. These findings can inform the design and application of telepresence robot systems to increase a remote operator's feelings of presence via robot.

Interactions between infants and their mothers can provide meaningful insight into the dyad's health and well-being. Previous work has shown that infant-mother coordination, within a single modality, varies significantly with age and interaction quality. However, as infants are still developing their motor, language, and social skills, they may differ from their mothers in the modes they use to communicate. This work examines how infant-mother coordination across modalities can expand researchers' abilities to observe meaningful trends in infant-mother interactions. Using automated feature extraction tools, we analyzed the head position, arm position, and vocal fundamental frequency of mothers and their infants during the Face-to-Face Still-Face (FFSF) procedure. A de-identified dataset including these features was made available online as a contribution of this work. Analysis of infant behavior over the course of the FFSF indicated that the amount and modality of infant behavior change evolves with age. Evaluating the interaction dynamics, we found that infant and mother behavioral signals are coordinated both within and across modalities, and that levels of both intramodal and intermodal coordination vary significantly with age and across stages of the FFSF. These results support the significance of intermodal coordination when assessing changes in infant-mother interaction across conditions.

Automated deception detection systems can enhance health, justice, and security in society by helping humans detect deceivers in high-stakes situations across medical and legal domains, among others. Existing machine learning approaches for deception detection have not leveraged dimensional representations of facial affect: valence and arousal. This paper presents a novel analysis of the discriminative power of facial affect for automated deception detection, along with interpretable features from visual, vocal, and verbal modalities. We used a video dataset of people communicating truthfully or deceptively in real-world, high-stakes courtroom situations. We leveraged recent advances in automated emotion recognition in-the-wild by implementing a state-of-the-art deep neural network trained on the Aff-Wild database to extract continuous representations of facial valence and facial arousal from speakers. We experimented with unimodal Support Vector Machines (SVM) and SVM-based multimodal fusion methods to identify effective features, modalities, and modeling approaches for detecting deception. Unimodal models trained on facial affect achieved an AUC of 80%, and facial affect contributed towards the highest-performing multimodal approach (adaptive boosting) that achieved an AUC of 91% when tested on speakers who were not part of training sets. This approach achieved a higher AUC than existing automated machine learning approaches that used interpretable visual, vocal, and verbal features to detect deception in this dataset, but did not use facial affect. Across all videos, deceptive and truthful speakers exhibited significant differences in facial valence and facial arousal, contributing computational support to existing psychological theories on relationships between affect and deception. The demonstrated importance of facial affect in our models informs and motivates the future development of automated, affect-aware machine learning approaches for modeling and detecting deception and other social behaviors in-the-wild.

Socially assistive robots have the potential to improve group dynamics when interacting with groups of people in social settings. This work contributes to the understanding of those dynamics through a user study of trust dynamics in the novel context of a robot mediated support group. For this study, a novel framework for robot mediation of a support group was developed and validated. To evaluate interpersonal trust in the multi-party setting, a dyadic trust scale was implemented and found to be uni-factorial, validating it as an appropriate measure of general trust. The results of this study demonstrate a significant increase in average interpersonal trust after the group interaction session, and qualitative post-session interview data report that participants found the interaction helpful and successfully supported and learned from one other. The results of the study validate that a robot-mediated support group can improve trust among strangers and allow them to share and receive support for their academic stress.

Since robots are increasingly expected to work in concert with humans in dynamic, unstructured environments, they will need to express information about their state and actions. We propose a formalism for planning robot communication that employs a probabilistic representation of the robots world. This representation takes on the form of a Markov Decision Process (MDP) and captures the uncertainty of interacting with humans. The key insight of this work is that humans preferences and time need to be carefully balanced against the robots in order to minimize human annoyance. The communication-MDP enables the robot to reason about the effects of its actions on a human interactor. We validated the model through a human subjects experiment (n=44) by learning communication policies for a loosely collaborative task. The results show that the communication-MDP improves participants' perceptions of the robot's thoughtfulness as an interactor.

The nexus of in-home intelligent assistants, activity tracking, and machine learning creates opportunities for personalized virtual and physical agents / robots that can positively impacts user health and quality of life. Well beyond providing information, such agents can serve as physical and mental health and education coaches and companions that support positive behavior change. However, sustaining user engagement and motivation over long-term interactions presents complex challenges. Our work over the past 15 years has addressed those challenges by developing human-machine (human-robot) interaction methods for socially assistive robotics that utilize multi-modal interaction data and expressive agent behavior to monitor, coach, and motivate users to engage in heath- and wellness-promoting activities. This talk will present methods and results of modeling, learning, and personalizing user motivation, engagement, and coaching of healthy children and adults, as well as stroke patients, Alzheimer's patients, and children with autism spectrum disorders, in short and long-term (month+) deployments in schools, therapy centers, and homes, and discuss research and commercial implications for technologies aimed at human daily use.

Although nonverbal expressive abilities are an essential element of human-to-human communication, telepresence robots support only select nonverbal behaviors. As a result, telepresence users can experience difficulties taking turns in conversation and using various cues to obtain the attention of others. To expand telepresence robot users' abilities to hold the floor during conversation, this work proposes and evaluates new types of expressive telepresence robot hardware. The described within-subjects study compared robot user and co-present person experiences during teamwork activity conditions involving basic robot functions, expressive LED lights, and an expressive robot arm. We found that among participants who preferred the arm-based expressiveness, individuals in both study roles felt the robot operator to be more in control of the robot during the arm condition, and participants co-located with the robot felt closer to their teammate during the arm phase. Participants also noted advantages of the LED lights for notification-type information and advantages of the arm for increasing perceptions of the robot as a human-like entity. Overall, these findings can inform future work on augmenting the nonverbal expressiveness of telepresence robots.

Telepresence robots can help to connect people by providing videoconferencing and navigation abilities in faraway environments. Despite this potential, current commercial telepresence robots lack certain nonverbal expressive abilities that are important for permitting the operator to communicate effectively in the remote environment. To help improve the utility of telepresence robots, we added an expressive, non-manipulating arm to our custom telepresence robot system and developed three user interfaces to control deictic gesturing by the arm: onscreen, dial-based, and skeleton tracking methods. A usability study helped us to evaluate user presence feelings, task load, preferences, and opinions while performing deictic gestures with the robot arm during a mock order packing task. The majority of participants preferred the dial-based method of controlling the robot, and survey responses revealed differences in physical demand and effort level across user interfaces. These results can guide robotics researchers interested in extending the nonverbal communication abilities of telepresence robots.

Expressivity-the use of multiple modalities to convey internal state and intent of a robot-is critical for interaction. Yet, due to cost, safety, and other constraints, many robots lack high degrees of physical expressivity. This paper explores using mixed reality to enhance a robot with limited expressivity by adding virtual arms that extend the robot's expressiveness. The arms, capable of a range of non-physically-constrained gestures, were evaluated in a between-subject study (n =34) where participants engaged in a mixed reality mathematics task with a socially assistive robot. The study results indicate that the virtual arms added a higher degree of perceived emotion, helpfulness, and physical presence to the robot. Users who reported a higher perceived physical presence also found the robot to have a higher degree of social presence, ease of use, usefulness, and had a positive attitude toward using the robot with mixed reality. The results also demonstrate the users' ability to distinguish the virtual gestures' valence and intent.

Robot tutors have great potential for supporting personalized learning, in both home and classroom settings. To be effective, robot tutors must encourage users to seek help as needed during the learning process. We conducted a between-subjects study with N=45 participants to compare different types of learner help-seeking behaviors-pressing an on-screen button, pressing a physical button, and raising a hand- and assess how help-seeking behavior preferences relate to perceptions of the robot tutor. The results indicate that hand raising was seen as the hardest method for a user to perform but the most useful and beneficial, with positive trends in students' intention to use a robot.

Early intervention to address developmental disability in infants has the potential to promote improved outcomes in neurodevelopmental structure and function [1]. Researchers are starting to explore Socially Assistive Robotics (SAR) as a tool for delivering early interventions that are synergistic with and enhance human-administered therapy. For SAR to be effective, the robot must be able to consistently attract the attention of the infant in order to engage the infant in a desired activity. This work presents the analysis of eye gaze tracking data from five 6-8 month old infants interacting with a Nao robot that kicked its leg as a contingent reward for infant leg movement. We evaluate a Bayesian model of low-level surprise on video data from the infants' head-mounted camera and on the timing of robot behaviors as a predictor of infant visual attention. The results demonstrate that over 67% of infant gaze locations were in areas the model evaluated to be more surprising than average. We also present an initial exploration using surprise to predict the extent to which the robot attracts infant visual attention during specific intervals in the study. This work is the first to validate the surprise model on infants; our results indicate the potential for using surprise to inform robot behaviors that attract infant attention during SAR interactions.

Socially assistive robotics (SAR) research has shown great potential for supplementing and augmenting therapy for children with autism spectrum disorders (ASD). However, the vast majority of SAR research has been limited to short-term studies in highly controlled environments. The design and development of a SAR system capable of interacting autonomously in situ for long periods of time involves many engineering and computing challenges. This paper presents the design of a fully autonomous SAR system for long-term, in-home use with children with ASD. We address design decisions based on robustness and adaptability needs, discuss the development of the robot's character and interactions, and provide insights from the month-long, in-home data collections with children with ASD. This work contributes to a larger research program that is exploring how SAR can be used for enhancing the social and cognitive development of children with ASD.

Auditory cues facilitate situational awareness by enabling humans to infer what is happening in the nearby environment. Unlike humans, many robots do not continuously produce perceivable state-expressive sounds. In this work, we propose the use of iconic auditory signals that mimic the sounds produced by a robot»s operations. In contrast to artificial sounds (e.g., beeps and whistles), these signals are primarily functional, providing information about the robot»s actions and state. We analyze the effects of two variations of robot sound, tonal and broadband, on auditory localization during a human-robot collaboration task. Results from 24 participants show that both signals significantly improve auditory localization, but the broadband variation is preferred by participants. We then present a computational formulation for auditory signaling and apply it to the problem of auditory localization using a human-subjects data collection with 18 participants to learn optimal signaling policies.

No abstract available.

We explored gesture recognition applied to the problem of classifying natural physical bullying behaviors by children. To capture natural bullying behavior data, we developed a humanoid robot that used hand-coded gesture recognition to identify basic physical bullying gestures and responded by explaining why the gestures were inappropriate. Children interacted with the robot by trying various bullying behaviors, thereby allowing us to collect a natural bullying behavior dataset for training the classifiers. We trained three different sequence classifiers using the collected data and compared their effectiveness at classifying different types of common physical bullying behaviors. Overall, Hidden Conditional Random Fields achieved the highest average F1 score (0.645) over all tested gesture classes.

Recent work has shown the potential for lights to act as simple, yet expressive signaling mechanisms for use in a variety of human-robot applications. However, the wide range of robot shapes and sizes makes it difficult for researchers to quickly prototype and evaluate different light configurations and signal designs. In this work, we present the design of ModLight, a modular research tool consisting of a set of low cost light blocks that can be easily reconfigured to fit a myriad of robots and applications. ModLight also provides researchers, designers, and students with open-source software tools that enable them to visually design new signals and easily integrate them into existing systems. This work also serves to motivate the need for further research in developing light behaviors for use in human-robot interaction. Towards this goal, we present our design rationale including a brief analysis of the signaling needs of several robots and applications.

Telepresence robots have the potential to improve access to K-12 education for students who are unable to attend school for a variety of reasons. Since previous telepresence research has largely focused on the needs of adult users in workplace settings, it is unknown what challenges must be addressed for these robots to be effective tools in classrooms. In this paper, we seek to better understand how a telepresence robot should function in the classroom when operated by a remote student. Toward this goal, we conducted field sessions in which four designers operated a telepresence robot in a real K-12 classroom. Using the results, we identify key research challenges and present design insights meant to inform the HRI community in particular and robot designers in general.

Telepresence robots have the potential to improve access to K-12 education. However, designing robots for classroom use presents unique challenges from both logistical and technological perspectives. To address these challenges, we created My Classroom Robot, an interactive game in which players can operate a virtual telepresence robot in a classroom environment. The virtual classroom environment allows us to collect data and prototype different designs prior to involving the high overhead required in going into the real classroom. In this work, we present the design of My Classroom Robot, an initial evaluation, and the lessons learned from its development.

This paper presents an algorithm for enabling a robot to act as the moderator in a group interaction centered around a tablet-based assembly game. The algorithm uses one of two different objective functions: one intended to be “performance equalizing”, wherein the robot attempts to equalize scoring among users, and another intended to be “performance reinforcing”, wherein the robot attempts to help the group score as many points as possible. In an evaluation study with ten groups of three participants, we found that the “performance equalizing” algorithm improved task performance and reduced group cohesion, while the “performance reinforcing” algorithm improved group cohesion and reduced task performance.

We present a pilot study of a socially assistive robot interacting with intergenerational groups. The system is designed to improve the social well-being of older adults by supporting interactions within families. Six intergenerational family groups interacted with the robot in four tablet-based games. Users' behavior during the sessions was used to compare the games and understand how members of different generations and different families interact with the robot. Interviews with users provide insight into users' priorities for in-home deployment of socially assistive robots, as well as preferences about the activities, appearance, and behavior of the robot.

Socially assistive robotics (SAR) has increasingly been shown to have potential as a tool for social skills therapy for children with autism, a developmental disorder associated with atypical social development. This work presents the results of a study of robot agency on child-robot interactions involving children with autism. We describe the development of a SAR interaction scenario with both agent-like and object-like robot behaviors and present the results of a pilot study of six children with autism interacting with a humanoid robot with the full controller, as well as three types of control: a non-humanoid box robot with similar behavior (reduced morphological agency), a humanoid robot with random behavior (reduced behavioral agency), and a robotic toy (reduced morphological and behavioral agency). We find that the children can be divided into two groups depending on their reaction to the robot; for some children, the robot was an engaging object that elicited social behavior by providing novel and appealing sensory experiences (primarily bubbleblowing), while for other children, the robot was an agent and elicited social behavior through agentlike actions such as autonomous movement. We found that the first group had small differences between robot conditions and vocalized most with the bubble-blowing toy, while the second group vocalized most with the humanoid robots and looked less at the humanoid portion of the robot with reduced behavioral agency.

Non-humanoid robots are becoming increasingly utilized for collaborative tasks across many domains, including industrial and service settings. Collaborative tasks between the human and robot rely on each collaborator's ability to effectively convey their mental state while accurately estimating and interpreting their partner's knowledge, intent, and actions. My research focuses on nonverbal communication signals that a non-humanoid robot can utilize during human-robot collaboration. We focus on motion, light and sound as they are commonly used communication channels across many domains and are available on most robot platforms. As a first step towards this goal, I present a completed study exploring how to use a simple multimodal light and sound signal to request help during a collaborative task. We then discuss future work to generate and utilize more complex signals to convey a variety of statuses to improve collaboration.

Our approach enables the robot to execute proxemic behaviors that improve speech and gesture recognition for both the robot and the human, even in noisy and cluttered environments. This work has been deployed on two different mobile robot platforms: (1) the PR22, and (2) the Bandit3upper body humanoid atop an iRobot Ava4mobile base. This demonstrates the portability of the approach.

Non-humanoid robots are becoming increasingly utilized for collaborative tasks that rely on each collaborator's ability to effectively convey their mental state while accurately estimating and interpreting their partner's knowledge, intent, and actions. During these tasks, it may be beneficial or even necessary for the human collaborator to assist the robot. Consequently, we explore the use of nonverbal signals to request help during a collaborative task. We focus on light and sound as they are commonly used communication channels across many domains. This paper analyzes the effectiveness of three nonverbal help signals that vary in urgency. Our results show that these signals significantly influence the human collaborator's and their perception of the collaboration.

We formalize and analyze a robotic tutor's elicitation of learning-sensitive information to be leveraged by interactive machine learning methods for personalized education. The user study presented in this paper is an initial exploration of elicitation frequency of learning sensitive information, and the social and computational implications thereof. Our results, evaluated using a variety of subjective measures, demonstrate that a humans-in-the-loop approach positively benefits the human-robotic tutor interaction, while minimizing the computational complexity of personalization.

We describe a study that compares two types of backstories, fictional and realistic, of an autonomous Socially Assistive Robot (SAR) exercise buddy in a four-session exercise intervention with overweight adolescents aged 11–14. We find a positive participant response to the robot as an exercise buddy and a trend toward increased intrinsic motivation for physical activity post-intervention. We do not find differences in ratings of the two backstories or between pre- and post-intervention measures of physical activity enjoyment and activity level.

Human preferences of distance (proxemics) to a robot significantly impact the performance of the robot's automated speech and gesture recognition during face-to-face, social human-robot interactions. This work investigated how people respond to a sociable robot based on its performance at different locations. We performed an experiment in which the robot's ability to understand social signals was artificially attenuated by distance. Participants (N = 180) instructed the robot using speech and pointing gestures, provided proxemic preferences before and after the interaction, and responded to a questionnaire. Our analysis of questionnaire responses revealed that robot performance factors---rather than human-robot proxemics---are significant predictors of user evaluations of robot competence, anthropomorphism, engagement, likability, and technology adoption. Our behavioral analysis suggests that human proxemic preferences change over time as users interact with and come to understand the needs of the robot, and those changes improve robot performance.

We detail an approach to planning effective verbal feedback during pairwise human-robot task collaboration. The approach is motivated by social science literature as well as existing work in robotics and is applicable to a variety of task scenarios. It consists of a dynamic, synthetic task implemented in an augmented reality environment. The result is combined robot task control and speech production, allowing the robot to actively participate and communicate with its teammate. A user study was conducted to experimentally validate the efficacy of the approach on a task in which a single user collaborates with an autonomous robot. The results demonstrate that the approach is capable of improving both objective measures of team performance and the user's subjective evaluation of both the task and the robot as a teammate.

An objective of an autonomous sociable robot is to meet the needs and preferences of a human user. However, this can sometimes be at the expense of the robot's own ability to understand social signals produced by the user. In particular, human preferences of distance (proxemics) to the robot can have significant impact on the performance rates of its automated speech and gesture recognition systems. In this work, we investigated how people perceive a sociable robot based on its performance at different locations. We performed an experiment in which a robot's ability to understand social signals was artificially attenuated across distance; robot maximum, minimum, and average performance rates were also varied. Participants (N = 160) instructed a robot using speech and pointing gestures, and then responded to a questionnaire to provide their subjective evaluations of the robot.We identified significant relationships between robot performance and user perceptions of robot competence, anthropomorphism, engagement, likability, and technology adoption; we identified no relationship between human-robot distance and subjective measures, which contrasts related work. Our results have significant implications for autonomous sociable robot design.

We present the design and analysis of a multi-session user study conducted with older adults to evaluate the effectiveness of a human-robot interaction (HRI) framework utilizing a neuroscience-inspired spatial language interpretation approach. The user study was designed to: 1) evaluate the effectiveness and feasibility of the spatial language interpretation framework with target users, and 2) collect data on the types of phrases, responses, and format of natural language instructions given by target users to help inform possible modifications to the HRI framework. In addition, an expanded study was conducted on Amazon's Mechanical Turk to gather further natural language instructions from general users for analysis. The results of both studies, evaluated across a variety of objective performance and participant evaluation measures, demonstrate the feasibility of the approach and its effectiveness in interpreting and following natural language instructions from target users, achieving high task success rates and high participant evaluations across multiple measures.

Socially assistive robotics (SAR) is a new subfield of robotics that bridges HRI, rehabilitation robotics, social robotics, and service robotics. SAR focuses on developing machines capable of assisting users, typically in health and education contexts, through social rather than physical interaction. The robot's physical embodiment is at the heart of SAR's effectiveness, as it leverages the inherently human tendency to engage with lifelike (but not necessarily humanlike or otherwise biomimetic) social behavior. This talk will describe research into embodiment, modeling and steering social dynamics, and long-term user adaptation for SAR. The research will be grounded in projects involving analysis of multi-modal activity data, modeling personality and engagement, formalizing social use of space and non-verbal communication, and personalizing the interaction with the user over a period of months. The presented methods and algorithms will be validated on implemented SAR systems evaluated by human subject cohorts from a variety of user populations, including stroke patients, children with autism spectrum disorder, and elderly with Alzheimers and other forms of dementia.

No abstract available.

We present a methodology for enabling service robots to interpret spatial language instruction sequences expressed through natural language discourse from non-expert users. As part of our approach, we propose a novel probabilistic algorithm for the automatic extraction of contextually and semantically valid instruction sequences from unconstrained spatial language discourse. Additionally, we present the design and implementation details of a procedure for reference resolution of anaphoric expressions encountered within the user discourse. Towards application of our human-robot interaction (HRI) methodology on robot platforms in practice with end users, we discuss a generalized procedure for transfer to physical systems and provide solutions for key pragmatic considerations including the generation of safe robot execution paths for both the robot and people in the environment. The paper concludes with an evaluation of our spatial language-based HRI framework implemented on a PR2 robot to demonstrate the generalizability and usefulness of our approach in real world applications.

We performed a study that examined the effects of a humanoid robot giving the minimum required feedback - graded cueing - during a one-on-one imitation game played children with autism spectrum disorders (ASD). 12 high-functioning participants with ASD, ages 7 to 10, each played “Copy-Cat” with a Nao robot 5 times over the span of 2.5 weeks. While the graded cueing model was not exercised in its fullest, using graded cueing-style feedback resulted in a nondecreasing trend in imitative accuracy when compared to a non-adaptive condition, where participants always received the same, most descriptive feedback whenever they made a mistake. These trends show promise for future work with robots encouraging autonomy in special needs populations.

This paper describes an extended (6-session) interaction between an ethnically and geographically diverse group of 26 first-grade children and the DragonBot robot in the context of learning about healthy food choices. We find that children demonstrate a high level of enjoyment when interacting with the robot, and a statistically significant increase in engagement with the system over the duration of the interaction. We also find evidence of relationship-building between the child and robot, and encouraging trends towards child learning. These results are promising for the use of socially assistive robotic technologies for long-term one-on-one educational interventions for younger children.

We present a methodology for enabling service robots to follow natural language commands from non-expert users, with and without user-specified constraints, with a particular focus on spatial language understanding. As part of our approach, we propose a novel extension to the semantic field model of spatial prepositions that enables the representation of dynamic spatial relations involving paths. The design, system modules, and implementation details of our robot software architecture are presented and the relevance of the proposed methodology to interactive instruction and task modification through the addition of constraints is discussed. The paper concludes with an evaluation of our robot software architecture implemented on a simulated mobile robot operating in both a 2D home environment and in real world environment maps to demonstrate the generalizability and usefulness of our approach in real world applications.

We present a methodology for the representation of dynamic spatial relations (DSRs) with global properties as part of an approach for enabling robots to follow natural language commands from non-expert users, with particular focus on the development of spatial language primitives. Our approach to modeling DSRs is based on related research in the fields of linguistics, cognitive science, and neuroscience, and contributes novel extensions to the semantic field model of spatial prepositions. We describe novel representations of the DSRs for “to”, “through”, and “around”, discuss their applicability in path classification scenarios, and provide implementation details of path generation routines instantiating these DSRs for use in robot task planning. The paper concludes with an evaluation of our robot architecture implemented on a simulated mobile robot in a 2D home environment.

We present a socially assistive robot (SAR) system designed to engage elderly users in physical exercise. We discuss the system approach, design methodology, and implementation details, which incorporate insights from psychology research on intrinsic motivation, and we present five clear design principles for SAR-based therapeutic interventions. We then describe the system evaluation, consisting of a multi-session user study with older adults (n = 33), to evaluate the effectiveness of our SAR exercise system and to investigate the role of embodiment by comparing user evaluations of similar physically and virtually embodied coaches. The results validate the system approach and effectiveness at motivating physical exercise in older adults according to a variety of user performance and outcomes measures. The results also show a clear preference by older adults for the physically embodied robot coach over the virtual coach in terms of enjoyableness, helpfulness, and social attraction, among other factors.

In this paper, we draw upon insights gained in our previous work on human-human proxemic behavior analysis to develop a novel method for human-robot proxemic behavior production. A probabilistic framework for spatial interaction has been developed that considers the sensory experience of each agent (human or robot) in a co-present social encounter. In this preliminary work, a robot attempts to maintain a set of human body features in its camera field-of-view. This methodology addresses the functional aspects of proxemic behavior in human-robot interaction, and provides an elegant connection between previous approaches.

In this paper, we present the design, implementation, and user study evaluation of a socially assistive robot (SAR) system designed to engage elderly users in physical exercise aimed at achieving health benefits and improving quality of life. We discuss our design methodology, which incorporates insights from psychology research in the area of intrinsic motivation, and focuses on maintaining engagement through personalized social interaction. We describe two user studies conducted to test the motivation theory in practice with our system. The first study investigated the role of praise and relational discourse in the exercise system by comparing a relational robot coach to a nonrelational robot coach. The second study evaluated participant preferences regarding user choice in the task scenario. Both studies served to evaluate the feasibility and overall effectiveness of the robot exercise system. The results of both studies are presented; they show a strong user preference for the relational over the nonrelational robot in terms of enjoyableness, companionship, and as an exercise coach, varying user preferences regarding choice, and high user ratings of the system across multiple metrics. The outcomes of the presented user studies, brought together, support the motivational capabilities of the robot, and demonstrate the viability and usefulness of the system in motivating exercise in elderly users.

Sensing and interpreting the user's activities and social behavior, monitoring the dynamics of the social context, and selecting and producing appropriate robot action are the core challenges involved in using robots as social tools in interaction scenarios. In social human-robot interaction, speech and gesture are the commonly considered interaction modalities. In human-human interactions, interpersonal distance between people can contain significant social and communicative information. Thus, if human-robot interaction reflects this human-human interaction property, then human-robot distances also convey social information. If a robot is to be an effective social agent, its actions, including those relating to interpersonal distance, must be appropriate for the given social situation. This becomes a greater challenge in playful and unstructured interactions, such as those involving children.

Recent feasibility studies involving children with autism spectrum disorders (ASD) interacting with socially assistive robots have shown that some children have positive reactions to robots, while others may have negative reactions. It is unlikely that children with ASD will enjoy any robot 100% of the time. It is therefore important to develop methods for detecting negative child behaviors in order to minimize distress and facilitate effective human-robot interaction. Our past work has shown that negative reactions can be readily identified and classified by a human observer from overhead video data alone, and that an automated position tracker combined with human-determined heuristics can differentiate between the two classes of reactions. This paper describes and validates an improved, non-heuristic method for determining if a child is interacting positively or negatively with a robot, based on Gaussian mixture models (GMM) and a naive-Bayes classifier of overhead camera observations. The approach achieves a 91.4% accuracy rate in classifying robot interaction, parent interaction, avoidance, and hiding against the wall behaviors and demonstrates that these classes are sufficient for distinguishing between positive and negative reactions of the child to the robot.

We present a user study and dataset designed and collected to analyze how humans use space in face-to-face interactions. In a proof-of-concept investigation into human spatial dynamics, a Hidden Markov Model (HMM) was trained over a subset of features to recognize each of three interaction cues - initiation, acceptance, and termination - in both dyadic and triadic scenarios; these cues are useful in predicting transitions into, during, and out of multi-party social encounters. It is shown that the HMM approach performed twice as well as a weighted random classifier, supporting the feasibility of recognizing and predicting social behavior based on spatial features.

Several machine learning techniques are used to model the behavior of children with autism interacting with a humanoid robot, comparing a static model to a dynamic model using hand-coded features. Good accuracy (over 80%) is achieved in predicting child vocalizations; directions for future approaches to modeling the behavior of children with autism are suggested.

This paper presents an application of a socially assistive robotics (SAR) system to hands-off post-stroke rehabilitation. We validate the technical feasibility and efficacy of our system in guiding, motivating, and administering an upper extremity rehabilitation task. The robot, which consists of a humanoid torso on a mobile base, monitors user performance on a wire puzzle task through a wearable inertial measurement unit and signals from the puzzle. Smoothness of stroke-affected limb movement is used as the evaluation metric. Five adults of mild to moderate functional ability in the chronic phase of stroke recovery interacted with our SAR system over three separate days. The inertial data from the five participants were analyzed using frequency domain techniques. Subsequently, the amount of power in frequency bands corresponding to voluntary (0.1 to 2Hz) and involuntary motion/jerk (4 to 8Hz) was evaluated. We found that, in adults of mild severity (Upper Extremity Fugl-Meyer Assessment scores greater than 40), the motion becomes smoother (the amount of jerk is reduced) over 3 days of task practice. In adults of moderate motor severity (scores below 40), the motion became less smooth. This may indicate that the combination of our task and SAR system is better suited for individuals with higher functional ability, and needs augmentation in order to aid those of lower functional ability levels.

In many collocated human-robot interaction scenarios, robots are required to accurately and unambiguously indicate an object or point of interest in the environment. Realistic, cluttered environments containing many visually salient targets can present a challenge for the observer of such pointing behavior. In this paper, we describe an experiment and results detailing the effects of visual saliency and pointing modality on human perceptual accuracy of a robot's deictic gestures (head and arm pointing) and compare the results to the perception of human pointing.

Automatic recognition of emotion is becoming an increasingly important component in the design process for affect-sensitive human-machine interaction (HMI) systems. Well-designed emotion recognition systems have the potential to augment HMI systems by providing additional user state details and by informing the design of emotionally relevant and emotionally targeted synthetic behavior. This paper describes an emotion classification paradigm, based on emotion profiles (EPs). This paradigm is an approach to interpret the emotional content of naturalistic human expression by providing multiple probabilistic class labels, rather than a single hard label. EPs provide an assessment of the emotion content of an utterance in terms of a set of simple categorical emotions: anger; happiness; neutrality; and sadness. This method can accurately capture the general emotional label (attaining an accuracy of 68.2% in our experiment on the IEMOCAP data) in addition to identifying underlying emotional properties of highly emotionally ambiguous utterances. This capability is beneficial when dealing with naturalistic human emotional expressions, which are often not well described by a single semantic label.

Recent feasibility studies involving children with autism spectrum disorders (ASD) interacting with socially assistive robots have shown that children can have both positive and negative reactions to robots. These reactions can be readily identified by a human observer watching videos from an overhead camera. Our goal is to automate the process of such behavior analysis. This paper shows how a heuristic classifier can be used to discriminate between children that are attempting to interact socially with a robot and children that are not.

We describe the design and implementation of a socially assistive robot that monitors the performance of a user during a seated arm exercise scenario, with the purpose of providing motivation to the user to complete the task and to improve performance. The visual arm pose recognition procedure used by the robot in tracking user performance, the three exercise games, and the methodology behind the human-robot interaction dialogue are presented. A two-condition experimental study was conducted with elderly participants to test the feasibility and effectiveness of the robot exercise system, the results of which demonstrate the viability and usefulness of the system in motivating exercise among elderly users.

New approaches to rehabilitation and health care have developed due to advances in technology and human robot interaction (HRI). Socially assistive robotics (SAR) is a subcategory of HRI that focuses on providing assistance through hands-off interactions. We have developed a SAR architecture that facilitates multiple task-oriented interactions between a user and a robot agent. The architecture accommodates a variety of inputs, tasks, and interaction modalities that are used to provide relevant, real-time feedback to the participant. We have implemented the architecture and validated its technological feasibility in a small pilot study in which a SAR agent led three post-stroke individuals through an exercise scenario. In the following, we present our architecture design, and the results of the feasibility study.

The proper representation of emotion is of vital importance for human-machine interaction. A correct understanding of emotion would allow interactive technology to appropriately respond and adapt to users. In human-machine interaction scenarios it is likely that over the course of an interaction, the human interaction partner will express an emotion not seen during the training of the machine's emotion models. It is therefore crucial to prepare for such eventualities by developing robust representations of emotion that can distinctly represent emotions regardless of whether the data were seen during training of the representation. This novel work demonstrates that an Emotion Profile (EP) representation introduced in [1], a representation composed of the confidences of four binary emotion-specific classifiers, can distinctly represent emotions unseen during training. The classification accuracy increases by only 0.35% over the full dataset when the data excluded from the EP training is included. The results demonstrate that EPs are a robust method for emotion representation.

We describe the design and implementation of a socially assistive robot that is able to monitor the performance of a user during a combined mental and physical task, with the purpose of motivating the user to complete the task and to improve performance. A three-condition experimental study was constructed for evaluation of the robot and preliminary results of the robot's interaction with human participants are presented.

We present a preliminary implementation of a robot within the context of social skills intervention. The robot engages a human user in an interactive and adaptive game-playing session that emphasizes a specific sequence of movements over time. Such games highlight joint attention and encourage forms of interaction that are useful within various assistive domains. Noteworthy robot activities include those that could be used to promote social cues in children with autism, sequences that maintain or improve memory in Alzheimer's patients, and movements that encourage exercises to increase range of motion in post-stroke rehabilitation.

Currently the 2 percent growth rate for the world's older population exceeds the 1.2 percent rate for the world's population as a whole. This difference is expected to increase rather than diminish so that by 2050, the number of individuals over the age 85 is projected to be three times what it is today. Most of these individuals will need physical, emotional, and cognitive assistance. In this paper, we present a new system based on the socially assistive robotics (SAR) technology that will play the role of a music therapist and will try to provide a customized help protocol through motivation, encouragements, and companionship to users suffering from cognitive changes related to aging and/or Alzheimer's disease.

Emotion perception is a complex process, often measured using stimuli presentation experiments that query evaluators for their perceptual ratings of emotional cues. These evaluations contain large amounts of variability both related and unrelated to the evaluated utterances. One approach to handling this variability is to model emotion perception at the individual level. However, the perceptions of specific users may not adequately capture the emotional acoustic properties of an utterance. This problem can be mitigated by the common technique of averaging evalu-ations from multiple users. We demonstrate that this averaging procedure improves classification performance when compared to classification results from models created using individual-specific evaluations. We also demonstrate that the performance increases are related to the consistency with which evaluators label data. These results suggest that the acoustic properties of emotional speech are better captured using models formed from averaged evaluations rather than from individual-specific eval-uations.

This paper describes a high-fidelity model of wireless propagation that integrates several existing models from the wireless communications literature. The model accounts for environmental features, including fading (large and small-scale, and multipath), link-layer models, and interference between radios. In addition to identification and integration of the complementary communication components, this paper's contribution is in demonstrating how discretization, approximation and batch pre-calculation allow the complete model to remain practicable for real-time robot simulation. The faithfulness of the simulated communications is assessed by showing how important qualitative aspects of the communication behavior are reproduced.

This research focuses on studying the possible role of a socially interactive robot as a tool for monitoring and encouraging cognitive activities of the elderly and/or individuals suffering from dementia. One of the aims of this work is to show the benefits of the robot's physical embodiment in human-robot social interactions. The social therapist robot tries to provide customized cognitive stimulation by playing a music game with the user. The results of the 8-month pilot study depict a more efficient, natural, and preferred interaction with the robot rather than with the simulated robot.

Computer simulated avatars and humanoid robots have an increasingly prominent place in today's world. Acceptance of these synthetic characters depends on their ability to properly and recognizably convey basic emotion states to a user population. This study presents an analysis of the interaction between emotional audio (human voice) and video (simple animation) cues. The emotional relevance of the channels is analyzed with respect to their effect on human perception and through the study of the extracted audio-visual features that contribute most prominently to human perception. As a result of the unequal level of expressivity across the two channels, the audio was shown to bias the perception of the evaluators. However, even in the presence of a strong audio bias, the video data were shown to affect human perception. The feature sets extracted from emotionally matched audio-visual displays contained both audio and video features while feature sets resulting from emotionally mismatched audio-visual displays contained only audio information. This result indicates that observers integrate natural audio cues and synthetic video cues only when the information expressed is in congruence. It is therefore important to properly design the presentation of audio-visual cues as incorrect design may cause observers to ignore the information conveyed in one of the channels.

Our research is the exploration of the social effects of human-robot interaction (HRI) on children with ASD, a population that has deficiencies in many types of social behavior. Computers and robots have been shown to be a catalyst for increased social interaction in children with ASD, yet that effect requires further study to be effectively employed as a therapeutic intervention.

Audio-visual emotion expression by synthetic agents is widely employed in research, industrial, and commercial applications. However, the mechanism through which people judge the multimodal emotional display of these agents is not yet well understood. This study is an attempt to provide a better understanding of the interaction between video and audio channels through the use of a continuous dimensional evaluation framework of valence, activation, and dominance. The results indicate that the congruent audio-visual presentation contains information allowing users to differentiate between happy and angry emotional expressions to a greater degree than either of the two channels individually. Interestingly, however, sad and neutral emotions which exhibit a lesser degree of activation show more confusion when presented using both channels. Furthermore, when faced with a conflicting emotional presentation, users predominantly attended to the vocal channel. It is speculated that this is most likely due to the limited level of facial emotion expression inherent in the current animated face. The results also indicate that there is no clear integration of audio and visual channels in emotion perception as in speech perception indicated by the McGurk effect. The final judgments were biased toward the modality with stronger expression power.

Expressive audio-visual synthetic characters are increasingly employed in research and commercial applications. However, the mechanism that people employ to interpret conflicting or uncertain multimodal emotional displays of these agents is not yet well understood. This study is an attempt to provide a better understanding of the interpretation of conflicting expressive displays in video and audio channels through the use of a continuous dimensional evaluation framework of emotional valence, activation, and dominance. The results indicate that when two conflicting emotions are presented to subjects using audio and video channels, the means of the dimensional evaluations of the resulting emotional judgments by the subjects is located in between the audio-only and video-only emotion perceptual centers. Furthermore, the deviation from the audio-only center is proportional to the distance between the audio and video centers. This indicates that the perceptual judgment of conflicting emotions involves the joint processing of both the audio and the video information irrespective of the perceptual bias toward the audio channel. In general the amount of interaction between audio and video channel seems proportional to the emotional disparity of the two channels in the continuous emotional space considered in this study.

Computer simulated avatars and humanoid robots have an increasingly prominent place in today's world. Acceptance of these synthetic characters depends on their ability to properly and recognizably convey basic emotion states to a user population. This study presents an analysis of audio-visual features that can be used to predict user evaluations of synthetic character emotion displays. These features include prosodic, spectral, and semantic properties of audio signals in addition to FACS-inspired video features. The goal of this paper is to identify the audio-visual features that explain the variance in the emotional evaluations of naive listeners through the utilization of information gain feature selection in conjunction with support vector machines. These results suggest that there exists an emotionally salient subset of the audio-visual feature space. The features that contribute most to the explanation of evaluator variance are the prior knowledge audio statistics (e.g., average valence rating), the high energy band spectral components, and the quartile pitch range. This feature subset should be correctly modeled and implemented in the design of synthetic expressive displays to convey the desired emotions.

This paper describes a novel control architecture, B3IA, designed to address the challenges of developing autonomous robot systems for use as behavior intervention tools for children with autism spectrum disorders (ASD). Our goal is to create a system that can be easily adapted for use by non-roboticists engaged in ASD therapy. B3IA is a behavior-based architecture for control of socially assistive robots using human-robot interaction in the ASD context. We hypothesize that the organization of a robot control architecture is important to the success of a robot-assisted intervention, because the success of such intervention hinges on the behavior of the robot. We detail the organization of B3IA and present preliminary results from experiments that begin to experimentally test this hypothesis.

This paper describes a hands-off therapist robot that monitors, assists, encourages, and socially interacts with post-stroke users in the process of rehabilitation exercises. We developed a behavior adaptation system that takes advantage of the users introversion-extroversion personality trait and the number of exercises performed in order to adjust its social interaction parameters (e.g., interaction distances/proxemics, speed, and vocal content) toward a customized post-stroke rehabilitation therapy. The experimental results demonstrate the robot's autonomous behavior adaptation to the user's personality and the resulting user improvements of the exercise task performance.

Achieving and maintaining user engagement is a key goal of human-robot interaction. This paper presents a method for determining user engagement state from physiological data (including galvanic skin response and skin temperature). In the reported study, physiological data were measured while participants played a wire puzzle game moderated by either a simulated or embodied robot, both with varying personalities. The resulting physiological data were segmented and classified based on position within trial using the K-Nearest Neighbors algorithm. We found it was possible to estimate the user's engagement state for trials of variable length with an accuracy of 84.73%. In future experiments, this ability would allow assistive robot moderators to estimate the user's likelihood of ending an interaction at any given point during the interaction. This knowledge could then be used to adapt the behavior of the robot in an attempt to re-engage the user.

In this work, we further test the hypothesis that physical embodiment has a measurable effect on performance and impression of social interactions. Support for this hypothesis would suggest fundamental differences between virtual agents and robots from a social standpoint and would have significant implications for human-robot interaction. We have refined our task-based metrics to give a measurement, not only of the participant's immediate impressions of a coach for a task, but also of the participant's performance in a given task. We measure task performance and participants' impression of a robot's social abilities in a structured task based on the Towers of Hanoi puzzle. Our experiment compares aspects of embodiment by evaluating: (1) the difference between a physical robot and a simulated one; and (2) the effect of physical presence through a co-located robot versus a remote, tele-present robot. With a participant pool (n=21) of roboticists and non- roboticists, we were able to show that participants felt that an embodied robot w as more appealing and perceptive of the world than non-embodied robots. A larger pool of participants (n=32) also demonstrated that the embodied robot was seen as most helpful, watchful, and enjoyable when compared to a remote tele-present robot and a simulated robot.

No abstract available.

This paper presents results toward our ongoing research program into hands-off assistive human-robot interaction [6]. Our work has focused on applications of socially assistive robotics in health care and education, where human supervision can be significantly augmented and complemented by intelligent machines. In this paper, we focus on the role of embodiment, empirically addressing the question: "In what ways can the robot's physical embodiment be used effectively to positively influence human task-related behavior?" We hypothesized that users' personalities would correlate with their preferences of robot behavior expression. To test this hypothesis, we implemented an autonomous mobile robot aimed at the role of a monitoring and encouragement system for stroke patient rehabilitation. We performed a pilot study that indicates that the presence and behavior of the robot can influence how well people comply with their physical therapy.

Automatic recognition of unstructured environments is an important problem for mobile robots. We focus on using audio features to recognize different auditory environments, where they are characterized by different types of sounds. The use of audio information provides a complementary means of scene recognition that can effectively augment visual information. In particular, audio can be used toward both the analysis and characterization of the environment at a higher level of abstraction. We begin our investigation of recognizing different auditory environments with the audio information. In this paper, we utilize low-level audio features from a mobile robot and investigate using high-level features based on spectral analysis for scene characterization, and a recognition system was built to discriminate between different environments based on these audio features found.

Physical interference limits the utility of large-scale multi-robot systems. We present an empirical study of the effects of such interference in systems with hundreds of minimalist robots. We consider the canonical multi-robot foraging task, and define a new parametrized controller. This controller allows for evaluation of spatial arbitration strategies along a continuum with the traditional homogeneous and bucket-brigading algorithms at each end. We present data from thousands of simulations which suggests that methods surprisingly close to homogeneous foraging, but augmented with limited arbitration, can improve both performance and reliability

Autonomous robots are agents with physical bodies that share our environment. In this work, we test the hypothesis that physical embodiment has a measurable effect on performance and perception of social interactions. Support of this hypothesis would suggest fundamental differences between virtual agents and robots from a social standpoint and have significant implications for human-robot interaction. We measure task performance and perception of a robot's social abilities in a structured but open-ended task based on the Towers of Hanoi puzzle. Our experiment compares aspects of embodiment by evaluating: (1) the difference between a physical robot and a simulated one; (2) the effect of physical presence through a co-located robot versus a remote tele-present robot. We present data from a pilot study with 12 subjects showing interesting differences in perception of remote physical robot's and simulated agent's attention to the task, and task enjoyment.

This paper describes a novel representation for two-dimensional maps, and shows how this representation may be applied to the problem of multirobot simultaneous localization and mapping. We are inspired by the notion of a manifold, which takes maps out of the two-dimensional plane and onto a surface embedded in a higher-dimensional space. The key advantage of the manifold representation is self-consistency: when closing loops, manifold maps do not suffer from the "cross over" problem exhibited in planar maps. This self-consistency, in turn, facilitates a number of important capabilities, including autonomous exploration, search, and retro-traverse. It also supports a very robust form of loop closure, in which pairs of robots act collectively to confirm or reject possible correspondence points. In this paper, we develop the basic formalism of the manifold representation, show how this may be applied to the multirobot simultaneous localization and mapping problem, and present experimental results obtained from teams of up to four robots in environments ranging in size from 400 to 900 m/sup 2/.

We present an extension of Isomap nonlinear dimension reduction (Tenenbaum et al., 2000) for data with both spatial and temporal relationships. Our method, ST-Isomap, augments the existing Isomap framework to consider temporal relationships in local neighborhoods that can be propagated globally via a shortest-path mechanism. Two instantiations of ST-Isomap are presented for sequentially continuous and segmented data. Results from applying ST-Isomap to real-world data collected from human motion performance and humanoid robot teleoperation are also presented.

We present a unified methodology for humanoid robot control and activity, classification using motor primitives (Mataric, M, 2002), computationally efficient behaviors capable of perception and control. These primitives constitute a vocabulary for humanoid control capable of generating a large variety of complex movement through sequencing and superposition. We demonstrate how such primitives can be automatically derived from human motion-capture data, how they can be used to construct upperbody controllers, and how they can be applied to classification of observed humanoid behavior in real time.

We evaluate the use of dynamic roadmaps for online motion planning in changing environments. When changes are detected in the workspace, the validity state of affected edges and nodes of a precompiled roadmap are updated accordingly. We concentrate in this paper on analyzing the tradeoffs between maintaining dynamic roadmaps and applying an on-line bidirectional rapidly-exploring random tree (RRT) planner alone, which requires no preprocessing or maintenance. We ground the analysis in several benchmarks in virtual environments with randomly moving obstacles. Different robotics structures are used, including a 17 degrees of freedom model of NASA's Robonaut humanoid. Our results show that dynamic roadmaps can be both faster and more capable for planning difficult motions than using on-line planning alone. In particular, we investigate its scalability to 3D workspaces and higher dimensional configurations spaces, as our main interest is the application of the method to interactive domains involving humanoids.

This paper addresses the problem of directional audio beacon deployment. We describe how these beacons can be used on mobile robots to produce a system that can self-deploy and aid in disaster recovery efforts. A distributed algorithm that uses explicit communication to coordinate the deployment process is presented. The algorithm employs existing multi-robot task allocation methodologies and a procedure for clustering potential deployment locations in a problem domain-specific manner. Results from a sensor-based multi-robot simulation demonstrate that self-deploying beacons are indeed feasible and have the potential to decrease expected egress time. Furthermore, we show that the implementation is free of simulator-specific anomalies through trials with a group of physical robots.

We propose a simple, reactive method for multiple robots carrying out sequential low-visibility navigation in the presence of an observer. Initially, the robots have no map of the environment but know the locations of the observer and goal. They generate an occupancy grid representation of the environment which is modeled using potential fields with embedded task information. These fields are combined and navigation waypoints extracted. Each robot carries out its traverse independently and shares its experience with its successor. The experience information consists of the occupancy grid and a filtered version of the traveled path used to assist the subsequent robot to traverse a lower visibility path. This produces a robust and reactive solution for stealthy navigation since there is no global path planning and the robots are not committed to any particular path. Experiments in simulation and real outdoor environments substantiate the approach and demonstrate the benefits of sharing information in reducing cumulative visibility. The experiments also demonstrate the algorithm's versatility in taking advantage of an environment that changes between robot traverses.

To enable the successful deployment of task-achieving multi-robot systems (MRS), coordination mechanisms must be utilized in order to effectively mediate the interactions between the robots and the task environment. Over the past decade, there have been a number of elegant experimentally demonstrated MRS coordination mechanisms. Most of these mechanisms have been task-specific in nature, typically providing only empirical insights into coordination design and little in the way of systematic techniques to assist in the design of coordinated MRS for new task domains. To fully realize the potentials of MRS, formally-grounded systematic techniques amenable to analysis are needed in order to facilitate the design of coordinated MRS. We address this problem by presenting a formal framework for describing and reasoning about coordination in a MRS. Using this principled foundation, we are developing a suite of general methods for automatically synthesizing the controllers of robots constituting a MRS such that the given task is performed in a coordinated fashion. This paper presents a method for the automatic synthesis of a specific type of controller, one that is stateless but capable of inter-robot communication. We also present a graph coloring-based approach for minimizing the number of necessary unique communication messages. The synthesis of such communicative controllers provides a means for assessing the uses and limitations of communication in MRS coordination. We present experimental validation of our formal approach of controller synthesis in a multi-robot construction domain through physically-realistic simulations and in real-robot demonstrations.

We present a laser range-finder-based system for tracking people in an outdoor environment and detecting interactions between them. The system does not use identities of people for tracking. Observed tracks are automatically segmented into individual activities using an entropy-based measure (Jensen-Shannon divergence (Lin, J, 1991)). Two people situated close to each other throughout the duration of an activity represents an interaction. The observed activities are combined using a hierarchical clustering algorithm to generate a representative set. The frequency of occurrence of these activities is modeled by a Poisson distribution. During the monitoring phase, this model is used to compute the probability of observing the detected activities and interactions; an anomaly is flagged if this probability falls below a threshold. Experimental results from an outdoor courtyard environment are described where the system indicates anomalies when there is a sudden increase in the number of people in the environment or in the number of interactions. This detection occurs without giving the system any a priori concepts of space occupancy.

Remaining elusive while navigating to a goal in a dynamic environment containing an observer requires taking advantage of opportunistic cover as it occurs. A reactive navigation approach is needed that recognizes the utility of environment features in offering protective cover. We present an approach that allows stealthy traverses in unknown environments containing dynamic objects. It is a frontier-based method that allows a robot to follow in the obscuring shadow of objects despite their dynamics, and take advantage of more opportunistic cover if it becomes available. An analysis of our approach in off-line modeling and experiments conducted in simulation and outdoor environments demonstrate its effectiveness in achieving high quality solutions for stealthy navigation.

An approach for model-free markerless motion capture of articulated kinematic structures is presented. This approach is centered our method for generating underlying nonlinear axes (or a skeleton curve) from the volume of an arbitrary rigid-body model. We describe the use of skeleton curves for deriving a kinematic model and motion (in the form of joint angles over time) from a captured volume sequence. Our motion capture method uses a skeleton curve, found in each frame of a volume sequence, to automatically determine kinematic postures. These postures are then aligned to determine a common kinematic model for the volume sequence. The derived kinematic model is then reapplied to each frame in the volume sequence to find the motion suited to this model. We demonstrate our method for several types of motion from synthetically generated volume sequences with arbitrary kinematic topology and human volume sequences captured from a set of multiple calibrated cameras.

This paper presents an algorithm for task allocation in groups of homogeneous robots. The algorithm is based on vacancy chains, a resource distribution strategy common in human and animal societies. We define a class of task-allocation problems for which the vacancy chain algorithm is suitable and demonstrate how reinforcement learning can be used to make vacancy chains emerge in a group of behavior-based robots. Experiments in simulation show that the vacancy chain algorithm consistently outperforms random and static task allocation algorithms when individual robots are prone to distractions or breakdowns, or when task priorities change.

Important theoretical aspects of multi-robot coordination mechanisms have, to date, been largely ignored. To address part of this negligence, we focus on the problem of multi-robot task allocation. We give a formal, domain-independent, statement of the problem and show it to be an instance of another, well-studied, optimization problem. In this light, we analyze several recently proposed approaches to multi-robot task allocation, describing their fundamental characteristics in such a way that they can be objectively studied, compared, and evaluated.

This paper describes a cooperative method for relative localization of mobile robot teams; that is, it describes a method whereby every robot in the team can estimate the pose of every other robot, relative to itself. This robot does not require the use of GPS, landmarks, or maps of any kind; instead, robots make direct measurement of the relative pose of nearby robots, and broadcast this information to the team as a whole. Each robot processes this information independently to generate ego-centric estimate for the pose of other robots. Our method uses Bayesian formalism with a particle filter implementation, and is, as a consequence, very robust. It is also completely distributed, yet requires relatively little communication between robots. This paper describes the basic ego-centric formalism, sketches the implementation, and presents experimental results obtained using a team of four mobile robots.

In this paper we present a method by which to organize the interactions of autonomous assembly agents in intelligent self-assembly (ISA). Each assembly agent has limited and local sensing and local rule-based control. A transition rule set (TRS) compiler is presented which takes a desired goal structure as input and gives as output a set of rules. When each assembly agent utilizes these rules, the desired goal structure is assembled. We show that the TRS compiler is scalable, efficient, and is capable of producing rules for a large class of goal structures.

Much of the current research in human-robot interaction is concerned with single systems and single or few users. These systems and their interfaces are generally tightly-coupled and well-defined. For large-scale human-robot applications, the systems may be unknown prior to designing the interface for potential human interaction. This presents a difficult goal for allowing multiple users to interact with many possibly unknown systems. In this paper, we present an interaction infrastructure aligned with providing this interface. It operates in two phases that accommodate both many-to-many interaction and generalized, one-to-one interaction between users and robotic systems. Our previous research has demonstrated the infrastructure to scale to a large number of users and several systems in simulation. The experiments in this paper substantiate these results in a smaller-scale real robotic environment.

We introduce a computationally efficient methodology for generating and recognizing free-space movements for humanoid robots. This methodology operates on exemplar-based representations of behaviors. Our method for actuating humanoid robots allows us to perform variations on a given behavior, resulting in a very humanlike movement appearance. Besides control, this method also facilitates classification of perceived human and humanoid robot movement. We demonstrate the method on a physically-simulated humanoid robot with 132 degrees-offreedom and evaluate our movement classification methodology on two data sets: human motion-capture data and joint-angle data sampled from the simulated robot.

A large-scale minimalist multi-robot system (LMMS) is one composed of a group of robots each with limited capabilities in terms of sensing, computation, and communication. Such systems have received increased attention due to their empirically demonstrated performance and beneficial characteristics, such as their robustness to environmental perturbations and individual robot failure and their scalability to large numbers of robots. However, little work has been done in investigating ways to endow such a LMMS with the capability to achieve a desired division of labor over a set of dynamically evolving concurrent tasks, important in many task-achieving LLMS. Such a capability can help to increase the efficiency and robustness of overall task performance as well as open new domains in which LMMS can be seen as a viable alternative to more complex control solutions. In this paper, we present a method for achieving a desired division of labor in a LMMS, experimentally validate it in a realistic simulation, and demonstrate its potential to scale to large numbers of robots and its ability to adapt to environmental perturbations.

This paper describes how environmental complexity measures can be employed in the process of validating experimental robotics work. We advocate the use of metrics that attempt to quantify the 'difficulty' of motion for a given environment. Space syntax methods (from the urban and building design literature) and fluid-flow models (used in crowd modeling) are described and proposed as relevant measures for mobile robotics domains. We show experimentally that these two metrics give very different expressions of complexity. We then discuss how, given their properties, these different metrics may be applied to robotics controller design and evaluation.

In this paper, we present a method for motion generation from automatically derived behaviors for a humanoid robot. Behaviors are derived automatically by using the underlying spatio-temporal structure in motion. The derived behaviors are stored in a robot's long-term (or procedural) memory. New motions are generated from the derived ones with a search mechanism. In our approach, vision, speech recognition, short-term memory and decision-making operate in parallel with long-term memory in a unique architecture. This organization is intended for autonomous robot control and learning.

The use of motor primitives for the generation of complex movements is a relatively new and interesting idea for dimensionality reduction in robot control. We propose a framework in which adaptive primitives learn and represent synergetic arm movements. A simple and fixed set of postural and oscillatory primitives form the substrate through which all control is elicited. Higher level adaptive primitives interact and control the primitive substrate in order to handle complex movement sequences. We implemented this model on a simulated 20 DOF humanoid character with dynamics. We present results of the experiments involving the presentation and learning of synergetic arm movements.

Describes a method for real-time tracking of people in everyday environments, using multiple planar laser range-finders. People tracking is a well-studied problem in machine vision; we adapt some of those methods to laser range-finders. We group range measurements into entities such as blobs and objects, and use a Kalman filter to estimate trajectories for these objects. The filter is able to generate smooth trajectories, even when objects are occluded. The paper presents our evaluation of the tracker's performance in a series of four experiments.

We present a graphical user interface (GUI) for easy, intuitive control of a group of mobile robots moving in formation, and we give a short description of the general, local, distributed formation algorithm underlying the GUI. In spite of the high level of abstraction of the GUI, it captures the full functionality of the algorithm. The GUI has been tested with robots running in a simulator, and the formation algorithm itself has also been tested with real robots. We show results from experiments in both scenarios.

We present a distributed planar object manipulation algorithm inspired by human behavior. The system, which we call pusher-watcher, enables the cooperative manipulation of large objects by teams of autonomous mobile robots. The robots are not equipped with gripping devices, but instead move objects by pushing against them. The pusher robots have no global positioning information, and cannot see over the object; thus a watcher robot has the responsibility for leading the team (and object) to the goal, which only it can perceive. The system is entirely distributed, with each robot under local control. Through the use of MURDOCH, an auction-based resource-centric general purpose task-allocation framework, roles in the team are automatically assigned in an efficient manner. Further, robot failures are easily tolerated and, when possible, automatically recovered. We present results and analysis from a battery of experiments with pusher-watcher implemented on a group of Pioneer 2 mobile robots.

Conventionally, mobile robots are controlled through an action selection mechanism (ASM) that chooses among multiple proposed actions. This choice can be made in a variety of ways, and ASMs have been developed that demonstrate many of them, from strict priority schemes to voting systems. We take a different approach, which we call concurrent control. Abandoning explicit action selection, we rely instead on the physical dynamics of the robot's actuators to achieve robust control. We claim that with many noisy controllers and no arbitration among commands we can elicit stable predictable behavior from a mobile robot. Specifically, we concern ourselves with the problem of driving a single planar mobile robot with multiple controlling agents that are concurrently sending commands directly to the robot's wheel motors. We state analytically conditions necessary for our concurrent control approach to be viable, and verify empirically its general effectiveness and robustness to error through experiments with a physical robot.

We describe an empirical study that sought general guidelines for task allocation strategies in multi-robot systems. We identify four distinct task allocation strategies, and demonstrate them in two versions of the multi-robot emergency handling task. We describe an experimental setup to compare results obtained from a simulated grid world to the results from real world experiments. Data resulting from eight hours of real mobile robot experiments are compared to the trend identified in simulation. The data from the simulations show that there is no single strategy that produces best performance in all cases, and that the best task allocation strategy changes as a function of the noise in the system. This result is significant, and shows the need for further investigation of task allocation strategies.

We describe a team of robots that uses a trail of landmarks to navigate between places of interest. The landmarks are not physical; they are waypoint coordinates generated online by each robot and shared with teammates over the network. Waypoints are specified with reference to features in the world that are relevant to the team's task and common to all robots. Using such task-level features as landmarks avoids the need to sense and name physical landmarks. Using these common landmarks, each robot can transform waypoint coordinates into its local reference frame, avoiding the cost of maintaining a fixed global coordinate system. The algorithm is tested in an experiment in which a team of 4 autonomous mobile robots run in our office building for more than 3 hours, travelling a total of 8.2 km (5.1 miles). Despite significant divergence of their local coordinate systems, they are able to share waypoints, forming and following a common trail between two fixed locations.

This paper introduces the concept of a temporal occupancy grid as a method for modeling and classifying spatial areas according to the time properties of their occupancy. The method extends the idea of occupancy grids by considering occupancy over a number of different timescales. This paper presents the basic formalism and its implementation using planar laser rangefinders. It includes the results of a number of validation experiments, and an experiment in which we demonstrate the ability to locate doors in a real-world setting.

This paper presents experiments, in simulation, with a group of robots that improve their performance on a straightforward transportation task by using reinforcement learning to associate input states with a set of abstract behaviors. We show that the improvement in performance is a result of the group adapting its spatio-temporal organization to the given environment. Spatio-temporal adaptation is a general form of adaptation in that it can improve performance over a range of different tasks and environments. Hence it increases the general applicability and autonomy of robotic systems. Lastly, we present two communication strategies that improve this ability to adapt by generally improving learning rates for cooperative robots in highly dynamic domains.

This paper describes an algorithm for deploying the members of a mobile robot team into an unknown environment. The algorithm deploys robots one-at-a-time, with each robot making use of information gathered by the previous robots to determine the next deployment location. The deployment pattern is designed to maximize the area covered by the robots' sensors, while simultaneously ensuring that the robots maintain line-of-sight contact with one another This paper describes the basic algorithm and presents results obtained from a series of experiments conducted using both real and simulated robots.

We address the problem of creating basis behaviors for modularizing humanoid robot control and representing human activity. These behaviors, called perceptual-motor primitives, serve as a substrate for linking a system's perception of human activities and the ability to perform those activities. We present a data-driven method for deriving perceptual-motor action and behavior primitives from human motion capture data. In order to find these primitives, we employ a spatio-temporal non-linear dimension reduction technique on a set of motion segments. From this transformation, motions representing the same action can be clustered and generalized. Further dimension reduction iterations are applied to derive extended-duration behaviors.

We study the problem of construction by autonomous mobile robots focusing on the coordination strategy employed by the robots to solve a simple construction problem efficiently. In particular we address the problem of constructing a linear 2D structure in a planar bounded environment. A "minimalist" single-robot solution to the problem is given, as well as two multi-robot solutions, which are natural extensions to the single-robot approach, with varying degrees of inter-robot communication. Results show that with minimal inter-robot communication (1 bit of state), there is a significant improvement in the system performance. This improvement is invariant with respect to the size of the environment.

In this paper, we present a method of quantifying ideas from proxemics into a set of spatial features that can be used for recognizing, classifying, and modeling multirobot and human activities. We validate the features first with position data from multirobot activities such as flocking, random walk, and boundary following, and then with human activity data. These features are simple and general, and can also be used as performance measures of robot controllers and in learning robot behaviors.

We study the problem of achieving global behavior in a group of distributed robots using only local sensing and minimal communication, in the context of formations. The goal is to have N mobile robots establish and maintain some predetermined geometric shape. We report results from extensive simulation experiments, and 40+ experiments with four physical robots, showing the viability of our approach. The key idea is that each robot keeps a single friend at a desired angle /spl theta/, using some appropriate sensor. By panning the sensor by /spl theta/ degrees, the goal for all formations becomes simply to center the friend in the sensor's field of view. We also present a general analytical measure for evaluating formations and apply it to the position data from both simulation and physical robot experiments. We used two lasers to track the physical robots to obtain ground truth validation data.

The key to utilizing the potential of multirobot systems is cooperation. How can we achieve cooperation in systems composed of failure-prone autonomous robots operating in noisy, dynamic environments? We present a method of dynamic task allocation for groups of such robots. We implemented and tested an auction-based task allocation system which we call MURDOCH, built upon a principled, resource centric, publish/subscribe communication model. A variant of the Contract Net Protocol, MURDOCH produces a distributed approximation to a global optimum of resource usage. We validated MURDOCH in two very different domains: a tightly coupled multirobot physical manipulation task and a loosely coupled multirobot experiment in long-term autonomy. The primary contribution of the paper is to show empirically that distributed negotiation mechanisms such as MURDOCH are viable and effective for coordinating physical multirobot systems.

We describe localization-space trails (LOST), a method that enables a team of robots to navigate between places of interest in an initially unknown environment using a trail of landmarks. The landmarks are not physical; they are waypoint coordinates generated online by each robot and shared with teammates. Waypoints are specified in each robot's local coordinate system, and contain references to features in the world that are relevant to the team's task and common to all robots. Using these task-level references, robots can share waypoints without maintaining a global coordinate system. The method is tested in a series of real-world multirobot experiments. The results demonstrate that the method: 1) copes with accumulating odometry error; 2) is robust to the failure of individual robots; 3) converges to the best route discovered by any robot in the team. In one experiment, a team of four autonomous mobile robots performs a resource transportation task in our uninstrumented office building. Despite significant divergence of their local coordinate systems, the robots are able to share waypoints, forming and following a common trail between two predetermined locations for more than three hours, traveling a total of 8.2 km (5.1 miles) before running out of power. Designed to scale to large populations, LOST is fully distributed, with low costs in processing, memory, and bandwidth. It combines metric data about the position of features in the world with instructions on how to get from one place to another; producing something between a map and a plan.

Successful distributed sensing and control require data to flow effectively between sensors, processors and actuators on single robots, in groups and across the Internet. We propose a mechanism for achieving this flow that we have found to be powerful and easy to use; we call it Player. Player combines an efficient message protocol with a simple device model. It is implemented as a multithreaded TCP socket server that provides transparent network access to a collection of sensors and actuators, often comprising a robot. The socket abstraction enables platform- and language-independent control of these devices, allowing the system designer to use the best tool for the task at hand Player is freely available from http://robotics.usc.edu/player.

We present an approach to the detection of global environmental regime changes by a mobile robot performing a task. The approach is based on the use of augmented Markov models (AMMs), a variation of semi-Markov process. We have developed an algorithm that constructs AMMs online and in real-time with little overhead. AMMs are a general tool for capturing the interaction dynamics between a robot and its environment using the history of behavior executed by the robot. We extend AMMs to regime detection, using multiple models to monitor events at different time scales and provide statistics to detect regime changes at those time scales. This approach has been successfully implemented using a physical mobile robot performing a land mine collection task. In the context of this task we present experimental results, first validating our approach, then demonstrating a more complex proportion-maintaining scenario of the land mine collection task. Finally, we present results using an alternative reward maximization decision criterion in the same task.

This paper considers two problems which at first sight appear to be quite distinct: localizing a robot in an unknown environment and calibrating an embedded sensor network. We show that both of these can be formulated as special cases of a generalized localization problem. In the standard localization problem, the aim is to determine the pose of some object (usually a mobile robot) relative to a global coordinate system. In our generalized version, the aim is to determine the pose of all elements in a network (both fixed and mobile) relative to an arbitrary global coordinate system. We have developed a physically inspired 'mesh-based' formalism for solving such problems. This paper outlines the formalism, and describes its application to the concrete tasks of multirobot mapping and calibration of a distributed sensor network. The paper presents experimental results for both tasks obtained using a set of Pioneer mobile robots equipped with scanning laser range-finders.

In this paper we address the problem of teaching robots to perform various tasks. We present a behavior-based approach that extends the capabilities of robots, allowing them to learn representations of complex tasks from their own experiences of interacting with a human, and to use the acquired knowledge to teach other robots in turn. A learner robot follows a human or robot teacher and maps its own observations of the environment to its internal behaviors, building at run-time a representation of the experienced task in the form of a behavior network. To enable this, we introduce an architecture that allows the representation and execution of complex and flexible sequences of behaviors and an online algorithm that builds the task representation from observations. We demonstrate our approach in a set of human(teacher)-robot(learner) and robot(teacher)-robot(learner) experiments, in which the robots learn representations for multiple tasks and are able to execute them even in environments with distractor objects that could hinder the learning and the execution process.

We describe a prototype task, emergency handling, for multi-robot coordination. The experiments reported measure the effects of individualism and opportunism in a physically-implemented multi-robot system. We use sound at multiple frequencies to simulate emergencies by producing several locally-sensable gradients in the environment. Our results show that opportunism affords a significant performance improvement over individualism. Our experiments also demonstrate the viability of sound for producing detectable local gradients in the environment.

We focus on a robotic domain in which a human acts both as a teacher and a collaborator to a mobile robot. First, we present an approach that allows a robot to learn task representations from its own experiences of interacting with a human. While most approaches to learning from demonstration have focused on acquiring policies (i.e., collections of reactive rules), we demonstrate a mechanism that constructs high-level task representations based on the robot's underlying capabilities. Next, we describe a generalization of the framework to allow a robot to interact with humans in order to handle unexpected situations that can occur in its task execution. Without using explicit communication, the robot is able to engage a human to aid it during certain parts of task execution. We demonstrate our concepts with a mobile robot learning various tasks from a human and, when needed, interacting with a human to get help performing them.

We propose an approach to multi-robot coordination in the context of cooperative target acquisition. The approach is based on multiple objective behavior coordination extended to multiple robots. It provides mechanisms for distributed command fusion across a group of robots to pursue multiple goals of multiple robots in parallel. The mechanisms enable each robot to select actions that not only benefit itself but also benefit the group as a whole. Experimental results with two mobile robots validate that, by using this method, a group of robots can successfully truck and acquire a moving target.

Learning in the mobile robot domain is a very challenging task, especially in nonstationary conditions. This paper presents an approach that allows a robot to learn a model of its interactions with its operating environment in order to manage them according to the experienced dynamics. The robot is initially given a set of "behavior-producing" modules to choose from, and the algorithm provides a means of making that choice intelligently and dynamically. The approach is validated using a vision- and sonar-based Pioneer I robot in non-stationary conditions, in the context of a multirobot foraging task. Results show the effectiveness of the approach in taking advantage of any regularities experienced in the world, leading to fast and adaptable specialization for the learning robot.

. This paper attempts to bridge the fields of machine learning, robotics, and distributed AI. It discusses the use of communication in reducing the undesirable effects of locality in fully distributed multi-agent systems with multiple agents robots learning in parallel while interacting with each other. Two key problems, hidden state and credit assignment, are addressed by applying local undirected broadcast communication in a dual role: as sensing and as reinforcement. The methodology is demonstrated on two multi-robot learning experiments. The first describes learning a tightly-coupled coordination task with two robots, the second a loosely-coupled task with four robots learning social rules. Communication is used to (1) share sensory data to overcome hidden state and (2) share reinforcement to overcome the credit assignment problem between the agents and bridge the gap between local individual and global group pay-off.

This work demonstrates the application of the distributed behavior-based approach to generating a multirobot controller for a group of mobile robots performing a clean-up and collection task. The paper studies a territorial approach to the task in which the robots are assigned individual territories that can be dynamically resized if one of the robots malfunctions, permitting the completion of the task. The described controller is implemented on a group of four IS Robotics R2e mobile robots. Using a collection of experimental robot data, we empirically derive and demonstrate most effective foraging in our domain, and show the decline of performance of the space division strategy with increased group size.

This paper raises the question of the connection between embodiment and higher level cognition, which has been eloquently addressed before but has not yet received much focus in the AI community. The paper then proceeds to break the question down into subparts and address how each can be approached and studied. Finally, the paper briefly overviews two directions of our work, group behavior and imitative behavior, and describes their relation to the issue of embodiment and cognition.

This paper describes the main properties of behaviour-based approaches to control. Different approaches to designing and using behaviours as basic units for control, representation, and learning are illustrated on three empirical examples of robots performing navigation and path-finding, group behaviours, and learning behaviour selection.

This paper deals with the communication in task-sharing between two autonomous six-legged robots equipped with object and goal sensing, and a repertoire of contact and light-following behaviors. The performance of pushing an elongated box towards a goal region is difficult for a single robot and improves significantly when performed cooperatively, but requires careful coordination between the robots. We present and experimentally demonstrate an approach that utilizes cooperation at three levels: sensing, action, and control, and takes the advantage of a simple communication protocol to compensate for the robots' noisy and uncertain sensing.

The author addresses the problem of distributing a task over a collection of homogeneous mobile robots. In contrast to hierarchical methods, which assign a leadership hierarchy and a priori roles to different robots, the proposed approach attempts to detect and utilize the run-time group dynamics within the robot collective. The authors apply a distributed control approach both on the level of the individual robot and on the level of the colony. This choice involves a number of tradeoffs. The complexity of a traditional centralized planner is replaced by the complexity of inter-robot and inter-behavior dynamics. The authors explore methods for not only overcoming interference but utilizing those dynamics to achieve super-linear improvements in task performance. They report the results of testing this architecture on a collection of homogeneous mobile robots. The robots were tested on a number of tasks with a series of more intelligent and efficient local control strategies.>

An architecture that integrates a map representation into a reactive, subsumption-based mobile robot is described. This fully integrated reactive system removes the distinction between the control program and the map. The method was implemented and tested on a mobile robot equipped with a ring of sonars and a compass, and programmed with a collection of simple, incrementally designed behaviors. The robot performs collision-free navigation, dynamic landmark detection, map construction and maintenance, and path planning. Given any known landmark as a goal, the robot plans and executes the shortest known path to it. If the goal is not reachable, the robot detects failure, updates the map, and finds an alternate route. The topological representation primitives are designed to suit the robot's sensors and its navigation behavior, thus minimizing the amount of stored information. Distributed over a collection of behaviors, the map itself performs constant-time localization and linear-time path planning. The approach is qualitative and robust.>

A method for robust mobile robot navigation and environmental learning is presented. It was implemented and tested on a physical robot. The method consists of a collection of simple, incrementally designed robot behaviors. The behaviors receive sonar and compass data which they use to dynamically detect landmarks and construct a distributed map of the environment. The map is represented as a graph in which each node is a collection of augmented finite state machines functioning in parallel. The distributed nature of the map allows for localization in constant time. The method utilizes a modified spreading of activation scheme to accomplish robust linear-time path planning. It is capable of generating both topologically and physically shortest paths to the goal. The method uses local information to achieve the global task without having to replan if the robot becomes lost or strays off the desired path.>