Virtual, augmented, and mixed reality for human-robot interaction (VAM-HRI) is a new and rapidly growing field of research. The field of socially assistive robot (SAR) has made impactful advances in educational settings, but has not yet benefited from VAM-HRI advances. We developed MoveToCode - an open-source, embodied (i.e., kinesthetic) learning visual programming language that aims to increase student (ages 8-12) curiosity during programming. MoveToCode uses an augmented reality (AR) autonomous robot tutor named Kuri that models the students’ kinesthetic curiosity and acts to promote their curiosity in programming. MoveToCode design was informed by pilot studies and tested in Los Angeles elementary classrooms $(n =21)$. Results from main study validated our design decisions compared to the pilot study which was conducted in a real elementary school classroom environment $(n =15)$, showing an improvement in perceived robot helpfulness (median $+ \Delta1.25$ out of 5) and number of completed exercises (median $+ \Delta1$, maximum of 11). While no significant changes were found in pre/post student curiosity or intention to program later in life, students wrote more open-ended questions post-study on topics related to robots, programming, research, and if they would like to do the activity again. This work demonstrates the potential of using VAM-HRI in a kinesthetic context for SAR tutors, and highlights the existing conventions and new design considerations for creating AR applications for SAR.

The 5th International Workshop on Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI) will bring together HRI, robotics, and mixed reality researchers to address challenges in mixed reality interactions between humans and robots. Topics relevant to the workshop include development of robots that can interact with humans in mixed reality, use of virtual reality for developing interactive robots, the design of augmented reality interfaces that mediate communication between humans and robots, social applications for virtual and mixed reality in HRI, the investigations of mixed reality interfaces for robot learning, comparisons of the capabilities and perceptions of robots and virtual agents, and best design practices. Special topics of interest this year include VAM-HRI research during the ongoing COVID-19 pandemic as well as the ethical implications of VAM-HRI research. VAM-HRI 2022 will follow on the success of VAM-HRI 2018–21 and advance the cause of this nascent research community. Website: https://vam-hri.github.io

Behavior trees (BTs) are hierarchical agent control architectures popular for robot task-level planning that can be autonomously learned from robot demonstrations via decision tree (DT) intermediaries, making them accessible to non-expert users. Conversion algorithms from DTs to BTs, such as the BT-Espresso algorithm, focus on replicating DT logic in a BT format but do not exploit the strengths of the BT architecture. We introduce the Representation Exploitation of BT-Espresso (RE:BT-Espresso) algorithm, which builds on BT-Espresso and improves the learned BT's interpretability and expressivity. RE:BT-Espresso improves interpretability by removing logical redundancies in the generated BTs and improves expressivity by exploiting desired BT structures, such as adding Inverter nodes, Repeater sequences, and Parallel Selector Action nodes that gives the user a choice of actions for state spaces that did not resolve to a concise action in the DT. The RE:BT-Espresso algorithm was evaluated against BT-Espresso using demonstration data synthesized by BTs. When compared to the synthesized BTs using graph edit distance (GED), RE:BT-Espresso outscored BT-Espresso on 54 subtrees, tied on 178, and lost on 2. Further, the proposed reduction strategies reduced the number of nodes in a generated tree by a median of 7.82%. The results validate improved interpretability and expressivity of learned RE:BT-Espresso task-level BT policies from robot demonstration.

Study reproducibility and generalizability of results to broadly inclusive populations is crucial in any research. Previous meta-analyses in HRI have focused on the consistency of reported information from papers in various categories. However, members of the HRI community have noted that much of the information needed for reproducible and generalizable studies is not found in published papers. We address this issue by surveying the reported study metadata over the main proceedings of the 2021 IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) and the past three years (2019 through 2021) of the main proceedings of the International Conference on Human-Robot Interaction (HRI) and alt.HRI. Based on the analysis results, we propose a set of recommendations for the HRI community that follow the longer-standing reporting guidelines from human-computer interaction (HCI), psychology, and other fields most related to HRI. Finally, we examine two key areas for user study reproducibility: recruitment details and participant compensation. We find a lack of reporting of both of these study metadata categories: of the 414 studies across both conferences and all years, 258 studies failed to report recruitment method and 255 studies failed to report compensation. This work provides guidance about specific types of needed reporting improvements for the field of HRI.

From RViz to augmented reality (AR), a wide variety of robot signal visualizations exist for conveying robot capabilities. Many of the visualizations designed for AR, however, have not isolated multiple salient Virtual Design Elements (VDEs) for a given signal and comparatively evaluated combinations of those VDEs. To address this, we identify multiple VDEs for AR signaling of the following core robot capabilities: navigation, light detection and ranging (LiDAR), camera, face detection, audio localization, and natural language processing. We evaluated each signal's VDE combinations with an Amazon Mechanical Turk study (n=150) where participants watched 4 videos for each signal (consisting of 2 independent VDE choices) and rated the clarity and visual appeal of each signal. The results define a set of the most clear and visually appealing signal visualization designs and inform about interaction effects among VDEs. The resulting VDEs offer design insights and a baseline for continued research into AR robot capability signalling.

Affect-aware socially assistive robotics (SAR) has shown great potential for augmenting interventions for children with autism spectrum disorders (ASD). However, current SAR cannot yet perceive the unique and diverse set of atypical cognitive-affective behaviors from children with ASD in an automatic and personalized fashion in long-term (multi-session) real-world interactions. To bridge this gap, this work designed and validated personalized models of arousal and valence for children with ASD using a multi-session in-home dataset of SAR interventions. By training machine learning (ML) algorithms with supervised domain adaptation (s-DA), the personalized models were able to tradeoff between the limited individual data and the more abundant less personal data pooled from other study participants. We evaluated the effects of personalization on a long-term multimodal dataset consisting of four children with ASD with a total of 19 sessions, and derived inter-rater reliability (IR) scores for binary arousal (IR = 83%) and valence (IR = 81%) labels between human annotators. Our results show that personalized Gradient Boosted Decision Trees (XGBoost) models with s-DA outperformed two non-personalized individualized and generic model baselines not only on the weighted average of all sessions, but also statistically ( p < .05) across individual sessions. This work paves the way for the development of personalized autonomous SAR systems tailored toward individuals with atypical cognitive-affective and socio-emotional needs.

Telepresence robots offer presence, embodiment, and mobility to remote users, making them promising options for homebound K-12 students. It is difficult, however, for robot operators to know how well they are being heard in remote and noisy classroom environments. One solution is to estimate the operator’s speech intelligibility to their listeners in order to provide feedback about it to the operator. This work contributes the first evaluation of a speech intelligibility feedback system for homebound K-12 students attending class remotely. In our four long-term, in-the-wild deployments we found that students speak at different volumes instead of adjusting the robot’s volume, and that detailed audio calibration and network latency feedback are needed. We also contribute the first findings about the types and frequencies of multimodal comprehension cues given to homebound students by listeners in the classroom. By annotating and categorizing over 700 cues, we found that the most common cue modalities were conversation turn timing and verbal content. Conversation turn timing cues occurred more frequently overall, whereas verbal content cues contained more information and might be the most frequent modality for negative cues. Our work provides recommendations for telepresence systems that could intervene to ensure that remote users are being heard.

Telepresence robots act as the remote embodiments of human operators, enabling people to stay connected to friends, family, and coworkers over lengthy physical separations. However, the factors affecting how humans can best make use of such systems are not yet well understood. This paper explores the effects of personalization and relationship closeness on telepresence via two studies. Study 1 was a between-participants experiment that investigated telepresence robot personalization. 32 pairs of friends (N = 64) participated in the study's team-building-style activities and answered questions about robot operator presence. The results unexpectedly indicated that relationship closeness influenced the interaction experience more than any other considered predictor variable. To study closeness more rigorously as the central manipulation, we conducted Study 2, a between-participants experiment with 24 pairs (N = 48) and a similar procedure. Robot operators who reported a closer relationship with their teammate felt more present in this investigation. These findings can inform the design and application of telepresence robot systems to increase a remote operator's feelings of presence via robot.

Expressivity-the use of multiple modalities to convey internal state and intent of a robot-is critical for interaction. Yet, due to cost, safety, and other constraints, many robots lack high degrees of physical expressivity. This paper explores using mixed reality to enhance a robot with limited expressivity by adding virtual arms that extend the robot's expressiveness. The arms, capable of a range of non-physically-constrained gestures, were evaluated in a between-subject study (n =34) where participants engaged in a mixed reality mathematics task with a socially assistive robot. The study results indicate that the virtual arms added a higher degree of perceived emotion, helpfulness, and physical presence to the robot. Users who reported a higher perceived physical presence also found the robot to have a higher degree of social presence, ease of use, usefulness, and had a positive attitude toward using the robot with mixed reality. The results also demonstrate the users' ability to distinguish the virtual gestures' valence and intent.