The inexorable progress of technology brought forth an era where robots increasingly integrate into human life which necessitates the understanding of human-robot interactions (HRI). This study unravels the details of HRI within interactive storytelling contexts. Through a between-subject experiment with 28 participants, we assessed response latency and utterance lengths to interactive story narrations delivered by either a human or a robot. Findings indicated that participants displayed longer response latency interacting with the robot narrator while articulating shorter utterances compared to the human condition where participants displayed longer utterances and shorter response latency. These observations suggest significant differences in cognitive and communicative strategies in human-human versus human-robot interactions. The results underscore the challenges and potential of designing social robots that are time-sensitive in interacting with humans. Future explorations should focus on the cognitive and emotional drivers behind these interactions.

Social robots are being introduced in a variety of educational domains with great success. These social robots are often designed by drawing inspiration from practices held by human teachers. Contingent mediations are a prime example of a high-quality teaching practice that supports better outcomes in human-human teaching. This can inform the design of social robots. Current research on robot use in education has focused on curriculum-level contingent mediations where the difficulty level of subsequent tasks are adjusted to the current capabilities of a learner. However, task-level contingent mediations that provide support for students’ learning during a specific task remain unexplored. This research investigates whether patterns of task-level mediations differ between a robot, virtual, and human agent, as well as their effects on learning outcomes. To investigate these research questions, we designed instruction that utilizes contingent mediational flows, based on formative assessment data, to be delivered by a human, robot, and virtual agent to teach grade 3-5 children science words. We identified 23 unique instructional patterns. Then, we compared these patterns across teaching agents and their effects on learning outcomes. Overall, our study demonstrated that high-quality contingent mediations support children’s science vocabulary in learning regardless of the teaching agent.

This qualitative study explored 20 grade 3-5 children’s perceptions of learning STEM vocabulary with an educational social robot. A semi-structured interview protocol was used to elicit children’s perceptions. Interviews were recorded and transcribed into a database reflecting one talk-turn per row (911 talk-turns total). Two coders used emergent coding and constant comparative method to identify talk-turns that reflected children’s perceptions of the assets and drawbacks of learning STEM vocabulary with the robot. Findings identified new assets and drawbacks about the robot’s instruction, and new drawbacks about the robot’s speech, which were not explored in previous research. Findings suggest design implications, including designing robots with the capacity for more individualization of instruction and adjustable movement and speech features based on learners’ preferences.

Backchanneling models, designed to enhance the interactive capabilities of robots, have primarily been trained on human-human interaction data. However, applying such data directly to social robots raises concerns due to dissimilarities in the way humans and robots exhibit verbal and nonverbal behaviors, particularly in the domain of emotional backchannels. This research aims to address this gap by conducting an exploratory study on the differences in human backchanneling behaviors during interactions with humans and social robots in various emotional contexts (e.g., happy and sad). Our findings reveal significant variations in emotionally specific backchannels between human-human and human-robot interactions under different emotional contexts. These results highlight the importance of designing backchanneling models that are tailored for human-robot interactions.

This study aimed to evaluate the effectiveness of robot-mediated training for job interviews for young adults with autism spectrum disorder (ASD). The six-week intervention involved mock job interviews with a Furhat social robot to target nonverbal behaviors and communication skills. To measure the efficacy of the intervention, four common nonverbal behavioral challenges among individuals with ASD were investigated: eye gaze, excessive body movement, atypical vocalization, and orientation toward the interviewer. Results indicated varying levels of improvement among participants, with some showing consistent improvement and others exhibiting unexpected results from session to session. This underscores the need for personalized, objective, and quantitative analysis. The study highlights the importance of addressing nonverbal communication challenges for individuals with ASD and equipping them with the necessary job market skills. While the pilot results from robot-mediated training appear promising, further research with a larger group including a wider range of participants with ASD is required to generalize the outcomes.

Turn-taking is a fundamental behavior during human interactions and robots must be capable of turn-taking to interact with humans. Current state-of-the-art approaches in turn-taking focus on developing general models to predict the end of turn (EoT) across all contexts. This demands an all-inclusive verbal and non-verbal behavioral dataset from all possible contexts of interaction. Before robot deployment, gathering such a dataset may be infeasible and/or impractical. More importantly, a robot needs to predict the EoT and decide on the best time to take a turn (i.e, start speaking). In this research, we present a learning from demonstration (LfD) system for a robot to learn from demonstrations, after it has been deployed, to make decisions on the appropriate time for taking a turn within specific social interaction contexts. The system captures demonstrations of turn-taking during social interactions and uses these demonstrations to train a LSTM RNN based model to replicate the turn-taking behavior of the demonstrator. We evaluate the system for teaching the turn-taking behavior of an interviewer during a job interview context. Furthermore, we investigate the efficacy of verbal, prosodic, and gestural cues for deciding when to begin a turn.

Learning from demonstration (LfD) approaches have garnered significant interest for teaching social robots a variety of tasks in healthcare, educational, and service domains after they have been deployed. These LfD approaches often require a significant number of demonstrations for a robot to learn a performant model from task demonstrations. However, requiring non-experts to provide numerous demonstrations for a social robot to learn a task is impractical in real-world applications. In this paper, we propose a method to improve the sample efficiency of existing learning from demonstration approaches via data augmentation, dynamic experience replay sizes, and hierarchical Deep Q-Networks (DQN). After validating our methods on two different datasets, results suggest that our proposed hierarchical DQN is effective for improving sample efficiency when learning tasks from demonstration. In the future, such a sample-efficient approach has the potential to improve our ability to apply LfD approaches for social robots to learn tasks in domains where demonstration data is limited, sparse, and imbalanced.

Children diagnosed with autism spectrum disorder (ASD) typically work towards acquiring skills to participate in a regular classroom setting such as attending and appropriately responding to an instructor’s requests. Social robots have the potential to support children with ASD in learning group-interaction skills. However, the majority of studies that target children with ASD’s interactions with social robots have been limited to one-on-one interactions. Group interaction sessions present unique challenges such as the unpredictable behaviors of the other children participating in the group intervention session and shared attention from the instructor. We present the design of a robot-mediated group interaction intervention for children with ASD to enable them to practice the skills required to participate in a classroom. We also present a study investigating differences in children’s learning behaviors during robot-led and human-led group interventions over multiple intervention sessions. Results of this study suggests that children with ASD’s learning behaviors are similar during human and robot instruction. Furthermore, preliminary results of this study suggest that a novelty effect was not observed when children interacted with the robot over multiple sessions.