Walking in natural environments is widely recognized as an effective stress reduction strategy, often offering greater benefits than walking in built environments. We examined the physiological and cognitive responses to walking in urban forest versus urban built environments in summer and in winter. This study utilized continuous heart rate monitoring with a wearable chest sensor and 2-back cognitive tests. Higher increases in heart rate during the walk and slower post-walk recovery were observed for walks in built environments compared to those in the forest, in both seasons. However, the magnitudes varied between the seasons, emphasizing the contextual nature of restorative benefits. Improvements in the accuracy of the cognitive tests were observed during the forest walks in summer, but the results were less conclusive in winter. Despite these differences, walking in built environments still conferred well-being benefits, supporting stress reduction regardless of the environment or season.

Virtual characters require the real-time streaming of verbal and nonverbal behaviors for the expression of dynamically generated humor. In this paper, we present the Live Link Animator, a real-time solution for multimodal animation of Unreal Engine characters using individual blendshapes. We demonstrate the tool through an example interaction with a MetaHuman character and outline potential areas of application in the domain of virtual agent humor research.

Existing research has shown that vocal and non-vocal human cues correlate with human trust and distrust behaviours, suggesting their potential to measure human trust in robots in real-time. However, there is a lack of research in Human-Robot Interaction that integrates vocal and non-vocal cues into a comprehensive model to measure trust. This paper aims to estimate human trust in robots by examining vocal and non-vocal cues differences between trust and distrust states across multiple sessions of collaborative game-based HRI with 40 participants. Our analysis revealed that vocal and non-vocal human cues can indeed predict trust in HRI, with certain facial expressions, facial movements, and pitch being significant factors. Random Forest classifier achieved the highest accuracy (84 %) in classifying trust states, with key features such as facial expressions (fear, angry), facial blendshapes (cheekSquintRight, jawRight), and vocal characteristics (Duration, Harmonicity std) being the most predictive of trust. These findings demonstrate the importance of combining vocal and non-vocal cues for accurate trust measurement and highlight the potential for real-time trust assessment in robotic systems.

Humor often serves as a catalyst for smoother interpersonal communication, enhancing interaction experience between individuals.While virtual characters can also gain from these benefits, implementing humor naturally in human-character interactions remains an open challenge.In this paper, we propose the Joking and Multimodally Amusing Real-Time Character (J-MARC) system, combining a photorealistic character with advanced large language model (LLM) techniques to contextualize jokes within small talk.In the real-time interaction, the character is able to present the jokes multimodally and to apply nonverbal behavior while listening.

Role-playing activities offer opportunities for developing individuals’ creativity, communication, and problem-solving skills. Recent advances in large language models (LLM) facilitate fluent conversations with machines. To investigate benefits and pitfalls of LLMs in a relatively unexplored context of human-agent role-play as a culturally contextualised activity, a dataset of twelve human-agent interactions produced by two researchers with two state-of-the-art LLMs was annotated based on a frame analysis scheme from literature. The pilot study shows that human-agent play has a similar complexity as human-human play in which players maintain identities of themselves, external observers and play characters simultaneously going beyond the pretend-reality dualism. Results suggest that, while the LLMs can maintain and shift between roles, they play some roles better than others, and display cultural and gender stereotypes. Additionally, the coding scheme shows potential to help identify LLM outputs that require embodied enactment, and to be used for LLM bench-marking for role-play.

Difficulty adjustment in practice exercises has been shown to be beneficial for learning. However, previous research has mostly investigated close-ended tasks, which do not offer the students multiple ways to reach a valid solution. Contrary to this, in order to learn in an open-ended learning task, students need to effectively explore the solution space as there are multiple ways to reach a solution. For this reason, the effects of difficulty adjustment could be different for open-ended tasks. To investigate this, as our first contribution, we compare different methods of difficulty adjustment in a user study conducted with 86 participants. Furthermore, as the practice behavior of the students is expected to influence how well the students learn, we additionally look at their practice behavior as a post-hoc analysis. Therefore, as a second contribution, we identify different types of practice behavior and how they link to students’ learning outcomes and subjective evaluation measures as well as explore the influence the difficulty adjustment methods have on the practice behaviors. Our results suggest the usefulness of taking into account the practice behavior in addition to only using the practice performance to inform adaptive intervention and difficulty adjustment methods.

This paper investigates the effects of gender neutralization on the perception of anthropomorphism, gender specificity, and novelty for human voices, comparing spoken and hummed voice modalities. We evaluated gender-neutralized and original voice samples in both spoken and hummed formats using an online survey. Our results confirm that gender-neutralizing filters effectively reduce perceived gender specificity in both modalities, supporting their use in creating gender-neutral voices for humanoid robots. Hummed voices were perceived as more anthropomorphic and less novel than spoken voices, suggesting that non-verbal sound modalities can enhance the human likeness of gender-neutral androids while maintaining gender ambiguity. The study contributes to HRI by highlighting the potential of humming to fulfill users’ expectations of interaction with android robots.

To make social robots effective in education, they need to be autonomous both in terms of assessing the student’s engagement state as well as intervening effectively in soft real-time when necessary. Hidden Markov Model (HMM) is an interpretable machine learning technique for modeling temporal data that is commonly used post-hoc to analyse latent learning processes. In this paper, we contribute by proposing an HMM-based intervention methodology for assessing and classifying the state of the student as either productive or unproductive in soft real-time. The system identifies and tracks states and patterns not conducive to learning, and a robot intervention is triggered whenever a too-high non-productive engagement is detected. In a pilot study with 22 children, we evaluate this methodology in terms of both 1) the effectiveness of the interventions on the students’ learning gains and on behaviors found conducive to learning, and 2) the students’ perception of the robotic interventions. Results suggest that the robot interventions have a positive effect on the post-test scores relative to the baseline robot, although there isn’t a significant difference in the learning gains. Moreover, interventions that try to induce reflective behaviors are most effective in inducing the required learning behavior, followed by communication-inducing interventions. Lastly, students’ perception of intervention usefulness does not reflect their actual effectiveness.

In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of productive engagement was developed by utilizing multimodal data of 32 dyads and learning profiles, namely, Expressive Explorers (EE), Calm Tinkerers (CT), and Silent Wanderers (SW) were identified which categorize learners according to their learning gain. Within the same framework, a PE score was constructed in a non-supervised manner for real-time evaluation. Here, we use these profiles and the PE score within an AutoML deep learning framework to personalize PE models. We investigate two approaches for this purpose: (1) Single-task Deep Neural Architecture Search (ST-NAS), and (2) Multitask NAS (MT-NAS). In the former approach, personalized models for each learner profile are learned from multimodal features and compared to non-personalized models. In the MT-NAS approach, we investigate whether jointly classifying the learners’ profiles with the engagement score through multi-task learning would serve as an implicit personalization of PE. Moreover, we compare the predictive power of two types of features: incremental and non-incremental features. Non-incremental features correspond to features computed from the participant’s behaviours in fixed time windows. Incremental features are computed by accounting to the behaviour from the beginning of the learning activity till the time window where productive engagement is observed. Our experimental results show that (1) personalized models improve the recognition performance with respect to non-personalized models when training models for the gainer vs. non-gainer groups, (2) multitask NAS (implicit personalization) also outperforms non-personalized models, (3) the speech modality has high contribution towards prediction, and (4) non-incremental features outperform the incremental ones overall.

Wizard of Oz, a very commonly employed technique in human-robot interaction, faces the criticism of being deceptive as the humans interacting with the robot are told, if at all, only at the end of their interaction that there was in fact a human behind the robot. What if the robot reveals the wizard behind itself very early in the interaction? We built a deep wizard of Oz setup to allow for a robot to play together with a human against a computer AI in the context of Connect 4 game. This cooperative game interaction against a common opponent is then followed by a conversation between the human and the robot. We conducted an exploratory user study with 29 adults with three conditions where the robot reveals the wizard, lies about the wizard, and does not say anything, respectively. We also split the data based on how the participants perceive the robot in terms of autonomy. Using different metrics, we evaluate how the users interact with and perceive the robot in both the experimental and perceived conditions. We find that while there is indeed a significant difference in the participants willingness to follow robots suggestions between the experimental conditions as well as in the effort they put to prove themselves as humans (reverse Turing test), there isn’t any significant difference in their robot perception. Additionally, how humans perceive whether the robot is tele-operated or autonomous seems to be indifferent to the robot revealing its identity, i.e., the pre-conceived notions may be uninfluenced even if the robot explicitly states otherwise. Lastly, interestingly in the perception based conditions, absence of statistical significance may suggest that, in certain contexts, wizard of oz may not require hiding the wizard after all.

Humanoid robots, with a focus on personalised social behaviours, are increasingly being deployed in educational settings to support learning. However, crafting pedagogical HRI designs and robot interventions that have a real, positive impact on participants' learning, as well as effectively measuring such impact, is still an open challenge. As a first effort in tackling the issue, in this paper we propose a novel robot-mediated, collaborative problem solving activity for school children, called JUSThink, aiming at improving their computational thinking skills. JUSThink will serve as a baseline and reference for investigating how the robot's behaviour can influence the engagement of the children with the activity, as well as their collaboration and mutual understanding while working on it. To this end, this first iteration aims at investigating (i) participants' engagement with the activity (Intrinsic Motivation Inventory-IMI), their mutual understanding (IMIlike) and perception of the robot (Godspeed Questionnaire); (ii) participants' performance during the activity, using several performance and learning metrics. We carried out an extensive user-study in two international schools in Switzerland, in which around 100 children participated in pairs in one-hour long interactions with the activity. Surprisingly, we observe that while a teams' performance significantly affects how team members evaluate their competence, mutual understanding and task engagement, it does not affect their perception of the robot and its helpfulness, a fact which highlights the need for baseline studies and multi-dimensional evaluation metrics when assessing the impact of robots in educational activities.

This paper presents the design of a novel and engaging collaborative learning activity for handwriting where a group of participants simultaneously tutor a Nao robot. This activity was intended to take advantage of both collaborative learning and the learning by teaching paradigm to improve children's meta-cognition (perception of their own skills). Multiple engagement probes were integrated into the activity as a first step towards fostering long term interactions. As a lot of research targets social interactions, the goal here was to determine whether an engagement strategy focused on the task could be as, or more efficient than one focused on social interactions and participants' introspection. To that effect, two engagement strategies were implemented. They differed in content but used the same multi-modal design in order to increase participants' meta-cognitive reflection, once on the task and performances, and once on participants' enjoyment and emotions. Both strategies were compared to a baseline by probing and assessing engagement at the individual and group level, along the behavioural, emotional and cognitive dimensions, in a between subject experiment with 12 groups of children. The experiments showed that the collaborative task pushed the children to adapt their manner of writing to the group, even though the adopted solution was not always correct. Furthermore, there was no significant difference between the strategies in terms of behaviour on task (behavioural engagement), satisfaction (emotional engagement) or performance (cognitive engagement) as the group dynamics had a stronger impact on the outcome of the collaborative teaching task. Therefore, the task and social engagement strategies can be considered as efficient in the context of collaboration.

In this paper, we propose that the data generated by educational robots can be better used by applying learning analytics methods and techniques which can lead to a deeper understanding of the learners' apprehension and behavior as well as refined guidelines for roboticists and improved interventions by the teachers. As a step towards this, we put forward analyzing behavior and task performance at team and/or individual levels by coupling robot data with the data from conventional methods of assessment through quizzes. Classifying learners/teams in the behavioral feature space with respect to the task performance gives insight into the behavior patterns relevant for high performance, which could be backed by feature ranking. As a use case, we present an open-ended learning activity using tangible haptic-enabled Cellulo robots in a classroom-level setting. The pilot study, spanning over approximately an hour, is conducted with 25 children in teams of two that are aged between 11-12. A linear separation is observed between the high and low performing teams where two of the behavioral features, namely number of distinct attempts and the visits to the destination, are found to be important. Although the pilot study in its current form has limitations, e.g. its low sample size, it contributes to highlighting the potential of the use of learning analytics in educational robotics.

Memory modeling has been a popular topic of research for improving the performance of autonomous agents in cognition related problems. Apart from learning distinct experiences correctly, significant or recurring experiences are expected to be learned better and be retrieved easier. In order to achieve this objective, this paper proposes a user preference-based dual-memory adaptive resonance theory network model, which makes use of a user preference to encode memories with various strengths and to learn and forget at various rates. Over a period of time, memories undergo a consolidation-like process at a rate proportional to the user preference at the time of encoding and the frequency of recall of a particular memory. Consolidated memories are easier to recall and are more stable. This dual-memory neural model generates distinct episodic memories and a flexible semantic-like memory component. This leads to an enhanced retrieval mechanism of experiences through two routes. The simulation results are presented to evaluate the proposed memory model based on various kinds of cues over a number of trials. The experimental results on Mybot are also presented. The results verify that not only are distinct experiences learned correctly but also that experiences associated with higher user preference and recall frequency are consolidated earlier. Thus, these experiences are recalled more easily relative to the unconsolidated experiences.