Human–robot collaboration increasingly frames robots as teammates rather than tools, yet there is limited guidance on how robots should respond when failures are attributed to the human collaborator. We investigate how robot collaborators should respond to support collaboration experience after a human-attributed failure. In a 4 × 2 mixed factorial design (N = 60), participants completed a collaborative block-stacking task with either a humanoid robot (NAO) or a human collaborator under four scenarios: success, affective feedback, cognitive feedback, and no feedback. We measured collaboration experience in terms of teamwork quality, perceived copresence, and intimacy. Both affective and cognitive feedback improved these outcomes compared with no feedback: affective cues yielded the strongest socio-relational gains (copresence, intimacy), whereas cognitive cues more strongly enhanced perceived teamwork quality. These patterns were consistent across human–robot and human–human collaboration, indicating shared team-level expectations that extend beyond the individual actor. The results provide empirical evidence for socially adaptive robots that pair brief emotional reassurance with concrete guidance to support collaboration after human-attributed failures.

We present TouchWalker, a real-time system for controlling fullbody avatar locomotion using finger-walking gestures on a touchscreen. The system comprises two main components: TouchWalker-MotionNet, a neural motion generator that synthesizes full-body avatar motion on a per-frame basis from temporally sparse twofinger input, and TouchWalker-UI, a compact touch interface that interprets user touch input to avatar-relative foot positions. Unlike prior systems that rely on symbolic gesture triggers or predefined motion sequences, TouchWalker uses its neural component to generate continuous, context-aware full-body motion on a per-frame basis-including airborne phases such as running, even without input during mid-air steps-enabling more expressive and immediate interaction. To ensure accurate alignment between finger contacts and avatar motion, it employs a MoE-GRU architecture with a dedicated foot-alignment loss. We evaluate TouchWalker in a user study comparing it to a virtual joystick baseline with predefined motion across diverse locomotion tasks. Results show that TouchWalker improves users' sense of embodiment, enjoyment, and immersion.

The ability to perform instrumental activities of daily living (IADL) allows individuals to live independently. Several evaluation tools can be used to evaluate IADL. Although questionnaire-based evaluation tools are widely used, they have the issue of potentially being reported differently than the participants' actual abilities. Virtual reality-based IADL (VR-IADL) programs have been proposed for improved assistance and evaluation. However, previous VR-IADL programs had limited scenarios and were difficult to use. To overcome this, we developed a ten-scenario version of a tablet-based VR-IADL program. To test the performance of the tablet-based VR-IADL program, two studies were performed. In Study 1, we recruited 20 healthy adults to develop the VR-IADL scoring system, and in Study 2, 31 patients with idiopathic rapid eye movement sleep behavior disorder were recruited to test its validity. Participants in Study 2 were considered to be at high risk of mild cognitive impairment, dementia, and Parkinson's disease. The results suggest that the VR-IADL score was positively correlated with cognitive tests and these correlations were better than those determined using existing tools. We believe that our tablet-based VR-IADL program is beneficial for older adults at high risk for neurodegenerative diseases.

Large language models (LLMs) have evolved into LLM agents that can use the user conversation context and respond according to the roles of the LLM agents. Recent studies have suggested that LLM-based agents can be used as human-like partners in social interactions. However, the role of the environmental context, particularly spatial information of user space, in the interaction between humans and LLM agents has not been explored. In this study, participants engaged in counselling conversations under three different conditions based on their spatial awareness levels. The dependent measures included copresence, trust, therapist alliances, and self-disclosure. The results suggested that participants in the condition where the LLM actively reflected spatial information generally reported higher levels of user experience. Interestingly, when the LLM actively reflected the spatial context of the user, the participants tended to describe themselves and express their emotions more. These findings suggest that spatially aware LLM agents can contribute to better social interactions between humans and LLM agents. Our findings can be used to design future augmented reality applications in the counselling, education, and healthcare industries.

Recent studies have demonstrated that brain synchrony can indicate the quality of social interaction in real-world communication. However, there is a lack of research on measurement of brain synchrony during social interactions in remote AR. In this study, we investigated the brain synchrony of remote augmented reality (AR; Study 1) and face-to-face (FTF; Study 2) interactions. Functional near-infrared spectroscopy was used to measure the brain synchrony during the tangram puzzle task. In a collaboration condition, participants worked together to solve the puzzle. In an individual condition, participants solved the puzzle independently. We recruited 46 participants in Study 1 and 48 participants in Study 2. Study 1 showed there was a significant difference in brain synchrony between the individual and collaboration conditions, and a positive correlation was observed between brain synchrony and the task performance in the collaboration condition. A comparison between Study 1 and 2 suggested that the difference between the collaboration and individual conditions was maintained, and some differences were observed in the brain synchrony between the AR and FTF interactions. These results suggest that measurement of brain synchrony is beneficial for social interaction in remote AR collaborations. The implications of these results on future remote interactions are discussed.

Previous education studies suggested that “warming up” activities featuring interpersonal synchrony components help improve educational quality and social closeness between teachers and students. While educators are eager to incorporate augmented reality (AR) into remote educational settings, prior research has not investigated the beneficial impacts of interpersonal synchrony on remote AR education. This study investigates the effects of prior physical synchrony (PS) on learning outcomes, inter brain synchrony (IBS), and social closeness in remote AR education. 41 participants were recruited for learning English vocabulary from a teacher. They conducted a movement learning task using prior PS and physical asynchrony (PA). The results showed that the positive effects of warming up activity on quiz score, IBS, and social closeness were observed in the PS section, but not in the PA section. Moreover, we found a high correlation between quiz score and IBS but not with social closeness. These results suggest that prior PS is a beneficial component of remote AR education and that IBS could be used as an index for evaluating educational quality. The implications of these results for remote AR education are discussed.

Emotion detection using features presented in the body has been comparatively understudied compared to other emotional modalities. This study investigated and compared how emotions are revealed through bodily sensations and body movement information. We propose a novel visualization method for addressing body part activation or deactivation associated with different emotions using motion capture data (bodily motion maps; BMMs) and aim to compare its emotional features with existing methods of addressing sensation towards body activation or deactivation using the computerised self-report method (bodily sensation maps; BSMs). A user study is conducted with twenty-nine participants to gather BSMs and BMMs under seven emotional statuses (happy, sad, surprise, angry, disgust, fearful, and neutral). The results indicate that people's BMMs as well as their BSMs contain shared features that can discriminate between emotions. The asynchronies between bodily sensation and body motion existed. Furthermore, convolutional neural network (CNN)-based automatic emotion recognition outperforms humans’ emotion recognition abilities. The implications of the current study's method of comparing emotional features between bodily sensation and body motion for future emotion-related studies for mental disorders are discussed.

As one of autonomous vehicles, conditional autonomous vehicles is expected to become popular in the near future. Conditional autonomous vehicles can send a take-over request (TOR) to a driver, and if they are immersed in non-driving-related tasks (NDRT), they will struggle to accommodate this request. Previous studies have shown that providing augmented reality (AR) information on traffic situations (status cues) or driver actions (command cues) can improve TOR performance. However, we are not aware of any studies comparing the types of AR cues (state versus command cues) and their interactions with environmental factors. Therefore, the current study investigated this and evaluated the TOR performance of 42 drivers. We used a 2 (environments: day and night) $\times$ 4 (AR cue types: without, status, command, and combined cues) mixed-subject experimental design, and dependent measures included driving, cognitive, and NDRT performances. The results suggest that overall driving and cognitive performance were significantly improved by the command AR cue. In contrast, the status AR cue improved the TOR performance in nighttime environments. The performance of AR cues can vary depending on environmental factors, and AR cue designs for autonomous vehicles should consider this interaction for successful collaboration between drivers and vehicles.

Conditional autonomous vehicles have attracted significant attention, and their wide-spread use is expected to further increase in the near future. However, such vehicles can send take-over requests (TORs) to a driver who may struggle to accommodate the request while immersed in non-driving-related tasks. Previous studies have focused on TOR times and cues; however, the effects of environmental conditions have not been examined rigorously. This study, with the aim of addressing the aforementioned issue, is divided into two parts: in Study 1, we examine driver responses to TORs under different environmental conditions (i.e., sunny, rainy, snowy, foggy, and night-time), and in Study 2, we examine the effects of the proposed TOR cues (augmented reality + smartphone alert) under different environmental conditions. For this investigation, a driving simulator was used for the participants’ safety. Each study involves 33 participants. The results of Study 1 indicate significant differences in the take-over time, lane-change time, time-to-collision, maximum acceleration, and subjective mental workload corresponding to different environmental conditions. Furthermore, the results of Study 2 suggest that the proposed TOR cues significantly reduce the effects of environmental conditions on various take-over performances. We discuss the implication of these results in terms of the improvements in responses to TORs and investigation of the effects of environmental conditions on the responses to TORs.

Recent advances in technology have allowed users to experience an illusory feeling of full body ownership of a virtual avatar. Such virtual embodiment has the power to elicit perceptual, behavioral or cognitive changes related to oneself, however, its emotional effects have not yet been rigorously examined. To address this issue, we investigated emotional changes as a function of the level of the illusion (Study 1) and whether changes in the facial expression of a virtual avatar can modulate the effects of the illusion (Study 2). The results revealed that stronger illusory feelings of full body ownership were induced in the synchronous condition, and participants reported higher valence in the synchronous condition in both Studies 1 and 2. The results from Study 2 suggested that the facial expression of a virtual avatar can modulate participants' emotions. We discuss the prospects of the development of therapeutic techniques using such illusions to help people with emotion-related symptoms such as depression and social anxiety.

For the several decades, researchers studied human-avatar interactions using a virtual reality (VR). However, speculation on the interaction between a human's recognition memory and interaction types/methods has not enough considered yet. In the current study, we designed a VR interaction paradigm with two different types of human-avatar interaction including initiating and responding interactions, and we also included two interaction methods including a head-gazing and a hand-pointing. The result indicated that there are significant differences in the recognition memory between the initiating and responding interactions. However, we couldn't find any significant effects of interaction methods in the current study. These results suggest that the human-avatar interaction may have similar patterns with the human-human interaction in the recognition memory, and methodological advances also required.

The goal of the current study was to investigate the effects of different virtual environment (VE) technologies on emotional arousal and task performance in a modified Stroop task presented under low- and high-stress conditions. Fifty-three participants were recruited from a college population. Reactivity to stressful VEs was examined in three representative VE systems from desktop to high-end fully immersive systems. The experiment was a 3 (desktop system, head mounted display (HMD), and six wall system) × 2 (high- and low-stressful VE) within subject design, with self-reported emotional arousal and valence, skin conductance, task performance, presence, and simulator sickness. Replicating previous studies, the fully immersive system induced the highest sense of presence and the HMD system elicited the highest amount of simulator sickness. Extending previous studies, the results showed that different technological platforms evoked unique response patterns. Our findings suggest that different VE systems may be appropriate for different scientific purposes when studying stress reactivity using emotionally evocative tasks.