Integrating non-verbal cues into Mixed Reality Agents (MiRAs) enhances their ability to engage users and foster socially rich interactions. This paper investigates the role of locomotion and body posture in shaping user engagement, social presence, and interaction quality through two user studies involving a turn-based Gobang game. From these studies we found that in a competitive context, MiRAs’ locomotion and posture enhanced social presence and engagement, but while in a cooperative context, these behaviors fostered rapport but not trust. By integrating subjective, behavioral, and physiological measures, including EEG, this study provides a holistic understanding of MiRAs’ impact. The findings offer actionable design implications for creating engaging and socially effective virtual agents, advancing the field of human-agent interaction in Mixed Reality. Future research directions include exploring long-term effects, using diverse application domains, and supporting multimodal interactions.

Mixed Reality Agents (MiRAs) have been extensively studied to enhance virtual-physical interactions, using their ability to exist in both virtual and physical environments. However, little research has focused on enhancing perceived empathy in MiRAs, despite its potential for agent-assisted therapy, education, and training. To fill this gap, we investigate the impact of an Empathic Mixed Reality agent (EMiRA) that adapts to users' physiological states and physical events in a shooting game. We found that this adaptation enhanced users' social perceptions of the agent, including social presence, social connectedness, and perceived empathy. Physiological adaptation increased paternalism and reduced user dominance, while physical adaptation had no such effect. We discuss these findings and provide design implications for future EMiRAs.

Non-verbal cues like locomotion and posture influence users’ perceptions of Mixed Reality Agents (MiRAs). While Electroencephalography (EEG) captures cognitive responses, the influence of MiRAs’ locomotion and postures on brain activity remains underexplored. Additionally, few studies integrate subjective and behavioral measures with EEG to evaluate these cues’ impact on social perception. To address this, we conducted a within-subject study where participants played Gobang against three virtual agents in mixed reality: 1) a speech-only agent (S), 2) an embodied agent with speech and locomotion (S + L), and 3) an embodied agent with speech, locomotion, and posture (S + L + P). Results showed the S + L + P agent had higher engagement measured by the questionnaire but a lower EEG-based engagement index at AF3 than the S + L agent. Besides, the S + L + P was also rated higher in social presence, engagement, and emotional arousal than the S condition; No behavioral differences were observed. We discuss how MiRAs’ locomotion and posture affect users’ social perception and provide design implications for future human-agent interactions.

In human-agent interaction, establishing trust and a social bond with the agent is crucial to improving communication quality and performance in collaborative tasks. This paper investigates how a Mixed Reality Agent’s (MiRA) ability to acknowledge a user’s physiological state affects perceptions such as empathy, social connectedness, presence, and trust. In a within-subject study with 24 subjects, we varied the companion agent’s awareness during a mixed-reality first-person shooting game. Three agents provided feedback based on the users’ physiological states: (1) No Awareness Agent (NAA), which did not acknowledge the user’s physiological state; (2) Random Awareness Agent (RAA), offering feedback with varying accuracy; and (3) Accurate Awareness Agent (AAA), which provided consistently accurate feedback. Subjects reported higher scores on perceived empathy, social connectedness, presence, and trust with AAA compared to RAA and NAA. Interestingly, despite exceeding NAA in perception scores, RAA was the least favored as a companion. The findings and implications for the design of MiRA interfaces are discussed, along with the limitations of the study and directions for future work.

In this research, we investigate the effectiveness of asymmetric interactions (HandStylus, HandController, and TwoHands) in Augmented Reality (AR), Virtual Reality (VR), and Extended Reality (XR) for 3D digital drawing overlaying on physical and virtual objects. We evaluate the input accuracy and fatigue of these object-based 3D drawing experiences using quantitative measurements and further explore the correlation between these outcomes with subjective questionnaires. We found significant independence between environments and interface types, which considerably influence the performance and usability of 3D immersive drawing. We noted discrepancies between users’ subjective experiences and objective performance. Specifically, although AR drawing on physical objects provides superior accuracy and minimal muscle fatigue due to tangible feedback, and the TwoHands interaction offers the highest precision, the subjective results show the reverse outcome. Based on these findings, we propose design recommendations and discuss directions for future research in immersive drawing environments.

Augmented Reality (AR) based manual assembly nowadays enables to guide the process of physical tasks, providing intuitive instructions and detailed information in real-time. However, very limited studies have explored AR manual adjustment tasks with precision requirements. In this paper, we develop an AR-assisted guidance system for manual adjustments with relatively high-precision requirements. We first assessed the accuracy of the special-set OptiTrack system to determine the threshold of precision requirements for our user study. We further evaluated the performance of Number-based and Bar-based precision feedback by comparing orienting assembly errors and task completion time, as well as the usability in the user study. We found that the assembly errors of orientation in the Number-based and Bar-based interfaces were significantly lower than the baseline condition, while there was no significant difference between the Number-based and Bar-based interfaces. Furthermore, the Number-based showed faster task completion time, lower workload, and higher usability than the Bar-based condition.

Combining dead reckoning and Ultra-Wideband (UWB) ranging information to achieve relative localization (RL) becomes prevalent in recent years. However, two main problems, i.e., pose initialization and distributed implementation, are rarely investigated in practical multi-robot systems. In this paper, a novel two-stage RL method is proposed to fill this gap, wherein an initialization strategy, using robot-to-robot measurements acquired at different vantage points during robot motion to determine initial pose, and a consensus-based distributed particle filter (DPF), fusing statistics from local robot and neighbors to realize RL, are designed. In our system, by computing the pairwise relative pose between all robots in a team, the initialization strategy can determine an estimate of the initial pose of all robots with respect to a common reference frame and the corresponding covariance. The consensus-based DPF allows determining the relative pose of all robots with significantly reduced computing costs. Experiments results on a team of differentially driven mobile robots show the effectiveness of the initialization strategy and highlight the low computation cost of the proposed approach.

Collaboration is an important application area for virtual reality (VR). However, unlike in the real world, collaboration in VR misses important empathetic cues that can make collaborators aware of each other’s emotional states. Providing physiological feedback, such as heart rate or respiration rate, to users in VR has been shown to create a positive impact in single user environments. In this paper, through a rigorous mixed-factorial user experiment, we evaluated how providing heart rate feedback to collaborators influences their collaboration in three different environments requiring different kinds of collaboration. We have found that when provided with real-time heart rate feedback participants felt the presence of the collaborator more and felt that they understood their collaborator’s emotional state more. Heart rate feedback also made participants feel more dominant when performing the task. We discuss the implication of this research for collaborative VR environments, provide design guidelines, and directions for future research.