Augmented Reality (AR) and Virtual Reality (VR) are essential tools for researchers and practitioners, serving purposes from training to entertainment: many of these applications rely on agents. This study explores the impact of agent characteristics on user reactions, focusing on gaze as a primary visual attention indicator in AR and VR. While existing research has investigated the agent’s gaze and its influence on the user, it is unclear how the agent’s auralization and visualization influence gaze behaviour. We investigate this by studying the impact of rendering style and type of audio on gaze behaviour during a narrative AR experience. Participants listened to a story with the agent visualized as a cartoon-style or realistic virtual human and auralized with spatial or non-spatial audio. The results revealed that the agent’s rendering style significantly influenced gaze behaviour, with cartoon-style agents capturing more visual attention. Audio variations did not yield significant differences. Together, our findings inform the design of AR user interfaces with agents, suggesting that low-realism visualizations are more captivating and, thus, more suitable for experiences where the user is supposed to look at the storyteller.

In this paper, we investigated the influence of video resolution and perceived conversational quality on nonverbal behaviors during dyadic videoconferencing (VC) conversations. We analyzed nonverbal behaviors at the individual and interpersonal level. At the individual level, we considered body motion, facial expressions, and gaze directivity. At the interpersonal level, we considered facial expression synchrony and body movement synchrony. For the analysis, we used webcam recordings from a VC experiment, extracting the aforementioned individual nonverbal behavioral features and using windowed lagged cross-correlation (WLCC) to quantify the degree of interpersonal synchronization. Our results indicate that high video resolution significantly increased individual body movements and encouraged gaze directivity toward the conversational partner, fostering greater engagement while paradoxically reducing body movement synchrony. Higher conversational quality was associated with increased facial expression synchrony between participants. Moreover, we observed that instantaneous synchrony (as quantified with lag-zero WLCC) for both body movement and facial expressions was significantly influenced by mutual gaze-like behavior. These findings indicate a complex relationship between technical settings and nonverbal behaviors, suggesting that while higher resolution enhances some nonverbal behaviors, especially body movement and mutual gaze, it may disrupt body movement synchronization. These insights could be applied to the VC setups to achieve a high level of interpersonal coordination and engagement.

In this paper, we investigated the effects of transmission delay on individual nonverbal behavior and interpersonal coordination during dyadic video conferencing conversations. For that, we assessed individual-level nonverbal behaviors, including body motion and gaze patterns, and examined participants' interpersonal coordination of body movement. Our results indicate that transmission delay significantly reduces individual body motion. No significant differences in gaze behaviors were found between different delay conditions, however, a trend of participants spending more time looking at their conversational partner in the high-delay condition than in the no-delay condition was observed. Also, we found that transmission delay significantly influences interpersonal body-movement coordination, enhancing structural organization and coordination stability while showing a threshold effect on movement similarity.

The appearance of virtual humans (avatars and agents) has been widely explored in immersive environments. However, virtual humans’ movements and associated sounds in real-world interactions, particularly in Augmented Reality (AR), are yet to be explored. In this paper, we investigate the influence of three distinct movement patterns (circle, side-to-side, and standing), two rendering styles (realistic and cartoon), and two types of audio (spatial audio and non-spatial audio) on emotional responses, social presence, appearance and behavior plausibility, audiovisual coherence, and auditory plausibility. To enable that, we conducted a study (N=36) where participants observed an agent reciting a short fictional story. Our results indicate an effect of the rendering style and the type of movement on the subjective perception of the agents behaving in an AR environment. Participants reported higher levels of excitement when they observed the realistic agent moving in a circle compared to the cartoon agent or the other two movement patterns. Moreover, we found an influence of agent’s movement pattern on social presence and higher appearance and behavior plausibility for the realistic rendering style. Regarding audiovisual spatial coherence, we found an influence of rendering style and type of audio only for the cartoon agent. Additionally, the spatial audio was perceived as more plausible than non-spatial audio. Our findings suggest that aligning realistic rendering styles with realistic auditory experiences may not be necessary for 1-1 listening experiences with moving sources. However, movement patterns of agents influence excitement and social presence in passive unidirectional communication scenarios.

Previous research from well before the Covid-19 pandemic had indicated little effect of delay on integral quality but a measurable one on user behavior, and a significant effect of resolution on quality but not on behavior in a two-party communication scenario. In this paper, we re-investigate the topic, after the times of the Covid-19 pandemic and its frequent and widespread videoconferencing usage. To this aim, we conducted a subjective test involving 23 pairs of participants, employing the Celebrity Name Guessing task. The focus was on impairments that may affect social (resolution) and communication cues (de-lay). Subjective data in the form of overall conversational quality and task performance satisfaction as well as objective data in the form of task correctness, user motion, and facial expressions were collected in the test. The analysis of the subjective data indicates that perceived conversational quality and performance satisfaction were mainly affected by video resolution, while delay (up to 1000 ms) had no significant impact. Furthermore, the analysis of the objective data shows that there is no impact of resolution and delay on user performance and behavior, in contrast to earlier findings.

Augmented Reality (AR) and Virtual Reality (VR) are pushing from the labs towards consumers, especially with social applications. These applications require visual representations of humans and intelligent entities. However, displaying and animating photo-realistic models comes with a high technical cost while low-fidelity representations may evoke eeriness and overall could degrade an experience. Thus, it is important to carefully select what kind of avatar to display. This article investigates the effects of rendering style and visible body parts in AR and VR by adopting a systematic literature review. We analyzed 72 papers that compare various avatar representations. Our analysis includes an outline of the research published between 2015 and 2022 on the topic of avatars and agents in AR and VR displayed using head-mounted displays, covering aspects like visible body parts (e.g., hands only, hands and head, full-body) and rendering style (e.g., abstract, cartoon, realistic); an overview of collected objective and subjective measures (e.g., task performance, presence, user experience, body ownership); and a classification of tasks where avatars and agents were used into task domains (physical activity, hand interaction, communication, game-like scenarios, and education/training). We discuss and synthesize our results within the context of today's AR and VR ecosystem, provide guidelines for practitioners, and finally identify and present promising research opportunities to encourage future research of avatars and agents in AR/VR environments.