Older adults, a growing demographic, face an increased risk of experiencing loneliness and are less exposed to emerging communication technologies. Augmented reality (AR) avatars and telepresence robots have been proposed as tools to foster social connection, yet their suitability for older users remains underexplored. We present an exploratory study with ten healthy older adults who engaged in both conversational and spatial collaboration tasks using AR avatar-mediated communication, robot-mediated communication, and face-to-face interaction. We collected self-reported measures of co-presence, social presence, closeness, uncanny valley, preferences, and open feedback. Our findings suggest that telepresence robots enhanced co-presence, while avatars were valued for their expressivity and humanlike qualities. Task type influenced co-presence in spatial collaboration only during communication using the telepresence robot. Other measures, such as social presence and closeness, were unaffected by task type or representation. While neither technology outperformed face-to-face interaction, both were positively received, underscoring their potential to address the social needs of older adults and highlighting the importance of enhancing nonverbal expressivity, particularly nonverbal cues in mediated communication. Ultimately, our results contribute to the fundamental understanding of mediated communication with older adults, motivating further empirical work to confirm and extend these findings.

Telepresence robots that allow communication between older adults and their remotely located social contacts can foster social integration. The present laboratory test study explores older adults’ successful use of a telepresence robot (Research Question 1 [RQ1]), as well as their perceived enjoyment (RQ2), perceived ease of use (RQ3), perceived usefulness (RQ4), perceived social presence (RQ5), and intention to use (RQ6) a telepresence robot for robot-mediated communication (RMC). Semi-structured interviews, observations, and questionnaires were applied with a group of N = 14 older adults living in Germany. Participants completed a navigational task (as remote users) and an interpersonal communication task (as local users). Results show older adults used the telepresence robot successfully (RQ1) during the tasks. Furthermore, in interviews, older adults described their perceived enjoyment (RQ2), perceived ease of use (RQ3), and perceived usefulness (RQ4) during RMC as generally high. Perceived social presence (RQ5) during RMC was generally described as high, with RMC being considered a viable substitute when face-to-face communication is not possible. Finally, only two participants (2/14) had no intention to use (RQ6) a telepresence robot in the long term. Future design recommendations are provided, such as adapting the telepresence robot’s functions to older adults physical, psychological, and social conditions.

Cobots are widespread among manufacturing industries, where humans and robots work together in assembly tasks. However, cobots are not free from errors, which may affect collaboration in terms of human-robot workflow fluency. We present a study (N = 21) where we simulate a manufacturing workcell in VR to investigate how a cobot’s faulty behavior can affect the perception of human-robot collaboration during assembly, considering fluency (human-robot fluency, robot relative contribution, positive teammate traits, trust and human-robot working alliance goal) and workload. For that, participants assembled, together with a cobot, a complex piece inside electric vehicles (gearbox e-axle) five times. The assembly process consisted of six different steps to be performed sequentially, and we implemented a faulty behavior where the robot would follow the wrong sequence. After an error occurred, we allowed participants to choose freely between two error recovery behaviors: an automatic error recovery (the robot would undo the last step and continue the assembly process) or a training mode (the human would indicate the right sequence to the robot). Our results confirm that a cobot’s faulty behavior negatively affects the perceived fluency and workload, but does not affect the perceived cobot’s relative contribution to the task. The temporal sequencing of an error (error timing) can affect the goal of the human-cobot working alliance, assembly duration, temporal demand, and effort. Also, error frequency impacts differently the perceived physical demand. These findings can inform design choices of human-robot collaborative workflows in manufacturing, specifically, in error handling and mitigation strategies that preserve a positive perception of human-robot teams.

Avatars and telepresence robots have long received attention for remote communication. However, the specific nature of their physicality, expressiveness, and mobility may affect their usefulness for different tasks. This work compares using an avatar (presented in augmented reality) and a telepresence robot to Face-to-Face (F2F) communication during different communication tasks: free conversation, negotiation, and referential communication with movement. We conducted a user study (split-plot design, N=54) with the type of representation of the conversational partner as the within variable and the communication task as the between variable. Our results show that the type of task, especially referential communication with movement, influenced the perceived attention to nonverbal cues and closeness. Generally, gestures and body movements received the least focus with telepresence robots. Gestures in avatars and F2F drew similar attention, which we attribute to the avatar's tracking fidelity. Gaze received less attention in both avatar- and robot-mediated communication compared to F2F, while facial expressions on the robot's screen heightened attention compared to avatars. These findings advance the fundamental understanding of mediated communication and support researchers and practitioners in shaping the design of communication applications beyond today's video calls.

Hand-tracking technologies allow us to use our own hands to interact with real and virtual objects in Augmented Reality (AR) environments. This enables us to explore the interplay between hand-visualizations and hand-object interactions. We present a user study that examines the effect of different hand visualizations (invisible, transparent, opaque) on manipulation performance when interacting with real and virtual objects. For this, we implemented video-see-through (VST) AR-based virtual building blocks and hot wire tasks with real one-to-one counterparts that require participants to use gross and fine motor hand movements. To evaluate manipulation performance, we considered three measures: task completion time, number of collisions (hot wire task), and percentage of object displacement (building block task). Additionally, we explored the sense of agency and subjective impressions (preference, ease of interaction, successful and awkwardness) evoked by the different hand-visualizations. The results show that (1) manipulation performance is significantly higher when interacting with real objects compared to virtual ones, (2) invisible hands lead to fewer errors, higher agency, higher perceived success and ease of interaction during fine manipulation tasks with real objects, and (3) having some visualization of the virtual hands (transparent or opaque) overlayed on the real hands is preferred when manipulating virtual objects even when there are no significant performance improvements. Our empirical findings about the differences when interacting with real and virtual objects can aid hand visualization choices for manipulation tasks in AR.

Augmented Reality (AR) and Virtual Reality (VR) are essential tools for researchers and practitioners, serving purposes from training to entertainment: many of these applications rely on agents. This study explores the impact of agent characteristics on user reactions, focusing on gaze as a primary visual attention indicator in AR and VR. While existing research has investigated the agent’s gaze and its influence on the user, it is unclear how the agent’s auralization and visualization influence gaze behaviour. We investigate this by studying the impact of rendering style and type of audio on gaze behaviour during a narrative AR experience. Participants listened to a story with the agent visualized as a cartoon-style or realistic virtual human and auralized with spatial or non-spatial audio. The results revealed that the agent’s rendering style significantly influenced gaze behaviour, with cartoon-style agents capturing more visual attention. Audio variations did not yield significant differences. Together, our findings inform the design of AR user interfaces with agents, suggesting that low-realism visualizations are more captivating and, thus, more suitable for experiences where the user is supposed to look at the storyteller.

In this paper, we investigated the influence of video resolution and perceived conversational quality on nonverbal behaviors during dyadic videoconferencing (VC) conversations. We analyzed nonverbal behaviors at the individual and interpersonal level. At the individual level, we considered body motion, facial expressions, and gaze directivity. At the interpersonal level, we considered facial expression synchrony and body movement synchrony. For the analysis, we used webcam recordings from a VC experiment, extracting the aforementioned individual nonverbal behavioral features and using windowed lagged cross-correlation (WLCC) to quantify the degree of interpersonal synchronization. Our results indicate that high video resolution significantly increased individual body movements and encouraged gaze directivity toward the conversational partner, fostering greater engagement while paradoxically reducing body movement synchrony. Higher conversational quality was associated with increased facial expression synchrony between participants. Moreover, we observed that instantaneous synchrony (as quantified with lag-zero WLCC) for both body movement and facial expressions was significantly influenced by mutual gaze-like behavior. These findings indicate a complex relationship between technical settings and nonverbal behaviors, suggesting that while higher resolution enhances some nonverbal behaviors, especially body movement and mutual gaze, it may disrupt body movement synchronization. These insights could be applied to the VC setups to achieve a high level of interpersonal coordination and engagement.

In this paper, we investigated the effects of transmission delay on individual nonverbal behavior and interpersonal coordination during dyadic video conferencing conversations. For that, we assessed individual-level nonverbal behaviors, including body motion and gaze patterns, and examined participants' interpersonal coordination of body movement. Our results indicate that transmission delay significantly reduces individual body motion. No significant differences in gaze behaviors were found between different delay conditions, however, a trend of participants spending more time looking at their conversational partner in the high-delay condition than in the no-delay condition was observed. Also, we found that transmission delay significantly influences interpersonal body-movement coordination, enhancing structural organization and coordination stability while showing a threshold effect on movement similarity.

Telepresence robots can enhance communication experiences by providing a sense of physical presence, embodiment and may evoke co-presence. In spite of that, telepresence robots have not made it fully to consumer markets. In this paper, we investigate how different levels of controlling a telepresence robot (teleoperation, shared control, and no control) influence presence. To this aim, we conducted a study (N=45) where participants were evenly distributed to one of the robot control conditions. The task involved navigating an unknown room and listening to stories told by a person co-located with the robot. We collected subjective impressions of presence using the temple presence inventory and performed a thematic content analysis on a post-experiment interview. Our results suggest nuances in perceived presence under different levels of robot control after performing a thematic content analysis. Copresence can be experienced during teleoperation and shared control, and teleoperation may evoke negative sentiments if it does not provide enough spatial information during navigation. However, our results did not point to significant differences in spatial or social presence. We consider that these findings encourage further discussions on how presence is perceived in robot-mediated communication.

The appearance of virtual humans (avatars and agents) has been widely explored in immersive environments. However, virtual humans’ movements and associated sounds in real-world interactions, particularly in Augmented Reality (AR), are yet to be explored. In this paper, we investigate the influence of three distinct movement patterns (circle, side-to-side, and standing), two rendering styles (realistic and cartoon), and two types of audio (spatial audio and non-spatial audio) on emotional responses, social presence, appearance and behavior plausibility, audiovisual coherence, and auditory plausibility. To enable that, we conducted a study (N=36) where participants observed an agent reciting a short fictional story. Our results indicate an effect of the rendering style and the type of movement on the subjective perception of the agents behaving in an AR environment. Participants reported higher levels of excitement when they observed the realistic agent moving in a circle compared to the cartoon agent or the other two movement patterns. Moreover, we found an influence of agent’s movement pattern on social presence and higher appearance and behavior plausibility for the realistic rendering style. Regarding audiovisual spatial coherence, we found an influence of rendering style and type of audio only for the cartoon agent. Additionally, the spatial audio was perceived as more plausible than non-spatial audio. Our findings suggest that aligning realistic rendering styles with realistic auditory experiences may not be necessary for 1-1 listening experiences with moving sources. However, movement patterns of agents influence excitement and social presence in passive unidirectional communication scenarios.

Despite remarkable advances, current Extended Reality (XR) applications are in their majority local and individual experiences. A plethora of interactive applications, such as teleconferencing, telesurgery, interconnection in new buildings project chain, Cultural Heritage, and Museum contents communication, are well on their way to integrating immersive technologies. However, interconnected, and interactive XR, where participants can virtually interact across vast distances, remains a distant dream. In fact, three great barriers stand between current technology and remote immersive interactive life-like experiences, namely (i) content realism, (ii) motion-to-photon latency, and accurate (iii) human-centric quality assessment and control. Overcoming these barriers will require novel solutions at all elements of the end-to-end transmission chain. This workshop focuses on the challenges, applications, and major advancements in multimedia, networks, and end-user infrastructures to enable the next generation of interactive XR applications and services.

Previous studies have explored the perception of various types of embodied avatars in immersive environments. However, the impact of eye and face tracking with personalized avatars is yet to be explored. In this paper, we investigate the impact of eye and face tracking on embodiment, enfacement, and the uncanny valley with four types of avatars using a VR-based mirroring task. We conducted a study (N=12) and created self-avatars with two rendering styles: a cartoon avatar (created in an avatar generator using a picture of the user’s face) and a photorealistic scanned avatar (created using a 3D scanner), each with and without eye and face tracking and respective adaptation of the mirror image. Our results indicate that adding eye and face tracking can be beneficial for certain enfacement scales (belonged), and we confirm that compared to a cartoon avatar, a scanned realistic avatar results in higher body ownership and increased enfacement (own face, belonging, mirror) — regardless of eye and face tracking. We critically discuss our experiences and outline the limitations of the applied hardware and software with respect to the provided level of control and the applicability for complex tasks such as displaying emotions. We synthesize these findings into a discussion about potential improvements for facial animation in VR and highlight the need for a better level of control, the integration of additional sensing and processing technologies, and an objective metric for comparing facial animation systems.

Virtual nature settings have demonstrated to provide benefits to mental well-being. However, most studies have focused on providing only audiovisual stimuli. We aim to evaluate the use of haptic feedback to simulate touching elements in nature-inspired settings. In this paper, we designed a VR gardening environment to investigate the impact of haptic feedback on spatial presence, perceptual realism, mental immersion, user experience, and task performance while interacting with gardening objects in a study (N=18, 9 female and 9 male). Our results suggest that haptic feedback can increase spatial presence and point to gender differences, i.e., female participants reported higher scores in spatial presence and perceptual realism, in the chosen VR experience. Although our main goal was to evaluate the role of haptics in a virtual garden, our findings highlight the importance of investigating and identifying factors that could lead to gender differences in VR experiences.

This paper presents a proof-of-concept study conducted to analyze the effect of simple diotic vs. spatial, position-dynamic binaural synthesis on social presence in VR, in comparison with face-to-face communication in the real world, for a sample two-party scenario. A conversational task with shared visual reference was realized. The collected data includes questionnaires for direct assessment, tracking data, and audio and video recordings of the individual participants’ sessions for indirect evaluation. While tendencies for improvements with binaural over diotic presentation can be observed, no significant difference in social presence was found for the considered scenario. The gestural analysis revealed that participants used the same amount and type of gestures in face-to-face as in VR, highlighting the importance of non-verbal behavior in communication. As part of the research, an end-to-end framework for conducting communication studies and analysis has been developed.

Augmented Reality (AR) and Virtual Reality (VR) are pushing from the labs towards consumers, especially with social applications. These applications require visual representations of humans and intelligent entities. However, displaying and animating photo-realistic models comes with a high technical cost while low-fidelity representations may evoke eeriness and overall could degrade an experience. Thus, it is important to carefully select what kind of avatar to display. This article investigates the effects of rendering style and visible body parts in AR and VR by adopting a systematic literature review. We analyzed 72 papers that compare various avatar representations. Our analysis includes an outline of the research published between 2015 and 2022 on the topic of avatars and agents in AR and VR displayed using head-mounted displays, covering aspects like visible body parts (e.g., hands only, hands and head, full-body) and rendering style (e.g., abstract, cartoon, realistic); an overview of collected objective and subjective measures (e.g., task performance, presence, user experience, body ownership); and a classification of tasks where avatars and agents were used into task domains (physical activity, hand interaction, communication, game-like scenarios, and education/training). We discuss and synthesize our results within the context of today's AR and VR ecosystem, provide guidelines for practitioners, and finally identify and present promising research opportunities to encourage future research of avatars and agents in AR/VR environments.

Teleoperating industrial manipulators in co-located spaces can be challenging. Facilitating robot teleoperation by providing additional visual information about the environment and the robot affordances using augmented reality (AR), can improve task performance in manipulation and grasping. In this paper, we present two designs of augmented visual cues, that aim to enhance the visual space of the robot operator through hints about the position of the robot gripper in the workspace and in relation to the target. These visual cues aim to improve the distance perception and thus, the task performance. We evaluate both designs against a baseline in an experiment where participants teleoperate a robotic arm to perform pick-and-place tasks. Our results show performance improvements in different levels, reflecting in objective and subjective measures with trade-offs in terms of time, accuracy, and participants’ views of teleoperation. These findings show the potential of AR not only in teleoperation, but in understanding the human-robot workspace.

Assistive technologies such as human-robot collaboration, have the potential to ease the life of people with physical mobility impairments in social and economic activities. Currently, this group of people has lower rates of economic participation, due to the lack of adequate environments adapted to their capabilities. We take a closer look at the needs and preferences of people with physical mobility impairments in a human-robot cooperative environment at the workplace. Specifically, we aim to design how to control a robotic arm in manufacturing tasks for people with physical mobility impairments. We present a case study of a sheltered-workshop as a prototype for an institution that employs people with disabilities in manufacturing jobs. Here, we collected data of potential end-users with physical mobility impairments, social workers, and supervisors using a participatory design technique (Future-Workshop). These stakeholders were divided into two groups, primary (end-users) and secondary users (social workers, supervisors), which were run across two separate sessions. The gathered information was analyzed using thematic analysis to reveal underlying themes across stakeholders. We identified concepts that highlight underlying concerns related to the robot fitting in the social and organizational structure, human-robot synergy, and human-robot problem management. In this paper, we present our findings and discuss the implications of each theme when shaping an inclusive human-robot cooperative workstation for people with physical mobility impairments.

Interacting with the physical and digital environment multimodally enhances user flexibility and adaptability to different scenarios. A body of research has focused on comparing the efficiency and effectiveness of different interaction modalities in digital environments. However, little is known about user behavior in an environment that provides freedom to choose from a range of modalities. That is why, we take a closer look at the factors that influence input modality choices. Building on the work by Jameson & Kristensson, our goal is to understand how different factors influence user choices. In this paper, we present a study that aims to explore modality choices in a hands-free interaction environment, wherein participants can choose and combine freely three hands-free modalities (Gaze, Head movements, Speech) to execute point and select actions in a 2D interface. On the one hand, our results show that users avoid switching modalities more often than we expected, particularly, under conditions that should prompt modality switching. On the other hand, when users make a modality switch, user characteristics and consequences of the experienced interaction have a higher impact in the choice, than the changes in environmental conditions. Further, when users switch between modalities, we identified different types of switching behaviors. Users who deliberately try to find and choose an optimal modality (single switcher), users who try to find optimal combinations of modalities (multiple switcher), and a switching behavior triggered by error occurrence (error biased switcher). We believe that these results help to further understand when and how to design for multimodal interaction in real-world systems.