ABSTRACT This study examines how subtitles and image visualizations influence gaze behavior, working alliance, and behavior change intentions in virtual health conversations with ECAs. Visualizations refer to images on a 3D model TV and text on a virtual whiteboard, both reinforcing key content conveyed by the ECA. Using a 2 2 factorial design, participants were randomly assigned to one of four conditions: no subtitles or visualizations (Control), subtitles only (SUB), visualizations only (VIS), or both subtitles and visualizations (VISSUB). Structural equation path modeling showed that SUB and VIS individually reduced gaze toward the ECA, whereas VISSUB moderated this reduction, resulting in less gaze loss than the sum of either condition alone. Gaze behavior was positively associated with working alliance, and perceptions of enjoyment and appropriateness influenced engagement, which in turn predicted behavior change intentions. VIS was negatively associated with behavior change intentions, suggesting that excessive visual input may introduce cognitive trade‐offs.

ABSTRACT This study examines how multimodal communication strategies (subtitles, visualizations, and their combination), affect memory retention, attitudes toward behavior change, and cognitive load during long conversations (+20 min) in immersive virtual reality (VR). Using embodied conversational agents to educate participants on diabetes and healthy eating, we found that all conditions effectively improved memory retention and behavior change attitudes. However, the combination of multimodal strategies increased cognitive load, suggesting a trade‐off between engagement and cognitive demands. These findings highlight the potential of long VR conversations for healthcare education, while emphasizing the importance of balancing cognitive demands and exploring personalization for diverse users.

Research on interpersonal relationships has shown that individuals often exhibit different gaze patterns when viewing faces of other races compared to those similar to their own. In the context of Virtual Humans (VHs), participatory design processes frequently include features that reflect the designer's identity. This paper explores gaze behavior towards demographically diverse VHs in the context of emerging educational technology. Using Augmented Reality (AR), 33 participants engaged in a within-subjects study where their eye movements, learning outcomes, and perceptions were compared. Results indicate that gaze (no. of fixations and durations) was influenced by demographic concordance (when race and sex matched that of the VH). Additionally, learning outcomes positively correlated with the number of fixations, suggesting that increased visual focus may enhance learning. However, participants frequently misidentified the race of the Hispanic, Asian, and Black VH, revealing a potential perceptual misalignment likely shaped by implicit social categorization biases.

Extensive HCI research has investigated how to prevent and mitigate harassment in virtual spaces, particularly by leveraging human-based and Artificial Intelligence (AI)-based moderation. However, social Virtual Reality (VR) constitutes a novel social space that faces both intensified harassment challenges and a lack of consensus on how moderation should be approached to address such harassment. Drawing on 39 interviews with social VR users with diverse backgrounds, we investigate the perceived opportunities and limitations for leveraging AI-based moderation to address emergent harassment in social VR, and how future AI moderators can be designed to enhance such opportunities and address limitations. We provide the first empirical investigation into re-envisioning AI’s new roles in innovating content moderation approaches to better combat harassment in social VR. We also highlight important principles for designing future AI-based moderation incorporating user-human-AI collaboration to achieve safer and more nuanced online spaces.

Harassment has long been considered a severe social issue and a culturally contextualized construct. More recently, understanding and mitigating emerging harassment in social Virtual Reality (VR) has become a growing research area in HCI and CSCW. Based on the perspective of harassment in the U.S. culture, in this paper we identify new characteristics of online harassment in social VR using 30 in-depth interviews. We especially attend to how people who are already considered marginalized in the gaming and virtual worlds contexts (e.g., women, LGBTQ, and ethnic minorities) experience such harassment. As social VR is still a novel technology, our proactive approach highlights embodied harassment as an emerging but understudied form of harassment in novel online social spaces. Our critical review of social VR users' experiences of harassment and recommendations to mitigate such harassment also extends the current conceptualization of online harassment in CSCW. We therefore contribute to the active prevention of future harassment in nuanced online environments, platforms, and experiences.

Understanding how and why users reveal information about their self in online social spaces and what they perceive as privacy online is a central research agenda in HCI. Drawing on 30 in-depth interviews, in this paper we focus on what type of information users disclose, to whom they reveal information, and concerns they had regarding self-disclosure in social Virtual Reality (VR) - where multiple users can interact with one another through VR head-mounted displays in 3D virtual spaces. Our findings show that overall, users felt comfortable to disclose their emotions, personal experience, and personal information in social VR. However, they also acknowledged that disclosing personal information in social VR was an inevitable trade-off: giving up bio-metric information in order to better use the system. We contribute to existing literature on self-disclosure and privacy online by focusing on social VR as an emerging novel online social space. We also explicate implications for designing and developing future social VR applications.