Intelligence and knowledgeability are sometimes treated interchangeably in virtual agents, yet they shape interaction in different ways. We disentangled these traits and tested how each drives human perceptions and interaction in virtual reality (VR). To address the lack of prior research examining both traits simultaneously, we created a VR application where participants collaborated with a virtual agent to complete a jigsaw puzzle while engaging in free-flowing conversation about the puzzle’s art piece. We manipulated intelligence through the virtual agent’s puzzle-solving ability and knowledgeability through its predefined depth of knowledge in art. Using a 2 × 2 within-group study, we collected perceptual responses, logged data, and qualitative feedback. Results showed intelligence significantly influenced perceptions of intelligence, knowledge, rapport, trust, co-presence, uncanny valley, and intelligence and knowledge comparisons, while knowledgeability impacted perceived knowledge, trust, and intelligence and knowledge comparisons. Interaction effects further highlighted their interdependence, offering design implications for virtual agents.

Playtesting is widely used in the game industry to identify design flaws and evaluate player experience, yet little research explores how to effectively visualize and analyze playtesting data. This challenge is particularly pronounced in motion-based VR games, which involve physical movements and interactions tracked through multimodal inputs, resulting in complex multidimensional data. To better understand the challenges designers face, we conducted a formative study with 30 practitioners in the VR domain to characterize playtesting workflows and associated tasks. Based on these findings, we present HieraVisVR, a hierarchical visual analytics framework that incorporates body-motion-related data to help designers identify player behaviors and critical game moments, simplifying their workflow. We demonstrate the applicability of HieraVisVR in three different applications and evaluate our system with playtesting experts through an analysis of motion-based game data. The study results suggest that our system enhances playtesters’ understanding of the gameplay and improves their data analysis workflow.

In virtual reality, creating intelligent virtual characters has been a long-lasting endeavor. However, while researchers have investigated several aspects of a virtual character’s intelligence, little attention has been paid to the impact of the implemented intelligence levels assigned to a virtual character during human–virtual character collaboration. Thus, we conducted a within-group study ( \(N=24\) ) to explore how three different intelligence levels (low vs. medium vs. high) assigned to a virtual character can impact how study participants perceive that virtual character and interact with the task they are instructed to complete. Specifically, for our study, we developed a jigsaw puzzle game and instructed our participants to solve it with the help of a virtual character. During the jigsaw puzzle solving process, we collected application logs related to how the participants executed the task and observed the virtual environment. Moreover, after each condition, we asked the participants to respond using a questionnaire that examined their social presence, how they perceived the character’s intelligence and compared it with their own, and how they rated the virtual character’s realism. Our results indicated that the different intelligence levels assigned to the virtual characters impacted participants’ responses on several variables, including co-presence, perceived intelligence, intelligence comparison, and character interaction and behavior realism. Moreover, based on the collected logged data, we found that the intelligence levels assigned to our virtual character significantly impacted the performance of our participants. Our results could be valuable to the research community for creating more engaging experiences with intelligent virtual characters for collaborative tasks in immersive environments.

Designing interactions between humans and virtual characters requires careful consideration of various human perceptions and user experiences. While numerous studies have explored the effects of several virtual characters' properties, the impacts of the virtual character's personality and age on human perceptions and experiences have yet to be thoroughly investigated. To address this gap, we conducted a within-group study (N = 28) following a 2 (personality: egoism vs. altruism) × 2 (age: child vs. adult) design to explore how the personality and age factors influence human perception and experience during interactions with virtual characters. In each condition of our study, our participants co-solved a jigsaw puzzle with a virtual character that embodied combinations of personality and age. After each condition, participants completed a survey. We also asked them to provide written feedback at the end of the study. Our statistical analyses revealed that the virtual character's personality and age significantly influenced participants' perceptions and experiences. The personality factor affected perceptions of altruism, anthropomorphism, likability, safety, and all aspects of user experience, including perceived collaboration, rapport, emotional reactivity, and the desire for future interaction. Additionally, the virtual character's age affected our participants' ratings of the uncanny valley and likability. We also identified an interaction effect between personality and age factors on the virtual character's anthropomorphism. Based on our findings, we offered guidelines and insights for researchers aiming to design collaborative experiences with virtual characters of different personalities and ages.

An animated agent’s believability, often defined as “the illusion of life,” has many facets, including the agent’s appearance, movements, and user interactions. Considering that human facial expressions are often not symmetrical, evaluating how facial expression asymmetries affect believability in animated agents would provide valuable insights for researchers and artists. Our research focused on facial movements and examined the impact of three dynamic facial asymmetries on users’ perception of the agent’s believability, appeal, and naturalness. Our research involved three studies: 1) a preliminary study in which we determined the maximum values to base asymmetry levels, 2) a study in which we identified what level of asymmetry yields the most perceived believability for each asymmetry location, and 3) a study in which we determined what effects different combinations of asymmetries have on perceived believability, appeal, and naturalness. The findings confirmed the effect of dynamic facial asymmetry on the agents’ perceived believability, appeal, and naturalness. Furthermore, our research suggests that including one or a combination of asymmetric eyebrow, mouth, or eyelid motions increases the agents’ perceived believability, appeal, and naturalness.

Researchers have provided insights into using virtual reality (VR) for visualization and interaction with 3D models and simulations. The interaction allows users to manipulate the 3D elements and visualize changes based on their inputs from movement with controllers or spatial actions. However, some users may find this interaction overwhelming, especially when immersed in a virtual environment. Additionally, the choice of dimensionality for visualizations influences user interaction, with potential implications for immersive experiences. Thus, we conducted a 2 (Immersion: Desktop vs. HMDVR) $\times 2$ (Dimensionality: 2 D vs. 3 D) within-group study $(N=32)$ to explore the impact of the utilized immersive degree and the dimensionality representation of the content on participants’ experience in terms of engagement, task load, usability, skill, and emotions when interacting with a science simulation. We designed and developed an application to simulate charged particles and electric field lines. We asked participants to complete a task of changing particles by matching them to a given simulation output. Our results indicated higher workload rates for HMDVR conditions, particularly with 3D representation, compared to Desktop. However, HMDVR conditions also showed greater engagement, emotional response, and presence. Based on our findings, we argue that participants prefer HMDVR over Desktop environments regardless of dimensionality.

Virtual character’s intelligence and task complexity are essential components for the design of collaboration between humans and virtual characters. However, studies have yet to explore the impact of a virtual character’s intelligence and task complexity during a collaboration task. To explore these impacts, we implemented a jigsaw puzzle co-solving experience and conducted a within-group study (N = 27) following a 2 (intelligence: low vs. high) × 2 (complexity: low vs. high) study design. During the puzzle co-solving process, we collected participants’ gaze data for each experimental condition. After our participants completed each condition (co-solved the jigsaw puzzle), they answered a survey that captured several variables. We found several significant main and interaction effect results indicating the impact of the virtual character’s intelligence and the task’s complexity. Specifically, the virtual character’s intelligence impacted most survey ratings, including the virtual character’s intelligence, perceived collaboration, self-confidence, and perceived contribution. Also, the task complexity affected perceived collaboration, the virtual character’s public awareness, and self-confidence. Furthermore, we found that the virtual character’s intelligence and complexity dominantly impacted our participants’ task load and perception of the virtual character’s confidence. Moreover, we found that our participants gazed at the virtual character and the puzzle piece more often than the puzzle goal across all experimental conditions. Our findings can inform the design of intelligent virtual characters that co-solve tasks with users.

Although researchers have explored how humans perceive the intelligence of virtual characters, few studies have focused on the ability of intelligent virtual characters to fix their mistakes. Thus, we explored the self-correction behavior of a virtual character with different intelligence capabilities in a within-group design ( \(N=23\) ) study. For this study, we developed a virtual character that can solve a jigsaw puzzle whose self-correction behavior is controlled by two parameters, namely, Intelligence and Accuracy of Self-correction . Then, we integrated the virtual character into our virtual reality experience and asked participants to co-solve a jigsaw puzzle. During the study, our participants were exposed to five experimental conditions resulting from combinations of the Intelligence and Accuracy of Self-correction parameters. In each condition, we asked our participants to respond to a survey examining their perceptions of the virtual character’s intelligence and awareness (private, public, and surroundings awareness) and user experiences, including trust, enjoyment, performance, frustration, and desire for future interaction. We also collected application logs, including participants’ dwell gaze data, completion times, and the number of puzzle pieces they placed to co-solve the jigsaw puzzle. The results of all the survey ratings and the completion time were statistically significant. Our results indicated that higher levels of Intelligence and Accuracy of Self-correction enhanced not only our participants’ perceptions of the virtual character’s intelligence, awareness (private, public, and surroundings), trustworthiness, and performance but also increased their enjoyment and desire for future interaction with the virtual character while reducing their frustration and completion time. Moreover, we found that as the Intelligence and Accuracy of Self-correction increased, participants had to place fewer puzzle pieces and needed less time to complete the jigsaw puzzle. Finally, regardless of the experimental condition to which we exposed our participants, they gazed at the virtual character for more time compared to the puzzle pieces and puzzle goal in the virtual environment.

We examined the effects of virtual characters’ body type and voice pitch on perceived audio-visual correspondence and believability. For our within-group study (N = 72), we developed nine experimental conditions using a 3 (body type: ectomorph vs. mesomorph vs. endomorph body types) × 3 (voice pitch: low vs. medium vs. high fundamental frequency [F0]) design. We found statistically significant main effects from voice pitch and statistically significant interaction effects between a virtual character’s body type and voice pitch on both the level of perceived audio-visual correspondence and believability of female and male virtual characters. For female virtual characters, we also observed an additional statistically significant main effect from body type and a statistically significant interaction effect between the participant’s biological sex and the virtual character’s voice pitch on both perceived audio-visual correspondence and believability. Moreover, the results show that perceived believability is highly correlated to perceived audio-visual correspondence. Our findings have important practical implications in applications where the virtual character is meant to be an emotional or informational guide that requires some level of perceived believability, as the findings suggest that it is possible to enhance the perceived believability of the virtual characters by generating appropriate voices through pitch manipulation of existing voices.

We developed a method to synthesize game levels that accounts for the degree of collaboration required by two players to finish a given game level. We first asked a game level designer to create playable game level chunks. Then, two artificial intelligence (AI) virtual agents driven by behavior trees played each game level chunk. We recorded the degree of collaboration required to accomplish each game level chunk by the AI virtual agents and used it to characterize each game level chunk. To synthesize a game level, we assigned to the total cost function cost terms that encode both the degree of collaboration and game level design decisions. Then, we used a Markov-chain Monte Carlo optimization method, called simulated annealing, to solve the total cost function and proposed a design for a game level. We synthesized three game levels (low, medium, and high degrees of collaboration game levels) to evaluate our implementation. We then recruited groups of participants to play the game levels to explore whether they would experience a certain degree of collaboration and validate whether the AI virtual agents provided sufficient data that described the collaborative behavior of players in each game level chunk. By collecting both in-game objective measurements and self-reported subjective ratings, we found that the three game levels indeed impacted the collaboration gameplay behavior of our participants. Moreover, by analyzing our collected data, we found moderate and strong correlations between the participants and the AI virtual agents. These results show that game developers can consider AI virtual agents as an alternative method for evaluating the degree of collaboration required to finish a game level.

Synthesizing game levels that evoke players’ curiosity, driving them to explore different level parts, is time-consuming and tedious. Typically, game level designers manually perform this synthesis using trial and error. In this paper, we propose a method with which to replace this manual, time-consuming process. We benefited from recent work that had proposed game level design patterns to evoke curiosity, and we propose an approach to automatically synthesizing game levels in order to encourage players to pursue designer-specified exploration goals. We started by creating a dataset of level assets, based on the four design patterns that evoke curiosity-driven exploration in games (reaching extreme points, resolving visual obstructions, out-of-place objects, and understanding spatial connections). We annotated the assets in our dataset with spatial exploration measurements (the time players took to explore an asset over their total time spent in the game level). We then formulated game level design as an optimization problem, encoding both spatial exploration (mean spatial exploration, spatial exploration variance, and spatial exploration distribution) and game level design (occupied area, adjacent penalty, and height distribution) decisions. Then, we solved this problem by implementing a reversible-jump Markov chain Monte Carlo method. We demonstrate our method’s ability to synthesize game level variations with different spatial exploration and level design decisions. Finally, a user study showed that our approach can automatically synthesize game levels, encouraging a certain amount of spatial exploration by players.

We have developed a mixed reality (MR)-based platform for basic mechanical engineering concepts as a learning environment for collaborative assembly tasks. In our platform, multiple co-located users interact with virtual objects simultaneously, and during that time, the platform collects data related to participants’ collaboration and team behavior. We implemented four main sections in the platform including setup, introduction, training, and assessment. The platform provides the opportunity for users to interact with virtual objects while also acquiring technical knowledge. Specifically, for the technical component of the platform, users are asked to assemble a hydraulic pump by manipulating and fitting various parts and pieces into a provided pre-assembled blueprint. We conducted a preliminary expert panel review composed of three experts and received positive feedback and suggestions for further development of the platform.