Robotics education fosters computational thinking, creativity, and problem-solving, but remains challenging due to technical complexity. Game-based learning (GBL) and gamification offer engagement benefits, yet their comparative impact remains unclear. We present the first PRISMA-aligned systematic review and comparative synthesis of GBL and gamification in robotics education, analyzing 95 studies from 12,485 records across four databases (2014–2025). We coded each study’s approach, learning context, skill level, modality, pedagogy, and outcomes (κ =.918). Three patterns emerged: (1) approach–context–pedagogy coupling (GBL more prevalent in informal settings, while gamification dominated formal classrooms [p <.001] and favored project-based learning [p =.009]); (2) emphasis on introductory programming and modular kits, with limited adoption of advanced software (~17%), advanced hardware (~5%), or immersive technologies (~22%); and (3) short study horizons, relying on self-report. We propose eight research directions and a design space outlining best practices and pitfalls, offering actionable guidance for robotics education.

Intelligence and knowledgeability are sometimes treated interchangeably in virtual agents, yet they shape interaction in different ways. We disentangled these traits and tested how each drives human perceptions and interaction in virtual reality (VR). To address the lack of prior research examining both traits simultaneously, we created a VR application where participants collaborated with a virtual agent to complete a jigsaw puzzle while engaging in free-flowing conversation about the puzzle’s art piece. We manipulated intelligence through the virtual agent’s puzzle-solving ability and knowledgeability through its predefined depth of knowledge in art. Using a 2 × 2 within-group study, we collected perceptual responses, logged data, and qualitative feedback. Results showed intelligence significantly influenced perceptions of intelligence, knowledge, rapport, trust, co-presence, uncanny valley, and intelligence and knowledge comparisons, while knowledgeability impacted perceived knowledge, trust, and intelligence and knowledge comparisons. Interaction effects further highlighted their interdependence, offering design implications for virtual agents.

Playtesting is widely used in the game industry to identify design flaws and evaluate player experience, yet little research explores how to effectively visualize and analyze playtesting data. This challenge is particularly pronounced in motion-based VR games, which involve physical movements and interactions tracked through multimodal inputs, resulting in complex multidimensional data. To better understand the challenges designers face, we conducted a formative study with 30 practitioners in the VR domain to characterize playtesting workflows and associated tasks. Based on these findings, we present HieraVisVR, a hierarchical visual analytics framework that incorporates body-motion-related data to help designers identify player behaviors and critical game moments, simplifying their workflow. We demonstrate the applicability of HieraVisVR in three different applications and evaluate our system with playtesting experts through an analysis of motion-based game data. The study results suggest that our system enhances playtesters’ understanding of the gameplay and improves their data analysis workflow.

In this study, we examined the impact of agent familiarity and knowledgeability on several variables spanning agent perceptions (i.e., perceived knowledge, familiarity, trust, anthropomorphism, uncanny valley effect, and likability), social and emotional experiences (i.e., co-presence, rapport, cognitive process expectations, and willingness for future interaction), and conversation dynamics (i.e., conversation transcript, participants’ response word count, and response time). We created two virtual agents for the study: a digital replica of a professor from our department (i.e., familiar agent) and an agent with similar demographic variables (i.e., age, gender, and ethnicity) but with a fabricated appearance and voice (i.e., unfamiliar agent). We implemented both agents to exhibit two levels of knowledgeability (i.e., low and high) in the domain of game development and course-specific information. We used large language models (LLMs) to provide the agents with persona information and domain knowledge through prompt engineering. For our user study, we followed a 2 (familiarity: unfamiliar vs. familiar agent) \(\times\) 2 (knowledgeability: low vs. high knowledgeability) within-group study design and recruited 32 participants who engaged in a 5-minute, conversation-based virtual reality (VR) interaction with all four experimental conditions: unfamiliar agent with low knowledgeability (ULK), unfamiliar agent with high knowledgeability (UHK), familiar agent with low knowledgeability (FLK), and familiar agent with high knowledgeability (FHK). The findings demonstrated a significant main effect of agent familiarity on perceived knowledge, suggesting that familiarity plays a crucial role in shaping users’ perception of the agent’s knowledgeability level. Besides perceived knowledge, familiarity also affected all other variables, apart from co-presence. Conversely, agent knowledgeability affected perceived familiarity, trust, anthropomorphism, cognitive process expectations, willingness for future interaction, conversation content, and participants’ response word count. Finally, we found an interaction effect between agent familiarity and perceived knowledge, indicating that familiarity has a significant influence on users’ perceptions of the agent’s knowledgeability. This study contributes to the field of conversational human-agent interaction in VR by providing empirical evidence on how adapting both familiarity and knowledgeability of virtual agents can significantly enhance user experience, offering valuable insights into designing more engaging, trustworthy, and effective embodied conversational agents.

This paper investigates how sinusoidal terrain characteristics and input modalities influence locomotion performance and user experience in virtual reality (VR). Two studies were conducted using a custom-built elastic balance board (EBB). The first study (Study 1) employed a 2 (amplitude: low vs. high) × 2 (frequency: low vs. high) within-group (N = 24) design to examine how slope characteristics affect locomotion efficiency (i.e., speed and completion time), postural stability (i.e., sway velocity), and subjective experiences (i.e., motion sickness, vection, usability, fear of movement, presence, and workload). High frequency slopes in terrains increased completion time, sway velocity, motion sickness, fear of movement, and vection, whereas amplitude alone had minimal effects. The second study (Study 2) used a 2 (amplitude: low vs. high) × 2 (input: EBB vs. joystick) within-group (N = 25) design to explore amplitude effects further and compare locomotion input methods. High amplitude increased completion time, sway velocity, and fear of movement, while EBB use enhanced presence and vection but required a higher workload compared to joystick input. Across both studies, frequency emerged as the primary factor influencing locomotion demands, and amplitude effects became more pronounced when combined with embodied locomotion interfaces. These findings provide design guidelines for VR locomotion systems, emphasizing the importance of aligning terrain complexity with input modality to optimize user comfort and control accuracy.

Motion tracking to project users into embodied virtual reality (VR) as avatars is an essential application of real-time computer graphics. Most current embodied VR systems rely on head-mounted displays (HMDs) to estimate user pose, as headset sensors can track the head and hands, thereby reconstructing the full body without the need for external hardware. However, measuring the quality of motion reconstruction algorithms from HMD-based tracking, particularly those intended for use in social settings, remains challenging due to the complex interaction between motion and perceived social signals. This paper compares two industrial tracking reconstruction solutions, called HMD1 (i.e., a basic HMD-based method that uses head tracking and hand positions estimated from HMD cameras) and HMD2 (i.e., an advanced HMD-based method with additional onboard camera streams), that estimate user motion using only an HMD against ground-truth motion capture (MoCap) data. It advocates for a social signal-based analysis that views motion as a communication medium and employs user observations to measure whether viewers successfully perceive the information encoded in motion. Across 156 socially expressive clips, Social Signal ratings were more effective than generic measures at revealing differences between the HMD methods. HMD2 preserved social meaning more accurately than HMD1, with fewer significant deviations from MoCap, while both HMD methods were frequently rated less natural than MoCap. A qualitative review localized recurrent failure modes, such as arm swivel/shoulder errors, posture reconstruction issues, and floating/stance artifacts, which help explain the misreading of social signals. We release a dashboard scorecard, motion capture data, and a benchmark protocol to enable consistent motion evaluation. More generally, this work advocates for an underexplored approach to motion evaluation that focuses on assessing the semantics of motion to determine quality. As reliance on generative artificial intelligence (AI) increases, it is essential to standardize evaluation to preserve the authenticity of the social signals conveyed. The developed dataset and the evaluation framework are provided on our project's website: https://github.com/facebookresearch/MotionIsTheMessageDataset.

We explored human-virtual agent collaboration during a layout design task in a virtual reality environment. Specifically, we developed a human-in-the-loop optimization-based method that drives the decision-making of the virtual agent. Our algorithm accounts for spatial constraints in furniture placement by evaluating boundary proximity, collision costs, and relationships between furniture items in real-time. It also considers the current configuration of the living room, as modified by the user during the co-design process, to guide the virtual agent's furniture placement decisions in the virtual living room. We compared our method (i.e., optimization) against two other co-design strategies (i.e., template and random) following a within-group ($N=24)$study design. We found the proposed optimization co-design strategy significantly enhanced perceived collaboration compared to the other two co-design strategies. Moreover, our participants attributed higher private and public awareness to the virtual agent in the optimization condition. In addition, the analysis of the logged data showed that participants placed more furniture items and made fewer corrections when codesigning the living room with a virtual agent whose decisions were based on the optimization method. Our results demonstrate that a virtual agent's behavior, which dynamically responds to user actions while maintaining spatial coherence, creates more effective collaborative experiences in an immersive co-design task.

Locomotion in virtual reality (VR) significantly impacts user immersion, spatial orientation, and overall experience. While traditional methods such as joystick-based movement and teleportation remain common, they present challenges like spatial disorientation, reduced presence, and motion sickness. In contrast, more embodied techniques, such as walking-in-place (WIP) and balance boards, aim to provide a more naturalistic form of locomotion. However, balance boards have proven to enhance the sense of control and presence in VR environments. Thus, we decided to further explore the benefits of isometric and elastic balance boards by examining their effects on VR locomotion movement behaviors, user experience, and motion sickness symptoms during a path-following task. We followed a 2 (feedback: isometric vs. elastic)$\times 3$(orientation: virtual vs. physical vs. conditional virtual) within-group experimental design involving 28 participants. The collected data included movement metrics (i.e., trajectory length, speed, time, root mean square error, sway velocity), user experience metrics (i.e., usability, spatial presence, task load, intrinsic motivation), and motion sickness metrics. Our results indicated that orientation significantly affected trajectory length, speed, and sway velocity, while feedback strongly influenced movement accuracy. Additionally, the user experience was influenced by the combination of feedback and orientation, with participants favoring elastic feedback, particularly when paired with virtual orientation. Finally, we found motion sickness to be less pronounced with elastic feedback, especially in virtual and conditional virtual orientations, highlighting the importance of feedback in reducing discomfort during VR locomotion.

We explored how responsiveness (i.e., the ability to answer questions) and awareness (i.e., the ability to navigate toward the user in the virtual environment) of a virtual agent acting as a tour guide impact study participants in a virtual museum. We followed a 2 (responsiveness: non-responsive vs. responsive virtual agent)$\times 2$(awareness: unaware vs. aware virtual agent) within-group ($N=29$) study design and conducted a study to explore several variables spanning: agent credibility and intelligence (i.e., perceived intelligence, perceived knowledge), social interaction and presence (i.e., co-presence, rapport), human-likeness (i.e., uncanny valley, anthropomorphism), awareness dimensions (i.e., private, public, and surrounding awareness), desire for future interaction, and behavioral responses (i.e., distance traveled, dwell gazes). We found that the responsive virtual agent positively impacted participants' perceived intelligence, perceived knowledgeability, co-presence, rapport, anthropomorphism, surrounding awareness, desire for future interaction, and dwell gaze on surroundings. However, the awareness factor did not impact our participants. Instead, we found responsiveness$\times$awareness interaction effects on distance traveled, dwell gaze on the virtual agent, and dwell gaze on surroundings. These findings offer valuable insights into designing intelligent virtual agents that act as museum tour guides, enhancing user experience in virtual museum settings.

ABSTRACT Interacting with small objects in virtual reality (VR) can be challenging due to the physical limitations of controllers and headsets, which often lead to unintended collisions and tracking loss when devices come too close, thereby disrupting the user's immersive experience. While researchers have developed techniques like translational gain, hand remapping, and specialized interaction to address these challenges, these approaches are often task‐specific or insufficient for precise, detailed interactions or observations. To address these challenges, in this paper, we introduce a novel interaction technique called dynamic translational gains manipulation (DTGM), which adjusts scaling in real‐time based on the user's proximity to objects. We conducted a user study to evaluate the effectiveness of improving precision during object manipulation and understanding the subjective mental workload of the proposed DTGM technique. Our results revealed that the DTGM technique improved interaction efficiency, making it suitable for various VR applications where precision and space optimization are crucial.

ABSTRACT This study examines how multimodal communication strategies (subtitles, visualizations, and their combination), affect memory retention, attitudes toward behavior change, and cognitive load during long conversations (+20 min) in immersive virtual reality (VR). Using embodied conversational agents to educate participants on diabetes and healthy eating, we found that all conditions effectively improved memory retention and behavior change attitudes. However, the combination of multimodal strategies increased cognitive load, suggesting a trade‐off between engagement and cognitive demands. These findings highlight the potential of long VR conversations for healthcare education, while emphasizing the importance of balancing cognitive demands and exploring personalization for diverse users.

In virtual reality, creating intelligent virtual characters has been a long-lasting endeavor. However, while researchers have investigated several aspects of a virtual character’s intelligence, little attention has been paid to the impact of the implemented intelligence levels assigned to a virtual character during human–virtual character collaboration. Thus, we conducted a within-group study ( \(N=24\) ) to explore how three different intelligence levels (low vs. medium vs. high) assigned to a virtual character can impact how study participants perceive that virtual character and interact with the task they are instructed to complete. Specifically, for our study, we developed a jigsaw puzzle game and instructed our participants to solve it with the help of a virtual character. During the jigsaw puzzle solving process, we collected application logs related to how the participants executed the task and observed the virtual environment. Moreover, after each condition, we asked the participants to respond using a questionnaire that examined their social presence, how they perceived the character’s intelligence and compared it with their own, and how they rated the virtual character’s realism. Our results indicated that the different intelligence levels assigned to the virtual characters impacted participants’ responses on several variables, including co-presence, perceived intelligence, intelligence comparison, and character interaction and behavior realism. Moreover, based on the collected logged data, we found that the intelligence levels assigned to our virtual character significantly impacted the performance of our participants. Our results could be valuable to the research community for creating more engaging experiences with intelligent virtual characters for collaborative tasks in immersive environments.

Designing interactions between humans and virtual characters requires careful consideration of various human perceptions and user experiences. While numerous studies have explored the effects of several virtual characters' properties, the impacts of the virtual character's personality and age on human perceptions and experiences have yet to be thoroughly investigated. To address this gap, we conducted a within-group study (N = 28) following a 2 (personality: egoism vs. altruism) × 2 (age: child vs. adult) design to explore how the personality and age factors influence human perception and experience during interactions with virtual characters. In each condition of our study, our participants co-solved a jigsaw puzzle with a virtual character that embodied combinations of personality and age. After each condition, participants completed a survey. We also asked them to provide written feedback at the end of the study. Our statistical analyses revealed that the virtual character's personality and age significantly influenced participants' perceptions and experiences. The personality factor affected perceptions of altruism, anthropomorphism, likability, safety, and all aspects of user experience, including perceived collaboration, rapport, emotional reactivity, and the desire for future interaction. Additionally, the virtual character's age affected our participants' ratings of the uncanny valley and likability. We also identified an interaction effect between personality and age factors on the virtual character's anthropomorphism. Based on our findings, we offered guidelines and insights for researchers aiming to design collaborative experiences with virtual characters of different personalities and ages.

An animated agent’s believability, often defined as “the illusion of life,” has many facets, including the agent’s appearance, movements, and user interactions. Considering that human facial expressions are often not symmetrical, evaluating how facial expression asymmetries affect believability in animated agents would provide valuable insights for researchers and artists. Our research focused on facial movements and examined the impact of three dynamic facial asymmetries on users’ perception of the agent’s believability, appeal, and naturalness. Our research involved three studies: 1) a preliminary study in which we determined the maximum values to base asymmetry levels, 2) a study in which we identified what level of asymmetry yields the most perceived believability for each asymmetry location, and 3) a study in which we determined what effects different combinations of asymmetries have on perceived believability, appeal, and naturalness. The findings confirmed the effect of dynamic facial asymmetry on the agents’ perceived believability, appeal, and naturalness. Furthermore, our research suggests that including one or a combination of asymmetric eyebrow, mouth, or eyelid motions increases the agents’ perceived believability, appeal, and naturalness.

We explored how study participants interacted with virtual characters in a virtual reality study. Specifically, we developed a 3 (proximity: close vs. middle vs. far) $\times 2$ (gesture: passive vs. active) experimental design (N = 26) to understand how combinations of proximity between two virtual characters and gestures assigned to them influence study participants’ self-reported ratings (co-presence, attentional allocation, behavioral interdependence, emotional reactivity, and perceived politeness). We also examined their avoidance movements (duration, trajectory length, and speed) and their avoidance decisions (passing through/around and minimum distance side). We collected both survey responses and our participants’ trajectories. Our study revealed that 1) the proximity factor impacted how our participants rated their co-presence and behavioral interdependence, as well as whether they decided to pass through or around the virtual characters, and 2) the gesture factor impacted how participants rated their behavioral interdependence, emotional reactivity, perceived politeness, and also affected their duration, trajectory length, and speed. Our research contributes to understanding personal space and social norms in virtual environments, offering valuable insights for virtual reality developers on the importance of social dynamics in designing interactions with virtual characters.

Researchers have provided insights into using virtual reality (VR) for visualization and interaction with 3D models and simulations. The interaction allows users to manipulate the 3D elements and visualize changes based on their inputs from movement with controllers or spatial actions. However, some users may find this interaction overwhelming, especially when immersed in a virtual environment. Additionally, the choice of dimensionality for visualizations influences user interaction, with potential implications for immersive experiences. Thus, we conducted a 2 (Immersion: Desktop vs. HMDVR) $\times 2$ (Dimensionality: 2 D vs. 3 D) within-group study $(N=32)$ to explore the impact of the utilized immersive degree and the dimensionality representation of the content on participants’ experience in terms of engagement, task load, usability, skill, and emotions when interacting with a science simulation. We designed and developed an application to simulate charged particles and electric field lines. We asked participants to complete a task of changing particles by matching them to a given simulation output. Our results indicated higher workload rates for HMDVR conditions, particularly with 3D representation, compared to Desktop. However, HMDVR conditions also showed greater engagement, emotional response, and presence. Based on our findings, we argue that participants prefer HMDVR over Desktop environments regardless of dimensionality.

Virtual character’s intelligence and task complexity are essential components for the design of collaboration between humans and virtual characters. However, studies have yet to explore the impact of a virtual character’s intelligence and task complexity during a collaboration task. To explore these impacts, we implemented a jigsaw puzzle co-solving experience and conducted a within-group study (N = 27) following a 2 (intelligence: low vs. high) × 2 (complexity: low vs. high) study design. During the puzzle co-solving process, we collected participants’ gaze data for each experimental condition. After our participants completed each condition (co-solved the jigsaw puzzle), they answered a survey that captured several variables. We found several significant main and interaction effect results indicating the impact of the virtual character’s intelligence and the task’s complexity. Specifically, the virtual character’s intelligence impacted most survey ratings, including the virtual character’s intelligence, perceived collaboration, self-confidence, and perceived contribution. Also, the task complexity affected perceived collaboration, the virtual character’s public awareness, and self-confidence. Furthermore, we found that the virtual character’s intelligence and complexity dominantly impacted our participants’ task load and perception of the virtual character’s confidence. Moreover, we found that our participants gazed at the virtual character and the puzzle piece more often than the puzzle goal across all experimental conditions. Our findings can inform the design of intelligent virtual characters that co-solve tasks with users.

We investigate how gender-anonymous voice avatars influence women’s performance in online computing group work. Female participants worked with two male confederates. Voices were filtered according to four voice gender anonymity conditions: (1) All unmasked, (2) Male confederates masked, (3) Female participant masked, and (4) All masked. When only male confederates used masked voices (compared to all unmasked), female participants spoke for a longer period of time and scored higher on computing problems. When everyone used masked voices (compared to all unmasked), female participants spoke for a longer period of time, spoke more words, and scored higher on computing problems. Effects were not significant on subjective measures and one behavioral measure. We discuss the implications for virtual interactions between people.

Although researchers have explored how humans perceive the intelligence of virtual characters, few studies have focused on the ability of intelligent virtual characters to fix their mistakes. Thus, we explored the self-correction behavior of a virtual character with different intelligence capabilities in a within-group design ( \(N=23\) ) study. For this study, we developed a virtual character that can solve a jigsaw puzzle whose self-correction behavior is controlled by two parameters, namely, Intelligence and Accuracy of Self-correction . Then, we integrated the virtual character into our virtual reality experience and asked participants to co-solve a jigsaw puzzle. During the study, our participants were exposed to five experimental conditions resulting from combinations of the Intelligence and Accuracy of Self-correction parameters. In each condition, we asked our participants to respond to a survey examining their perceptions of the virtual character’s intelligence and awareness (private, public, and surroundings awareness) and user experiences, including trust, enjoyment, performance, frustration, and desire for future interaction. We also collected application logs, including participants’ dwell gaze data, completion times, and the number of puzzle pieces they placed to co-solve the jigsaw puzzle. The results of all the survey ratings and the completion time were statistically significant. Our results indicated that higher levels of Intelligence and Accuracy of Self-correction enhanced not only our participants’ perceptions of the virtual character’s intelligence, awareness (private, public, and surroundings), trustworthiness, and performance but also increased their enjoyment and desire for future interaction with the virtual character while reducing their frustration and completion time. Moreover, we found that as the Intelligence and Accuracy of Self-correction increased, participants had to place fewer puzzle pieces and needed less time to complete the jigsaw puzzle. Finally, regardless of the experimental condition to which we exposed our participants, they gazed at the virtual character for more time compared to the puzzle pieces and puzzle goal in the virtual environment.

We introduce a novel co-design method for autonomous moving agents' shape attributes and locomotion by combining deep reinforcement learning and evolution with user control. Our main inspiration comes from evolution, which has led to wide variability and adaptation in Nature and has significantly improved design and behavior simultaneously. Our method takes an input agent with optional user-defined constraints, such as leg parts that should not evolve or are only within the allowed ranges of changes. It uses physics-based simulation to determine its locomotion and finds a behavior policy for the input design that is used as a baseline for comparison. The agent is randomly modified within the allowed ranges, creating a new generation of several hundred agents. The generation is trained by transferring the previous policy, which significantly speeds up the training. The best-performing agents are selected, and a new generation is formed using their crossover and mutations. The next generations are then trained until satisfactory results are reached. We show a wide variety of evolved agents, and our results show that even with only 10% of allowed changes, the overall performance of the evolved agents improves by 50%. If more significant changes to the initial design are allowed, our experiments' performance will improve even more to 150%. Our method significantly improved motion tasks without changing body structures, and it does not require considerable computation resources as it works on a single GPU and provides results by training thousands of agents within 30 minutes.

We explored the impact of depth of knowledge on conversational agents and human perceptions in a virtual reality (VR) environment. We designed experimental conditions with low, medium, and high depths of knowledge in the domain of game development and tested them among 27 game development students. We aimed to understand how the agent's predefined knowledge levels affected the participants' perceptions of the agent and its knowledge. Our findings showed that participants could distinguish between different knowledge levels of the virtual agent. Moreover, the agent's depth of knowledge significantly impacted participants' perceptions of intelligence, rapport, factuality, the uncanny valley effect, anthropomorphism, and willingness for future interaction. We also found strong correlations between perceived knowledge, perceived intelligence, factuality, and willingness for future interactions. We developed design guidelines for creating conversational agents from our data and observations. This study contributes to the human-agent interaction field in VR settings by providing empirical evidence on the importance of tailoring virtual agents' depth of knowledge to improve user experience, offering insights into designing more engaging and effective conversational agents.

We examined the effects of virtual characters’ body type and voice pitch on perceived audio-visual correspondence and believability. For our within-group study (N = 72), we developed nine experimental conditions using a 3 (body type: ectomorph vs. mesomorph vs. endomorph body types) × 3 (voice pitch: low vs. medium vs. high fundamental frequency [F0]) design. We found statistically significant main effects from voice pitch and statistically significant interaction effects between a virtual character’s body type and voice pitch on both the level of perceived audio-visual correspondence and believability of female and male virtual characters. For female virtual characters, we also observed an additional statistically significant main effect from body type and a statistically significant interaction effect between the participant’s biological sex and the virtual character’s voice pitch on both perceived audio-visual correspondence and believability. Moreover, the results show that perceived believability is highly correlated to perceived audio-visual correspondence. Our findings have important practical implications in applications where the virtual character is meant to be an emotional or informational guide that requires some level of perceived believability, as the findings suggest that it is possible to enhance the perceived believability of the virtual characters by generating appropriate voices through pitch manipulation of existing voices.

We explored the avoidance movement behaviors of study participants immersed in a virtual reality environment. We placed a static virtual character at the midpoint between the start and target spot for the avoidance task, and a virtual walker character in front of the starting spot and scripted it to reach the target spot. Participants were placed behind the virtual walker in order to measure its influence on participants’ behavior. We developed nine experimental conditions assigned to the virtual walker character by following a 3 (speed: slow vs. normal vs. fast walking speed) $\times 3$ (proximity: close vs. middle vs. far proximity to the static virtual character) study design. For this within-group study, we collected data from 22 study participants to explore how speed and proximity walking patterns assigned to a virtual walker character could impact participants’ avoidance movement behaviors and decisions. Our data revealed that 1) the speed factor impacted the participants’ avoidance movement behavior; 2) the proximity factor did not significantly impact the participants’ avoidance movement behavior; 3) the virtual walker character did not significantly impact participants’ avoidance decisions regarding the static virtual character; 4) in all examined conditions, the side-by-side distances between the participants and the static virtual character were inside the social space according to the proxemics model; and 5) in conditions in which a slow virtual walker character was present or in the condition of normal speed and far proximity, we observed an increased number of participants pass the virtual walker character.

Summary Midair typing has gradually become the potential mainstream of virtual reality (VR) typing, as it eliminates the setup of additional hardware devices and provides convenience for portable head‐mounted displays (HMDs). In midair typing, users type on a virtual keyboard while using their virtual hands. In addition to the lack of a dominant method for VR‐based midair typing, we still do not know the impact that different appearances of virtual hands could have on text input in VR. Thus, we picked a widely used midair typing method (i.e., the VR‐rendered QWERTY keyboard) and conducted a VR study to explore the effects of three hand appearances (i.e., abstract, mannequin, realistic hands) on typing efficiency in terms of typing accuracy, typing speed, and task load. Our within‐group study revealed that (1) the mannequin hand caused significantly higher performance on typing speed than the abstract and realistic hands, (2) the abstract hand caused significantly lower performance (more typing errors) on typing accuracy compared to the mannequin and realistic hands, and (3) participants rated the task load of using the abstract hand significantly higher than the mannequin and realistic hands.

Summary This paper focuses on understanding how study participants interact and perceive a virtual crowd in an immersive virtual environment. Specifically, our within‐group exploratory study investigated how avoidance proximity variations (i.e., low, medium, and high avoidance proximity [defined as avoidance radius]) assigned to crowd agents impacted participants' interaction with the virtual crowd. During the study, we instructed our participants to walk in a virtual environment. At the same time, we had a virtual crowd scripted to walk toward the start position of the participant following a straight path. During the participants' walking task, we collected movement data (i.e., trajectory length and completion time) and immediately after each experimental condition, we asked participants to self‐report their experience (i.e., co‐presence, behavioral independence, crowd realism, crowd interaction realism, perceived politeness, and emotional reactivity). Based on the collected data, we found that when we exposed our participants to the high avoidance proximity condition, they: (1) followed longer paths, (2) spent more time reaching the target goal, (3) rated the virtual crowd less polite, (4) rated the virtual crowd and their interaction with the virtual crowd less realistic, (5) rated the behavior independence of the virtual crowd lower, and (6) self‐reported higher emotional reactivity. We discuss our findings and suggestions for further research on human‐virtual crowd interaction.

We developed a method to synthesize game levels that accounts for the degree of collaboration required by two players to finish a given game level. We first asked a game level designer to create playable game level chunks. Then, two artificial intelligence (AI) virtual agents driven by behavior trees played each game level chunk. We recorded the degree of collaboration required to accomplish each game level chunk by the AI virtual agents and used it to characterize each game level chunk. To synthesize a game level, we assigned to the total cost function cost terms that encode both the degree of collaboration and game level design decisions. Then, we used a Markov-chain Monte Carlo optimization method, called simulated annealing, to solve the total cost function and proposed a design for a game level. We synthesized three game levels (low, medium, and high degrees of collaboration game levels) to evaluate our implementation. We then recruited groups of participants to play the game levels to explore whether they would experience a certain degree of collaboration and validate whether the AI virtual agents provided sufficient data that described the collaborative behavior of players in each game level chunk. By collecting both in-game objective measurements and self-reported subjective ratings, we found that the three game levels indeed impacted the collaboration gameplay behavior of our participants. Moreover, by analyzing our collected data, we found moderate and strong correlations between the participants and the AI virtual agents. These results show that game developers can consider AI virtual agents as an alternative method for evaluating the degree of collaboration required to finish a game level.

Avatar customization is known to positively affect crucial outcomes in numerous domains. However, it is unknown whether audial customization can confer the same benefits as visual customization. We conducted a preregistered 2 x 2 (visual choice vs. visual assignment x audial choice vs. audial assignment) study in a Java programming game. Participants with visual choice experienced higher avatar identification and autonomy. Participants with audial choice experienced higher avatar identification and autonomy, but only within the group of participants who had visual choice available. Visual choice led to an increase in time spent, and indirectly led to increases in intrinsic motivation, immersion, time spent, future play motivation, and likelihood of game recommendation. Audial choice moderated the majority of these effects. Our results suggest that audial customization plays an important enhancing role vis-à-vis visual customization. However, audial customization appears to have a weaker effect compared to visual customization. We discuss the implications for avatar customization more generally across digital applications.

Synthesizing game levels that evoke players’ curiosity, driving them to explore different level parts, is time-consuming and tedious. Typically, game level designers manually perform this synthesis using trial and error. In this paper, we propose a method with which to replace this manual, time-consuming process. We benefited from recent work that had proposed game level design patterns to evoke curiosity, and we propose an approach to automatically synthesizing game levels in order to encourage players to pursue designer-specified exploration goals. We started by creating a dataset of level assets, based on the four design patterns that evoke curiosity-driven exploration in games (reaching extreme points, resolving visual obstructions, out-of-place objects, and understanding spatial connections). We annotated the assets in our dataset with spatial exploration measurements (the time players took to explore an asset over their total time spent in the game level). We then formulated game level design as an optimization problem, encoding both spatial exploration (mean spatial exploration, spatial exploration variance, and spatial exploration distribution) and game level design (occupied area, adjacent penalty, and height distribution) decisions. Then, we solved this problem by implementing a reversible-jump Markov chain Monte Carlo method. We demonstrate our method’s ability to synthesize game level variations with different spatial exploration and level design decisions. Finally, a user study showed that our approach can automatically synthesize game levels, encouraging a certain amount of spatial exploration by players.

Games are now ubiquitous, and educational games are becoming increasingly prevalent. Like other games, educational video games attract participants from different ethnicities and with different gender expressions. As such, educational game designers face a necessity to develop inclusive games. In this paper, we focus on inclusivity, diversity, and equity (DEI) issues by investigating if the computer programming game Mazzy benefited participants from broad demographic backgrounds. We highlight inclusive features present in Mazzy, and, focusing on the participants’ self-reported gender and race/ethnicity, reflect on their play experience and learning outcomes. We found evidence that the game supported learning outcomes and facilitated an engaging play experience for participants from diverse demographic backgrounds. We discuss challenges and implications for the broader literature.

In virtual reality (VR) applications, humans experience the provided content not only through the visual and auditory systems but also through the somatosensory system. Thus, we decided to conduct a VR study to further explore the just noticeable difference (JND) of tactile feedback to understand humans’ perceptions of tactile stimuli. Our VR study examined the JND in terms of the intensity, duration, and frequency of tactile feedback provided through commercially available vibrotactile motion controllers, the Oculus Quest 2 controllers. We instructed participants to report whether they perceived a difference between a reference (variation) and a testing stimulus at each point in the experiment for a different property (intensity, duration, and frequency) of tactile feedback. We report both positive and negative JND values for the three properties of tactile feedback. We discuss our findings and limitations and provide directions for future studies regarding tactile perception for commercially available tactile feedback devices.

We have developed a mixed reality (MR)-based platform for basic mechanical engineering concepts as a learning environment for collaborative assembly tasks. In our platform, multiple co-located users interact with virtual objects simultaneously, and during that time, the platform collects data related to participants’ collaboration and team behavior. We implemented four main sections in the platform including setup, introduction, training, and assessment. The platform provides the opportunity for users to interact with virtual objects while also acquiring technical knowledge. Specifically, for the technical component of the platform, users are asked to assemble a hydraulic pump by manipulating and fitting various parts and pieces into a provided pre-assembled blueprint. We conducted a preliminary expert panel review composed of three experts and received positive feedback and suggestions for further development of the platform.

Abstract We explored user interactions with a holographic sign language interpreter in a mixed reality (MR) classroom for deaf and hard of hearing students. The developed MR application projects a holographic signing avatar that translates in real time the lecture while a speaking instructor is teaching. Our study explored user interaction with the MR system, intending to provide design guidelines for digital MR sign language interpreters. We recruited eight participants and conducted a usability test focused on avatar framing (full‐body vs. half‐body) and avatar manipulation (fixed position, scale, and orientation vs. user‐adjustable position, scale, and orientation) in the MR classroom. We used a mixed‐method approach to analyze quantitative and qualitative data through recordings, surveys, and interviews. The results show user preferences toward viewing holographic signing avatars in the MR environment and user acceptability toward such applications.

In this work, we introduce a Denser Feature Network(DenserNet) for visual localization. Our work provides three principal contributions. First, we develop a convolutional neural network (CNN) architecture which aggregates feature maps at different semantic levels for image representations. Using denser feature maps, our method can produce more key point features and increase image retrieval accuracy. Second, our model is trained end-to-end without pixel-level an-notation other than positive and negative GPS-tagged image pairs. We use a weakly supervised triplet ranking loss to learn discriminative features and encourage keypoint feature repeatability for image representation. Finally, our method is computationally efficient as our architecture has shared features and parameters during forwarding propagation. Our method is flexible and can be crafted on a light-weighted backbone architecture to achieve appealing efficiency with a small penalty on accuracy. Extensive experiment results indicate that our method sets a new state-of-the-art on four challenging large-scale localization benchmarks and three image retrieval benchmarks with the same level of supervision. The code is available at https://github.com/goodproj13/DenserNet

COVID-19 has upended lives everywhere, causing millions of deaths and tens of millions of infections worldwide. Nevertheless, for many people, staying in permanent isolation is neither desirable nor possible. To mitigate the spread of the disease, we iteratively developed a game that teaches hygienic best practices for preventing COVID-19. We consulted professional game designers, health experts, and educational technology designers. We then compared the effectiveness of the game to an equivalent video in two longitudinal experiments during the pandemic: 1) an experiment in a programming lab (N=11), and 2) an online-only experiment (N=475). In Experiment #1, we observe that participants in the game condition had higher intrinsic motivation, and a more sustained rise in hygienic self-efficacy, compared to participants in the video condition. Both conditions saw a rise in COVID-19 knowledge and positive hygienic attitude. Both conditions were relatively unchanged in COVID-19 anxiety and hygienic behavior. In Experiment #2, participants in the game condition experienced greater intrinsic motivation than participants in the video condition. Both conditions saw a sustained rise in COVID-19 hygienic self-efficacy, positive hygienic attitudes, and knowledge. Neither condition saw an effect on COVID-19 anxiety. Our work demonstrates that game-based learning can be an effective approach for teaching COVID-19 hygienic knowledge, for improving COVID-19 hygienic self-efficacy, and for fostering COVID-19 hygienic positive attitudes, and is more intrinsically motivating than video-based learning.

This virtual reality study was conducted to assess the impact of the appearance of virtual characters on the avoidance movement behavior of participants. Five experimental conditions were examined. Under each condition, one of the five different virtual characters (classified as mannequin, human, cartoon, robot, and zombie) was studied. Each participant had to experience only one condition and was asked to perform the collision avoidance tasks two times. During the walking task, the motion of participants was recorded. After finishing the collision avoidance segment of the study, a questionnaire that examined different concepts (emotional reactivity, emotional contagion, attentional allocation, behavioral independence, perceived skill, presence, immersion, virtual character realism, and virtual character unpleasantness) was distributed to the participants. Based on the collected measurements (avoidance movement behavior and self-reported ratings), we tried to understand the effects of the appearance of a virtual character on the avoidance movement behavior, and its possible correlation to subjective ratings. The results obtained from this study indicated that the appearance of the virtual characters did affect the avoidance movement behavior and also some of the examined concepts. Additionally, participant avoidance movement behavior correlates with some subjective ratings.

A virtual reality (VR) study was conducted both in our lab setting and remotely to investigate the effects of virtual hand appearance and tactile feedback on ownership, external appearance, and tactile sensation embodiment dimensions when participants were instructed to perform an assembly task in a virtual environment. Six experimental conditions that combine hand appearances (abstract, mannequin, and realistic) and tactile feedback (tactile and no tactile) levels were examined. The results of our study indicated that: (1) the more realistic hand had a stronger effect on tactile sensation and external appearance, while the mannequin hand was associated with a greater ownership effect; (2) tactile feedback was associated with a stronger effect on external appearance compared to no tactile feedback; (3) the realistic hand model in conjunction with tactile feedback significantly affected the tactile sensation of the participants; (4) the participants in the lab setting rated the external appearance of the realistic hand model higher than the remote participants; (5) the participants exposed for less than one hour per week to VR rated the tactile sensation in the presence of tactile feedback higher, while they similarly rated the external appearance of the virtual hand model in the presence of a realistic hand model higher when compared to those who were exposed for more than one hour per week to VR; and finally, (6) the younger participants rated the tactile sensation higher than the older participants in the presence of the realistic hand model. In this paper, we discuss our findings and provide design considerations for virtual reality applications that consider virtual hands and tactile feedback.

The COVID-19 pandemic forced many countries worldwide to take restrictive measures from securing borders and minimising movement, cancelling public gatherings, controlling citizens’ movement, and e...

Abstract This between‐group study investigated participants' experiences of tactile feedback patterns when asked to hug a virtual character. Five experimental conditions were developed, one with no tactile feedback and four with tactile feedback. The participants were placed in a virtual city and informed they would be meeting a virtual friend, who they were instructed to hug once the character came close to them. During the virtual hug, one of the five experimental conditions was examined. Immediately after the hug, participants were asked to complete a questionnaire to capture their experiences. The results obtained from this study indicated that: (1) even if the tactile feedback is not considered to be highly accurate in terms of timing, duration, and position, as long as it is perceived as less persistent, it provides a more positive experience; (2) the perceived realism of the virtual hug is strongly correlated with the perceived realism of the tactile feedback; and (3) the female participants had a more intense interaction with the virtual character (friend) compared with the male participants. Limitations and future study directions are discussed.

Abstract Previous research has shown that student‐instructor interaction is vital to motivating students to learn a second language. However, it is unclear whether learners' demographics affect in‐game immersion and interactions with virtual instructors. This study's purpose is to investigate whether the number of years learning Japanese (foreign language familiarity) influences students' immersion levels in serious games and their interactions with virtual instructors. We developed a 3D animated Japanese roleplaying game with a virtual in‐game instructor. Eighty‐four college students enrolled in 200‐ and 300‐level Japanese language courses voluntarily participated in the study. Participants played the game and then answered a questionnaire concerning virtual character appearance, attentiveness to the instructor, and immersion in the game. The findings indicated that gender and the number of years studying Japanese significantly impact multiple measurements.

Abstract This article presents a methodology for automatically synthesizing a virtual population (pedestrians placed in a virtual environment) that impacts a user with a specified affective experience. The pipeline began by developing a dataset of behaviors that could be assigned to virtual characters. Next, an annotation phase assigned affective responses of participants to each character's behavior. The design considerations of our affective multicharacter virtual reality experience were then encoded to cost terms and assigned to a total cost function. This method allowed the developer to control the priority and the targets of the cost terms, and given the user inputs, our application could optimize the multicharacter experience using a Markov chain Monte Carlo method known as simulated annealing. A user study was conducted to investigate whether our method could synthesize virtual reality multicharacter experiences that affect participants in an expected way. The results of our study showed that the three different synthesized multicharacter experiences (low, medium, and high negative affect) were perceived as expected by participants; therefore, we argue that we can indeed automatically synthesize virtual reality multicharacter experiences that impact participants' affect levels in an expected way. Limitations and future research directions are discussed.

Virtual reality (VR) has disrupted the gaming market and is rapidly becoming ubiquitous. Yet differences between VR and traditional mediums, such as controllers that are visible in the virtual world, enable entirely new approaches to instruction. In this paper, we present four studies, each using a different VR game. Within each study, we compared three different modalities of tutorials: Text (text-only), Text+Diagram (text with controller diagrams), and Text+Spatial (text with controller tooltips appearing on top of the player's virtual controllers). Data from our studies show that the importance of tutorial modality depends greatly on game type. In a third-person shooter, Text+Spatial led to significantly higher controls learnability than Text and Text+Diagram, and also led to significantly higher performance, player experience, and intrinsic motivation than Text. In a puzzle game, Text+Spatial led to significantly higher controls learnability and performance than Text. Additionally, Text+Diagram led to significantly higher controls learnability than Text. However, in a wave shooter and a rhythm game, differences between conditions were negligible on all measures. Our studies show that game type is an important factor to consider when designing tutorial modality.

Avatar identification is one of the most promising research areas in games user research. Greater identification with one's avatar has been associated with improved outcomes in the domains of health, entertainment, and education. However, existing studies have focused almost exclusively on the visual appearance of avatars. Yet audio is known to influence immersion/presence, performance, and physiological responses. We perform one of the first studies to date on avatar self-similar audio. We conducted a 2 x 3 (similar/dissimilar x modulation upwards/downwards/none) study in a Java programming game. We find that voice similarity leads to a significant increase in performance, time spent, similarity identification, competence, relatedness, and immersion. Similarity identification acts as a significant mediator variable between voice similarity and all measured outcomes. Our study demonstrates the importance of avatar audio and has implications for avatar design more generally across digital applications.

Synthesizing photo products such as photo strips and slideshows using a database of images is a time-consuming and tedious process that requires significant manual work. To overcome this limitation, we developed a method that automatically synthesizes photo sequences based on several design parameters. Our method considers the valence and arousal ratings of images in conjunction with parameters related to both the visual consistency of the synthesized photo sequence and the progression of valence and arousal throughout the photo sequence. Our method encodes valence, arousal, and visual consistency parameters as cost terms into a total cost function while applying a Markov chain Monte Carlo optimization techniques called simulated annealing to synthesize the photo sequence based on user-defined target objectives in a few seconds. As our method was developed for the synthesis of photo sequences using the valence-arousal emotional model, a user study was conducted to evaluate the efficacy of the synthesized photo sequences in triggering valence-arousal ratings as expected. Our results indicate that the proposed method synthesizes photo sequences in which valence and arousal dimensions are perceived as expected by participants; however, valence may be more appropriately perceived than arousal.

We have developed a modular virtual reality gaming application that can be used to synthesize exercise drills for racket sports. By defining cost terms that are related to the gameplay and the mechanics of the game, as well as by allowing a user to control the parameters of the cost terms, users can easily adjust the objectives and the intensity levels of the exercise drills. Based on the user-defined exercise objectives, a Markov chain Monte Carlo optimization method called “simulated annealing” was used to optimize the exercise drill. The effectiveness of the developed virtual reality gaming application was measured in two studies by using virtual reality table tennis as the evaluation tool. The first study investigated the potential usefulness of the developed virtual reality gaming application as an exercise tool by comparing its workout effectiveness at three intensity levels (low, medium, and high) through the collection of heart rate readings. The second study explored the potential utility of the virtual reality gaming application as a training tool by exploring whether there was any improvement in participants' performance across the three conditions (no training, virtual reality training, and real-world training). The results indicate that a virtual reality gaming application, such as the examined virtual reality table tennis exergame, could indeed be used effectively as both an exercise and a training tool. Limitations and future research directions are discussed further below.

This paper examines a common problem found in a number of virtual reality setups—mismatches between real and virtual environments. Specifically, this paper investigates whether the mismatching between a real and a virtual environment in terms of appearance and physical constraints can affect the arousal (electrodermal activity) and movement behavior in the participants. For this study, one baseline condition and four mismatch conditions that examine different mismatching types were developed and tested in a between-group study design. The participants were immersed in a virtual environment and were asked to walk in a direction given to them along a provided path. During that time, electrodermal activity and the walking motion of participants were captured to assess potential alterations in their arousal and movement behavior respectively. Results obtained from this study indicate significant differences in the electrodermal activity and movement behavior of participants, especially when walking in a virtual environment that is mismatched both in appearance and physical constraints. Even though to a lesser degree, evidence was also found that correlates electrodermal activity with movement behavior. Limitations and future research directions are discussed.

Abstract The aim of our study was to investigate whether different tactile feedback conditions could affect the behavior of participants who were instructed to walk within a virtual reality environment surrounded by a virtual crowd of people. A road crossing scenario that takes place in a virtual metropolitan city was developed for this study. Participants were asked to walk toward the opposite sidewalk while wearing a tactile vest. At each road crossing, one of several tactile feedback conditions was generated, including No Tactile, Side Tactile, Back Tactile, Front Tactile, Accurate Tactile, and Random Tactile. During the virtual road crossing, the movement of the participants was captured, and movement‐related measurements were extracted and analyzed to evaluate the effects of tactile feedback on the participant's movement behavior. Additional data were collected through the distribution of a questionnaire to consider self‐reported ratings of the experimental conditions. The results revealed that tactile feedback conditions had significant effects on movement behavior, while the participants' ratings also indicated that they were affected by the tactile feedback conditions. We found that when the participants were immersed in a high‐density crowd simulation, they became sensitive to tactile feedback. However, they were not able to distinguish between the accurate feedback and the random feedback.

Abstract We investigated the movement behavior of participants walking within a virtual crowd in an immersive virtual environment. We investigated three different parameters that characterize a moving virtual crowd: density, speed, and direction. An immersive road‐crossing scenario that took place in a virtual metropolitan city was created. In this scenario, the participants were instructed to walk toward the opposite sidewalk. Three measurements (speed, deviation, and trajectory length) were used to evaluate the impact of the parameters assigned to the virtual crowd on the movement behavior of the participants. Significant results were found for both the main and interaction effects. The results suggested that the high density, low speed, and diagonal direction situations associated with the virtual crowd had the greatest impacts on the speed, deviation, and trajectory lengths of participants when they walked in a virtual environment and were surrounded by a moving virtual population.

Summary As motion‐sensing devices become more accessible to consumers, it is important to understand how users experience embodiment when using these devices. In our 3 × 2 between‐groups study, we explored eye fixations and electrodermal activity (EDA) in order to more objectively understand potential interaction effects between the self‐avatar body, and the presence of a mirror within the context of low‐budget embodiment. We developed six experimental conditions concerning Body (human, mannequin, and zombie self‐avatars) and Mirror (mirror and no mirror) factors, and presented participants with a virtual environment in which they could control their self‐avatars by using HTC Vive controllers and trackers. In addition to eye fixations and EDA, we assessed self‐reported data concerning body ownership, agency, self‐location, as well as enjoyment of the experience. Our results suggest that the Body may have been more influential in eliciting body ownership than the Mirror, and that an interaction effect between Gender and Body may influence eye gaze behavior. Additionally, female participants reported significantly higher agency than males. We consider logical next steps for similar research which might elaborate upon our findings.

A virtual reality study was conducted to understand participants' anxiety when immersed in a virtual reality trip with a self-driving car. Participants were placed as passengers in a virtual car, and they were seated in the co-driver seat. Five different conditions were developed and examined. For this experiment, the Anxiety Modality Questionnaire that captures the cognitive anxiety of participants was used. The obtained results indicated that the participants' level of anxiety for the partial awareness of the driver condition is influenced less than expected. Specifically, lower levels of anxiety were found when the driver is either fully or partially aware of the traffic and the behavior of the car, and higher anxiety levels were found when the driver is completely unaware.

How does haptic feedback during a human-virtual human interaction affect emotional arousal in virtual reality? In this between-subjects study, we compare haptic feedback and no haptic feedback conditions in which a virtual human “bumps” into the participant in order to determine the influence of haptic feedback on emotional arousal, sense of presence, and embodiment in virtual reality, as well as compare self-report measures of emotional arousal to those objectively collected via event-related galvanic skin response (GSR) recordings. We plan to extend the current preliminary study by adding three more conditions as described in the future work section. Participants are students age 18-32 with at least moderate experience in virtual reality. Preliminary results indicate significant differences in presence and embodiment between haptic feedback and no haptic feedback groups. With our small sample size at the current time, GSR does not show significant differences between haptic and no haptic feedback conditions.

The present study investigates users' movement behavior in a virtual environment when they attempted to avoid a virtual character. At each iteration of the experiment, four conditions (Self-Avatar LookAt, No Self-Avatar LookAt, Self-Avatar No LookAt, and No Self-Avatar No LookAt) were applied to examine users' movement behavior based on kinematic measures. During the experiment, 52 participants were asked to walk from a starting position to a target position. A virtual character was placed at the midpoint. Participants were asked to wear a head-mounted display throughout the task, and their locomotion was captured using a motion capture suit. We analyzed the captured trajectories of the participants' routes on four kinematic measures to explore whether the four experimental conditions influenced the paths they took. The results indicated that the Self-Avatar LookAt condition affected the path the participants chose more significantly than the other three conditions in terms of length, duration, and deviation, but not in terms of speed. Overall, the length and duration of the task, as well as the deviation of the trajectory from the straight line, were greater when a self-avatar represented participants. An additional effect on kinematic measures was found in the LookAt (Gaze) conditions. Implications for future research are discussed.

Abstract In this study, we compare haptic feedback and nonhaptic feedback conditions in which virtual characters bump into the participant who is immersed in a virtual environment. A questionnaire was developed to determine the influence of haptic feedback on a number of concepts (presence, embodiment, positive and negative affect, interaction realism with virtual character, and haptic feedback realism). Physiological data were also collected using galvanic skin response (GSR) to investigate the influence of haptic feedback on physiological arousal during human–virtual character interaction. Five conditions were developed (no haptic feedback, full intensity, half intensity, incorrect position, and delayed timing) to determine which aspects of haptic feedback are most important in influencing participant responses. Significant differences were found in embodiment, realism of virtual character interaction, and haptic feedback realism. In addition, significant differences were found in GSR amplitude after the first interaction with the virtual character. Implications for further research are discussed.

Taking advantage of motion capture and display technologies, a method giving a user the ability to control the dance motions of a virtual partner in an immersive setup was developed and is presented in this paper. The method utilizes a dance motion dataset containing the motion of both dancers (leader and partner). A hidden Markov model (HMM) was used to learn the structure of the dance motions. The HMM was trained on the motion of a chosen dancer (leader or partner), and during runtime, the system predicts the progress of the chosen dance motion, which corresponds to the progress of the user's motion. The regular structure of the HMM was extended by utilizing a jump state transition, allowing the user to improvise dance motions during the runtime. Since the jump state addition increases the model's complexity, an effort was made to optimize the prediction process to ensure runtime efficiency. A few corrective steps were also implemented to ensure the partner character's motions appear natural. A user study was conducted to understand the naturalness of the synthesized motion as well as the control that the user has on the partner character's synthesized motion.

Despite rapid advances in 3D printing, fabricating large, durable and robust artifacts is impractical with current technology. We focus on a particularly challenging environment-scale artifact: rock climbing routes. We propose a prototype fabrication method to replicate part of an outdoor climbing route and enable the same sensorimotor experience in an indoor gym. We start with 3D reconstruction of the rock wall using multi-view stereo and use reference videos of a climber in action to identify localized rock features that are necessary for ascent. We create 3D models akin to traditional indoor climbing holds, fabricated using rapid prototyping, molding and casting techniques. This results in robust holds accurately replicating the features and configuration of the original rock route. Validation was performed on two rock climbing sites in New Hampshire and Utah. We verified our results by comparing climbers' moves on the indoor replicas and original outdoor routes.