Interaction between humans and virtual agents is becoming increasingly prevalent. This research investigates the influence of virtual agents on users' opinions within a virtual stand-up comedy club setting, focusing on the socio-psychological mechanisms of emotional contagion and group conformity. We aimed to combine these two areas in a novel study approach. To this end, we designed and conducted an exploratory pilot study in which participants ($\mathrm{N}=20$) rated the funniness of jokes in VR, under varying conditions of virtual agent laughter (absence, medium intensity, strong intensity). Our analysis shows a significant positive linear effect on the log odds of higher funniness ratings when virtual agents exhibit laughter. Moreover, our study shows that this effect is mainly due to emotional contagion processes, individual differences in susceptibility to group conformity, and the interaction of both concepts, showing that emotional contagion happens only when the individual susceptibility to group conformity is high. The results offer valuable insights into the social impact of VR environments and the potential of virtual agents to shape user perception and opinion.

Sensorimotor contingencies (SC) refer to the rules by which we use our body to perceive. It has been argued that to the extent that a virtual reality (VR) application affords natural SC so the greater likelihood that participants will experience Place Illusion (PI), the illusion of 'being there' (a component of presence) in the virtual environment. However, notwithstanding numerous studies this only has anecdotal support. Here we used a reinforcement learning (RL) paradigm where 26 participants experienced a VR scenario where the RL agent could sequentially propose changes to 5 binary factors: mono or stereo vision, 3 or 6 degrees of freedom head tracking, mono or spatialised sound, low or high display resolution, or one of two color schemes. The first 4 are SC, whereas the last is not. Participants could reject or accept each change proposed by the RL, until convergence. Participants were more likely to accept changes from low to high SC than changes to the color. Additionally, theory suggests that increased PI should be associated with lower eye scanpath entropy. Our results show that mean entropy did decrease over time and the final level of entropy was negatively correlated with a post exposure questionnaire-based assessment of PI.

It is becoming increasingly easier to set up multi-user virtual reality sessions, and these can become viable alternatives to video conference in events such as international conferences. Moreover, it is possible to enhance such events with automated virtual humans, who may participate in the discussion. This paper presents the behind-the-scenes work of a panel session titled "Is virtual reality genuine reality?", which was held during a physical symposium, "XR for the people," in June 2022. The panel featured a virtual Albert Einstein, based on a large language model (LLM), as a panelist, alongside three international experts having a live conference panel discussion. The VR discussion was broadcast live on stage, and a moderator was able to communicate with both the live audience, the virtual world participants, and the virtual agent (Einstein). We provide lessons learned from the implementation and from the live production, and discuss the potential and pitfalls of using LLM-based virtual humans for multi-user VR in live hybrid events.

This work deals with the automatic 3D reconstruction of objects from frontal RGB images. This aims at a better understanding of the reconstruction of 3D objects from RGB images and their use in immersive virtual environments. We propose a complete workflow that can be easily adapted to almost any other family of rigid objects. To explain and validate our method, we focus on guitars. First, we detect and segment the guitars present in the image using semantic segmentation methods based on convolutional neural networks. In a second step, we perform the final 3D reconstruction of the guitar by warping the rendered depth maps of a fitted 3D template in 2D image space to match the input silhouette. We validated our method by obtaining guitar reconstructions from real input images and renders of all guitar models available in the ShapeNet database. Numerical results for different object families were obtained by computing standard mesh evaluation metrics such as Intersection over Union, Chamfer Distance, and the F-score. The results of this study show that our method can automatically generate high-quality 3D object reconstructions from frontal images using various segmentation and 3D reconstruction techniques.

This paper deals with automatic 3D reconstruction of objects from frontal RGB images. We propose a complete workflow that can be easily adapted to other object families. First, we detect and segment the object present in the image using Convolutional Neural Networks. In a second step, we perform the final 3D reconstruction of the object by warping the rendered depth maps of a fitted 3D template in 2D image space to match the input silhouette. To explain and validate our method, we focus on 3D guitar reconstructions from real input images and renders of guitar models available in the ShapeNet database. We also show qualitative examples of applying our reconstruction method to other family of objects. The results of this study show that our method can automatically produce high-quality 3D object reconstructions from frontal images using a set of segmentation and 3D reconstruction techniques.

We present methods used to produce and study a first version of an attempt to reconstruct a 1983 live rock concert in virtual reality. An approximately 10 minute performance by the rock band Dire Straits was rendered in virtual reality, based on the use of computer vision techniques to extract the appearance and movements of the band, and crowd simulation for the audience. An online pilot study was conducted where participants experienced the scenario and freely wrote about their experience. The documents produced were analyzed using sentiment analysis, and groups of responses with similar sentiment scores were found and compared. The results showed that some participants were disturbed not by the band performance but by the accompanying virtual audience that surrounded them. The results point to a profound level of plausibility of the experience, though not in the way that the authors expected. The findings add to our understanding of plausibility of virtual environments.

We present an automatic animatable 3D character reconstruction method from frontal and lateral RGB pictures of the person to reconstruct. We propose a method where, after obtaining a 3D reconstruction from a frontal view, we adapt the template of the Skinned Multi-Person Linear Model (SMPL) to it. This frontal modified SMPL is then used in a second step as an input to reconstruct the character from the lateral view, thus obtaining a correct person reconstruction in all dimensions. Our method enables the creation of 3D animatable characters focusing on two main aspects: shape and texture. The obtained results show that the proposed method achieves an accurate automatic 3D character reconstruction from only two views that enhances current methods while offering a reproducible work-flow.

When building virtual reality applications teams must choose between different configurations of the hardware and/or software aspects, and other factors, of the experience. In this paper we extend a framework for assessing how these factors contribute to quality of experience in an example evaluation. We consider how four factors related to avatar expressiveness affect quality of experience: Eye Gaze, Eye Blinking, Mouth Animation, and Microexpressions. 55 participants experienced an avatar delivering a presentation in virtual reality. At fixed times participants had the opportunity to spend a virtual budget to modify the factors to incrementally improve their quality of experience. They could stop making transitions when they felt further changes would make no further difference. From these transitions a Markov matrix was built, along with probabilities of a factor being present at a given level on participants’ final configurations. Most participants did not spend the full budget, suggesting that there was a point of equilibrium which did not require maximizing all factor levels. We discuss that point of equilibrium and present this work as an extended contribution to the evaluation of people’s responses to immersive virtual environments.

With the increasing use of avatars (i.e. the virtual representation of the user in a virtual environment) in virtual reality, it is important to identify the factors eliciting the sense of embodiment or the factors that can disrupt this feeling. This paper reports an exploratory study aiming at identifying internal factors (personality traits and body awareness) that might cause either a resistance or a predisposition to feel a sense of embodiment towards a virtual avatar. To this purpose, we conducted an experiment (n=123) in which participants were immersed in a virtual environment and embodied in a gender-matched generic virtual avatar through a head-mounted display. After an exposure phase in which they had to perform a number of visuomotor tasks (during 2 minutes) a virtual character entered the virtual scene and stabbed the participants' virtual hand with a knife. The participants' sense of embodiment was measured, as well as several personality traits (Big Five traits and locus of control) and body awareness, to evaluate the influence of participants' personality on the acceptance of the virtual body. The major finding of the experiment is that the locus of control is linked to several components of embodiment: the sense of agency is positively correlated with an internal locus of control and the sense of body ownership is positively correlated with an external locus of control. Interestingly, both components are not influenced by the same traits, which confirms that they can appear independently. Taken together our results suggest that the locus of control could be a good predictor of the sense of embodiment when the user embodies an avatar with a similar physical appearance. Yet, further studies are required to confirm these results.

We describe an experiment that explores the contribution of auditory and other features to the illusion of plausibility in a virtual environment that depicts the performance of a string quartet. 'Plausibility' refers to the component of presence that is the illusion that the perceived events in the virtual environment are really happening. The features studied were: Gaze (the musicians ignored the participant, the musicians sometimes looked towards and followed the participant's movements), Sound Spatialization (Mono, Stereo, Spatial), Auralization (no sound reflections, reflections corresponding to a room larger than the one perceived, reflections that exactly matched the virtual room), and Environment (no sound from outside of the room, birdsong and wind corresponding to the outside scene). We adopted the methodology based on color matching theory, where 20 participants were first able to assess their feeling of plausibility in the environment with each of the four features at their highest setting. Then five times participants started from a low setting on all features and were able to make transitions from one system configuration to another until they matched their original feeling of plausibility. From these transitions a Markov transition matrix was constructed, and also probabilities of a match conditional on feature configuration. The results show that Environment and Gaze were individually the most important factors influencing the level of plausibility. The highest probability transitions were to improve Environment and Gaze, and then Auralization and Spatialization. We present this work as both a contribution to the methodology of assessing presence without questionnaires, and showing how various aspects of a musical performance can influence plausibility.

We report on the design and results of an experiment investigating factors influencing Slater's Plausibility Illusion (Psi) in virtual environments (VEs). Slater proposed Psi and Place Illusion (PI) as orthogonal components of virtual experience which contribute to realistic response in a VE. PI corresponds to the traditional conception of presence as "being there," so there exists a substantial body of previous research relating to PI, but very little relating to Psi. We developed this experiment to investigate the components of plausibility illusion using subjective matching techniques similar to those used in color science. Twenty-one participants each experienced a scenario with the highest level of coherence (the extent to which a scenario matches user expectations and is internally consistent), then in eight different trials chose transitions from lower-coherence to higher-coherence scenarios with the goal of matching the level of Psi they felt in the highest-coherence scenario. At each transition, participants could change one of the following coherence characteristics: the behavior of the other virtual humans in the environment, the behavior of their own body, the physical behavior of objects, or the appearance of the environment. Participants tended to choose improvements to the virtual body before any other improvements. This indicates that having an accurate and well-behaved representation of oneself in the virtual environment is the most important contributing factor to Psi. This study is the first to our knowledge to focus specifically on coherence factors in virtual environments.

This study presents an evaluation of a mobile game with physiologically aware virtual humans as an approach to modulate the participant's affective and physiological state. We developed a mobile version of a virtual reality scenario where the participants were able to interact with virtual human characters through their psychophysiological activity. Music was played in the background of the scenario and, depending on the experimental condition, the virtual humans were initially either barely dancing or dancing very euphorically. The task of the participants was to encourage the apathetic virtual humans to dance or to calm down the frenetically dancing characters, through the modulation of their own mood and physiological activity. Results from our study show that by using this mobile game with the physiologically aware and affective virtual humans the participants were able to emotionally arouse themselves in the Activation condition and were able to relax themselves in the Relaxation condition, during the same session with only a brief break between conditions. The self-reported affective data was also corroborated by the physiological data (heart rate, respiration and skin conductance) which significantly differed between the Activation and Relaxation conditions.

Is it their movements or appearance that helps us to identify a child as a child? In this work we created four video clips with a Virtual Character walking, but with different combinations of either child or adult animation applied on either a child or adult body. An experimental study was conducted with 53 participants who viewed all four videos in random orders. The results show that participants could easily identify the consistent video clips (child animation with child body, and adult animation with adult body). With the inconsistent video clips, both animation and body shape had an effect on participants' judgments. They also reported higher level of empathy, care, and feeling of protection towards the child character as compared to the adult character. Finally, compared to appearance, animation seems to be playing a bigger role in invoking participants' emotional responses.

Summary form only given. Immersive virtual reality (IVR) has been successfully exploited in the study of body ownership illusions - a topic that contributes to the question of how the human brain represents the body. This is made possible because with a head-tracked wide field of view head-mounted display, and other tracking and stimulation equipment, we can visually substitute a person's real body by a virtual body (VB), that is coincident in space with the real body, moves synchronously with it, and where visuotactile synchronisation is possible with respect to seen events on the VB and the corresponding stimuli felt on the real one. Such a process of virtual embodiment typically gives rise to the perceptual illusion of ownership with respect to the virtual body. Here we explore how IVR may be used to transform the self, providing examples ranging from illusory time travel, psychological problems and illusory agency.

The International Air Transport Association forecasts that there will be at least a 30% increase in passenger demand for flights over the next five years. In these circumstances the aircraft industry is looking for new ways to keep passengers occupied, entertained and healthy, and one of the methods under consideration is immersive virtual reality. It is therefore becoming important to understand how motion sickness and presence in virtual reality are influenced by physical motion. We were specifically interested in the use of head-mounted displays (HMD) while experiencing in-flight motions such as turbulence. 50 people were tested in different virtual environments varying in their context (virtual airplane versus magic carpet ride over tropical islands) and the way the physical motion was incorporated into the virtual world (matching visual and auditory stimuli versus no incorporation). Participants were subjected to three brief periods of turbulent motions realized with a motion simulator. Physiological signals (postural stability, heart rate and skin conductance) as well as subjective experiences (sickness and presence questionnaires) were measured. None of our participants experienced severe motion sickness during the experiment and although there were only small differences between conditions we found indications that it is beneficial for both wellbeing and presence to choose a virtual environment in which turbulent motions could be plausible and perceived as part of the scenario. Therefore we can conclude that brief exposure to turbulent motions does not get participants sick.

We easily adapt to changes in the environment that involve cross-sensory discrepancies (e.g., between vision and proprioception). Adaptation can lead to changes in motor commands so that the experienced sensory consequences are appropriate for the new environment (e.g., we program a movement differently while wearing prisms that shift our visual space). In addition to these motor changes, perceptual judgments of space can also be altered (e.g., how far can I reach with my arm?). However, in previous studies that assessed perceptual judgments of space after visuomotor adaptation, the manipulation was always a planar spatial shift, whereas changes in body perception could not directly be assessed. In this study, we investigated the effects of velocity-dependent (spatiotemporal) and spatial scaling distortions of arm movements on space and body perception, taking advantage of immersive virtual reality. Exploiting the perceptual illusion of embodiment in an entire virtual body, we endowed subjects with new spatiotemporal or spatial 3D mappings between motor commands and their sensory consequences. The results imply that spatiotemporal manipulation of 2 and 4 times faster can significantly change participants’ proprioceptive judgments of a virtual object’s size without affecting the perceived body ownership, although it did affect the agency of the movements. Equivalent spatial manipulations of 11 and 22 degrees of angular offset also had a significant effect on the perceived virtual object’s size; however, the mismatched information did not affect either the sense of body ownership or agency. We conclude that adaptation to spatial and spatiotemporal distortion can similarly change our perception of space, although spatiotemporal distortions can more easily be detected.

Our vision is that regardless of future variations in the interior of airplane cabins, we can utilize ever-advancing state-of-the-art virtual and mixed reality technologies with the latest research in neuroscience and psychology to achieve high levels of comfort for passengers. Current surveys on passenger's experience during air travel reveal that they are least satisfied with the amount and effectiveness of their personal space, and their ability to work, sleep or rest. Moreover, considering current trends it is likely that the amount of available space is likely to decrease and therefore the passenger's physical comfort during a flight is likely to worsen significantly. Therefore, the main challenge is to enable the passengers to maintain a high level of comfort and satisfaction while being placed in a restricted physical space.

Summary form only given. It has been shown that it is possible to generate perceptual illusions of ownership in immersive virtual reality (IVR) over a virtual body that visually substitutes a person's real body, independently of appearance differences between the two [1, 2]. However, the psychological, behavioral and attitudinal consequences of such body transformations remain unknown [3]. Thirty six Caucasian people participated in a between-groups experiment where they played a West-African Djembe hand drum accompanying another avatar inside IVR. Participants were represented by plainly shaded white hands in the baseline condition, and either a casually dressed dark-skinned virtual body (CD) or a formal suited light-skinned body (FL) in the experimental conditions. Although the experienced body ownership illusion was strong for both groups, only the CD representation produced significant increases in participants' movement patterns compared to the baseline and compared to the FL representation. Further analysis showed that the observed behavioral changes were a function of the illusion strength and the perceived appropriateness of the virtual body for drumming. These results demonstrate that full body ownership illusions can lead to substantial behavioral and possibly cognitive changes depending on the virtual body appearance, with important implications for learning, education, training, psychotherapy and rehabilitation applications using IVR.

This paper describes the impact of display resolution and luminance on the responses of participants in a behavioral study that used a projection-based Immersive Virtual Reality System. The scenario was a virtual bar where participants witnessed a violent attack of one person on another due to an argument about support for a soccer club. The major response variable was the number of interventions made by participants. The study was between-groups with 10 participants in two groups pre-upgrade and post-upgrade, both in the same 4-screen Cave-like system. However, the post-upgrade group experienced the scenario with projectors that had a significantly higher level of resolution and luminance than those experienced by the pre-upgrade group. The results show that, other things being equal, the number of both verbal and physical interventions was greater amongst those in the post-upgrade group compared to the pre-upgrade group.

We built a Collaborative Virtual Environment (CVE) allowing one person, the 'visitor' to be digitally transported to a remote destination to interact with local people there. This included full body tracking, vibrotactile feedback and voice. This allowed interactions in the same CVE between multiple people situated in different physical remote locations. This system was used for an experiment to study whether the conveyance of touch has an impact on the willingness of participants embodied in the CVE to sing in public.

We describe a system that allows for controlling different robots and avatars from a real time motion stream. The underlying problem is that motion data from tracking systems is usually represented differently to the motion data required to drive an avatar or a robot: there may be different joints, motion may be represented by absolute joint positions and rotations or by a root position, bone lengths and relative rotations in the skeletal hierarchy. Our system resolves these issues by remapping in real time the tracked motion so that the avatar or robot performs motions that are visually close to those of the tracked person. The mapping can also be reconfigured interactively at run-time. We demonstrate the effectiveness of our system by case studies in which a tracked person is embodied as an avatar in immersive virtual reality or as a robot in a remote location. We show this with a variety of tracking systems, humanoid avatars and robots.

We present a novel technique for animating self-avatar eye movements in an immersive virtual environment without the use of eye-tracking hardware, and evaluate our technique via a two-alternative, forced-choice-with-confidence experiment that compares this simulated-eye-tracking condition to a no-eye-tracking condition and a real-eye-tracking condition in which the avatar's eyes were rotated with an eye tracker. Viewing the reflection of a tracked self-avatar is often used in virtual-embodiment scenarios to induce in the participant the illusion that the virtual body of the self-avatar belongs to them, however current tracking methods do not account for the movements of the participants eyes, potentially lessening this body-ownership illusion. The results of our experiment indicate that, although blind to the experimental conditions, participants noticed differences between eye behaviors, and found that the real and simulated conditions represented their behavior better than the no-eye-tracking condition. Additionally, no statistical difference was found when choosing between the real and simulated conditions. These results suggest that adding eye movements to self-avatars produces a subjective increase in self-identification with the avatar due to a more complete representation of the participant's behavior, which may be beneficial for inducing virtual embodiment, and that effective results can be obtained without the need for any specialized eye-tracking hardware.

It has been shown that it is possible to generate perceptual illusions of ownership in immersive virtual reality (IVR) over a virtual body seen from first person perspective, in other words over a body that visually substitutes the person's real body. This can occur even when the virtual body is quite different in appearance from the person's real body. However, investigation of the psychological, behavioral and attitudinal consequences of such body transformations remains an interesting problem with much to be discovered. Thirty six Caucasian people participated in a between-groups experiment where they played a West-African Djembe hand drum while immersed in IVR and with a virtual body that substituted their own. The virtual hand drum was registered with a physical drum. They were alongside a virtual character that played a drum in a supporting, accompanying role. In a baseline condition participants were represented only by plainly shaded white hands, so that they were able merely to play. In the experimental condition they were represented either by a casually dressed dark-skinned virtual body (Casual Dark-Skinned - CD) or by a formal suited light-skinned body (Formal Light-Skinned - FL). Although participants of both groups experienced a strong body ownership illusion towards the virtual body, only those with the CD representation showed significant increases in their movement patterns for drumming compared to the baseline condition and compared with those embodied in the FL body. Moreover, the stronger the illusion of body ownership in the CD condition, the greater this behavioral change. A path analysis showed that the observed behavioral changes were a function of the strength of the illusion of body ownership towards the virtual body and its perceived appropriateness for the drumming task. These results demonstrate that full body ownership illusions can lead to substantial behavioral and possibly cognitive changes depending on the appearance of the virtual body. This could be important for many applications such as learning, education, training, psychotherapy and rehabilitation using IVR.

This paper explores body ownership and control of an 'extended' humanoid avatar that features a distinct and flexible tail-like appendage protruding from its coccyx. Thirty-two participants took part in a between-groups study to puppeteer the avatar in an immersive CAVE™ -like system. Participants' body movement was tracked, and the avatar's humanoid body synchronously reflected this motion. However, sixteen participants experienced the avatar's tail moving around randomly and asynchronous to their own movement, while the other participants experienced a tail that they could, potentially, control accurately and synchronously through hip movement. Participants in the synchronous condition experienced a higher degree of body ownership and agency, suggesting that visuomotor synchrony enhanced the probability of ownership over the avatar body despite of its extra-human form. Participants experiencing body ownership were also more likely to be more anxious and attempt to avoid virtual threats to the tail and body. The higher task performance of participants in the synchronous condition indicates that people are able to quickly learn how to remap normal degrees of bodily freedom in order to control virtual bodies that differ from the humanoid form. We discuss the implications and applications of extended humanoid avatars as a method for exploring the plasticity of the brain's representation of the body and for gestural human-computer interfaces.

A reinforcement learning (RL) method was used to train a virtual character to move participants to a specified location. The virtual environment depicted an alleyway displayed through a wide field-of-view head-tracked stereo head-mounted display. Based on proxemics theory, we predicted that when the character approached within a personal or intimate distance to the participants, they would be inclined to move backwards out of the way. We carried out a between-groups experiment with 30 female participants, with 10 assigned arbitrarily to each of the following three groups: In the Intimate condition the character could approach within 0.38m and in the Social condition no nearer than 1.2m. In the Random condition the actions of the virtual character were chosen randomly from among the same set as in the RL method, and the virtual character could approach within 0.38m. The experiment continued in each case until the participant either reached the target or 7 minutes had elapsed. The distributions of the times taken to reach the target showed significant differences between the three groups, with 9 out of 10 in the Intimate condition reaching the target significantly faster than the 6 out of 10 who reached the target in the Social condition. Only 1 out of 10 in the Random condition reached the target. The experiment is an example of applied presence theory: we rely on the many findings that people tend to respond realistically in immersive virtual environments, and use this to get people to achieve a task of which they had been unaware. This method opens up the door for many such applications where the virtual environment adapts to the responses of the human participants with the aim of achieving particular goals.

To what extent do people behave in immersive virtual environments as they would in similar situations in a physical environment? There are many ways to address this question, ranging from questionnaires, behavioral studies, and the use of physiological measures. Here, we compare the onsets of muscle activity using surface electromyography (EMG) while participants were walking under three different conditions: on a normal floor surface, on a narrow ribbon along the floor, and on a narrow platform raised off the floor. The same situation was rendered in an immersive virtual environment (IVE) Cave-like system, and 12 participants did the three types of walking in a counter-balanced within-groups design. The mean number of EMG activity onsets per unit time followed the same pattern in the virtual environment as in the physical environment-significantly higher for walking on the platform compared to walking on the floor. Even though participants knew that they were in fact really walking at floor level in the virtual environment condition, the visual illusion of walking on a raised platform was sufficient to influence their behavior in a measurable way. This opens up the door for this technique to be used in gait and posture related scenarios including rehabilitation.

This paper reports an experiment that investigated people's body ownership of an avatar that was observed in a virtual mirror. Twenty subjects were recruited in a within-groups study where 10 first experienced a virtual character that synchronously reflected their upper-body movements as seen in a virtual mirror, and then an asynchronous condition where the mirror avatar displayed prerecorded actions, unrelated to those of the participant. The other 10 subjects experienced the conditions in the opposite order. In both conditions the participant could carry out actions that led to elevation above ground level, as seen from their first person perspective and correspondingly in the mirror. A rotating virtual fan eventually descended to 2 m above the ground. The hypothesis was that synchronous mirror reflection would result in higher subjective sense of ownership. A questionnaire analysis showed that the body ownership illusion was significantly greater for the synchronous than asynchronous condition. Additionally participants in the synchronous condition avoided collision with the descending fan significantly more often than those in the asynchronous condition. The results of this experiment are put into context within similar experiments on multisensory correlation and body ownership within cognitive neuroscience.

We describe a system that shows how to substitute a person's body in virtual reality by a virtual body (or avatar). The avatar is seen from a first person perspective, moves as the person moves and the system generates touch on the real person's body when the avatar is touched. Such replacement of the person's real body by a virtual body requires a wide field-of-view head-mounted display, real-time whole body tracking, and tactile feedback. We show how to achieve this with a variety of off-the-shelf hardware and software, and also custom systems for real-time avatar rendering and collision detection. We present an overview of the system and detail on some of its components. We provide examples of how such a system is being used in some of our current experimental studies of embodiment.

We have integrated the Graz brain–computer interface (BCI) system with a highly immersive virtual reality (VR) Cave-like system. This setting allows for a new type of experience, whereby participants can control a virtual world using imagery of movement. However, current BCI systems still have many limitations. In this article we present two experiments exploring the different constraints posed by current BCI systems when used in VR. In the first experiment we let the participants make free choices during the experience and compare their BCI performance with participants using BCI without free choice; this is unlike most previous work in this area, in which participants are requested to obey cues. In the second experiment we allowed participants to control a virtual body with motor imagery. We provide both quantitative and subjective results, regarding both BCI accuracy and the nature of the subjective experience in this new type of setting.

An experiment was carried out to examine the impact on electrodermal activity of people when approached by groups of one or four virtual characters at varying distances. It was premised on the basis of proxemics theory that the closer the approach of the virtual characters to the participant, the greater the level of physiological arousal. Physiological arousal was measured by the number of skin conductance responses within a short time period after the approach, and the maximum change in skin conductance level 5 s after the approach. The virtual characters were each either female or a cylinder of human size, and one or four characters approached each subject a total of 12 times. Twelve male subjects were recruited for the experiment. The results suggest that the number of skin conductance responses after the approach and the change in skin conductance level increased the closer the virtual characters approached toward the participants. Moreover, these response variables were inversely correlated with the number of visits, showing a typical adaptation effect. There was some evidence to suggest that the number of characters who simultaneously approached (one or four) was positively associated with the responses. Surprisingly there was no evidence of a difference in response between the humanoid characters and cylinders on the basis of this physiological data. It is suggested that the similarity in this quantitative arousal response to virtual characters and virtual objects might mask a profound difference in qualitative response, an interpretation supported by questionnaire and interview results. Overall the experiment supported the premise that people exhibit heightened physiological arousal the closer they are approached by virtual characters.

A new definition of immersion with respect to virtual environment (VE) systems has been proposed in earlier work, based on the concept of simulation. One system ( A ) is said to be more immersive than another ( B ) if A can be used to simulate an application as if it were running on B. Here we show how this concept can be used as the basis for a psychophysics of presence in VEs, the sensation of being in the place depicted by the virtual environment displays (Place Illusion, PI), and also the illusion that events occurring in the virtual environment are real (Plausibility Illusion, Psi). The new methodology involves matching experiments akin to those in color science. Twenty participants first experienced PI or Psi in the initial highest level immersive system, and then in 5 different trials chose transitions from lower to higher order systems and declared a match whenever they felt the same level of PI or Psi as they had in the initial system. In each transition they could change the type of illumination model used, or the field-of-view, or the display type (powerwall or HMD) or the extent of self-representation by an avatar. The results showed that the 10 participants instructed to choose transitions to attain a level of PI corresponding to that in the initial system tended to first choose a wide field-of-view and head-mounted display, and then ensure that they had a virtual body that moved as they did. The other 10 in the Psi group concentrated far more on achieving a higher level of illumination realism, although having a virtual body representation was important for both groups. This methodology is offered as a way forward in the evaluation of the responses of people to immersive virtual environments, a unified theory and methodology for psychophysical measurement.

Abstract Humans use their bodies in a highly expressive way during conversation, and animated characters that lack this form of non‐verbal expression can seem stiff and unemotional. An important aspect of non‐verbal expression is that people respond to each other's behavior and are highly attuned to picking up this type of response. This is particularly important for the feedback given while listening to some one speak. However, automatically generating this type of behavior is difficult as it is highly complex and subtle. This paper takes a data driven approach to generating interactive social behavior. Listening behavior is motion captured, together with the audio being listened to. These data are used to learn an animation model of the responses of one person to the other. This allows us to create characters that respond in real‐time during a conversation with a real human. Copyright ? 2008 John Wiley & Sons, Ltd.

We present a participant study that compares biological exploration tasks using volume renderings of laser confocal microscopy data across three environments which vary in level of immersion. For the tasks, data, and visualization approach used in our study, we found that subjects qualitatively preferred and quantitatively performed better in environments with greater levels of immersion. Subjects performed real-world biological data analysis tasks that emphasized understanding spatial relationships including characterizing the general features in a volume, identifying co-located features, and reporting geometric relationships such as whether clusters of cells were coplanar. After analyzing data in each environment, subjects were asked to choose which environment they wanted to analyze additional data sets in--subjects uniformly selected the Cave environment.

This paper presents the quantitative and qualitative findings from an experiment designed to evaluate a developing model of affective postures for full-body virtual characters in immersive virtual environments (IVEs). Forty-nine participants were each requested to explore a virtual environment by asking two virtual characters for instructions. The participants used a CAVE-like system to explore the environment. Participant responses and their impression of the virtual characters were evaluated through a wide variety of both quantitative and qualitative methods. Combining a controlled experimental approach with various data-collection methods provided a number of advantages such as providing a reason to the quantitative results. The quantitative results indicate that posture plays an important role in the communication of affect by virtual characters. The qualitative findings indicated that participants attribute a variety of psychological states to the behavioral cues displayed by virtual characters. In addition, participants tended to interpret the social context portrayed by the virtual characters in a holistic manner. This suggests that one aspect of the virtual scene colors the perception of the whole social context portrayed by the virtual characters. We conclude by discussing the importance of designing holistically congruent virtual characters especially in immersive settings.

Global illumination in VR applications remains an elusive goal. While it potentially has a positive impact on presence, the significant real-time computation and integration complexities involved have been stumbling blocks. In this paper we present recent and ongoing work in the Virtual Light Field paradigm for global illumination as a solution to this problem. We discuss its suitability for real-time VR applications and detail recent work in integrating it with the XVR system for real-time GPU-based rendering in a CAVE™. This rendering method achieves real-time rendering of L(S|D)* E solutions in time independent of illumination complexity and largely independent of geometric complexity.

This paper presents a study of the fine grain physiological responses of participants to an immersive virtual simulation of an urban environment. The analysis of differences in participant responses at various stages of the experiment (baseline recordings, training, first half and second half of the urban simulation) are examined in detail. It was found that participants typically show a stress response during the training phase and a stress response towards the end of the simulation of the urban experience.There is also some evidence that variations in the level of visual realism based the texture strategy used was associated with changes in mental stress.

This paper describes an experiment that examines the influence of visual realism on reported presence. 33 participants experienced two different renderings of a virtual environment that depicts a pit in the centre of a room, in a head-tracked head-mounted display. The environment was rendered using parallel ray tracing at 15fps,mbut in one condition ray casting (RC) was used achieving a result equivalent to OpenGL based per-pixel local illumination, and in the second full recursive ray tracing (RT). The participants were randomly allocated to two groups -- one that experienced RC first followed by RT, and the second group in the opposite order. Reported presence was obtained by questionnaires following each session. The results indicate that reported presence, in terms of the 'sense of being there' was significantly higher for the RT than for the RC condition.

This paper describes an algorithm that provides real-time walkthrough for globally illuminated scenes that contain mixtures of ideal diffuse and specular surfaces. A type of light field data structure is used for propagating radiance outward from light emitters through the scene, accounting for any kind of L(S\D)* light path. The light field employed is constructed by choosing a regular point subdivision over a hemisphere, to give a set of directions, and then corresponding to each direction there is a rectangular grid of parallel rays. Each rectangular grid of rays is further subdivided into rectangular tiles, such that each tile references a sequence of 2D images containing colour values corresponding to the outgoing radiances of surfaces intersected by the rays in that tile. This structure is then used for final image rendering. Propagation times can be very long and the memory requirements very high. This algorithm, however, offers a global illumination solution for real-time walkthrough even on a single processor

Abstract This paper describes a behavioural model used to simulate realistic eye‐gaze behaviour and body animations for avatars representing participants in a shared immersive virtual environment (IVE). The model was used in a study designed to explore the impact of avatar realism on the perceived quality of communication within a negotiation scenario. Our eye‐gaze model was based on data and studies carried out on the behaviour of eye‐gaze during face‐to‐face communication. The technical features of the model are reported here. Information about the motivation behind the study, experimental procedures and a full analysis of the results obtained are given in [ 17 ].

This paper presents an experiment designed to investigate the impact of visual and behavioral realism in avatars on perceived quality of communication in an immersive virtual environment. Participants were paired by gender and were randomly assigned to a CAVE‘-like system or a head-mounted display. Both were represented by a humanoid avatar in the shared 3D environment. The visual appearance of the avatars was either basic and genderless (like a "match-stick" figure), or more photorealistic and gender-specific. Similarly, eye gaze behavior was either random or inferred from voice, to reflect different levels of behavioral realism. Our comparative analysis of 48 post-experiment questionnaires confirms earlier findings from non-immersive studies using semi-photorealistic avatars, where inferred gaze significantly outperformed random gaze. However responses to the lower-realism avatar are adversely affected by inferred gaze, revealing a significant interaction effect between appearance and behavior. We discuss the importance of aligning visual and behavioral realism for increased avatar effectiveness.

The Internet, as an accessible network with global connectivity, presents itself as the likely solution for the supporting infrastructure of cyberspace. In contrast, the field of virtual environments (VE) is fragmented with a wide proliferation of different systems, as a result of the current development trend. However, these divergent systems present a significant overlap of functionality that represents a waste of development resources and a detriment to innovation. This paper proposes Mayhem as a new approach to the design and development of VE systems. Our approach relies heavily on component design methodology coupled with a simple layered approach to system architectures. The combination of the two promotes the construction of systems from smaller building blocks, which may have been developed by different sources, each specializing in the supported functionality. Although Mayhem promotes the trend of middleware, it requires a common base infrastructure - the Java Adaptive Dynamic Environment (JADE). To demonstrate how to build a VE system using Mayhem, the paper describes a simple prototype of a VE and some of its key building blocks.

This paper demonstrates a technique for visualising multidimensional data sets holistically. The EVA algorithm provides an automatic mapping from semantically important features of the data to emotionally or perceptually significant features of the corresponding visual structure (such as a face). In other words a single glance at the visual structure informs the observer of the global state of the data, since the visual structure has an emotional impact on the observer The visualisation is designed to correspond to the impact that would have been generated had the observer been able to access the relevant semantics of the underlying data.

In this paper we describe an experiment designed to investigate the importance of eye gaze in humanoid avatar's representing people engaged in conversation. We compare responses to dyadic conversations in four mediated conditions: video, audio-only, and two avatar conditions. The avatar conditions differed only in their treatment of eye gaze. In the random-gaze condition the avatars head and eye animations were unrelated to conversational flow. In the informed-gaze condition, they were related to turn-taking during the conversation. The head animations were tracked and the eye animations were inferred from the audio stream. Our comparative analysis of 100 post-experiment questionnaires showed that the random-gaze avatar did not improve on audio-only communication. The informed-gaze avatar significantly outperformed the random-gaze model and also outperformed audio-only on several response measures. We conclude that an avatar whose gaze behaviour is related to the conversation provides a marked improvement on an avatar that merely exhibits liveliness.

This paper presents an approach to visibility called the Viewpoint Movement Space (VpMS) algorithm which supports the concept of dynamic polygon visibility orderings for head‐slaved viewing in virtual environments (VE). The central idea of the approach is that the visibility, in terms of back‐to‐front polygon visibility ordering, does not change dramatically as the viewpoint moves. Moreover, it is possible to construct a partition of the space into cells, where for each cell the ordering is invariant. As the viewpoint moves across a cell boundary typically only a small and predictable change is made to the visibility ordering. The cost to perform this operation represents a notable reduction when compared with the cost of resolving the visibility information from the BSP tree where the classification of the viewpoint with every node plane has to be performed. The paper demonstrates how the subdivision into such cells can represent the basic source for an acceleration of the rendering process. We also discuss how the same supportive data structure can be exploited to solve other tasks in the graphics pipeline.

Investigating virtual environments has become an increasingly interesting research topic for engineers, computer and cognitive scientists, and psychologists. Although there have been several recent studies focused on the development of multimodal virtual environments (VEs) to study human-machine interactions, less attention has been paid to human-human and human-machine interactions in shared virtual environments (SVEs), and to our knowledge, no attention paid at all to what extent the addition of haptic communication between people would contribute to the shared experience. We have developed a multimodal shared virtual environment and performed a set of experiments with human subjects to study the role of haptic feedback in collaborative tasks and whether haptic communication through force feedback can facilitate a sense of being and collaborating with a remote partner. The study concerns a scenario where two participants at remote sites must cooperate to perform a joint task in an SVE. The goals of the study are (1) to assess the impact of force feedback on task performance, (2) to better understand the role of haptic communication in human-human interactions, (3) to study the impact of touch on the subjective sense of collaborating with a human as reported by the participants based on what they could see and feel, and (4) to investigate if gender, personality, or emotional experiences of users can affect haptic communication in SVEs. The outcomes of this research can have a powerful impact on the development of next-generation human-computer interfaces and network protocols that integrate touch and force feedback technology into the internet, development of protocols and techniques for collaborative teleoperation such as hazardous material removal, space station.

A study by Slater, et al., [1995] indicated that naive subjects in an immersive virtual environment experience a higher subjective sense of presence when they locomote by walking-in-place (virtual walking) than when they push-button-fly (along the floor plane).We replicated their study, adding real walking as a third condition.Our study confirmed their findings.We also found that real walking is significantly better than both virtual walking and flying in ease (simplicity, straightforwardness, naturalness) as a mode of locomotion.The greatest difference in subjective presence was between flyers and both kinds of walkers.In addition, subjective presence was higher for real walkers than virtual walkers, but the difference was statistically significant only in some models.Follow-on studies show virtual walking can be substantially improved by detecting footfalls with a head accelerometer.As in the Slater study, subjective presence significantly correlated with subjects' degree of association with their virtual bodies (avatars).This, our strongest statistical result, suggests that substantial potential presence gains can be had from tracking all limbs and customizing avatar appearance.An unexpected by-product was that real walking through our enhanced version of Slater's visual-cliff virtual environment (Figure 1) yielded a strikingly compelling virtual experience-the strongest we and most of our visitors have yet experienced.The most needed system improvement is the substitution of wireless technology for all links to the user.

We present an experiment that investigates the behaviour of small groups of participants in a wide-area distributed collaborative virtual environment (CVE). This is the third and largest study in a series of experiments that have examined trios of participants carrying out a highly collaborative puzzle-solving task. The results reproducing those of earlier studies suggest a positive relationship between place-presence and co-presence, between co-presence and group accord, with evidence supporting the notion that immersion confers leadership advantage.

In virtual reality simulations the speed of rendering is vitally important. One of the techniques for controlling the frame rate is the assignment of different levels of detail for each object within a scene. The most well-known level of detail assignment algorithms are the Funkhouser[1] algorithm and the algorithm where the level of detail is assigned with respect to the distance of the object from the viewer.

The majority of existing systems supporting Large Scale Virtual Environments (LSVE) are based on monolithic architectures, making maintenance, reusability and extensibility difficult at best. An overview of the Java Adaptive Dynamic Environment (JADE) is presented as an alternative to the traditional approach for developing a core infrastructure for VE systems. JADE consists of a light-weight cross-platform kernel with inherent capabilities for dynamic extensibility in run-time. Although the design principles are applicable to other areas, the implementation is targeted at LSVE. The JADE implementation is 100% Java, taking full advantage of the following properties: wide deployment across major operating systems and browsers; comprehensive set of packages; networking capabilities and security mechanisms.

This paper introduces a new text input device called the chording glove. The keys of a chord keyboard are mounted on the fingers of a glove. A chord can be made by pressing the fingers against any surface. Shift buttons placed on the index finger enable the glove to enter the full ASCII character set. The chording glove is designed as a text input device for wearable computers and virtual environments. An experiment was conducted to assess the performance of the glove. After an average of 80 min of a tutorial, ten subjects reached a continuous text input speed of 8.9/spl plusmn/1.4 words/min, and after 10 1-hr sessions, they achieved 16.8/spl plusmn/2.5 words/min.

All in-text\treferences\tunderlined\tin\tblue\tare\tlinked\tto\tpublications\ton\tResearchGate, letting you\taccess\tand\tread\tthem\timmediately.

This paper describes an experiment to assess the influence of immersion on performance in immersive virtual environments. The task involved Tri-Dimensional Chess, and required subjects to reproduce on a real chess board the state of board learned from a sequence of moves witnessed in a virtual environment. Twenty four subjects were allocated to a factorial design consisting of two levels of immersion (exocentric screen based, and egocentric HMD based), and two kinds of environment (plain and realistic). The results suggest that egocentric subjects performed better than exocentric, and those in the more realistic environment performed better than those in the less realistic environment. Previous knowledge of chess, and amount of virtual practice were also significant, and may be considered as control variables to equalise these factors amongst the subjects. Other things being equal, males remembered the moves better than females, although female performance improved with higher spatial ability test score. The paper also attempts to clarify the relationship between immersion, presence and performance, and locates the experiment within such a theoretical framework.

This paper describes the creation of a Western shootout animation for a BBC2 television production. This provides a case study describing the interaction between the TV production team and the animators. The animation was produced by explicitly programming transformations applied to nodes in hierarchical models of simple human characters. A subsequent project has attempted to provide character animation making use of the human body tracking and immersion afforded by virtual reality systems. In the new system a human actor can enter into a virtual environment (VE), act out a role, resulting in a file containing a script. While the script is replayed the actor can enter again into the VE, and interact with the first virtual character, and so on for subsequent replays. The paper describes a first version of this system, and provides a critical examination of the use of VR for creating animations.>

This paper presents an algorithm for shadow calculation in dynamic polyhedral scenes illuminated by point light sources. It is based on a modification of Shadow Volume Binary Space Partition trees, to allow these to be constructed from the original scene polygons in arbitrary order and to support for fast reconstruction after a change in scene geometry. Timings using sample scenes are presented that indicate substantial savings both in terms of computation time and shadows produced.

Abstract Many applications now demand interaction with visualizations of 3D scenes and data sets. Current flat 2D displays are limited in their capacity to provide this not only by the display technology but the interaction metaphors and devices used. The Desktop Bat is a device that has 5 degrees of freedom whilst retaining the simplicity of use o fa mouse. To use it for general 3D interaction several metaphors were created for the tasks of navigation and cursor manipulation and a set of experiments were conducted to determine which metaphors were the most efficient in use. Of these metaphors, a velocity control metaphor was the best for navigation and a metaphor that applied rotations and translations relative to the eyepoint coordinate system was best for object control.

This article presents an interactive technique for moving through an immersive virtual environment (or “virtual reality”). The technique is suitable for applications where locomotion is restricted to ground level. The technique is derived from the idea that presence in virtual environments may be enhanced the stronger the match between proprioceptive information from human body movements and sensory feedback from the computer-generated displays. The technique is an attempt to simulate body movements associated with walking. The participant “walks in place” to move through the virtual environment across distances greater than the physical limitations imposed by the electromagnetic tracking devices. A neural network is used to analyze the stream of coordinates from the head-mounted display, to determine whether or not the participant is walking on the spot. Whenever it determines the walking behavior, the participant is moved through virtual space in the direction of his or her gaze. We discuss two experimental studies to assess the impact on presence of this method in comparison to the usual hand-pointing method of navigation in virtual reality. The studies suggest that subjective rating of presence is enhanced by the walking method provided that participants associate subjectively with the virtual body provided in the environment. An application of the technique to climbing steps and ladders is also presented.

Abstract This paper describes a multi‐processor implementation of form factor computation in the radiosity method. Form factors are computed using the ray casting method and the algorithm is enhanced with Binary Space Partition (BSP) Trees and bounding boxes. Experimental results are presented together with a discussion of load balance efficiency.

Abstract This paper investigates a new method for dynamically changing Binary Space Partition (BSP) trees. A BSP tree representation of a 3D polygonal scene provides an ideal data structure for rapidly performing the hidden surface computations involved in changing the viewpoint. However, BSP trees have generally been thought to be unsuitable for applications where the geometry of objects in the scene changes dynamically. The purpose of this paper is to introduce a dynamic BSP tree algorithm which does allow for such changes, and which maintains the simplicity and integrity of the BSP tree representation. The algorithm is extended to include dynamic changes to shadows. We calibrate the algorithms by transforming a range of objects in a scene, and reporting on the observed timing results.

Abstract This paper presents a new methodfor providing interactive shape control of interpolating B‐splines. The CAD designer can directly interact with geometric entities defined on the B‐spline at any interpolated data point; shape adjustments can be performed either globally or locally. Our approach is based on Bλ‐splines of order k (λ,k ≥1), i.e. λ‐reparametrized, classical B‐splines. The method presented can be easily generalised to surfaces defined either as tensor products or by using the skinning technique; interactive shape control can be provided in both surface parametric directions.

Abstract This paper compares two image space hidden surface removal algorithms for polygonal scenes. These are the z‐buffer and scan‐line algorithms. There is first an overview of each algorithm, followed by a simulation experiment, designed to compare the number of polygons per second which can be rendered by each algorithm. The simulation varies the number of polygons in the scene, and the size and distribution of polygons over the display. The results suggest that the z‐buffer is preferred for a large enough number of polygons, however, smaller polygons and uniform distribution of polygons favour the scan‐line approach. The analysis does not take into account the complexity of the shading rule, which is likely also to favour the scan‐line method.

A novel device for 3D interaction is introduced, the Desktop Bat. This device is an evolutionary development of a mouse. It is like a mouse in that it is used on a desktop, but provides 5 degrees of freedom since in conjunction with translations it can be used to effect rotations about any combination of the principal axes. We discuss the requirements for 3D interaction which led to the design of the Desktop Bat, and the interaction models and software which allow it to be used as a 6 degrees of freedom input device.

Abstract This paper describes a library for the creation and manipulation of geometrically defined graphical objects in The X Window System Objects are arranged in a 2.5D space, and can be transformed, made invisible or visible, or have their priority changed. Transformation of objects occurs immediately and with full damage repair, thus providing an essential requirement for the support of interactive graphics. The paper reports on an experimental implementation of the library and also discusses the implications of graphical objects in the X environment.

When graphics objects (also called segments) are used on raster display hardware, problems arise because of the mismatch between the high level requirements of object manipulation, and the low level representation from which the image is refreshed. This paper describes a tiling algorithm which provides a solution to the problems of damage repair and hit detection. In particular methods for handling object priority are discussed and experimental results showing the performance of the algorithm for a number of cases are presented.

Abstract Traditional interactive drawing programs adopt a bottom‐up approach, allowing the user to construct a picture by the use of discrete tools, for example, lines, circles, rectangles, and so on. This paper presents a different approach, which allows users to construct graphical objects by stretching and cutting existing objects. The representation is simply implemented, based on a ring of cubic Bezier curves, and use of the de Casteljau algorithm.

Abstract The problems associated with using picture segments in interactive applications on workstations with bit‐mapped displays are discussed. A method for displaying and updating segments is introduced.

Abstract The design of the Pascal interface to the Graphics Kernel System is discussed and illustrated. The principles underlying the binding are given and examples show how some of the major features of GKS will look in Pascal. The examples cover graphical output, setting primitive representation, graphical input and inquiries.

Abstract This paper describes a program which provides an interactive command driven interface to the Graphical Kernel System (GKS).