Immersion in virtual reality is a powerful sensory experience. It often leads to striking situations, such as experiencing virtual contacts on your avatar, despite the absence of physical contact on your real body. However, as of today, it is still unknown whether purely-virtual touch remains fictive/imaginative, or whether our brain processes it the same way as real touch. In this paper, we conducted an experiment comparing purely visual touch simulation (a virtual robot arm touching the avatar’s hand) with a congruent visuo-tactile simulation (visual feedback is coupled with a real robot arm touching the participant’s hand in the physical world). Using this setup, we investigated the neural responses to purely-virtual and visuo-tactile touch, and examined how these responses were modulated by cognitive workload induced by tasks within the virtual environment. The analysis of the electroencephalography (EEG recording) revealed a strong activation of the sensorimotor cortex in both conditions, although stronger and faster when physical touch occurs. Our results also revealed a decrease in evoked potentials under high cognitive workload, with the reduction being more pronounced for physical touch. Our results show that a purely-virtual touch is processed similarly to a physical touch, i.e, engaging sensorimotor regions despite the absence of direct somatosensory stimulation. This suggests that the perception of the body in virtual environments can extend beyond simple illusions, involving neural processes similar to those engaged during real physical experiences. Besides, in the age of metaverses becoming accessible to a wider audience, knowing that our brain perceives virtual contacts in VR similarly to physical contacts further underlines the need to design immersive environments that prioritize physical integrity and user safety.

Social Virtual Reality (SVR) enables users to embody avatars and interact with others in shared virtual environments. On platforms such as Rec Room, VRChat, and Meta Horizon Worlds, inappropriate social behaviours are common and can lead to harmful emotional experiences for users. To prevent or mitigate these negative effects, SVR platforms offer a range of actions, referred to as safety tools, that allow users to control the information and interactions they are exposed to. Although prior research has analysed and proposed initial classifications of these tools, a structured, theory-driven approach to their characterization is still lacking. In this paper, we propose a taxonomy of safety tools for SVR platforms inspired by existing literature in social psychology and human-computer interaction. In order to characterise these tools, we focus on the area of social interaction, which occurs between a sender and a receiver, within a social context of the SVR medium. Our approach not only provides a model of social interaction in SVR, but also describes how safety tools function, including the processes of selecting and manipulating them, factors that are critical to their effectiveness. Finally, we apply our framework to characterise existing safety tools, identify their limitations, and present design guidelines for future SVR platforms.

Virtual reality is becoming increasingly popular, and modern haptic equipment, such as vibrotactile suits, haptic gloves, and force-feedback controllers, offers new means of interaction within virtual environments, significantly enhancing user experience. When interacting with virtual objects, combined visual and haptic feedback simulates the physical sensations of grasping, lifting, or moving real objects. This sensorimotor feedback is essential for inducing a sense of presence and agency, yet it remains challenging to reproduce in the absence of reliable haptic cues. In this study, we design and evaluate several haptic metaphors using combinations of vibrotactile design parameters to simulate the lifting effort associated with light to heavy objects. These parameters include primitive signals, intensity, spatial density, propagation, and temporal density. Our contribution is threefold. First, we propose a method for modulating perceived physical effort by extending signal intensity with spatial and temporal density, which together reflect the effort required to lift an object. Second, we present a user study in which participants compared haptic effects and ranked them according to perceived lifting effort, comfort, and confidence, allowing us to assess the influence of each parameter. Third, we report the results of a second study in which participants evaluated vibrotactile effects when lifting different virtual objects. The findings confirm the importance of intensity and spatial density, as well as the influence of graphical representation on perceived effort. This research provides practical insights for designing haptic-enabled virtual reality systems and offers guidance for developers seeking to create more expressive and believable vibrotactile interactions.

Pseudo-haptics techniques are interesting alternatives for inducing haptic perceptions, achieved by manipulating haptic perception through the appropriate alteration of primarily visual feedback in response to body movements. However, the use of pseudo-haptics techniques with a motion-input system can sometimes be limited. This paper investigates a novel approach for extending the potential of pseudo-haptics techniques in virtual reality (VR), focusing on pseudoweight perception as the target case. The proposed approach utilizes a reaction force from force-input as a substitution of haptic cue for the pseudo-haptic perception. The paper introduced a manipulation method in which the vertical acceleration of the virtual hand is controlled by the extent of push-in of a force sensor. Such a force-input manipulation of a virtual body cannot only present pseudo-haptics with smaller physical spaces and be used by various users including physically handicapped people, but can present the reaction force proportional to the user's input to the user. We hypothesized that such a haptic force cue would contribute to the pseudo-haptic, here, the pseudoweight perception. Therefore, the paper endeavors to investigate the force-input pseudo-haptic perception in comparison with the motion-input pseudo-haptics. The paper compared force-input and motion-input manipulation in a point of achievable range and resolution of pseudo-weight. The experimental results suggest that forceinput manipulation successfully extends the range of perceptible pseudo-weight by 80% in comparison to the motion-input manipulation. On the other hand, it is revealed that motion-input manipulation has 1 step larger number of distinguishable weight levels and is easier to operate.

Does seeing avatars during social interaction in virtual reality (VR) affect the neural synchrony of interacting people? To address this question, we assessed inter-brain synchronization (IBS), which reflects synchronization of brain activities among interacting people and is linked to interaction success. We designed a social interaction task in VR where participants' appearances were represented by virtual cursors only or additional human avatars on them. Participants performed a finger position tracking interaction under both conditions. We compared the IBS across interaction conditions by collecting electroencephalography (EEG), positional data, and virtual embodiment questionnaires. The results from 30 participants (15 pairs) show that seeing the avatars resulted in significantly higher virtual body ownership but comparable agency and positional synchronization compared to the virtual cursors-only condition. In the inter-brain analysis, we found a significant increase in IBS strength and an increasing trend in the number of IBS in the alpha band after interacting with avatar-embodied virtual cursors. Correlation analyses further revealed a significant association between alpha-band IBS and subjective body ownership, and a trend with social inclusion scores. While the influence of visual complexity cannot be ruled out, these findings suggest that avatar-induced embodiment may contribute to enhanced inter-brain synchrony during VR-mediated social interaction.

Pseudo-haptics alters vision to evoke haptic perception without dedicated hardware. We introduce a method that reproduces sticky-surface interactions during attaching-detaching 3D direct manipulation in VR by blending three cues—motion gain, surfacedeformation, and vibration—in various combinations. Accurately sensing how strongly a surface “clings” is vital for realistic grasping, adhesion training and material evaluation, yet it has been largely overlooked in pseudo-haptic research. Three studies evaluated these cues. The first experiment compared individual and combined cues on perceived stickiness, confirming that additional cues reliably strengthened perceived stickiness. The second experiment tested how cue number affects tolerance for visual-physical mismatch, indicating that they lowered the minimum detectable threshold though they did not widen the overall tolerated mismatch. The third experiment measured whether and how much added cues sharpen perceptual resolution, showing multiple cues improved perceptual resolution by reducing just noticeable differences by 44 % ($1.8 \times$finer), doubling discriminable levels from roughly eight with a single cue to sixteen with all cues.

Augmented Reality (AR) is a growing technology that enables interaction with both virtual and real objects. However, in order to support the future development of efficient and usable AR interactions, there is still a lack of systematic knowledge establishing basic interaction performance across different conditions. Therefore, in this paper, we report a user study measuring the impact of objects' physicalization (object's set composed of (i) virtual, (ii) real, or (iii) a composite mix of real and virtual objects) and hand appearance (hand's appearance displayed as (i) the real hand, (ii) an avatar, or (iii) dynamically adapting to the surrounding objects' physicalization) on the speed performance of a pick-and-place task. Overall, our results reveal that objects' physicalization plays a significant role in interaction performance, with the more real objects in a set the better the performance. Moreover, our results also suggest that pick-and-place interaction performances are mostly unaffected by the hand appearance. Interestingly, we also observed that interactions with real objects were less efficient as the object condition required the user to alternate between interactions with virtual and real objects (object condition (iii)), which provides novel insights into an important - mostly AR-specific - factor to consider for designing future AR interactions. Taken together, our results provide a rich characterization of different factors influencing different phases of a pick-and-place interaction, which could be employed to improve the design of future AR applications.

Abstract One singularity of VR is its capacity to generate a strong sensation of being present in a dangerous environment, without risking the physical consequences. However, this absence of consequences when interacting with virtual dangers might also limit the induction of realistic responses, such as avoidance behaviors, which are a key factor of various VR applications (e.g., training, journalism, or exposure therapies). To address this limitation, we propose avatar-centered feedback, a novel, device-free approach, consisting of dynamically altering the avatar’s appearance and movements to offer coherent feedback from interactions with the virtual environment, including dangers. To begin with, we present a design space clarifying the range of potential implementations for this approach. Then, we tested this approach in a metallurgy scenario, where participants’ virtual hands would redden and display burns as they got closer to a virtual fire (appearance alteration), and simulate short withdrawal reflexes movements when a spark burst next to them (movement alteration). Our results show that in comparison to a control group, participants receiving avatar-centered feedback demonstrated significantly more avoidance behaviors to the virtual fire. Interestingly, we also found that experiencing avatar-centered feedback of fire significantly increased avoidance behaviors toward a following danger of different nature (a circular saw). These results suggest that avatar-centered feedback can also impact the general perception of the avatar vulnerability to the virtual environment.

In this paper, we study how thermal haptic feedback can influence social interactions in virtual environments, with an emphasis on persuasion, focus, co-presence, and friendliness. Physical and social warmth have been repeatedly linked in psychological literature, which allows for speculations on the effect of thermal haptics on virtual social interactions. To that effect, we conducted a study on thermal feedback during simulated social interactions with a virtual agent. We tested three conditions: warm, cool, and neutral. Results showed that warm feedback positively influenced users’ perception of the agent and significantly enhanced persuasion and thermal comfort. Multiple users reported the agent feeling less ‘robotic’ and more ‘human’ during the warm condition. Moreover, multiple studies have previously shown the potential of vibrotactile feedback for social interactions. A second study thus evaluated the combination of warmth and vibrations for social interactions. The study included the same protocol and three similar conditions: warmth, vibrations, and warm vibrations. Warmth was perceived as more friendly, while warm vibrations heightened the agent’s virtual presence and persuasion. These results encourage the study of thermal haptics to support positive social interactions. Moreover, they suggest that some haptic feedback are more suited to certain types of social interactions and communication than others.

Unintended Positional Drift (UPD) is a phenomenon that occurs during navigation in Virtual Reality (VR). It is characterized by the user’s unconscious or unintentional physical movements in the workspace while using a locomotion technique (LT) that does not require physical displacement (e.g., steering, teleportation). Recent work showed that some factors, such as the LT used and the type of trajectory, can influence UPD. However, little is known about the influence of rotation gains (commonly used in redirection-based LTs) on UPD during navigation in VR. In this paper, we conducted two user studies to assess the influence of rotation gains on UPD. In the first study, participants had to perform consecutive turns in a corridor virtual environment. In the second study, participants had to explore a large office floor and collect spheres freely. We compared the conditions between rotation gains and without gains, and we also varied the turning angle to perform the turns while considering factors such as sensitivity to cybersickness and the learning effect. We found that rotation gains and lower turning angles decreased UPD during the first study, but the presence of rotation gains increased UPD in the second study. This work contributes to the understanding of UPD, which tends to be an overlooked topic and discusses the design implications of these results for improving navigation in VR.

Pseudo-haptic techniques are used to modify haptic perception by appropriately changing visual feedback to body movements. Based on the knowledge that tendon vibration can affect our somatosensory perception, this article proposes a method for leveraging tendon vibration to enhance pseudo-haptics during free arm motion. Three experiments were performed to examine the impact of tendon vibration on the range and resolution of pseudo-haptics. The first experiment investigated the effect of tendon vibration on the detection threshold of the discrepancy between visual and physical motion. The results indicated that vibrations applied to the inner tendons of the wrist and elbow increased the threshold, suggesting that tendon vibration can augment the applicable visual motion gain by approximately 13% without users detecting the visual/physical discrepancy. Furthermore, the results demonstrate that tendon vibration acts as noise on haptic motion cues. The second experiment assessed the impact of tendon vibration on the resolution of pseudo-haptics by determining the just noticeable difference in pseudo-weight perception. The results suggested that the tendon vibration does not largely compromise the resolution of pseudo-haptics. The third experiment evaluated the equivalence between the weight perception triggered by tendon vibration and that by visual motion gain, that is, the point of subjective equality. The results revealed that vibration amplifies the weight perception and its effect was equivalent to that obtained using a gain of 0.64 without vibration, implying that the tendon vibration also functions as an additional haptic cue. Our results provide design guidelines and future work for enhancing pseudo-haptics with tendon vibration.

Walking-in-place (WIP) is a locomotion technique that enables users to "walk infinitely" through vast virtual environments using walking-like gestures within a limited physical space. This article investigates alternative interaction schemes for WIP, addressing successively the control, input, and output of WIP. First, we introduce a novel height-based control to increase advanced speed. Second, we introduce a novel input system for WIP based on elastic and passive strips. Third, we introduce the use of pseudo-haptic feedback as a novel output for WIP meant to alter walking sensations. The results of a series of user studies show that height and frequency based control of WIP can facilitate higher virtual speed with greater efficacy and ease than in frequency-based WIP. Second, using an upward elastic input system can result in a stable virtual speed control, although excessively strong elastic forces may impact the usability and user experience. Finally, using a pseudo-haptic approach can improve the perceived realism of virtual slopes. Taken together, our results suggest that, for future VR applications, there is value in further research into the use of alternative interaction schemes for walking-in-place.

During mid-air interactions, common approaches (such as the god-object method) typically rely on visually constraining the user's avatar to avoid visual interpenetrations with the virtual environment in the absence of kinesthetic feedback. This paper explores two methods which influence how the position mismatch (positional offset) between users' real and virtual hands is recovered when releasing the contact with virtual objects. The first method (sticky) constrains the user's virtual hand until the mismatch is recovered, while the second method (unsticky) employs an adaptive offset recovery method. In the first study, we explored the effect of positional offset and of motion alteration on users' behavioral adjustments and users' perception. In a second study, we evaluated variations in the sense of embodiment and the preference between the two control laws. Overall, both methods presented similar results in terms of performance and accuracy, yet, positional offsets strongly impacted motion profiles and users' performance. Both methods also resulted in comparable levels of embodiment. Finally, participants usually expressed strong preferences toward one of the two methods, but these choices were individual-specific and did not appear to be correlated solely with characteristics external to the individuals. Taken together, these results highlight the relevance of exploring the customization of motion control algorithms for avatars.

Consumer-grade virtual reality (VR) controllers are typically equipped with one vibrotactile actuator, allowing to create simple and non-spatialized tactile sensations through the vibration of the entire controller. Leveraging the funneling effect, an illusion in which multiple vibrations are perceived as a single one, we propose MultiVibes, a VR controller capable of rendering spatialized sensations at different locations on the user’s hand and fingers. The designed prototype includes ten vibrotactile actuators, directly in contact with the skin of the hand, limiting the propagation of vibrations through the controller. We evaluated MultiVibes through two controlled experiments. The first one focused on the ability of users to recognize spatio-temporal patterns, while the second one focused on the impact of MultiVibes on the users’ haptic experience when interacting with virtual objects they can feel. Taken together, the results show that MultiVibes is capable of providing accurate spatialized feedback and that users prefer MultiVibes over recent VR controllers.

In Virtual Reality (VR), a growing number of applications involve verbal communications with avatars, such as for teleconference, entertainment, virtual training, social networks, etc. In this context, our paper aims to investigate how tactile feedback consisting in vibrations synchronized with speech could influence aspects related to VR social interactions such as persuasion, co-presence and leadership. We conducted two experiments where participants embody a first-person avatar attending a virtual meeting in immersive VR. In the first experiment, participants were listening to two speaking virtual agents and the speech of one agent was augmented with vibrotactile feedback. Interestingly, the results show that such vibrotactile feedback could significantly improve the perceived co-presence but also the persuasiveness and leadership of the haptically-augmented agent. In the second experiment, the participants were asked to speak to two agents, and their own speech was augmented or not with vibrotactile feedback. The results show that vibrotactile feedback had again a positive effect on co-presence, and that participants perceive their speech as more persuasive in presence of haptic feedback. Taken together, our results demonstrate the strong potential of haptic feedback for supporting social interactions in VR, and pave the way to novel usages of vibrations in a wide range of applications in which verbal communication plays a prominent role.

In virtual reality, the avatar - the user's digital representation - is an important element which can drastically influence the immersive experience. In this paper, we especially focus on the use of "dissimilar" avatars i.e., avatars diverging from the real appearance of the user, whether they preserve an anthropomorphic aspect or not. Previous studies reported that dissimilar avatars can positively impact the user experience, in terms for example of interaction, perception or behaviour. However, given the sparsity and multi-disciplinary character of research related to dissimilar avatars, it tends to lack common understanding and methodology, hampering the establishment of novel knowledge on this topic. In this paper, we propose to address these limitations by discussing: (i) a methodology for dissimilar avatars characterization, (ii) their impacts on the user experience, (iii) their different fields of application, and finally, (iv) future research direction on this topic. Taken together, we believe that this paper can support future research related to dissimilar avatars, and help designers of VR applications to leverage dissimilar avatars appropriately.

Recent research has attempted to identify methods to mitigate cybersickness and examine its aftereffects. In this direction, this paper examines the effects of cybersickness on cognitive, motor, and reading performance in VR. Also, this paper evaluates the mitigating effects of music on cybersickness, as well as the role of gender, and the computing, VR, and gaming experience of the user. This paper reports two studies. In the 1st study, 92 participants selected the music tracks considered most calming (low valence) or joyful (high valence) to be used in the 2nd study. In the 2nd study, 39 participants performed an assessment four times, once before the rides (baseline), and then once after each ride (3 rides). In each ride either Calming, or Joyful, or No Music was played. During each ride, linear and angular accelerations took place to induce cybersickness in the participants. In each assessment, while immersed in VR, the participants evaluated their cybersickness symptomatology and performed a verbal working memory task, a visuospatial working memory task, and a psychomotor task. While responding to the cybersickness questionnaire (3D UI), eye-tracking was conducted to measure reading time and pupillometry. The results showed that Joyful and Calming music substantially decreased the intensity of nausea-related symptoms. However, only Joyful music significantly decreased the overall cybersickness intensity. Importantly, cybersickness was found to decrease verbal working memory performance and pupil size. Also, it significantly decelerated psychomotor (reaction time) and reading abilities. Higher gaming experience was associated with lower cybersickness. When controlling for gaming experience, there were no significant differences between female and male participants in terms of cybersickness. The outcomes indicated the efficiency of music in mitigating cybersickness, the important role of gaming experience in cybersickness, and the significant effects of cybersickness on pupil size, cognition, psychomotor skills, and reading ability.

In Virtual Reality, self-touch (ST) stimulation is a promising method of sense of body ownership (SoBO) induction that does not require an external effector. However, its applicability to dysmorphic bodies has not been explored yet and remains uncertain due to the requirement to provide incongruent visuomotor sensations. In this, paper, we studied the effect of ST stimulation on dysmorphic hands via haptic retargeting, as compared to a classical external-touch (ET) stimulation, on the SoBO. Our results indicate that ST can induce similar levels of dysmorphic SoBO than ET stimulation, but that some types of dysmorphism might decrease the ST stimulation accuracy due to the nature of the re-targeting that they induce.

When designing virtual embodiment studies, one of the key choices is the nature of the experimental factors, either between-subjects or within-subjects. However, it is well known that each design has ad-vantages and disadvantages in terms of statistical power, sample size requirements and confounding factors. This paper reports a within-subjects experiment with 92 participants comparing self-reported embodiment scores under a visuomotor task with two conditions: synchronous motions and asynchronous motions with a latency of 300 ms. With the gathered data, using a Monte-Carlo method, we created numerous simulations of within- and between-subjects experiments by selecting subsets of the data. In particular, we explored the impact of the number of participants on the replicability of the results from the 92 within-subjects experiment. For the between-subjects simulations, only the first condition for each user was considered to create the simulations. The results showed that while the replicability of the results increased as the number of participants increased for the within-subjects simulations, no matter the number of participants, between-subjects simulations were not able to replicate the initial results. We discuss the potential reasons that could have led to this surprising result and potential methodological practices to mitigate them.

Timelines are common visualizations to represent and manipulate temporal data. However, timeline visualizations rarely consider spatio-temporal 3D data (e.g. mesh or volumetric models) directly. In this paper, leveraging the increased workspace and 3D interaction capabilities of virtual reality (VR), we first propose a timeline design space for 3D temporal data extending the timeline design space proposed by Brehmer et al. [7]. The proposed design space adapts the scale, layout and representation dimensions to account for the depth dimension and how the 3D temporal data can be partitioned and structured. Moreover, an additional dimension is introduced, the support, which further characterizes the 3D dimension of the visualization. The design space is complemented by discussing the interaction methods required for the efficient visualization of 3D timelines in VR. Secondly, we evaluate the benefits of 3D timelines through a formal evaluation (n=21). Taken together, our results showed that time-related tasks can be achieved more comfortably using timelines, and more efficiently for specific tasks requiring the analysis of the surrounding temporal context. Finally, we illustrate the use of 3D timelines with a use-case on morphogenetic analysis in which domain experts in cell imaging were involved in the design and evaluation process.

The use of electrotactile feedback in Virtual Reality (VR) has shown promising results for providing tactile information and sensations. While progress has been made to provide custom electrotactile feedback for specific interaction tasks, it remains unclear which modulations and rendering algorithms are preferred in rich interaction scenarios. In this paper, we propose a unified tactile rendering architecture and explore the most promising modulations to render finger interactions in VR. Based on a literature review, we designed six electrotactile stimulation patterns/effects (EFXs) striving to render different tactile sensations. In a user study (N=18), we assessed the six EFXs in three diverse finger interactions: 1) tapping on a virtual object; 2) pressing down a virtual button; 3) sliding along a virtual surface. Results showed that the preference for certain EFXs depends on the task at hand. No significant preference was detected for tapping (short and quick contact); EFXs that render dynamic intensities or dynamic spatio-temporal patterns were preferred for pressing (continuous dynamic force); EFXs that render moving sensations were preferred for sliding (surface exploration). The results showed the importance of the coherence between the modulation an the interaction being performed and the study proved the versatility of electrotactile feedback and its efficiency in rendering different haptic information and sensations.

In virtual reality, several manipulation techniques distort users' motions, for example to reach remote objects or increase precision. These techniques can become problematic when used with avatars, as they create a mismatch between the real performed action and the corresponding displayed action, which can negatively impact the sense of embodiment. In this paper, we propose to use a dual representation during anisomorphic interaction. A co-located representation serves as a spatial reference and reproduces the exact users' motion, while an interactive representation is used for distorted interaction. We conducted two experiments, investigating the use of dual representations with amplified motion (with the Go-Go technique) and decreased motion (with the PRISM technique). Two visual appearances for the interactive representation and the co-located one were explored. This exploratory study investigating dual representations in this context showed that people globally preferred having a single representation, but opinions diverged for the Go-Go technique. Also, we could not find significant differences in terms of performance. While interacting seemed more important than showing exact movements for agency during out-of-reach manipulation, people felt more in control of the realistic arm during close manipulation.

While user's perception and performance are predominantly examined independently in virtual reality, the Action-Specific Perception (ASP) theory postulates that the performance of an individual on a task modulates this individual's spatial and time perception pertinent to the task's components and procedures. This paper examines the association between performance and perception and the potential effects that tactile feedback modalities could generate. This paper reports a user study (N=24), in which participants performed a standardized Fitts's law target acquisition task by using three feedback modalities: visual, visuo-electrotactile, and visuo-vibrotactile. The users completed 3 Target Sizes × 2 Distances × 3 feedback modalities = 18 trials. The size perception, distance perception, and (movement) time perception were assessed at the end of each trial. Performance-wise, the results showed that electrotactile feedback facilitates a significantly better accuracy compared to vibrotactile and visual feedback, while vibrotactile provided the worst accuracy. Electrotactile and visual feedback enabled a comparable reaction time, while the vibrotactile offered a substantially slower reaction time than visual feedback. Although amongst feedback types the pattern of differences in perceptual aspects were comparable to performance differences, none of them was statistically significant. However, performance indeed modulated perception. Significant action-specific effects on spatial and time perception were detected. Changes in accuracy modulate both size perception and time perception, while changes in movement speed modulate distance perception. Also, the index of difficulty was found to modulate all three perceptual aspects. However, individual differences appear to affect the magnitude of action-specific effects. These outcomes highlighted the importance of haptic feedback on performance, and importantly the significance of action-specific effects on spatial and time perception in VR, which should be considered in future VR studies.

In Virtual Reality (VR), users can be immersed in emotionally intense and cognitively engaging experiences. Yet, despite strong interest from scholars and a large amount of work associating VR and Affective and Cognitive States (ACS), there is a clear lack of structured and systematic form in which this research can be classified. We define "Affective and Cognitive VR" to relate to works which (1) induce ACS, (2) recognize ACS, or (3) exploit ACS by adapting virtual environments based on ACS measures. This survey clarifies the different models of ACS, presents the methods for measuring them with their respective advantages and drawbacks in VR, and showcases Affective and Cognitive VR studies done in an Immersive Virtual Environment (IVE) in a non-clinical context. Our article covers the main research lines in Affective and Cognitive VR. We provide a comprehensive list of references with the analysis of 63 research articles and summarize future works directions.

In this special issue ofIEEE Transactions on Visualization and Computer Graphics(TVCG), we are pleased to present a subset of papers from the 29th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2022), held virtually March 12-16, 2022, in Christchurch, New Zealand.

Avatars, the users’ virtual representations, are becoming ubiquitous in virtual reality applications. In this context, the avatar becomes the medium which enables users to manipulate objects in the virtual environment. It also becomes the users’ main spatial reference, which can not only alter their interaction with the virtual environment, but also the perception of themselves. In this paper, we review and analyse the current state-of-the-art for 3D object manipulation and the sense of embodiment. Our analysis is twofold. First, we discuss the impact that the avatar can have on object manipulation. Second, we discuss how the different components of a manipulation technique (i.e. input, control and feedback) can influence the user’s sense of embodiment. Throughout the analysis, we crystallise our discussion with practical guidelines for VR application designers and we propose several research topics towards “avatar-friendly’’ manipulation techniques.

We report on a controlled user study in which we investigated and compared three selection techniques in discovering and traversing 3D objects in densely packed environments. We apply this to cell division history marking as required by plant biologists who study the development of embryos, for whom existing selection techniques do not work due to the occlusion and tight packing of the cells to be selected. We specifically compared a list-based technique with an additional 3D view, a 3D selection technique that relies on an exploded view, and a combination of both techniques. Our results indicate that the combination was most preferred. List selection has advantages for traversing cells, while we did not find differences for surface cells. Our participants appreciated the combination because it supports discovering 3D objects with the 3D explosion technique while using the lists to traverse 3D cells.

Virtual steering techniques enable users to navigate in larger Virtual Environments (VEs) than the physical workspace available. Even though these techniques do not require physical movement of the users (e.g. using a joystick and the head orientation to steer towards a virtual direction), recent work observed that users might unintentionally move in the physical workspace while navigating, resulting in Unintended Positional Drift (UPD). This phenomenon can be a safety issue since users may unintentionally reach the physical boundaries of the workspace while using a steering technique. In this context, as a necessary first step to improve the design of navigation techniques minimizing the UPD, this paper aims at analyzing and modeling the UPD during a virtual navigation task. In particular, we characterize and analyze the UPD for a dataset containing the positions and orientations of eighteen users performing a virtual slalom task using virtual steering techniques. Participants wore a head-mounted display and had to follow three different sinusoidal-like trajectories (with low, medium and high curvature) using a torso-steering navigation technique. We analyzed the performed motions and proposed two UPD models: the first based on a linear regression analysis and the second based on a Gaussian Mixture Model (GMM) analysis. Then, we assessed both models through a simulation-based evaluation where we reproduced the same navigation task using virtual agents. Our results indicate the feasibility of using simulation-based evaluations to study UPD. The paper concludes with a discussion of potential applications of the results in order to gain a better understanding of UPD during steering and therefore improve the design of navigation techniques by compensating for UPD.

In this article, we introduce a concept called "virtual co-embodiment", which enables a user to share their virtual avatar with another entity (e.g., another user, robot, or autonomous agent). We describe a proof-of-concept in which two users can be immersed from a first-person perspective in a virtual environment and can have complementary levels of control (total, partial, or none) over a shared avatar. In addition, we conducted an experiment to investigate the influence of users' level of control over the shared avatar and prior knowledge of their actions on the users' sense of agency and motor actions. The results showed that participants are good at estimating their real level of control but significantly overestimate their sense of agency when they can anticipate the motion of the avatar. Moreover, participants performed similar body motions regardless of their real control over the avatar. The results also revealed that the internal dimension of the locus of control, which is a personality trait, is negatively correlated with the user's perceived level of control. The combined results unfold a new range of applications in the fields of virtual-reality-based training and collaborative teleoperation, where users would be able to share their virtual body.

While digital humans are key aspects of the rapidly evolving areas of virtual reality, gaming, and online communications, many applications would benefit from using digital personalized (stylized) representations of users, as they were shown to highly increase immersion, presence and emotional response. In particular, depending on the target application, one may want to look like a dwarf or an elf in a heroic fantasy world, or like an alien on another planet, in accordance with the style of the narrative. While creating such virtual replicas requires stylization of the user’s features onto the virtual character, no formal study has however been conducted to assess the ability to recognize stylized characters. In this paper, we present a perceptual study investigating the effect of the degree of stylization on the ability to recognize an actor, and the subjective acceptability of stylizations. Results show that recognition rates decrease when the degree of stylization increases, while acceptability of the stylization increases. These results provide recommendations to achieve good compromises between stylization and recognition, and pave the way to new stylization methods providing a tradeoff between stylization and recognition of the actor.

Virtual embodiment and navigation are two topics widely studied in the virtual reality community. Despite the potential inter-relation between embodiment and locomotion, studies on virtual navigation rarely supply users with a virtual representation, while studies on virtual embodiment rarely allow users to virtually navigate. This paper therefore explores this potential inter-relation by focusing on the two following questions: Does the locomotion technique have an impact on the user's sense of embodiment? Does embodying an avatar have an impact on the user's preference and performance depending on the locomotion technique?To address these questions, we conducted a user study (N=60) exploring the relationship between the locomotion technique and the user's sense of embodiment over a virtual avatar seen from a first-person perspective. Three widely used locomotion techniques were evaluated: real walking, walking-in-place and virtual steering. All participants performed four different tasks involving a different awareness of their virtual avatar. Participants also performed the same tasks without being embodied in an avatar. The results show that participants had a comparable sense of embodiment with all techniques when embodied in an avatar, and that the presence or absence of the virtual avatar did not alter their performance while navigating, independently of the technique. Taken together, our results represent a first attempt to qualify the inter-relation between virtual navigation and virtual embodiment, and suggest that the 3D locomotion technique used has little influence on the user's sense of embodiment in VR.

This paper describes an “all-in-one” solution for the real-time recognition of users' mental workloads in virtual reality through the customization of a commercial HMD with physiological sensors. First, we describe the hardware and software solution employed to build the system. Second, we detail the machine learning methods used for the automatic recognition of the users' mental workload, which are based on the well-known Random Forest algorithm. In order to gather data to train the system, we conducted an extensive user study with 75 participants using a VR flight simulator to induce different levels of mental workload. In contrast to previous works which label the data based on a standardized task (e.g., n-back task) or on a pre-defined task-difficulty, participants were asked about their perceived mental workload level along the experiment. With the data collected, we were able to train the system in order to classify four different levels of mental workload with an accuracy up to 65%. In addition, we discuss the role of the signal normalization procedures, the contribution of the different physiological signals on the recognition accuracy and compare the results obtained with the sensors embedded in the HMD with commercial grade systems. Preliminary results show our pipeline is able to recognize mental workload in real-time. Taken together, our results suggest that such all-in-one approach, with physiological sensors directly embedded in the HMD, is a promising path for VR applications in which the real-time or off-line estimation of Mental Workload assessment is beneficial.

Commonly used Head Mounted Displays (HMDs) in Augmented Reality (AR), namely Optical See-Through (OST) displays, suffer from a main drawback: their focal lenses can only provide a fixed focal distance. Such a limitation is suspected to be one of the main factors for distance misperception in AR. In this paper, we studied the use of an emerging new kind of AR display to tackle such perception issues: Retinal Projection Displays (RPDs). With RPDs, virtual images have no focal distance and the AR content is always in focus. We conducted the first reported experiment evaluating egocentric distance perception of observers using Retinal Projection Displays. We compared the precision and accuracy of the depth estimation between real and virtual targets, displayed by either OST HMDs or RPDs. Interestingly, our results show that RPDs provide depth estimates in AR closer to real ones compared to OST HMDs. Indeed, the use of an OST device was found to lead to an overestimation of the perceived distance by 16%, whereas the distance overestimation bias dropped to 4% with RPDs. Besides, the task was reported with the same level of difficulty and no difference in precision. As such, our results shed the first light on retinal projection displays' benefits in terms of user's perception in Augmented Reality, suggesting that RPD is a promising technology for AR applications in which an accurate distance perception is required.

Training is one of the major use cases of Virtual Reality (VR) due to the flexibility and reproducibility of VR simulations. However, the use of the user’s cognitive state, and in particular mental workload (MWL), remains largely unexplored in the design of training scenarios. In this paper, we propose to consider MWL for the design of complex training scenarios involving multiple parallel tasks in VR. The proposed approach is based on the assessment of the MWL elicited by each potential task configuration in the training application. Following the assessment, the resulting model is then used to create training scenarios able to modulate the user’s MWL over time. This approach is illustrated by a VR flight training simulator based on the Multi-Attribute Task Battery II, which solicits different cognitive resources, able to generate 12 different tasks configurations. A first user study (N = 38) was conducted to assess the MWL for each task configuration using self-reports and performance measurements. This assessment was then used to generate three training scenarios in order to induce different levels of MWL over time. A second user study (N = 14) confirmed that the proposed approach was able to induce the expected mental workload over time for each training scenario. These results pave the way to further studies exploring how MWL modulation can be used to improve VR training applications.

When persons interact with the environment and experience or witness an error (e.g. an unexpected event), a specific brain pattern, known as error-related potential (ErrP) can be observed in the electroencephalographic signals (EEG). Virtual Reality (VR) technology enables users to interact with computer-generated simulated environments and to provide multi-modal sensory feedback. Using VR systems can, however, be error-prone. In this paper, we investigate the presence of ErrPs when Virtual Reality users face 3 types of visualization errors: (Te) tracking errors when manipulating virtual objects, (Fe) feedback errors, and (Be) background anomalies. We conducted an experiment in which 15 participants were exposed to the 3 types of errors while performing a center-out pick and place task in virtual reality. The results showed that tracking errors generate error-related potentials, the other types of errors did not generate such discernible patterns. In addition, we show that it is possible to detect the ErrPs generated by tracking losses in single trial, with an accuracy of 85%. This constitutes a first step towards the automatic detection of error-related potentials in VR applications, paving the way to the design of adaptive and self-corrective VR/AR applications by exploiting information directly from the user’s brain.

In Virtual Reality, a number of studies have been conducted to assess the influence of avatar appearance, avatar control and user point of view on the Sense of Embodiment (SoE) towards a virtual avatar. However, such studies tend to explore each factor in isolation. This paper aims to better understand the inter-relations among these three factors by conducting a subjective matching experiment. In the presented experiment ( n=40), participants had to match a given "optimal" SoE avatar configuration (realistic avatar, full-body motion capture, first-person point of view), starting by a "minimal" SoE configuration (minimal avatar, no control, third-person point of view), by iteratively increasing the level of each factor. The choices of the participants provide insights about their preferences and perception over the three factors considered. Moreover, the subjective matching procedure was conducted in the context of four different interaction tasks with the goal of covering a wide range of actions an avatar can do in a VE. The paper also describes a baseline experiment ( n=20) which was used to define the number and order of the different levels for each factor, prior to the subjective matching experiment (e.g. different degrees of realism ranging from abstract to personalised avatars for the visual appearance). The results of the subjective matching experiment show that point of view and control levels were consistently increased by users before appearance levels when it comes to enhancing the SoE. Second, several configurations were identified with equivalent SoE as the one felt in the optimal configuration, but vary between the tasks. Taken together, our results provide valuable insights about which factors to prioritize in order to enhance the SoE towards an avatar in different tasks, and about configurations which lead to fulfilling SoE in VE.

Brain-Computer Interfaces (BCIs) enable users to interact with computers without any dedicated movement, bringing new hands-free interaction paradigms. In this paper we study the combination of BCI and Augmented Reality (AR). We first tested the feasibility of using BCI in AR settings based on Optical See-Through Head-Mounted Displays (OST-HMDs). Experimental results showed that a BCI and an OST-HMD equipment (EEG headset and Hololens in our case) are well compatible and that small movements of the head can be tolerated when using the BCI. Second, we introduced a design space for command display strategies based on BCI in AR, when exploiting a famous brain pattern called Steady-State Visually Evoked Potential (SSVEP). Our design space relies on five dimensions concerning the visual layout of the BCI menu; namely: orientation, frame-of-reference, anchorage, size and explicitness. We implemented various BCI-based display strategies and tested them within the context of mobile robot control in AR. Our findings were finally integrated within an operational prototype based on a real mobile robot that is controlled in AR using a BCI and a HoloLens headset. Taken together our results (4 user studies) and our methodology could pave the way to future interaction schemes in Augmented Reality exploiting 3D User Interfaces based on brain activity and BCIs.

With the increasing use of avatars (i.e. the virtual representation of the user in a virtual environment) in virtual reality, it is important to identify the factors eliciting the sense of embodiment or the factors that can disrupt this feeling. This paper reports an exploratory study aiming at identifying internal factors (personality traits and body awareness) that might cause either a resistance or a predisposition to feel a sense of embodiment towards a virtual avatar. To this purpose, we conducted an experiment (n=123) in which participants were immersed in a virtual environment and embodied in a gender-matched generic virtual avatar through a head-mounted display. After an exposure phase in which they had to perform a number of visuomotor tasks (during 2 minutes) a virtual character entered the virtual scene and stabbed the participants' virtual hand with a knife. The participants' sense of embodiment was measured, as well as several personality traits (Big Five traits and locus of control) and body awareness, to evaluate the influence of participants' personality on the acceptance of the virtual body. The major finding of the experiment is that the locus of control is linked to several components of embodiment: the sense of agency is positively correlated with an internal locus of control and the sense of body ownership is positively correlated with an external locus of control. Interestingly, both components are not influenced by the same traits, which confirms that they can appear independently. Taken together our results suggest that the locus of control could be a good predictor of the sense of embodiment when the user embodies an avatar with a similar physical appearance. Yet, further studies are required to confirm these results.

While perceptual biases have been widely investigated in Virtual Reality (VR), very few studies have considered the challenging environment of Optical See-through Augmented Reality (OST-AR). Moreover, regarding distance perception, existing works mainly focus on the assessment of egocentric distance perception, i.e. distance between the observer and a real or a virtual object. In this paper, we study exocentric distance perception in AR, hereby considered as the distance between two objects, none of them being directly linked to the user. We report a user study (n=29) aiming at estimating distances between two objects lying in a frontoparallel plane at 2.1m from the observer (i.e. in the medium-field perceptual space). Four conditions were tested in our study: real objects on the left and on the right of the participant (called real-real), virtual objects on both sides (virtual-virtual), a real object on the left and a virtual one on the right (real-virtual) and finally a virtual object on the left and a real object on the right (virtual-real). Participants had to reproduce the distance between the objects by spreading two real identical objects presented in front of them. The main findings of this study are the overestimation (20%) of exocentric distances for all tested conditions. Surprisingly, the real-real condition was significantly more overestimated (by about 4%, p=.0166) compared to the virtual-virtual condition, i.e. participants obtained better estimates of the exocentric distance for the virtual-virtual condition. Finally, for the virtual-real/real-virtual conditions, the analysis showed a non-symmetrical behavior, which suggests that the relationship between real and virtual objects with respect to the user might be affected by other external factors. Considered together, these unexpected results illustrate the need for additional experiments to better understand the perceptual phenomena involved in exocentric distance perception with real and virtual objects.

This paper investigates whether the body anticipation synergies in real environments (REs) are preserved during navigation in virtual environments (VEs). Experimental studies related to the control of human locomotion in REs during curved trajectories report a top-down reorientation strategy with the reorientation of the gaze anticipating the reorientation of head, the shoulders and finally the global body motion. This anticipation behavior provides a stable reference frame to the walker to control and reorient his/her body according to the future walking direction. To assess body anticipation during navigation in VEs, we conducted an experiment where participants, wearing a head-mounted display, performed a lemniscate trajectory in a virtual environment (VE) using five different navigation techniques, including walking, virtual steering (head, hand or torso steering) and passive navigation. For the purpose of this experiment, we designed a new control law based on the power-law relation between speed and curvature during human walking. Taken together our results showed a similar ordered top-down sequence of reorientation of the gaze, head and shoulders during curved trajectories between walking in REs and in VEs (for all the evaluated techniques). However, the anticipation mechanism was significantly higher for the walking condition compared to the others. The results presented in this paper pave the way to the better understanding of the underlying mechanisms of human navigation in VEs and to the design of navigation techniques more adapted to humans.

Is there an effect of Virtual Reality (VR) Head-Mounted Display (HMD) on the user's mental effort? In this paper, we compare the mental effort in VR versus in real environments. An experiment (N=27) was conducted to assess the effect of being immersed in a virtual environment (VE) using an HMD on the user's mental effort while performing a standardized cognitive task (the well-known N-back task, with three levels of difficulty, N ∈ {1,2,3}). In addition to test the effect of the environment (i.e., virtual versus real), we also explored the impact of performing a dual task (i.e., sitting versus walking) in both environments on mental effort. The mental effort was assessed through self-report, task performance, behavioural and physiological measures. In a nutshell, the analysis of all measurements revealed no significant effect of being immersed in the VE on the users' mental effort. In contrast, natural walking significantly increased the users' mental effort. Taken together, our results support the fact there is no specific additional mental effort related to the immersion in a VE using a VR HMD.

The topic of distance perception has been widely investigated in Virtual Reality (VR). However, the vast majority of previous work mainly focused on distance perception of objects placed in front of the observer. Then, what happens when the observer looks on the side? In this paper, we study differences in distance estimation when comparing objects placed in front of the observer with objects placed on his side. Through a series of four experiments (n=85), we assessed participants' distance estimation and ruled out potential biases. In particular, we considered the placement of visual stimuli in the field of view, users' exploration behavior as well as the presence of depth cues. For all experiments a two-alternative forced choice (2AFC) standardized psychophysical protocol was employed, in which the main task was to determine the stimuli that seemed to be the farthest one. In summary, our results showed that the orientation of virtual stimuli with respect to the user introduces a distance perception bias: objects placed on the sides are systematically perceived farther away than objects in front. In addition, we could observe that this bias increases along with the angle, and appears to be independent of both the position of the object in the field of view as well as the quality of the virtual scene. This work sheds a new light on one of the specificities of VR environments regarding the wider subject of visual space theory. Our study paves the way for future experiments evaluating the anisotropy of distance perception in real and virtual environments.

In immersive projection systems (IPS), the presence of the user's real body limits the possibility to elicit a virtual body ownership illusion. But, is it still possible to embody someone else in an IPS even though the users are aware of their real body? In order to study this question, we propose to consider using a virtual shadow in the IPS, which can be similar or different from the real user's morphology. We have conducted an experiment (N=27) to study the users' sense of embodiment whenever a virtual shadow was or was not present. Participants had to perform a 3D positioning task in which accuracy was the main requirement. The results showed that users widely accepted their virtual shadow (agency and ownership) and felt more comfortable when interacting with it (compare to no virtual shadow). Yet, due to the awareness of their real body, the users have less acceptance of the virtual shadow whenever the shadow gender differs from their own. Furthermore, the results showed that virtual shadows increase the users' spatial perception of the virtual environment by decreasing the inter-penetrations between the user and the virtual objects. Taken together, our results promote the use of dynamic and realistic virtual shadows in IPS and pave the way for further studies on "virtual shadow ownership" illusion.

This paper describes three novel 3D interaction metaphors conceived to fulfill the three tasks proposed in the current edition of the IEEE VR 3DUI Contest. We propose the VladdeR, a tangible interface for Virtual laddeR climbing, the FPDrone, a First Person Drone control flying interface, and the Dice Cup, a tangible interface for virtual object stacking. All three interactions take advantage of body proprioception and previous knowledge of real life interactions without the need of complex interaction mechanics: climbing a tangible ladder through arm and leg motions, control a drone like a child flies an imaginary plane by extending your arms or stacking objects as you will grab and stack dice with a dice cup.

In this paper, we explore the influence of sharing a virtual environment with another user on the sense of embodiment in virtual reality. For this aim, we conducted an experiment where users were immersed in a virtual environment while being embodied in an anthropomorphic virtual representation of themselves. To evaluate the influence of the presence of another user, two situations were studied: either users were immersed alone, or in the company of another user. During the experiment, participants performed a virtual version of the well-known whac-a-mole game, therefore interacting with the virtual environment, while sitting at a virtual table. Our results show that users were significantly more “efficient” (i.e., faster reaction times), and accordingly more engaged, in performing the task when sharing the virtual environment, in particular for the more competitive tasks. Also, users experienced comparable levels of embodiment both when immersed alone or with another user. These results are supported by subjective questionnaires but also through behavioural responses, e.g. users reacting to the introduction of a threat towards their virtual body. Taken together, our results show that competition and shared experiences involving an avatar do not influence the sense of embodiment, but can increase user engagement. Such insights can be used by designers of virtual environments and virtual reality applications to develop more engaging applications.

While the Sense of Agency (SoA) has so far been predominantly characterised in VR as a component of the Sense of Embodiment, other communities (e.g., in psychology or neurosciences) have investigated the SoA from a different perspective proposing complementary theories. Yet, despite the acknowledged potential benefits of catching up with these theories a gap remains. This paper first aims to contribute to fill this gap by introducing a theory according to which the SoA can be divided into two components, the feeling and the judgment of agency, and relies on three principles, namely the principles of priority, exclusivity and consistency. We argue that this theory could provide insights on the factors influencing the SoA in VR systems. Second, we propose novel approaches to manipulate the SoA in controlled VR experiments (based on these three principles) as well as to measure the SoA, and more specifically its two components based on neurophysiological markers, using ElectroEncephaloGraphy (EEG). We claim that these approaches would enable us to deepen our understanding of the SoA in VR contexts. Finally, we validate these approaches in an experiment. Our results (N=24) suggest that our approach was successful in manipulating the SoA as the modulation of each of the three principles induced significant decreases of the SoA (measured using questionnaires). In addition, we recorded participants' EEG signals during the VR experiment, and neurophysiological markers of the SoA, potentially reflecting the feeling and judgment of agency specifically, were revealed. Our results also suggest that users' profile, more precisely their Locus of Control (LoC), influences their level of immersion and SoA.

BCIs are presumably supposed to require the full attention of their users and to lose accuracy if they pay attention to another task. This assertion has been verified with several BCI paradigms (e.g. P300). But the cognitive demand of the promising SSVEP paradigm had never been specifically assessed yet. We measured the accuracy of an SSVEP-based BCI used by 26 participants in various conditions of mental workload. Our analysis revealed that surprisingly, for this type of BCI, little attention is actually needed from participants to reach optimal accuracy: participants were able to successfully perform a complex secondary task (N-back) without degrading the BCI accuracy. The same observation was made whether visual or auditive attention was solicited. These results indicate that SSVEP is a low-demanding paradigm in terms of cognitive resources, and are encouraging for its use in complex interaction settings.

Due to the nature of Virtual Reality (VR) research, conducting experiments in order to validate the researcher's hypotheses is a must. However, the development of such experiments is a tedious and time-consuming task. In this work, we propose to make this task easier, more intuitive and faster with a method able to describe and generate the most tedious components of VR experiments. The main objective is to let experiment designers focus on their core tasks: designing, conducting, and reporting experiments. To that end, we propose the use of Domain-Specific Languages (DSLs) to ease the description and generation of VR experiments. An analysis of published VR experiments is used to identify the main properties that characterize VR experiments. This allowed us to design AGENT (Automatic Generation of ExperimeNtal proTocol runtime), a DSL for specifying and generating experimental protocol runtimes. We demonstrated the feasibility of our approach by using AGENT on two experiments published in the VRST'16 proceedings.

Does it feel the same when you touch an object in Augmented Reality (AR) or in Virtual Reality (VR)? In this paper we study and compare the haptic perception of stiffness of a virtual object in two situations: (1) a purely virtual environment versus (2) a real and augmented environment. We have designed an experimental setup based on a Microsoft HoloLens and a haptic force-feedback device, enabling to press a virtual piston, and compare its stiffness successively in either Augmented Reality (the virtual piston is surrounded by several real objects all located inside a cardboard box) or in Virtual Reality (the same virtual piston is displayed in a fully virtual scene composed of the same other objects). We have conducted a psychophysical experiment with 12 participants. Our results show a surprising bias in perception between the two conditions. The virtual piston is on average perceived stiffer in the VR condition compared to the AR condition. For instance, when the piston had the same stiffness in AR and VR, participants would select the VR piston as the stiffer one in 60% of cases. This suggests a psychological effect as if objects in AR would feel "softer" than in pure VR. Taken together, our results open new perspectives on perception in AR versus VR, and pave the way to future studies aiming at characterizing potential perceptual biases.

Brain-Computer Interfaces (BCIs) are still much less reliable than other input devices. The error rates of BCIs range from 5% up to 60%. In this paper, we assess the subjective frustration, motivation, and fatigue of BCI users, when confronted to different levels of error rate. We conducted a BCI experiment in which the error rate was artificially controlled. Our results first show that a prolonged use of BCI significantly increases the perceived fatigue, and induces a drop in motivation. We also found that user frustration increases with the error rate of the system but this increase does not seem critical for small differences of error rate. Thus, for future BCIs, we would advise to favor user comfort over accuracy when the potential gain of accuracy remains small.

How do people appropriate their virtual hand representation when interacting in virtual environments? In order to answer this question, we conducted an experiment studying the sense of embodiment when interacting with three different virtual hand representations, each one providing a different degree of visual realism but keeping the same control mechanism. The main experimental task was a Pick-and-Place task in which participants had to grasp a virtual cube and place it to an indicated position while avoiding an obstacle (brick, barbed wire or fire). An additional task was considered in which participants had to perform a potentially dangerous operation towards their virtual hand: place their virtual hand close to a virtual spinning saw. Both qualitative measures and questionnaire data were gathered in order to assess the sense of agency and ownership towards each virtual hand. Results show that the sense of agency is stronger for less realistic virtual hands which also provide less mismatch between the participant's actions and the animation of the virtual hand. In contrast, the sense of ownership is increased for the human virtual hand which provides a direct mapping between the degrees of freedom of the real and virtual hand.

Navigation in multi-scale virtual environments (MSVE) requires the adjustment of the navigation parameters to ensure optimal navigation experiences at each level of scale. In particular, in immersive stereoscopic systems, e.g. when performing zoom-in and zoom-out operations, the navigation speed and the stereoscopic rendering parameters have to be adjusted accordingly. Although this adjustment can be done manually by the user, it can be complex, tedious and strongly depends on the virtual environment. In this work we propose a new multi-scale navigation technique named GiAnt (GIant/ANT) which automatically and seamlessly adjusts the navigation speed and the scale factor of the virtual environment based on the user's perceived navigation speed. The adjustment ensures an almost-constant perceived navigation speed while avoiding diplopia effects or diminished depth perception due to improper stereoscopic rendering configurations. The results from the conducted user evaluation shows that GiAnt is an efficient multi-scale navigation which minimizes the changes of the scale factor of the virtual environment compared to state-of-the-art multi-scale navigation techniques.

The main objective of this paper is to study and formalize the Take-Over Control in Collaborative Virtual Environments for Training (CVET). The Take-Over Control represents the transfer (the take over) of the interaction control of an object between two or more users. This paradigm is particularly useful for training scenarios, in which the interaction control could be continuously exchanged between the trainee and the trainer, e.g. the latter guiding and correcting the trainee's actions. The paper presents the formalization of the Take-Over Control followed by an illustration focusing in a use-case of collaborative maritime navigation. In the presented use-case, the trainee has to avoid an under-water obstacle with the help of a trainer who has additional information about the obstacle. The use-case allows to highlight the different elements a Take-Over Control situation should enforce, such as user's awareness. Different Take-Over Control techniques were provided and evaluated focusing on the transfer exchange mechanism and the visual feedback. The results show that participants preferred the Take-Over Control technique which maximized the user awareness.

In this paper, we investigate obstacle avoidance behavior during real walking in a large immersive projection setup. We analyze the walking behavior of users when avoiding real and virtual static obstacles. In order to generalize our study, we consider both anthropomorphic and inanimate objects, each having his virtual and real counterpart. The results showed that users exhibit different locomotion behaviors in the presence of real and virtual obstacles, and in the presence of anthropomorphic and inanimate objects. Precisely, the results showed a decrease of walking speed as well as an increase of the clearance distance (i. e., the minimal distance between the walker and the obstacle) when facing virtual obstacles compared to real ones. Moreover, our results suggest that users act differently due to their perception of the obstacle: users keep more distance when the obstacle is anthropomorphic compared to an inanimate object and when the orientation of anthropomorphic obstacle is from the profile compared to a front position. We discuss implications on future large shared immersive projection spaces.

We describe an experiment whose goal is to investigate the usage of different audio rendering techniques delivered through headphones while walking inside a wide four-side CAVE environment. In our experiment, participants had to physically walked along a virtual path exposed to different auditory stimuli. Each subject was exposed to three conditions: Stereo, Binaural sound spatially congruent with visual and binaural sound spatially incongruent with visuals and had to rate subjectively each. The results of the experiment showed increased preference ratings for the binaural audio rendering, followed by stereo rendering. As expected incongruent spatial cues were ranked significantly lower. Binaural rendering can deliver an increased immersive experience and do no require specialized hardware.

In four-sided CAVE-like VR systems, the absence of the rear wall has been shown to decrease the level of immersion and can introduce breaks in presence. With this experiment we analyse how user attention is diverted while physically walking in a virtual environment, when audio and/or visual attractors are present. Key features of the experiment are the fact that auditory feedback was delivered through binaural audio rendering dependent on the user's head position and orientation and that the four-sided CAVE used for the experiment allowed users to walk up to 9m in straight line. In the experiment we analysed how different “attractors” (audio and/or visual, static or dynamic) modify the user's attention. Results of the experiment show that audio-visual attractors are the most efficient attractors in order to keep the user's attention toward the inside of the CAVE.

In this paper we study how the visual animation of a self-avatar can be artificially modified in real-time in order to generate different haptic perceptions. In our experimental setup, participants could watch their self-avatar in a virtual environment in mirror mode while performing a weight lifting task. Users could map their gestures on the self-animated avatar in real-time using a Kinect. We introduce three kinds of modification of the visual animation of the self-avatar according to the effort delivered by the virtual avatar: 1) changes on the spatial mapping between the user’s gestures and the avatar, 2) different motion profiles of the animation, and 3) changes in the posture of the avatar (upper-body inclination). The experimental task consisted of a weight lifting task in which participants had to order four virtual dumbbells according to their virtual weight. The user had to lift each virtual dumbbells by means of a tangible stick, the animation of the avatar was modulated according to the virtual weight of the dumbbell. The results showed that the altering the spatial mapping delivered the best performance. Nevertheless, participants globally appreciated all the different visual effects. Our results pave the way to the exploitation of such novel techniques in various VR applications such as sport training, exercise games, or industrial training scenarios in single or collaborative mode.

We introduce the Elastic Images, a novel pseudo-haptic feedback technique which enables the perception of the local elasticity of images without the need of any haptic device. The proposed approach focus on whether visual feedback is able to induce a sensation of stiffness when the user interacts with an image using a standard mouse. The user, when clicking on a Elastic Image, is able to deform it locally according to its elastic properties. To reinforce the effect, we also propose the generation of procedural shadows and creases to simulate the compressibility of the image and several mouse cursors replacements to enhance pressure and stiffness perception. A psychophysical experiment was conducted to quantify this novel pseudo-haptic perception and determine its perceptual threshold (or its Just Noticeable Difference). The results showed that users were able to recognize up to eight different stiffness values with our proposed method and confirmed that it provides a perceivable and exploitable sensation of elasticity. The potential applications of the proposed approach range from pressure sensing in product catalogs and games, or its usage in graphical user interfaces for increasing the expressiveness of widgets.

The act of pointing to graphical elements is one of the fundamental tasks in Human-Computer Interaction. In this paper we analyze visual feedback techniques for accurate pointing on stereoscopic displays. Virtual feedback techniques must provide precise information about the pointing tool and its spatial relationship with potential targets. We show both analytically and empirically that current approaches provide poor feedback on stereoscopic displays, resulting in low user performance when accurate pointing is required. We propose a new feedback technique following a camera viewfinder metaphor. The key idea is to locally flatten the scene objects around the pointing direction to facilitate their selection. We present the results of a user study comparing cursor-based and ray-based visual feedback techniques with our approach. Our user studies indicate that our viewfinder metaphor clearly outperforms competing techniques in terms of user performance and binocular fusion.

Most pointing techniques for 3D selection on virtual environments rely on a ray originating at the user's hand whose direction is controlled by the hand orientation. In this paper we study the potential mismatch between visible objects (those which appear unoccluded from the user's eye position) and selectable objects (those which appear unoccluded from the user's hand position). We study the impact of such eye-hand visibility mismatch on selection performance, and propose a new technique for ray control which attempts to overcome this problem. We present an experiment to compare our ray control technique with classic raycasting in selection tasks with complex 3D scenes. Our user studies show promising results of our technique in terms of speed and accuracy.