Data visualization benefits from non-visual cues to enable people to understand information by engaging with it through multimodality, yet most approaches rely on cumbersome technologies or large scale artifacts, making them difficult to adapt to dynamic or complex datasets. In this paper, we explore the use of cutaneous haptics as a lightweight quantitative channel for visualization tasks, allowing users to feel data and interact with it dynamically. We present Haptixel, an open-source DIY encountered-type wearable providing force-feedback on the users’ fingertips’ pulp. We propose an interaction framework illustrating how Haptixel can be used to complement visualization tasks through combinations of force levels and contact types. We evaluate our approach in a pixel-art-like VR user study (n=16) where pixels color/height are associated to forces as a univariate value mapping. Results show that participants can retrieve information with Haptixel, and significantly discriminate 3D-data with at least four levels of forces; suggesting that cutaneous force-feedback can function for quantitative distinctions in visualization tasks.

Haptic interfaces enable shape simulation in virtual reality, making such content "tangible". Among existing haptic technologies, "grounded" interfaces simulate convincing grounded surfaces (like walls, tables, or other grounded shapes) but have limited workspaces, while "portable" interfaces are usable everywhere but struggle to display force feedback. In this paper, we formalize an alternative approach that we call "anchor-based haptics", which consists of portable interfaces capable of temporarily making contact with a real surface and leveraging it for grounded force feedback. To illustrate this approach, we present two new interfaces: T-Hank, which uses a reel-based extension mechanism to give height feedback from the surface, and R-Hank, which provides orientation feedback. The design of these interfaces is presented, and their potential interest in virtual reality has been investigated within two user studies by comparing them with vibrations in various 3D tasks. Overall, T-Hank and R-Hank enhance user experience for the exploration of 3D shapes and softness sensations. Taken together, results promote the use of the anchor-based haptic approach for VR applications in which both a large workspace and shape simulation play a prominent role.

Immersion in virtual reality is a powerful sensory experience. It often leads to striking situations, such as experiencing virtual contacts on your avatar, despite the absence of physical contact on your real body. However, as of today, it is still unknown whether purely-virtual touch remains fictive/imaginative, or whether our brain processes it the same way as real touch. In this paper, we conducted an experiment comparing purely visual touch simulation (a virtual robot arm touching the avatar’s hand) with a congruent visuo-tactile simulation (visual feedback is coupled with a real robot arm touching the participant’s hand in the physical world). Using this setup, we investigated the neural responses to purely-virtual and visuo-tactile touch, and examined how these responses were modulated by cognitive workload induced by tasks within the virtual environment. The analysis of the electroencephalography (EEG recording) revealed a strong activation of the sensorimotor cortex in both conditions, although stronger and faster when physical touch occurs. Our results also revealed a decrease in evoked potentials under high cognitive workload, with the reduction being more pronounced for physical touch. Our results show that a purely-virtual touch is processed similarly to a physical touch, i.e, engaging sensorimotor regions despite the absence of direct somatosensory stimulation. This suggests that the perception of the body in virtual environments can extend beyond simple illusions, involving neural processes similar to those engaged during real physical experiences. Besides, in the age of metaverses becoming accessible to a wider audience, knowing that our brain perceives virtual contacts in VR similarly to physical contacts further underlines the need to design immersive environments that prioritize physical integrity and user safety.

Virtual reality is becoming increasingly popular, and modern haptic equipment, such as vibrotactile suits, haptic gloves, and force-feedback controllers, offers new means of interaction within virtual environments, significantly enhancing user experience. When interacting with virtual objects, combined visual and haptic feedback simulates the physical sensations of grasping, lifting, or moving real objects. This sensorimotor feedback is essential for inducing a sense of presence and agency, yet it remains challenging to reproduce in the absence of reliable haptic cues. In this study, we design and evaluate several haptic metaphors using combinations of vibrotactile design parameters to simulate the lifting effort associated with light to heavy objects. These parameters include primitive signals, intensity, spatial density, propagation, and temporal density. Our contribution is threefold. First, we propose a method for modulating perceived physical effort by extending signal intensity with spatial and temporal density, which together reflect the effort required to lift an object. Second, we present a user study in which participants compared haptic effects and ranked them according to perceived lifting effort, comfort, and confidence, allowing us to assess the influence of each parameter. Third, we report the results of a second study in which participants evaluated vibrotactile effects when lifting different virtual objects. The findings confirm the importance of intensity and spatial density, as well as the influence of graphical representation on perceived effort. This research provides practical insights for designing haptic-enabled virtual reality systems and offers guidance for developers seeking to create more expressive and believable vibrotactile interactions.

Pseudo-haptics techniques are interesting alternatives for inducing haptic perceptions, achieved by manipulating haptic perception through the appropriate alteration of primarily visual feedback in response to body movements. However, the use of pseudo-haptics techniques with a motion-input system can sometimes be limited. This paper investigates a novel approach for extending the potential of pseudo-haptics techniques in virtual reality (VR), focusing on pseudoweight perception as the target case. The proposed approach utilizes a reaction force from force-input as a substitution of haptic cue for the pseudo-haptic perception. The paper introduced a manipulation method in which the vertical acceleration of the virtual hand is controlled by the extent of push-in of a force sensor. Such a force-input manipulation of a virtual body cannot only present pseudo-haptics with smaller physical spaces and be used by various users including physically handicapped people, but can present the reaction force proportional to the user's input to the user. We hypothesized that such a haptic force cue would contribute to the pseudo-haptic, here, the pseudoweight perception. Therefore, the paper endeavors to investigate the force-input pseudo-haptic perception in comparison with the motion-input pseudo-haptics. The paper compared force-input and motion-input manipulation in a point of achievable range and resolution of pseudo-weight. The experimental results suggest that forceinput manipulation successfully extends the range of perceptible pseudo-weight by 80% in comparison to the motion-input manipulation. On the other hand, it is revealed that motion-input manipulation has 1 step larger number of distinguishable weight levels and is easier to operate.

Vection, the visual illusion of self-motion, provides a strong marker of the VR user experience and plays an important role in both presence and cybersickness. Traditional measurements have been conducted using questionnaires, which exhibit inherent limitations due to their subjective nature and prevent real-time adjustments. Detecting vection in real time would allow VR systems to adapt to users' needs, improving comfort and minimizing negative effects like cybersickness. This paper investigates the presence of vection markers in electroencephalographic (EEG) brain signals using evoked potentials (brain responses to external stimuli). We designed a VR experiment that induces vection using two conditions: (1) forward acceleration or (2) backward acceleration. We recorded electroencephalographic (EEG) signals and gathered subjective reports on thirty (30) participants. We found an evoked potential of vection characterized by a positive peak around 600 ms (P600) after stimulus onset in the parietal region and a simultaneous negative peak in the frontal region. This result paves the way for the automatic detection of vection using EEG as well as a better understanding of vection. It also provides insights into the functional role of the visual system and its integration with the vestibular system during motion-perception. It has the potential to help enhance VR user experience by qualifying users' perceived vection and adapting the VR environments accordingly.

Does seeing avatars during social interaction in virtual reality (VR) affect the neural synchrony of interacting people? To address this question, we assessed inter-brain synchronization (IBS), which reflects synchronization of brain activities among interacting people and is linked to interaction success. We designed a social interaction task in VR where participants' appearances were represented by virtual cursors only or additional human avatars on them. Participants performed a finger position tracking interaction under both conditions. We compared the IBS across interaction conditions by collecting electroencephalography (EEG), positional data, and virtual embodiment questionnaires. The results from 30 participants (15 pairs) show that seeing the avatars resulted in significantly higher virtual body ownership but comparable agency and positional synchronization compared to the virtual cursors-only condition. In the inter-brain analysis, we found a significant increase in IBS strength and an increasing trend in the number of IBS in the alpha band after interacting with avatar-embodied virtual cursors. Correlation analyses further revealed a significant association between alpha-band IBS and subjective body ownership, and a trend with social inclusion scores. While the influence of visual complexity cannot be ruled out, these findings suggest that avatar-induced embodiment may contribute to enhanced inter-brain synchrony during VR-mediated social interaction.

Pseudo-haptics alters vision to evoke haptic perception without dedicated hardware. We introduce a method that reproduces sticky-surface interactions during attaching-detaching 3D direct manipulation in VR by blending three cues—motion gain, surfacedeformation, and vibration—in various combinations. Accurately sensing how strongly a surface “clings” is vital for realistic grasping, adhesion training and material evaluation, yet it has been largely overlooked in pseudo-haptic research. Three studies evaluated these cues. The first experiment compared individual and combined cues on perceived stickiness, confirming that additional cues reliably strengthened perceived stickiness. The second experiment tested how cue number affects tolerance for visual-physical mismatch, indicating that they lowered the minimum detectable threshold though they did not widen the overall tolerated mismatch. The third experiment measured whether and how much added cues sharpen perceptual resolution, showing multiple cues improved perceptual resolution by reducing just noticeable differences by 44 % ($1.8 \times$finer), doubling discriminable levels from roughly eight with a single cue to sixteen with all cues.

When walking, our head does not travel on a straight path but oscillates in a swaying pattern. This pattern has been implemented in Virtual Reality (VR) as "viewpoint oscillations" - which can be defined as periodic changes in position and/or orientation of the point of view to enhance walking simulations and make them feel closer to real walking. Viewpoint oscillations are especially beneficial when users cannot physically walk because of limitations of space or hardware, disability, or to avoid fatigue. In this paper, we provide new experimental work on the effects of viewpoint oscillations on walking sensation, as well as cybersickness and virtual embodiment, since such results are scarce in immersive VR, especially when using an avatar in first-person view. To do so, we also propose a technical improvement of viewpoint oscillations in embodied VR. Our technique makes use of an HMD-embedded gaze tracker to artificially add rotations that stabilize the target of the gaze in the users' field of view. We conducted a user study on 24 participants, which showed that our implementation of viewpoint oscillations successfully increased walking sensation and did not impact cybersickness or agency, compared to a linear motion without oscillations. In addition, a novel positive effect of stabilized viewpoint oscillations was found on virtual body ownership. As such, this study demonstrates the feasibility and viability of implementing gaze tracking-based stabilization with standard commercial HMDs, and, taken together, our results promote the use of viewpoint oscillations during walking simulations in embodied VR with an HMD.

Viewpoint oscillations are periodic changes in the position and/or orientation of the point of view in a virtual environment. They can be implemented in Virtual Reality (VR) walking simulations to make them feel closer to real walking. This is especially useful in simulations where users remain in place because of space or hardware constraints. As for today, it remains unclear what exact benefit they bring to user experience during walking simulations, and with what characteristics they should be implemented. To answer these questions, we conduct a systematic literature review focusing on five main dimensions of user experience (walking sensation, vection, cybersickness, presence and embodiment) and discuss 44 articles from the fields of VR, Vision, and Human-Computer Interaction. Overall, the literature suggests that viewpoint oscillations benefit vection, and with less evidence, walking sensation and presence. As for cybersickness, the literature contains contrasted results. Based on these results, we recommend using viewpoint oscillations in applications that require accurate distance or speed perception, or that aim to provide compelling walking simulations without a walking avatar, and a particular attention should be paid to cybersickness. Taken together, this work gives recommendations for enhancing walking simulations in VR, which may be applied to entertainment, virtual visits, and medical rehabilitation.

Humans experience social interactions partly through nonverbal communication, including proxemic behaviors and haptic sensations. Body language, facial expressions, personal spaces, and social touch are multiple factors influencing how a stranger's approach is experienced. Furthermore, the rise of virtual social platforms raises concerns about virtual harassment and the perception of personal space in VR: harassment is felt much more strongly in virtual spaces, and the psychological effects can be just as severe. While most virtual platforms have a 'personal bubble' feature that keeps strangers at a distance, it does not seem to suffice: personal space violations seem influenced by more than simply distance. With this paper, we aim to further clarify the variability of personal spaces. We focus on haptic stimulation, elaborating our hypotheses on the relationship between social touch and the perception of personal spaces. Users wore a haptic compression belt and were immersed in a virtual dark alley. Virtual agents approached them while exhibiting either neutral or threatening body language. In half of all trials, as the agent advanced, the compression belt tightened around the users' torsos with three different pressures. Participants could press a response button when uncomfortable with the agent's proximity. Peripersonal space violations occurred 31% earlier on average when the agent was visibly angry and the compression belt activated. A greater tightening pressure also slightly increased the personal sphere radius by up to 13%. Overall, our results are consistent with previous works on peripersonal spaces. They help further define our relationship to personal space boundaries and encourage using haptic devices during simulated social interactions in VR.

Abstract One singularity of VR is its capacity to generate a strong sensation of being present in a dangerous environment, without risking the physical consequences. However, this absence of consequences when interacting with virtual dangers might also limit the induction of realistic responses, such as avoidance behaviors, which are a key factor of various VR applications (e.g., training, journalism, or exposure therapies). To address this limitation, we propose avatar-centered feedback, a novel, device-free approach, consisting of dynamically altering the avatar’s appearance and movements to offer coherent feedback from interactions with the virtual environment, including dangers. To begin with, we present a design space clarifying the range of potential implementations for this approach. Then, we tested this approach in a metallurgy scenario, where participants’ virtual hands would redden and display burns as they got closer to a virtual fire (appearance alteration), and simulate short withdrawal reflexes movements when a spark burst next to them (movement alteration). Our results show that in comparison to a control group, participants receiving avatar-centered feedback demonstrated significantly more avoidance behaviors to the virtual fire. Interestingly, we also found that experiencing avatar-centered feedback of fire significantly increased avoidance behaviors toward a following danger of different nature (a circular saw). These results suggest that avatar-centered feedback can also impact the general perception of the avatar vulnerability to the virtual environment.

In this paper, we study how thermal haptic feedback can influence social interactions in virtual environments, with an emphasis on persuasion, focus, co-presence, and friendliness. Physical and social warmth have been repeatedly linked in psychological literature, which allows for speculations on the effect of thermal haptics on virtual social interactions. To that effect, we conducted a study on thermal feedback during simulated social interactions with a virtual agent. We tested three conditions: warm, cool, and neutral. Results showed that warm feedback positively influenced users’ perception of the agent and significantly enhanced persuasion and thermal comfort. Multiple users reported the agent feeling less ‘robotic’ and more ‘human’ during the warm condition. Moreover, multiple studies have previously shown the potential of vibrotactile feedback for social interactions. A second study thus evaluated the combination of warmth and vibrations for social interactions. The study included the same protocol and three similar conditions: warmth, vibrations, and warm vibrations. Warmth was perceived as more friendly, while warm vibrations heightened the agent’s virtual presence and persuasion. These results encourage the study of thermal haptics to support positive social interactions. Moreover, they suggest that some haptic feedback are more suited to certain types of social interactions and communication than others.

The IEEE VR Steering Committee congratulates and offers gratitude to the enormous efforts of the IEEE VR Conference Organizing Committees—with special recognition for the multiple years of hard work of the General Chairs, Carolina Cruz-Neira, Greg Welch, and Xubo Yang. To realize the conference requires a sizeable, motivated team of volunteers to work with the IEEE conference management team. Maintaining the conference’s status as the premier international virtual reality conference is a testament to the commitment of all the organizers and participants that make up the IEEE Virtual Reality community.

Haptic exoskeleton gloves are a widespread solution for providing force-feedback in Virtual Reality (VR), especially for 3D object manipulations. However, they are still lacking an important feature regarding in-hand haptic sensations: the palmar contact. In this paper, we present PalmEx, a novel approach which incorporates palmar force-feedback into exoskeleton gloves to improve the overall grasping sensations and manual haptic interactions in VR. PalmEx's concept is demonstrated through a self-contained hardware system augmenting a hand exoskeleton with an encountered palmar contact interface - physically encountering the users' palm. We build upon current taxonomies to elicit PalmEx's capabilities for both the exploration and manipulation of virtual objects. We first conduct a technical evaluation optimising the delay between the virtual interactions and their physical counterparts. We then empirically evaluate PalmEx's proposed design space in a user study (n=12) to assess the potential of a palmar contact for augmenting an exoskeleton. Results show that PalmEx offers the best rendering capabilities to perform believable grasps in VR. PalmEx highlights the importance of the palmar stimulation, and provides a low-cost solution to augment existing high-end consumer hand exoskeletons.

Pseudo-haptic techniques are used to modify haptic perception by appropriately changing visual feedback to body movements. Based on the knowledge that tendon vibration can affect our somatosensory perception, this article proposes a method for leveraging tendon vibration to enhance pseudo-haptics during free arm motion. Three experiments were performed to examine the impact of tendon vibration on the range and resolution of pseudo-haptics. The first experiment investigated the effect of tendon vibration on the detection threshold of the discrepancy between visual and physical motion. The results indicated that vibrations applied to the inner tendons of the wrist and elbow increased the threshold, suggesting that tendon vibration can augment the applicable visual motion gain by approximately 13% without users detecting the visual/physical discrepancy. Furthermore, the results demonstrate that tendon vibration acts as noise on haptic motion cues. The second experiment assessed the impact of tendon vibration on the resolution of pseudo-haptics by determining the just noticeable difference in pseudo-weight perception. The results suggested that the tendon vibration does not largely compromise the resolution of pseudo-haptics. The third experiment evaluated the equivalence between the weight perception triggered by tendon vibration and that by visual motion gain, that is, the point of subjective equality. The results revealed that vibration amplifies the weight perception and its effect was equivalent to that obtained using a gain of 0.64 without vibration, implying that the tendon vibration also functions as an additional haptic cue. Our results provide design guidelines and future work for enhancing pseudo-haptics with tendon vibration.

Walking-in-place (WIP) is a locomotion technique that enables users to "walk infinitely" through vast virtual environments using walking-like gestures within a limited physical space. This article investigates alternative interaction schemes for WIP, addressing successively the control, input, and output of WIP. First, we introduce a novel height-based control to increase advanced speed. Second, we introduce a novel input system for WIP based on elastic and passive strips. Third, we introduce the use of pseudo-haptic feedback as a novel output for WIP meant to alter walking sensations. The results of a series of user studies show that height and frequency based control of WIP can facilitate higher virtual speed with greater efficacy and ease than in frequency-based WIP. Second, using an upward elastic input system can result in a stable virtual speed control, although excessively strong elastic forces may impact the usability and user experience. Finally, using a pseudo-haptic approach can improve the perceived realism of virtual slopes. Taken together, our results suggest that, for future VR applications, there is value in further research into the use of alternative interaction schemes for walking-in-place.

During mid-air interactions, common approaches (such as the god-object method) typically rely on visually constraining the user's avatar to avoid visual interpenetrations with the virtual environment in the absence of kinesthetic feedback. This paper explores two methods which influence how the position mismatch (positional offset) between users' real and virtual hands is recovered when releasing the contact with virtual objects. The first method (sticky) constrains the user's virtual hand until the mismatch is recovered, while the second method (unsticky) employs an adaptive offset recovery method. In the first study, we explored the effect of positional offset and of motion alteration on users' behavioral adjustments and users' perception. In a second study, we evaluated variations in the sense of embodiment and the preference between the two control laws. Overall, both methods presented similar results in terms of performance and accuracy, yet, positional offsets strongly impacted motion profiles and users' performance. Both methods also resulted in comparable levels of embodiment. Finally, participants usually expressed strong preferences toward one of the two methods, but these choices were individual-specific and did not appear to be correlated solely with characteristics external to the individuals. Taken together, these results highlight the relevance of exploring the customization of motion control algorithms for avatars.

Consumer-grade virtual reality (VR) controllers are typically equipped with one vibrotactile actuator, allowing to create simple and non-spatialized tactile sensations through the vibration of the entire controller. Leveraging the funneling effect, an illusion in which multiple vibrations are perceived as a single one, we propose MultiVibes, a VR controller capable of rendering spatialized sensations at different locations on the user’s hand and fingers. The designed prototype includes ten vibrotactile actuators, directly in contact with the skin of the hand, limiting the propagation of vibrations through the controller. We evaluated MultiVibes through two controlled experiments. The first one focused on the ability of users to recognize spatio-temporal patterns, while the second one focused on the impact of MultiVibes on the users’ haptic experience when interacting with virtual objects they can feel. Taken together, the results show that MultiVibes is capable of providing accurate spatialized feedback and that users prefer MultiVibes over recent VR controllers.

Kinesthetic motor imagery (KMI) based brain-computer interfaces hold great potential for post-stroke motor rehabilitation. However, KMI is a complex task to perform, mainly due to the absence of sensory and kinesthetic feedback. To address this issue, sensory neurofeedback solutions have been proposed. One of them is tactile vibration, which has been used in BCI paradigms, but the optimal design choices of the vibration remain unclear. In this study, we present a novel bimodal stimulus that combines a vibrotactile device for the upper limb with a visual animation of a hand grasping a bottle. We aimed to investigate the effects of four different vibration patterns and four stimulation intensities to study the effects of the stimuli on the electroencephalography (EEG) signals of 17 healthy subjects without performing KMI. Our findings revealed EEG activity in the alpha, beta, and delta frequency bands within the somatosensory cortex during the sensory stimulation. While we did not observe significant differences in brain activity among the vibration patterns, we did observe statistically significant variations based on the vibration intensity. We observed central activity in both brain hemispheres, with stronger activity occurring in the contralateral hemisphere. Additionally, EEG activity seemed to involve the contralateral occipital areas. Based on these results, it is worth considering the delivery of feedback after participants have completed KMI tasks to differentiate and comprehensively study the brain activity resulting from the mental and sensory stimulation tasks.

The IEEE VR Steering Committee congratulates and offers gratitude to the enormous efforts of the IEEE VR Conference Organizing Committees — with special recognition for the multiple years of hard work of the General Chairs, Xubo Yang, Kun Zhou, Tobias Langlotz, and Stephan Lukosch. To realize the conference requires a sizeable, motivated team of volunteers to work with the IEEE conference management team. Maintaining the conference's status as the premier international virtual reality conference is a testament to the commitment of all the organizers and participants that make up the IEEE Virtual Reality community.

Virtual avatars are more and more often featured in Virtual Reality (VR) and Augmented Reality (AR) applications. When embodying a virtual avatar, one may desire to change of appearance over the course of the embodiment. However, switching suddenly from one appearance to another can break the continuity of the user experience and potentially impact the sense of embodiment (SoE), especially when the new appearance is very different. In this paper, we explore how applying smooth visual transitions at the moment of the change can help to maintain the SoE and benefit the general user experience. To address this, we implemented an AR system allowing users to embody a regular-shaped avatar that can be transformed into a muscular one through a visual effect. The avatar's transformation can be triggered either by the user through physical action (“active” transition), or automatically launched by the system (“passive” transition). We conducted a user study to evaluate the effects of these two types of transformations on the SoE by comparing them to control conditions where there was no visual feedback of the transformation. Our results show that changing the appearance of one's avatar with an active transition (with visual feedback), compared to a passive transition, helps to maintain the user's sense of agency, a component of the SoE. They also partially suggest that the Proteus effects experienced during the embodiment were enhanced by these transitions. Therefore, we conclude that visual effects controlled by the user when changing their avatar's appearance can benefit their experience by preserving the SoE and intensifying the Proteus effects.

In Virtual Reality (VR), a growing number of applications involve verbal communications with avatars, such as for teleconference, entertainment, virtual training, social networks, etc. In this context, our paper aims to investigate how tactile feedback consisting in vibrations synchronized with speech could influence aspects related to VR social interactions such as persuasion, co-presence and leadership. We conducted two experiments where participants embody a first-person avatar attending a virtual meeting in immersive VR. In the first experiment, participants were listening to two speaking virtual agents and the speech of one agent was augmented with vibrotactile feedback. Interestingly, the results show that such vibrotactile feedback could significantly improve the perceived co-presence but also the persuasiveness and leadership of the haptically-augmented agent. In the second experiment, the participants were asked to speak to two agents, and their own speech was augmented or not with vibrotactile feedback. The results show that vibrotactile feedback had again a positive effect on co-presence, and that participants perceive their speech as more persuasive in presence of haptic feedback. Taken together, our results demonstrate the strong potential of haptic feedback for supporting social interactions in VR, and pave the way to novel usages of vibrations in a wide range of applications in which verbal communication plays a prominent role.

Pseudo-Haptic techniques, or visuo-haptic illusions, leverage user's visual dominance over haptics to alter the users' perception. As they create a discrepancy between virtual and physical interactions, these illusions are limited to a perceptual threshold. Many haptic properties have been studied using pseudo-haptic techniques, such as weight, shape or size. In this paper, we focus on estimating the perceptual thresholds for pseudo-stiffness in a virtual reality grasping task. We conducted a user study (n = 15) where we estimated if compliance can be induced on a non-compressible tangible object and to what extent. Our results show that (1) compliance can be induced in a rigid tangible object and that (2) pseudo-haptics can simulate beyond 24 N/cm stiffness (k > 24N/cm, between a gummy bear and a raisin, up to rigid objects). Pseudo-stiffness efficiency is (3) enhanced by the objects' scales, but mostly (4) correlated to the user input force. Taken altogether, our results offer novel opportunities to simplify the design of future haptic interfaces, and extend the haptic properties of passive props in VR.

In virtual reality, the avatar - the user's digital representation - is an important element which can drastically influence the immersive experience. In this paper, we especially focus on the use of "dissimilar" avatars i.e., avatars diverging from the real appearance of the user, whether they preserve an anthropomorphic aspect or not. Previous studies reported that dissimilar avatars can positively impact the user experience, in terms for example of interaction, perception or behaviour. However, given the sparsity and multi-disciplinary character of research related to dissimilar avatars, it tends to lack common understanding and methodology, hampering the establishment of novel knowledge on this topic. In this paper, we propose to address these limitations by discussing: (i) a methodology for dissimilar avatars characterization, (ii) their impacts on the user experience, (iii) their different fields of application, and finally, (iv) future research direction on this topic. Taken together, we believe that this paper can support future research related to dissimilar avatars, and help designers of VR applications to leverage dissimilar avatars appropriately.

Virtual reality (VR) provides users with the ability to substitute their physical appearance by embodying virtual characters (avatars) using head-mounted displays and motion-capture technologies. Previous research demonstrated that the sense of embodiment toward an avatar can impact user behavior and cognition. In this paper, we present an experiment designed to investigate whether embodying a well-known creative genius could enhance participants' creative performance. Following a preliminary online survey ( N = 157) to select a famous character suited to the purpose of this study, we developed a VR application allowing participants to embody Leonardo da Vinci or a self-avatar. Self-avatars were approximately matched with participants in terms of skin tone and morphology. 40 participants took part in three tasks seamlessly integrated in a virtual workshop. The first task was based on a Guilford's Alternate Uses test (GAU) to assess participants' divergent abilities in terms of fluency and originality. The second task was based on a Remote Associates Test (RAT) to evaluate convergent abilities. Lastly, the third task consisted in designing potential alternative uses of an object displayed in the virtual environment using a 3D sketching tool. Participants embodying Leonardo da Vinci demonstrated significantly higher divergent thinking abilities, with a substantial difference in fluency between the groups. Conversely, participants embodying a self-avatar performed significantly better in the convergent thinking task. Taken together, these results promote the use of our virtual embodiment approach, especially in applications where divergent creativity plays an important role, such as design and innovation.

In this paper, we investigate the use of a motorized bike to support the walk of a self-avatar in virtual reality (VR). While existing walking-in-place (WIP) techniques render compelling walking experiences, they can be judged repetitive and strenuous. Our approach consists in assisting a WIP technique so that the user does not have to actively move in order to reduce effort and fatigue. We chose to assist a technique called walking-by-cycling, which consists in mapping the cycling motion of a bike onto the walking of the user's self-avatar, by using a motorized bike. We expected that our approach could provide participants with a compelling walking experience while reducing the effort required to navigate. We conducted a within-subjects study where we compared "assisted walking-by-cycling" to a traditional active walking-by-cycling implementation, and to a standard condition where the user is static. In the study, we measured embodiment, including ownership and agency, walking sensation, perceived effort and fatigue. Results showed that assisted walking-by-cycling induced more ownership, agency, and walking sensation than the static simulation. Additionally, assisted walking-by-cycling induced levels of ownership and walking sensation similar to that of active walking-by-cycling, but it induced less perceived effort. Taken together, this work promotes the use of assisted walking-by-cycling in situations where users cannot or do not want to exert much effort while walking in embodied VR such as for injured or disabled users, for prolonged uses, medical rehabilitation, or virtual visits.

Technological developments provide solutions to alleviate the tremendous impact on the health and autonomy due to the impact of dementia on navigation abilities. We systematically reviewed the literature on devices tested to provide assistance to people with dementia during indoor, outdoor and virtual navigation (PROSPERO ID number: 215585). Medline and Scopus databases were searched from inception. Our aim was to summarize the results from the literature to guide future developments. Twenty-three articles were included in our study. Three types of information were extracted from these studies. First, the types of navigation advice the devices provided were assessed through: (i) the sensorial modality of presentation, e.g., visual and tactile stimuli, (ii) the navigation content, e.g., landmarks, and (iii) the timing of presentation, e.g., systematically at intersections. Second, we analyzed the technology that the devices were based on, e.g., smartphone. Third, the experimental methodology used to assess the devices and the navigation outcome was evaluated. We report and discuss the results from the literature based on these three main characteristics. Finally, based on these considerations, recommendations are drawn, challenges are identified and potential solutions are suggested. Augmented reality-based devices, intelligent tutoring systems and social support should particularly further be explored.

A seated user watching his avatar walking in Virtual Reality (VR) may have an impression of walking. In this paper, we show that such an impression can be extended to other postures and other locomotion exercises. We present two user studies in which participants wore a VR headset and observed a first-person avatar performing virtual exercises. In the first experiment, the avatar walked and the participants (n=36) tested the simulation in 3 different postures (standing, sitting and Fowler's posture). In the second experiment, other participants (n=18) were sitting and observed the avatar walking, jogging or stepping over virtual obstacles. We evaluated the impression of locomotion by measuring the impression of walking (respectively jogging or stepping) and embodiment in both experiments. The results show that participants had the impression of locomotion in either sitting, standing and Fowler's posture. However, Fowler's posture significantly decreased both the level of embodiment and the impression of locomotion. The sitting posture seems to decrease the sense of agency compared to standing posture. Results also show that the majority of the participants experienced an impression of locomotion during the virtual walking, jogging, and stepping exercises. The embodiment was not influenced by the type of virtual exercise. Overall, our results suggest that an impression of locomotion can be elicited in different users' postures and during different virtual locomotion exercises. They provide valuable insight for numerous VR applications in which the user observes a self-avatar moving, such as video games, gait rehabilitation, training, etc.

We investigate how underfoot vibrotactile feedback can be used to increase the impression of walking and embodiment of static users represented by a first- or third-person avatar. We designed a multi-sensory setup involving avatar displayed on an HMD, and a set of vibrotactile effects displayed at every footstep. In a first study (N = 44), we compared the impression of walking in 3 vibrotactile conditions : 1) with a "constant" vibrotactile rendering reproducing simple contact information, 2) with a more sophisticated "phase-based" vibrotactile rendering the successive contacts of a walking cycle and 3) without vibrotactile feedback. The results show that overall both constant and phase-based rendering significantly improve the impression of walking in first and third-person perspective. Interestingly, the more realistic phase-based rendering seems to increase significantly the impression of walking in the third-person condition, but not in the first-person condition. In a second study (N=28), we evaluated the embodiment towards first- and third-person avatar while receiving no vibrotactile feedback or by receiving vibrotactile feedback. The results show that vibrotactile feedback improves embodiment in both perspectives of the avatar. Taken together, our results support the use of vibrotactile feedback when users observe first- and third-person avatar. They also suggest that constant and phase-based rendering could be used with first-person avatar and support the use of phase-based rendering with third-person avatar. They provide valuable insight for stimulations in any VR applications in which the impression of walking is prominent such as for virtual visits, walking rehabilitation, video games, etc.

In Virtual Reality, self-touch (ST) stimulation is a promising method of sense of body ownership (SoBO) induction that does not require an external effector. However, its applicability to dysmorphic bodies has not been explored yet and remains uncertain due to the requirement to provide incongruent visuomotor sensations. In this, paper, we studied the effect of ST stimulation on dysmorphic hands via haptic retargeting, as compared to a classical external-touch (ET) stimulation, on the SoBO. Our results indicate that ST can induce similar levels of dysmorphic SoBO than ET stimulation, but that some types of dysmorphism might decrease the ST stimulation accuracy due to the nature of the re-targeting that they induce.

This work explores a new usage of Augmented Reality (AR) to extend perception and interaction within physical areas ahead of ourselves. To do so, we propose to detach ourselves from our physical position by creating a controllable “digital copy”; of our body that can be used to navigate in local space from a third-person perspective. With such a viewpoint, we aim to improve our mental representation of distant space and understanding of action possibilities (called affordances), without requiring us to physically enter this space. Our approach relies on AR to virtually integrate the user’s body in remote areas in the form of an avatar. We discuss concrete application scenarios and propose several techniques to manipulate avatars in the third person as a part of a larger conceptual framework. Finally, through a user study employing one of the proposed techniques (puppeteering), we evaluate the validity of using third-person embodiment to extend our perception of the real world to areas outside of our proximal zone. We found that this approach succeeded in enhancing the user’s accuracy and confidence when estimating their action capabilities at distant locations.

When designing virtual embodiment studies, one of the key choices is the nature of the experimental factors, either between-subjects or within-subjects. However, it is well known that each design has ad-vantages and disadvantages in terms of statistical power, sample size requirements and confounding factors. This paper reports a within-subjects experiment with 92 participants comparing self-reported embodiment scores under a visuomotor task with two conditions: synchronous motions and asynchronous motions with a latency of 300 ms. With the gathered data, using a Monte-Carlo method, we created numerous simulations of within- and between-subjects experiments by selecting subsets of the data. In particular, we explored the impact of the number of participants on the replicability of the results from the 92 within-subjects experiment. For the between-subjects simulations, only the first condition for each user was considered to create the simulations. The results showed that while the replicability of the results increased as the number of participants increased for the within-subjects simulations, no matter the number of participants, between-subjects simulations were not able to replicate the initial results. We discuss the potential reasons that could have led to this surprising result and potential methodological practices to mitigate them.

In absence of haptic feedback, the perception of contact with virtual objects can rapidly become a problem in extended reality (XR) applications. XR developers often rely on visual feedback to inform the user and display contact information. However, as for today, there is no clear path on how to design and assess such visual techniques. In this paper, we propose a design space for the creation of visual feedback techniques meant to represent contact with virtual surfaces in XR. Based on this design space, we conceived a set of various visual techniques, including novel approaches based on onomatopoeia and inspired by cartoons, or visual effects based on physical phenomena. Then, we conducted an online preliminary user study with 60 participants, consisting in assessing 6 visual feedback techniques in terms of user experience. We could notably assess, for the first time, the potential influence of the interaction context by comparing the participants’ answers in two different scenarios: industrial versus entertainment conditions. Taken together, our design space and initial results could inspire XR developers for a wide range of applications in which the augmentation of contact seems prominent, such as for vocational training, industrial assembly/maintenance, surgical simulation, videogames, etc.

In virtual reality, several manipulation techniques distort users' motions, for example to reach remote objects or increase precision. These techniques can become problematic when used with avatars, as they create a mismatch between the real performed action and the corresponding displayed action, which can negatively impact the sense of embodiment. In this paper, we propose to use a dual representation during anisomorphic interaction. A co-located representation serves as a spatial reference and reproduces the exact users' motion, while an interactive representation is used for distorted interaction. We conducted two experiments, investigating the use of dual representations with amplified motion (with the Go-Go technique) and decreased motion (with the PRISM technique). Two visual appearances for the interactive representation and the co-located one were explored. This exploratory study investigating dual representations in this context showed that people globally preferred having a single representation, but opinions diverged for the Go-Go technique. Also, we could not find significant differences in terms of performance. While interacting seemed more important than showing exact movements for agency during out-of-reach manipulation, people felt more in control of the realistic arm during close manipulation.

Virtual self-avatars have been increasingly used in Augmented Reality (AR) where one can see virtual content embedded into physical space. However, little is known about the perception of self-avatars in such a context. The possibility that their embodiment could be achieved in a similar way as in Virtual Reality opens the door to numerous applications in education, communication, entertainment, or the medical field. This article aims to review the literature covering the embodiment of virtual self-avatars in AR. Our goal is (i) to guide readers through the different options and challenges linked to the implementation of AR embodiment systems, (ii) to provide a better understanding of AR embodiment perception by classifying the existing knowledge, and (iii) to offer insight on future research topics and trends for AR and avatar research. To do so, we introduce a taxonomy of virtual embodiment experiences by defining a "body avatarization" continuum. The presented knowledge suggests that the sense of embodiment evolves in the same way in AR as in other settings, but this possibility has yet to be fully investigated. We suggest that, whilst it is yet to be well understood, the embodiment of avatars has a promising future in AR and conclude by discussing possible directions for research.

In Virtual Reality (VR), users can be immersed in emotionally intense and cognitively engaging experiences. Yet, despite strong interest from scholars and a large amount of work associating VR and Affective and Cognitive States (ACS), there is a clear lack of structured and systematic form in which this research can be classified. We define "Affective and Cognitive VR" to relate to works which (1) induce ACS, (2) recognize ACS, or (3) exploit ACS by adapting virtual environments based on ACS measures. This survey clarifies the different models of ACS, presents the methods for measuring them with their respective advantages and drawbacks in VR, and showcases Affective and Cognitive VR studies done in an Immersive Virtual Environment (IVE) in a non-clinical context. Our article covers the main research lines in Affective and Cognitive VR. We provide a comprehensive list of references with the analysis of 63 research articles and summarize future works directions.

In this paper we explore the multi-sensory display of self-avatars' physiological state in Virtual Reality (VR), as a means to enhance the connection between the users and their avatar. Our approach consists in designing and combining a coherent set of visual, auditory and haptic cues to represent the avatar's cardiac and respiratory activity. These sensory cues are modulated depending on the avatar's simulated physical exertion. We notably introduce a novel haptic technique to represent respiratory activity using a compression belt simulating abdominal movements that occur during a breathing cycle. A series of experiments was conducted to evaluate the influence of our multi-sensory rendering techniques on various aspects of the VR user experience, including the sense of virtual embodiment and the sensation of effort during a walking simulation. A first study ($\mathrm{N}=30$) that focused on displaying cardiac activity showed that combining sensory modalities significantly enhances the sensation of effort. A second study ($\mathrm{N}=20$) that focused on respiratory activity showed that combining sensory modalities significantly enhances the sensation of effort as well as two sub-components of the sense of embodiment. Interestingly, the user's actual breathing tended to synchronize with the simulated breathing, especially with the multi-sensory and haptic displays. A third study ($\mathrm{N}=18$) that focused on the combination of cardiac and respiratory activity showed that combining both rendering techniques significantly enhances the sensation of effort. Taken together, our results promote the use of our novel breathing display technique and multi-sensory rendering of physiological parameters in VR applications where effort sensations are prominent, such as for rehabilitation, sport training, or exergames.

Avatars, the users’ virtual representations, are becoming ubiquitous in virtual reality applications. In this context, the avatar becomes the medium which enables users to manipulate objects in the virtual environment. It also becomes the users’ main spatial reference, which can not only alter their interaction with the virtual environment, but also the perception of themselves. In this paper, we review and analyse the current state-of-the-art for 3D object manipulation and the sense of embodiment. Our analysis is twofold. First, we discuss the impact that the avatar can have on object manipulation. Second, we discuss how the different components of a manipulation technique (i.e. input, control and feedback) can influence the user’s sense of embodiment. Throughout the analysis, we crystallise our discussion with practical guidelines for VR application designers and we propose several research topics towards “avatar-friendly’’ manipulation techniques.

In this article, we introduce a concept called "virtual co-embodiment", which enables a user to share their virtual avatar with another entity (e.g., another user, robot, or autonomous agent). We describe a proof-of-concept in which two users can be immersed from a first-person perspective in a virtual environment and can have complementary levels of control (total, partial, or none) over a shared avatar. In addition, we conducted an experiment to investigate the influence of users' level of control over the shared avatar and prior knowledge of their actions on the users' sense of agency and motor actions. The results showed that participants are good at estimating their real level of control but significantly overestimate their sense of agency when they can anticipate the motion of the avatar. Moreover, participants performed similar body motions regardless of their real control over the avatar. The results also revealed that the internal dimension of the locus of control, which is a personality trait, is negatively correlated with the user's perceived level of control. The combined results unfold a new range of applications in the fields of virtual-reality-based training and collaborative teleoperation, where users would be able to share their virtual body.

In this article, we propose an approach towards an infinite surface haptic display. Our approach, named ENcountered-Type ROtating Prop Approach (ENTROPiA) is based on a cylindrical spinning prop attached to a robot's end-effector serving as an encountered-type haptic display (ETHD). This type of haptic display permits the users to have an unconstrained, free-hand contact with a surface being provided by a robotic device for the users' to encounter a surface to be touched. In our approach, the sensation of touching a virtual surface is given by an interaction technique that couples with the sliding movement of the prop under the users' finger by tracking their hand location and establishing a path to be explored. This approach enables large motion for a larger surface rendering, permits to render multi-textured haptic feedback, and leverages the ETHD approach introducing large motion and sliding/friction sensations. As a part of our contribution, a proof of concept was designed for illustrating our approach. A user study was conducted to assess the perception of our approach showing a significant performance for rendering the sensation of touching a large flat surface. Our approach could be used to render large haptic surfaces in applications such as rapid prototyping for automobile design.

We combine tracking information from a tangible object instrumented with capacitive sensors and an optical tracking system, to improve contact rendering when interacting with tangibles in VR. A human-subject study shows that combining capacitive sensing with optical tracking significantly improves the visuohaptic synchronization and immersion of the VR experience.

While digital humans are key aspects of the rapidly evolving areas of virtual reality, gaming, and online communications, many applications would benefit from using digital personalized (stylized) representations of users, as they were shown to highly increase immersion, presence and emotional response. In particular, depending on the target application, one may want to look like a dwarf or an elf in a heroic fantasy world, or like an alien on another planet, in accordance with the style of the narrative. While creating such virtual replicas requires stylization of the user’s features onto the virtual character, no formal study has however been conducted to assess the ability to recognize stylized characters. In this paper, we present a perceptual study investigating the effect of the degree of stylization on the ability to recognize an actor, and the subjective acceptability of stylizations. Results show that recognition rates decrease when the degree of stylization increases, while acceptability of the stylization increases. These results provide recommendations to achieve good compromises between stylization and recognition, and pave the way to new stylization methods providing a tradeoff between stylization and recognition of the actor.

Virtual embodiment and navigation are two topics widely studied in the virtual reality community. Despite the potential inter-relation between embodiment and locomotion, studies on virtual navigation rarely supply users with a virtual representation, while studies on virtual embodiment rarely allow users to virtually navigate. This paper therefore explores this potential inter-relation by focusing on the two following questions: Does the locomotion technique have an impact on the user's sense of embodiment? Does embodying an avatar have an impact on the user's preference and performance depending on the locomotion technique?To address these questions, we conducted a user study (N=60) exploring the relationship between the locomotion technique and the user's sense of embodiment over a virtual avatar seen from a first-person perspective. Three widely used locomotion techniques were evaluated: real walking, walking-in-place and virtual steering. All participants performed four different tasks involving a different awareness of their virtual avatar. Participants also performed the same tasks without being embodied in an avatar. The results show that participants had a comparable sense of embodiment with all techniques when embodied in an avatar, and that the presence or absence of the virtual avatar did not alter their performance while navigating, independently of the technique. Taken together, our results represent a first attempt to qualify the inter-relation between virtual navigation and virtual embodiment, and suggest that the 3D locomotion technique used has little influence on the user's sense of embodiment in VR.

This paper describes an “all-in-one” solution for the real-time recognition of users' mental workloads in virtual reality through the customization of a commercial HMD with physiological sensors. First, we describe the hardware and software solution employed to build the system. Second, we detail the machine learning methods used for the automatic recognition of the users' mental workload, which are based on the well-known Random Forest algorithm. In order to gather data to train the system, we conducted an extensive user study with 75 participants using a VR flight simulator to induce different levels of mental workload. In contrast to previous works which label the data based on a standardized task (e.g., n-back task) or on a pre-defined task-difficulty, participants were asked about their perceived mental workload level along the experiment. With the data collected, we were able to train the system in order to classify four different levels of mental workload with an accuracy up to 65%. In addition, we discuss the role of the signal normalization procedures, the contribution of the different physiological signals on the recognition accuracy and compare the results obtained with the sensors embedded in the HMD with commercial grade systems. Preliminary results show our pipeline is able to recognize mental workload in real-time. Taken together, our results suggest that such all-in-one approach, with physiological sensors directly embedded in the HMD, is a promising path for VR applications in which the real-time or off-line estimation of Mental Workload assessment is beneficial.

Commonly used Head Mounted Displays (HMDs) in Augmented Reality (AR), namely Optical See-Through (OST) displays, suffer from a main drawback: their focal lenses can only provide a fixed focal distance. Such a limitation is suspected to be one of the main factors for distance misperception in AR. In this paper, we studied the use of an emerging new kind of AR display to tackle such perception issues: Retinal Projection Displays (RPDs). With RPDs, virtual images have no focal distance and the AR content is always in focus. We conducted the first reported experiment evaluating egocentric distance perception of observers using Retinal Projection Displays. We compared the precision and accuracy of the depth estimation between real and virtual targets, displayed by either OST HMDs or RPDs. Interestingly, our results show that RPDs provide depth estimates in AR closer to real ones compared to OST HMDs. Indeed, the use of an OST device was found to lead to an overestimation of the perceived distance by 16%, whereas the distance overestimation bias dropped to 4% with RPDs. Besides, the task was reported with the same level of difficulty and no difference in precision. As such, our results shed the first light on retinal projection displays' benefits in terms of user's perception in Augmented Reality, suggesting that RPD is a promising technology for AR applications in which an accurate distance perception is required.

Training is one of the major use cases of Virtual Reality (VR) due to the flexibility and reproducibility of VR simulations. However, the use of the user’s cognitive state, and in particular mental workload (MWL), remains largely unexplored in the design of training scenarios. In this paper, we propose to consider MWL for the design of complex training scenarios involving multiple parallel tasks in VR. The proposed approach is based on the assessment of the MWL elicited by each potential task configuration in the training application. Following the assessment, the resulting model is then used to create training scenarios able to modulate the user’s MWL over time. This approach is illustrated by a VR flight training simulator based on the Multi-Attribute Task Battery II, which solicits different cognitive resources, able to generate 12 different tasks configurations. A first user study (N = 38) was conducted to assess the MWL for each task configuration using self-reports and performance measurements. This assessment was then used to generate three training scenarios in order to induce different levels of MWL over time. A second user study (N = 14) confirmed that the proposed approach was able to induce the expected mental workload over time for each training scenario. These results pave the way to further studies exploring how MWL modulation can be used to improve VR training applications.

Encountered-Type Haptic Displays (ETHDs) represent a field of haptic displays with the premise of not using any type of actuator directly in contact with the user skin, thus providing an alternative integration of haptic displays in virtual environments. In this paper, we present novel interaction techniques (ITs) dedicated to ETHDs. The techniques aim at addressing the issues commonly presented for these devices such as limited contact areas, lags and unexpected collisions with the user. First, our paper proposes a design framework based on several parameters defining the interactive process between user and ETHD (input, movement control, displacement and contact). Five techniques based on different ramifications of the design space framework were conceived, respectively named: Swipe, Drag, Clutch, Bubble and Follow. Then, a use-case scenario was designed to depict the usage of these techniques on the task of touching and coloring a wide, flat surface. Finally, a user study based on the coloring task was conducted to assess the performance and user experience for each IT. Results were in favor of Drag and Clutch techniques which are based on manual surface displacement, absolute position selection and intermittent contact interaction. Taken together our results and design methodology pave the way to the design of future ITs for ETHDs in virtual environments.

When persons interact with the environment and experience or witness an error (e.g. an unexpected event), a specific brain pattern, known as error-related potential (ErrP) can be observed in the electroencephalographic signals (EEG). Virtual Reality (VR) technology enables users to interact with computer-generated simulated environments and to provide multi-modal sensory feedback. Using VR systems can, however, be error-prone. In this paper, we investigate the presence of ErrPs when Virtual Reality users face 3 types of visualization errors: (Te) tracking errors when manipulating virtual objects, (Fe) feedback errors, and (Be) background anomalies. We conducted an experiment in which 15 participants were exposed to the 3 types of errors while performing a center-out pick and place task in virtual reality. The results showed that tracking errors generate error-related potentials, the other types of errors did not generate such discernible patterns. In addition, we show that it is possible to detect the ErrPs generated by tracking losses in single trial, with an accuracy of 85%. This constitutes a first step towards the automatic detection of error-related potentials in VR applications, paving the way to the design of adaptive and self-corrective VR/AR applications by exploiting information directly from the user’s brain.

In Virtual Reality, a number of studies have been conducted to assess the influence of avatar appearance, avatar control and user point of view on the Sense of Embodiment (SoE) towards a virtual avatar. However, such studies tend to explore each factor in isolation. This paper aims to better understand the inter-relations among these three factors by conducting a subjective matching experiment. In the presented experiment ( n=40), participants had to match a given "optimal" SoE avatar configuration (realistic avatar, full-body motion capture, first-person point of view), starting by a "minimal" SoE configuration (minimal avatar, no control, third-person point of view), by iteratively increasing the level of each factor. The choices of the participants provide insights about their preferences and perception over the three factors considered. Moreover, the subjective matching procedure was conducted in the context of four different interaction tasks with the goal of covering a wide range of actions an avatar can do in a VE. The paper also describes a baseline experiment ( n=20) which was used to define the number and order of the different levels for each factor, prior to the subjective matching experiment (e.g. different degrees of realism ranging from abstract to personalised avatars for the visual appearance). The results of the subjective matching experiment show that point of view and control levels were consistently increased by users before appearance levels when it comes to enhancing the SoE. Second, several configurations were identified with equivalent SoE as the one felt in the optimal configuration, but vary between the tasks. Taken together, our results provide valuable insights about which factors to prioritize in order to enhance the SoE towards an avatar in different tasks, and about configurations which lead to fulfilling SoE in VE.

Brain-Computer Interfaces (BCIs) enable users to interact with computers without any dedicated movement, bringing new hands-free interaction paradigms. In this paper we study the combination of BCI and Augmented Reality (AR). We first tested the feasibility of using BCI in AR settings based on Optical See-Through Head-Mounted Displays (OST-HMDs). Experimental results showed that a BCI and an OST-HMD equipment (EEG headset and Hololens in our case) are well compatible and that small movements of the head can be tolerated when using the BCI. Second, we introduced a design space for command display strategies based on BCI in AR, when exploiting a famous brain pattern called Steady-State Visually Evoked Potential (SSVEP). Our design space relies on five dimensions concerning the visual layout of the BCI menu; namely: orientation, frame-of-reference, anchorage, size and explicitness. We implemented various BCI-based display strategies and tested them within the context of mobile robot control in AR. Our findings were finally integrated within an operational prototype based on a real mobile robot that is controlled in AR using a BCI and a HoloLens headset. Taken together our results (4 user studies) and our methodology could pave the way to future interaction schemes in Augmented Reality exploiting 3D User Interfaces based on brain activity and BCIs.

With the increasing use of avatars (i.e. the virtual representation of the user in a virtual environment) in virtual reality, it is important to identify the factors eliciting the sense of embodiment or the factors that can disrupt this feeling. This paper reports an exploratory study aiming at identifying internal factors (personality traits and body awareness) that might cause either a resistance or a predisposition to feel a sense of embodiment towards a virtual avatar. To this purpose, we conducted an experiment (n=123) in which participants were immersed in a virtual environment and embodied in a gender-matched generic virtual avatar through a head-mounted display. After an exposure phase in which they had to perform a number of visuomotor tasks (during 2 minutes) a virtual character entered the virtual scene and stabbed the participants' virtual hand with a knife. The participants' sense of embodiment was measured, as well as several personality traits (Big Five traits and locus of control) and body awareness, to evaluate the influence of participants' personality on the acceptance of the virtual body. The major finding of the experiment is that the locus of control is linked to several components of embodiment: the sense of agency is positively correlated with an internal locus of control and the sense of body ownership is positively correlated with an external locus of control. Interestingly, both components are not influenced by the same traits, which confirms that they can appear independently. Taken together our results suggest that the locus of control could be a good predictor of the sense of embodiment when the user embodies an avatar with a similar physical appearance. Yet, further studies are required to confirm these results.

While perceptual biases have been widely investigated in Virtual Reality (VR), very few studies have considered the challenging environment of Optical See-through Augmented Reality (OST-AR). Moreover, regarding distance perception, existing works mainly focus on the assessment of egocentric distance perception, i.e. distance between the observer and a real or a virtual object. In this paper, we study exocentric distance perception in AR, hereby considered as the distance between two objects, none of them being directly linked to the user. We report a user study (n=29) aiming at estimating distances between two objects lying in a frontoparallel plane at 2.1m from the observer (i.e. in the medium-field perceptual space). Four conditions were tested in our study: real objects on the left and on the right of the participant (called real-real), virtual objects on both sides (virtual-virtual), a real object on the left and a virtual one on the right (real-virtual) and finally a virtual object on the left and a real object on the right (virtual-real). Participants had to reproduce the distance between the objects by spreading two real identical objects presented in front of them. The main findings of this study are the overestimation (20%) of exocentric distances for all tested conditions. Surprisingly, the real-real condition was significantly more overestimated (by about 4%, p=.0166) compared to the virtual-virtual condition, i.e. participants obtained better estimates of the exocentric distance for the virtual-virtual condition. Finally, for the virtual-real/real-virtual conditions, the analysis showed a non-symmetrical behavior, which suggests that the relationship between real and virtual objects with respect to the user might be affected by other external factors. Considered together, these unexpected results illustrate the need for additional experiments to better understand the perceptual phenomena involved in exocentric distance perception with real and virtual objects.

Is there an effect of Virtual Reality (VR) Head-Mounted Display (HMD) on the user's mental effort? In this paper, we compare the mental effort in VR versus in real environments. An experiment (N=27) was conducted to assess the effect of being immersed in a virtual environment (VE) using an HMD on the user's mental effort while performing a standardized cognitive task (the well-known N-back task, with three levels of difficulty, N ∈ {1,2,3}). In addition to test the effect of the environment (i.e., virtual versus real), we also explored the impact of performing a dual task (i.e., sitting versus walking) in both environments on mental effort. The mental effort was assessed through self-report, task performance, behavioural and physiological measures. In a nutshell, the analysis of all measurements revealed no significant effect of being immersed in the VE on the users' mental effort. In contrast, natural walking significantly increased the users' mental effort. Taken together, our results support the fact there is no specific additional mental effort related to the immersion in a VE using a VR HMD.

The topic of distance perception has been widely investigated in Virtual Reality (VR). However, the vast majority of previous work mainly focused on distance perception of objects placed in front of the observer. Then, what happens when the observer looks on the side? In this paper, we study differences in distance estimation when comparing objects placed in front of the observer with objects placed on his side. Through a series of four experiments (n=85), we assessed participants' distance estimation and ruled out potential biases. In particular, we considered the placement of visual stimuli in the field of view, users' exploration behavior as well as the presence of depth cues. For all experiments a two-alternative forced choice (2AFC) standardized psychophysical protocol was employed, in which the main task was to determine the stimuli that seemed to be the farthest one. In summary, our results showed that the orientation of virtual stimuli with respect to the user introduces a distance perception bias: objects placed on the sides are systematically perceived farther away than objects in front. In addition, we could observe that this bias increases along with the angle, and appears to be independent of both the position of the object in the field of view as well as the quality of the virtual scene. This work sheds a new light on one of the specificities of VR environments regarding the wider subject of visual space theory. Our study paves the way for future experiments evaluating the anisotropy of distance perception in real and virtual environments.

Tangible objects are a simple yet effective way for providing haptic sensations in Virtual Reality. For achieving a compelling illusion, there should be a good correspondence between what users see in the virtual environment and what they touch in the real world. The haptic features of the tangible object should indeed match those of the corresponding virtual one in terms of, e.g., size, local shape, mass, texture. A straightforward solution is to create perfect tangible replicas of all the virtual objects in the scene. However, this is often neither feasible nor desirable. This paper presents an innovative approach enabling the use of few tangible objects to render many virtual ones. The proposed algorithm analyzes the available tangible and virtual objects to find the best grasps in terms of matching haptic sensations. It starts by identifying several suitable pinching poses on the considered tangible and virtual objects. Then, for each pose, it evaluates a series of haptically-salient characteristics. Next, it identifies the two most similar pinching poses according to these metrics, one on the tangible and one on the virtual object. Finally, it highlights the chosen pinching pose, which provides the best matching sensation between what users see and touch. The effectiveness of our approach is evaluated through a user study. Results show that the algorithm is able to well combine several haptically-salient object features to find convincing pinches between the given tangible and virtual objects.

In immersive projection systems (IPS), the presence of the user's real body limits the possibility to elicit a virtual body ownership illusion. But, is it still possible to embody someone else in an IPS even though the users are aware of their real body? In order to study this question, we propose to consider using a virtual shadow in the IPS, which can be similar or different from the real user's morphology. We have conducted an experiment (N=27) to study the users' sense of embodiment whenever a virtual shadow was or was not present. Participants had to perform a 3D positioning task in which accuracy was the main requirement. The results showed that users widely accepted their virtual shadow (agency and ownership) and felt more comfortable when interacting with it (compare to no virtual shadow). Yet, due to the awareness of their real body, the users have less acceptance of the virtual shadow whenever the shadow gender differs from their own. Furthermore, the results showed that virtual shadows increase the users' spatial perception of the virtual environment by decreasing the inter-penetrations between the user and the virtual objects. Taken together, our results promote the use of dynamic and realistic virtual shadows in IPS and pave the way for further studies on "virtual shadow ownership" illusion.

In this paper, we explore the influence of sharing a virtual environment with another user on the sense of embodiment in virtual reality. For this aim, we conducted an experiment where users were immersed in a virtual environment while being embodied in an anthropomorphic virtual representation of themselves. To evaluate the influence of the presence of another user, two situations were studied: either users were immersed alone, or in the company of another user. During the experiment, participants performed a virtual version of the well-known whac-a-mole game, therefore interacting with the virtual environment, while sitting at a virtual table. Our results show that users were significantly more “efficient” (i.e., faster reaction times), and accordingly more engaged, in performing the task when sharing the virtual environment, in particular for the more competitive tasks. Also, users experienced comparable levels of embodiment both when immersed alone or with another user. These results are supported by subjective questionnaires but also through behavioural responses, e.g. users reacting to the introduction of a threat towards their virtual body. Taken together, our results show that competition and shared experiences involving an avatar do not influence the sense of embodiment, but can increase user engagement. Such insights can be used by designers of virtual environments and virtual reality applications to develop more engaging applications.

This paper studies the combination of tangible objects and wearable haptics for improving the display of stiffness sensations in virtual environments. Tangible objects enable to feel the general shape of objects, but they are often passive or unable to simulate several varying mechanical properties. Wearable haptic devices are portable and unobtrusive interfaces able to generate varying tactile sensations, but they often fail at providing convincing stiff contacts and distributed shape sensations. We propose to combine these two approaches in virtual and augmented reality (VR/AR), becoming able of arbitrarily augmenting the perceived stiffness of real/tangible objects by providing timely tactile stimuli at the fingers. We developed a proof-of-concept enabling to simulate varying elasticity/stiffness sensations when interacting with tangible objects by using wearable tactile modules at the fingertips. We carried out a user study showing that wearable haptic stimulation can well alter the perceived stiffness of real objects, even when the tactile stimuli are not delivered at the contact point. We illustrated our approach both in VR and AR, within several use cases and different tangible settings, such as when touching surfaces, pressing buttons and pistons, or holding an object. Taken together, our results pave the way for novel haptic sensations in VR/AR by better exploiting the multiple ways of providing simple, unobtrusive, and low-cost haptic displays.

While the Sense of Agency (SoA) has so far been predominantly characterised in VR as a component of the Sense of Embodiment, other communities (e.g., in psychology or neurosciences) have investigated the SoA from a different perspective proposing complementary theories. Yet, despite the acknowledged potential benefits of catching up with these theories a gap remains. This paper first aims to contribute to fill this gap by introducing a theory according to which the SoA can be divided into two components, the feeling and the judgment of agency, and relies on three principles, namely the principles of priority, exclusivity and consistency. We argue that this theory could provide insights on the factors influencing the SoA in VR systems. Second, we propose novel approaches to manipulate the SoA in controlled VR experiments (based on these three principles) as well as to measure the SoA, and more specifically its two components based on neurophysiological markers, using ElectroEncephaloGraphy (EEG). We claim that these approaches would enable us to deepen our understanding of the SoA in VR contexts. Finally, we validate these approaches in an experiment. Our results (N=24) suggest that our approach was successful in manipulating the SoA as the modulation of each of the three principles induced significant decreases of the SoA (measured using questionnaires). In addition, we recorded participants' EEG signals during the VR experiment, and neurophysiological markers of the SoA, potentially reflecting the feeling and judgment of agency specifically, were revealed. Our results also suggest that users' profile, more precisely their Locus of Control (LoC), influences their level of immersion and SoA.

BCIs are presumably supposed to require the full attention of their users and to lose accuracy if they pay attention to another task. This assertion has been verified with several BCI paradigms (e.g. P300). But the cognitive demand of the promising SSVEP paradigm had never been specifically assessed yet. We measured the accuracy of an SSVEP-based BCI used by 26 participants in various conditions of mental workload. Our analysis revealed that surprisingly, for this type of BCI, little attention is actually needed from participants to reach optimal accuracy: participants were able to successfully perform a complex secondary task (N-back) without degrading the BCI accuracy. The same observation was made whether visual or auditive attention was solicited. These results indicate that SSVEP is a low-demanding paradigm in terms of cognitive resources, and are encouraging for its use in complex interaction settings.

Optical tracking is widely used in robotics applications such as unmanned aerial vehicle (UAV) localization. Unfortunately, such systems require many cameras and are, consequently, expensive. In this paper, we propose an approach to considerably increase the optical tracking volume without adding cameras. First, when the target becomes no longer visible by at least two cameras we propose a single-view tracking mode which requires only one camera. Furthermore, we propose to rely on controlled cameras able to track the UAV all around the volume to provide 6DoF tracking data through multi-view registration. This is achieved by using a visual servoing scheme. The two methods can be combined in order to maximize the tracking volume. We propose a proof-of-concept of such an optical tracking system based on two consumer-grade cameras and a pan-tilt actuator and we used this approach on UAV localization.

Producing sensations of motion in driving simulators often requires using cumbersome and expensive motion platforms. In this article we present a novel and alternative approach for producing self-motion sensations in driving simulations by relying on haptic-feedback. The method consists in applying a force-feedback proportional to the acceleration of the virtual vehicle directly to the hands of the driver, by means of a haptic device attached to the manipulated controller (or a steering wheel). We designed a proof-of-concept based on a standard gamepad physically attached at the extremity of a standard 3DOF haptic display. Haptic effects were designed to match notably the acceleration/braking (longitudinal forces) and left/right turns (lateral forces) of the virtual vehicle. A preliminary study conducted with 23 participants, engaged in gamepad-based active VR navigations in a straight line, showed that haptic motion effects globally improved the involvement and realism of motion sensation for participants with prior experience with haptic devices. Taken together, our results suggest that our approach could be further tested and used in driving simulators in entertainment and/or professional contexts.

Does it feel the same when you touch an object in Augmented Reality (AR) or in Virtual Reality (VR)? In this paper we study and compare the haptic perception of stiffness of a virtual object in two situations: (1) a purely virtual environment versus (2) a real and augmented environment. We have designed an experimental setup based on a Microsoft HoloLens and a haptic force-feedback device, enabling to press a virtual piston, and compare its stiffness successively in either Augmented Reality (the virtual piston is surrounded by several real objects all located inside a cardboard box) or in Virtual Reality (the same virtual piston is displayed in a fully virtual scene composed of the same other objects). We have conducted a psychophysical experiment with 12 participants. Our results show a surprising bias in perception between the two conditions. The virtual piston is on average perceived stiffer in the VR condition compared to the AR condition. For instance, when the piston had the same stiffness in AR and VR, participants would select the VR piston as the stiffer one in 60% of cases. This suggests a psychological effect as if objects in AR would feel "softer" than in pure VR. Taken together, our results open new perspectives on perception in AR versus VR, and pave the way to future studies aiming at characterizing potential perceptual biases.

Brain-Computer Interfaces (BCIs) are still much less reliable than other input devices. The error rates of BCIs range from 5% up to 60%. In this paper, we assess the subjective frustration, motivation, and fatigue of BCI users, when confronted to different levels of error rate. We conducted a BCI experiment in which the error rate was artificially controlled. Our results first show that a prolonged use of BCI significantly increases the perceived fatigue, and induces a drop in motivation. We also found that user frustration increases with the error rate of the system but this increase does not seem critical for small differences of error rate. Thus, for future BCIs, we would advise to favor user comfort over accuracy when the potential gain of accuracy remains small.

How do people appropriate their virtual hand representation when interacting in virtual environments? In order to answer this question, we conducted an experiment studying the sense of embodiment when interacting with three different virtual hand representations, each one providing a different degree of visual realism but keeping the same control mechanism. The main experimental task was a Pick-and-Place task in which participants had to grasp a virtual cube and place it to an indicated position while avoiding an obstacle (brick, barbed wire or fire). An additional task was considered in which participants had to perform a potentially dangerous operation towards their virtual hand: place their virtual hand close to a virtual spinning saw. Both qualitative measures and questionnaire data were gathered in order to assess the sense of agency and ownership towards each virtual hand. Results show that the sense of agency is stronger for less realistic virtual hands which also provide less mismatch between the participant's actions and the animation of the virtual hand. In contrast, the sense of ownership is increased for the human virtual hand which provides a direct mapping between the degrees of freedom of the real and virtual hand.

We are pleased to present the technical papers for the 2016 IEEE Virtual Reality Conference (IEEE VR 2016), held March 19–23, 2016, in Greenville, South Carolina, USA.

The main objective of this paper is to study and formalize the Take-Over Control in Collaborative Virtual Environments for Training (CVET). The Take-Over Control represents the transfer (the take over) of the interaction control of an object between two or more users. This paradigm is particularly useful for training scenarios, in which the interaction control could be continuously exchanged between the trainee and the trainer, e.g. the latter guiding and correcting the trainee's actions. The paper presents the formalization of the Take-Over Control followed by an illustration focusing in a use-case of collaborative maritime navigation. In the presented use-case, the trainee has to avoid an under-water obstacle with the help of a trainer who has additional information about the obstacle. The use-case allows to highlight the different elements a Take-Over Control situation should enforce, such as user's awareness. Different Take-Over Control techniques were provided and evaluated focusing on the transfer exchange mechanism and the visual feedback. The results show that participants preferred the Take-Over Control technique which maximized the user awareness.

The hands of virtual characters are highly complex 3D models that can be tedious and time-consuming to animate with current methods. This paper introduces THING, a novel tablet-based approach that leverages multi-touch interaction for a quick and precise control of a 3D hand's pose. The flexion/extension and abduction/adduction of the virtual fingers can be controlled for each finger individually or for several fingers in parallel through sliding motions on the tablet's surface. We designed two variants of THING: (1) MobileTHING, which maps the spatial location and orientation of the tablet to that of the virtual hand, and (2) DesktopTHING, which combines multi-touch controls of fingers with traditional mouse controls for the hand's global position and orientation. We compared the usability of THING against mouse-only controls and a data glove in two controlled experiments. Results show that DesktopTHING was significantly preferred by users while providing performance similar to data gloves. Together, these results could pave the way to the introduction of novel hybrid user interfaces based on tablets and mice in future animation pipelines.

Haptic feedback is known to improve 3D interaction in virtual environments but current haptic interfaces remain complex and tailored to desktop interaction. In this paper, we introduce the “Elastic-Arm”, a novel approach for incorporating haptic feedback in immersive virtual environments in a simple and cost-effective way. The Elastic-Arm is based on a body-mounted elastic armature that links the user's hand to her shoulder. As a result, a progressive resistance force is perceived when extending the arm. This haptic feedback can be incorporated with various 3D interaction techniques and we illustrate the possibilities offered by our system through several use cases based on well-known examples such as the Bubble technique, Redirected Touching and pseudo-haptics. These illustrative use cases provide users with haptic feedback during selection and navigation tasks but they also enhance their perception of the virtual environment. Taken together, these examples suggest that the Elastic-Arm can be transposed in numerous applications and with various 3D interaction metaphors in which a mobile hap-tic feedback can be beneficial. It could also pave the way for the design of new interaction techniques based on “human-scale” egocentric haptic feedback.

In this paper, we investigate obstacle avoidance behavior during real walking in a large immersive projection setup. We analyze the walking behavior of users when avoiding real and virtual static obstacles. In order to generalize our study, we consider both anthropomorphic and inanimate objects, each having his virtual and real counterpart. The results showed that users exhibit different locomotion behaviors in the presence of real and virtual obstacles, and in the presence of anthropomorphic and inanimate objects. Precisely, the results showed a decrease of walking speed as well as an increase of the clearance distance (i. e., the minimal distance between the walker and the obstacle) when facing virtual obstacles compared to real ones. Moreover, our results suggest that users act differently due to their perception of the obstacle: users keep more distance when the obstacle is anthropomorphic compared to an inanimate object and when the orientation of anthropomorphic obstacle is from the profile compared to a front position. We discuss implications on future large shared immersive projection spaces.

When walking within an immersive projection environment, accommodation distance, parallax and angular resolution vary according to the distance between the user and the projection walls which can influence spatial perception. As CAVE-like virtual environments get bigger, accurate spatial perception within the projection setup becomes increasingly important for application domains that require the user to be able to naturally explore a virtual environment by moving through the physical interaction space. In this paper we describe an experiment which analyzes how distance estimation is biased when the distance to the screen and parallax vary. The experiment was conducted in a large immersive projection setup with up to ten meter interaction space. The results showed that both the screen distance and parallax have a strong asymmetric effect on distance judgments. We found an increased distance underestimation for positive parallax conditions. In contrast, we found less distance overestimation for negative and zero parallax conditions. We conclude the paper discussing the results with view on future large immersive projection environments.

In this paper we introduce a novel approach for stereoscopic rendering of virtual environments with a wide Field-of-View (FoV) up to 360°. Handling such a wide FoV implies the use of non-planar projections and generates specific problems such as for rasterization and clipping of primitives. We propose a novel pre-clip stage specifically adapted to geometric approaches for which problems occur with polygons spanning across the projection discontinuities. Our approach integrates seamlessly with immersive virtual reality systems as it is compatible with stereoscopy, head-tracking, and multi-surface projections. The benchmarking of our approach with different hardware setups could show that it is well compliant with real-time constraint, and capable of displaying a wide range of FoVs. Thus, our geometric approach could be used in various VR applications in which the user needs to extend the FoV and apprehend more visual information.

We describe an experiment whose goal is to investigate the usage of different audio rendering techniques delivered through headphones while walking inside a wide four-side CAVE environment. In our experiment, participants had to physically walked along a virtual path exposed to different auditory stimuli. Each subject was exposed to three conditions: Stereo, Binaural sound spatially congruent with visual and binaural sound spatially incongruent with visuals and had to rate subjectively each. The results of the experiment showed increased preference ratings for the binaural audio rendering, followed by stereo rendering. As expected incongruent spatial cues were ranked significantly lower. Binaural rendering can deliver an increased immersive experience and do no require specialized hardware.

In four-sided CAVE-like VR systems, the absence of the rear wall has been shown to decrease the level of immersion and can introduce breaks in presence. With this experiment we analyse how user attention is diverted while physically walking in a virtual environment, when audio and/or visual attractors are present. Key features of the experiment are the fact that auditory feedback was delivered through binaural audio rendering dependent on the user's head position and orientation and that the four-sided CAVE used for the experiment allowed users to walk up to 9m in straight line. In the experiment we analysed how different “attractors” (audio and/or visual, static or dynamic) modify the user's attention. Results of the experiment show that audio-visual attractors are the most efficient attractors in order to keep the user's attention toward the inside of the CAVE.

Imagine you are facing a mirror, seeing at the same time both your real body and a virtual display of your brain in activity and perfectly superimposed to your real image “inside your real skull”. In this paper, we introduce a novel augmented reality paradigm called “Mind-Mirror” which enables the experience of seeing “through your own head”, visualizing your brain “in action and in situ”. Our approach relies on the use of a semi-transparent mirror positioned in front of a computer screen. A virtual brain is displayed on screen and automatically follows the head movements using an optical face-tracking system. The brain activity is extracted and processed in real-time with the help of an electroencephalography cap (EEG) worn by the user. A rear view is also proposed thanks to an additional webcam recording the rear of the user's head. The use of EEG classification techniques enables to test a Neurofeedback scenario in which the user can train and progressively learn how to control different mental states, such as “concentrated” versus “relaxed”. The results of a user study comparing a standard visualization used in Neurofeedback to our approach showed that the Mind-Mirror could be successfully used and that the participants have particularly appreciated its innovation and originality. We believe that, in addition to applications in Neurofeedback and Brain-Computer Interfaces, the Mind-Mirror could also be used as a novel visualization tool for education, training or entertainment applications.

In this paper we study how the visual animation of a self-avatar can be artificially modified in real-time in order to generate different haptic perceptions. In our experimental setup, participants could watch their self-avatar in a virtual environment in mirror mode while performing a weight lifting task. Users could map their gestures on the self-animated avatar in real-time using a Kinect. We introduce three kinds of modification of the visual animation of the self-avatar according to the effort delivered by the virtual avatar: 1) changes on the spatial mapping between the user’s gestures and the avatar, 2) different motion profiles of the animation, and 3) changes in the posture of the avatar (upper-body inclination). The experimental task consisted of a weight lifting task in which participants had to order four virtual dumbbells according to their virtual weight. The user had to lift each virtual dumbbells by means of a tangible stick, the animation of the avatar was modulated according to the virtual weight of the dumbbell. The results showed that the altering the spatial mapping delivered the best performance. Nevertheless, participants globally appreciated all the different visual effects. Our results pave the way to the exploitation of such novel techniques in various VR applications such as sport training, exercise games, or industrial training scenarios in single or collaborative mode.

Haptic interaction with different types of materials in the same scene is a challenging task, mainly due to the specific coupling mechanisms that are usually required for either fluid, deformable or rigid media. Dynamically-changing materials, such as melting or freezing objects, present additional challenges by adding another layer of complexity in the interaction between the scene and the haptic proxy. In this paper, we address these issues through a common simulation framework, based on Smoothed-Particle Hydrodynamics, and enable haptic interaction simultaneously with fluid, elastic and rigid bodies, as well as their melting or freezing. We introduce a mechanism to deal with state changes, allowing the perception of haptic feedback during the process, and a set of dynamic mechanisms to enrich the interaction through the proxy. We decouple the haptic and visual loops through a dual GPU implementation. An initial evaluation of the approach is performed through performance and feedback measurements, as well as a small user study assessing the capability of users to recognize the different states of matter they interact with.

Brain-Computer Interfaces (BCI) introduce a novel way of interacting with real and virtual environments by directly exploiting cerebral activity. However in most setups using a BCI, the user is explicitly asked to remain as motionless as possible, since muscular activity is commonly admitted to add noise and artifacts in brain electrical signals. Thus, as for today, people have been rarely let using other classical input devices such as mice or joysticks simultaneously to a BCI-based interaction. In this paper, we present an experimental study on the influence of manipulating an input device such as a standard computer mouse on the performance of a BCI system. We have designed a 2-class BCI which relies on Alpha brainwaves to discriminate between focused versus relaxed mental activities. The study uses a simple virtual environment inspired by the well-known Pac-Man videogame and based on BCI and mouse controls. The control of mental activity enables to eat pellets in a simple 2D virtual maze. Different levels of motor activity achieved with the mouse are progressively introduced in the gameplay: 1) no motor activity (control condition), 2) a semi-automatic motor activity, and 3) a highly-demanding motor activity. As expected the BCI performance was found to slightly decrease in presence of motor activity. However, we found that the BCI could still be successfully used in all conditions, and that relaxed versus focused mental activities could still be significantly discriminated even in presence of a highly-demanding mouse manipulation. These promising results pave the way to future experimental studies with more complex mental and motor activities, but also to novel 3D interaction paradigms that could mix BCI and other input devices for virtual reality and videogame applications.

We introduce the Elastic Images, a novel pseudo-haptic feedback technique which enables the perception of the local elasticity of images without the need of any haptic device. The proposed approach focus on whether visual feedback is able to induce a sensation of stiffness when the user interacts with an image using a standard mouse. The user, when clicking on a Elastic Image, is able to deform it locally according to its elastic properties. To reinforce the effect, we also propose the generation of procedural shadows and creases to simulate the compressibility of the image and several mouse cursors replacements to enhance pressure and stiffness perception. A psychophysical experiment was conducted to quantify this novel pseudo-haptic perception and determine its perceptual threshold (or its Just Noticeable Difference). The results showed that users were able to recognize up to eight different stiffness values with our proposed method and confirmed that it provides a perceivable and exploitable sensation of elasticity. The potential applications of the proposed approach range from pressure sensing in product catalogs and games, or its usage in graphical user interfaces for increasing the expressiveness of widgets.

How can we connect two brains to a video game by means of a brain-computer interface (BCI), and what will happen when we do so? How will the two users behave, and how will they perceive this novel common experience? In this paper, we are concerned with the design and evaluation of multiuser BCI applications. We created a multiuser videogame called BrainArena in which two users can play a simple football game by means of two BCIs. They can score goals on the left or right side of the screen by simply imagining left or right hand movements. To add another interesting element, the gamers can play in a collaborative manner (their two mental activities are combined to score in the same goal), or in a competitive manner (the gamers must push the ball in opposite directions). Two experiments were conducted to evaluate the performance and subjective experience of users in the different conditions. In the first experiment, we compared a single-user situation with one multiuser situation: the collaborative task. Experiment 1 showed that multiuser conditions are significantly preferred, in terms of fun and motivation, compared to the single-user condition. The performance of some users was even significantly improved in the multiuser condition. A subset of well-performing subjects was involved in the second experiment, where we added the competitive task. Experiment 2 suggested that competitive and collaborative conditions may lead to similar performances and motivations. However, the corresponding gaming experiences can be perceived differently among the participants. Taken together our results suggest that multiuser BCI applications can be operational, effective, and more engaging for participants.

In this paper we introduce novel 'Camera Motions' (CMs) to improve the sensations related to locomotion in virtual environments (VE). Traditional Camera Motions are artificial oscillating motions applied to the subjective viewpoint when walking in the VE, and they are meant to evoke and reproduce the visual flow generated during a human walk. Our novel camera motions are: (1) multistate, (2) personified, and (3) they can take into account the topography of the virtual terrain. Being multistate, our CMs can account for different states of locomotion in VE namely: walking, but also running and sprinting. Being personified, our CMs can be adapted to avatar's physiology such as to its size, weight or training status. They can then take into account avatar's fatigue and recuperation for updating visual CMs accordingly. Last, our approach is adapted to the topography of the VE. Running over a strong positive slope would rapidly decrease the advance speed of the avatar, increase its energy loss, and eventually change the locomotion mode, influencing the visual feedback of the camera motions. Our new approach relies on a locomotion simulator partially inspired by human physiology and implemented for a real-time use in Desktop VR. We have conducted a series of experiments to evaluate the perception of our new CMs by naive participants. Results notably show that participants could discriminate and perceive transitions between the different locomotion modes, by relying exclusively on our CMs. They could also perceive some properties of the avatar being used and, overall, very well appreciated the new CMs techniques. Taken together, our results suggest that our new CMs could be introduced in Desktop VR applications involving first-person navigation, in order to enhance sensations of walking, running, and sprinting, with potentially different avatars and over uneven terrains, such as for: training, virtual visits or video games.

Brain computer interfaces (BCIs) allow the direct human-computer interaction without the need of motor intervention. To properly and efficiently decode brain signals into computer commands the application of machine-learning techniques is required. Evolutionary algorithms have been increasingly applied in different steps of BCI implementations. In this paper we introduce the use of the covariance matrix adaptation evolution strategy (CMA-ES) for BCI systems based on motor imagery. The optimization algorithm is used to evolve linear classifiers able to outperform other traditional classifiers. We also analyze the role of modeling variables interactions for additional insight in the understanding of the BCI paradigms.

Have you ever dreamed of having eyes in the back of your head? In this paper we present a novel display device called FlyVIZ which enables humans to experience a real-time 360° vision of their surroundings for the first time. To do so, we combine a panoramic image acquisition system (positioned on top of the user's head) with a Head-Mounted Display (HMD). The omnidirectional images are transformed to fit the characteristics of HMD screens. As a result, the user can see his/her surroundings, in real-time, with 360° images mapped into the HMD field-ofview. We foresee potential applications in different fields where augmented human capacity (an extended field-of-view) could benefit, such as surveillance, security, or entertainment. FlyVIZ could also be used in novel perception and neuroscience studies.

We introduce a novel way of simulating sensations of motion which does not require an expensive and cumbersome motion platform. Multiple force-feedbacks are applied to the seated user's body to generate a sensation of motion experiencing passive navigation. A set of force-feedback devices such as mobile armrests or headrests are arranged around a seat so that they can apply forces to the user. We have dubbed this new approach HapSeat. A proof of concept has been designed which uses three low-cost force-feedback devices, and two control models have been implemented. Results from the first user study suggest that subjective sensations of motion are reliably generated using either model. Our results pave the way to a novel device to generate consumer motion effects based on our prototype.

Immersive spaces such as 4-sided displays with stereo viewing and high-quality tracking provide a very engaging and realistic virtual experience. However, walking is inherently limited by the restricted physical space, both due to the screens (limited translation) and the missing back screen (limited rotation). In this paper, we propose three novel locomotion techniques that have three concurrent goals: keep the user safe from reaching the translational and rotational boundaries; increase the amount of real walking and finally, provide a more enjoyable and ecological interaction paradigm compared to traditional controller-based approaches. We notably introduce the "Virtual Companion", which uses a small bird to guide the user through VEs larger than the physical space. We evaluate the three new techniques through a user study with travel-to-target and path following tasks. The study provides insight into the relative strengths of each new technique for the three aforementioned goals. Specifically, if speed and accuracy are paramount, traditional controller interfaces augmented with our novel warning techniques may be more appropriate; if physical walking is more important, two of our paradigms (extended Magic Barrier Tape and Constrained Wand) should be preferred; last, fun and ecological criteria would favor the Virtual Companion.

This paper studies the design and application of a novel visual attention model designed to compute user's gaze position automatically, i.e., without using a gaze-tracking system. The model we propose is specifically designed for real-time first-person exploration of 3D virtual environments. It is the first model adapted to this context which can compute in real time a continuous gaze point position instead of a set of 3D objects potentially observed by the user. To do so, contrary to previous models which use a mesh-based representation of visual objects, we introduce a representation based on surface-elements. Our model also simulates visual reflexes and the cognitive processes which take place in the brain such as the gaze behavior associated to first-person navigation in the virtual environment. Our visual attention model combines both bottom-up and top-down components to compute a continuous gaze point position on screen that hopefully matches the user's one. We conducted an experiment to study and compare the performance of our method with a state-of-the-art approach. Our results are found significantly better with sometimes more than 100 percent of accuracy gained. This suggests that computing a gaze point in a 3D virtual environment in real time is possible and is a valid approach, compared to object-based approaches. Finally, we expose different applications of our model when exploring virtual environments. We present different algorithms which can improve or adapt the visual feedback of virtual environments based on gaze information. We first propose a level-of-detail approach that heavily relies on multiple-texture sampling. We show that it is possible to use the gaze information of our visual attention model to increase visual quality where the user is looking, while maintaining a high-refresh rate. Second, we introduce the use of the visual attention model in three visual effects inspired by the human visual system namely: depth-of-field blur, camera- motions, and dynamic luminance. All these effects are computed based on the simulated gaze of the user, and are meant to improve user's sensations in future virtual reality applications.

Pseudo-haptics, a form of haptic illusion exploiting the brain's capabilities and limitations, has been studied for about a decade. Various interaction techniques making use of it emerged in different fields. However, important questions remain unanswered concerning the nature and the fundamentals of pseudo-haptics, the problems frequently encountered, and sophisticated means supporting the development of new systems and applications. We provide the theoretical background needed to understand the key mechanisms involved in the perception of / interaction with pseudo-haptic phenomena. We synthesise a framework resting on two theories of human perception, cognition and action: The Interacting Cognitive Subsystems model by Barnard et al. and the Bayesian multimodal cue integration framework by Ernst et al. Based on this synthesis and in order to test its utility, we discuss a recent pseudo-haptics example. Finally, we derive system design recommendations meant to facilitate the advancement in the field of pseudo-haptics for user interface researchers and practitioners.

In recent years many advances have enabled users to naturally navigate large-scale graphical worlds. The entertainment industry is increasingly providing visual and body-based cues to users to increase the natural feel of their navigational experience. So far, however, none of the existing solutions fully support the most natural locomotion through virtual worlds. Techniques and technologies which have the advantage of insights into human perceptual sensitivity thus have to be considered. In this context, by far the most natural way to move through the real world is via a full body experience where we receive sensory stimulation to all of our senses, i.e. when walking, running, biking or driving. With some exciting technological advances, people are now beginning to get this same full body sensory experience when navigating computer-generated, three-dimensional environments. Enabling an active and dynamic ability to navigate large-scale virtual scenes is of great interest for many 3D applications demanding locomotion, such as video games, edutainment, simulation, rehabilitation, military, tourism or architecture. Today it is still mostly impossible to freely move through computer generated environments in exactly the same way as the real world. Unnatural and artificial approaches are instead applied, providing only the visual sensation of self-motion. Computer graphics environments were initially restricted to visual displays combined with interaction devices - for example the joystick or mouse - providing often unnatural inputs to generate self-motion. Today, more and more interaction devices like Nintendo Wii, Microsoft Kinect or Sony EyeToy enable intuitive and natural interaction. In this context many research groups are investigating natural, multimodal methods of generating self-motion in virtual worlds based on such consumer hardware.

Haptic interaction with virtual objects is a major concern in the virtual reality field. There are many physically-based efficient models that enable the simulation of a specific type of media, e.g. fluid volumes, deformable and rigid bodies. However, combining these often heterogeneous algorithms in the same virtual world in order to simulate and interact with different types of media can be a complex task. In this paper, we propose the first haptic rendering technique for the simulation and the interaction with multistate media, namely fluids, deformable bodies and rigid bodies, in real-time and with 6DoF haptic feedback. Based on the Smoothed-Particle Hydrodynamics (SPH) physical model for all three types of media, our method avoids the complexity of dealing with different algorithms and their coupling. We achieve high update rates while simulating a physically-based virtual world governed by fluid and elasticity theories, and show how to render interaction forces and torques through a 6DoF haptic device.

We often interact with fluids in our daily life, either through tools such as when holding a glass of water or directly with our body when we swim or we wash our hands. Multimodal interactions with virtual fluids would greatly improve the simulations realism, particularly through haptic interaction. However, achieving realistic, stable, and real-time force feedback from fluids is particularly challenging. In this work, we propose a novel approach that allows real-time six Degrees of Freedom (DoF) haptic interaction with fluids of variable viscosity. Our haptic rendering technique, based on a Smoothed-Particle Hydrodynamics physical model, provides a realistic haptic feedback through physically based forces. 6DoF haptic interaction with fluids is made possible thanks to a new coupling scheme and a unified particle model, allowing the use of arbitrary-shaped rigid bodies. Particularly, fluid containers can be created to hold fluid and hence transmit to the user force feedback coming from fluid stirring, pouring, shaking, and scooping, to name a few. Moreover, we adapted an existing visual rendering algorithm to meet the frame rate requirements of the haptic algorithms. We evaluate and illustrate the main features of our approach through different scenarios, highlighting the 6DoF haptic feedback and the use of containers.

The concept of affordance, introduced by the psychologist James Gibson, can be defined as the functional utility of an object, a surface or an event. The purpose of this article was to evaluate the perception of affordances in virtual environments (VE). In order to test this perception, we considered the affordances for standing on a virtual slanted surface. The participants were asked to judge whether a virtual slanted surface supported upright stance. The perception was investigated by manipulating the texture of the slanted surface (Wooden texture vs. Ice texture). Results showed an effect of the texture: the perceptual boundary (or critical angle) with the Ice texture was significantly lower than with the Wooden texture. These results reveal that perception of affordances for standing on a slanted surface in virtual reality is possible and comparable to previous studies conducted in real environments.

In this paper we introduce the concept of a Reconfigurable Tangible Device for manipulation of 3D objects in virtual environments by single or multiple users. This Reconfigurable Tangible Device (RTD) provides points of manipulation rigidly linked together. The shape of the RTD can be reconfigured at any time as its arms can be compressed or stretched by users at will. Due to its simple shape the Reconfigurable Tangible Device can be attached to any 3D virtual object. Then, it can fully define the motion of the virtual object in 6 Degrees of Freedom. Two examples of Reconfigurable Tangible Device were developed: one with three points of manipulation (a reconfigurable triangle) and one with four points. We illustrate how these two simple devices can match many different shapes of 3D objects, and in different contexts. Preliminary testing was conducted with the RTD based on three points of manipulation involving a collaborative manipulation task in virtual reality. It produced better subjective appreciation for the RTD compared to more classical 3D collaborative techniques.

This paper introduces a novel visual attention model to compute user's gaze position automatically, i.e. without using a gaze-tracking system. Our model is specifically designed for real-time first-person exploration of 3D virtual environments. It is the first model adapted to this context which can compute, in real-time, a continuous gaze point position instead of a set of 3D objects potentially observed by the user. To do so, contrary to previous models which use a mesh-based representation of visual objects, we introduce a representation based on surface-elements. Our model also simulates visual reflexes and the cognitive process which takes place in the brain such as the gaze behavior associated to first-person navigation in the virtual environment. Our visual attention model combines the bottom-up and top-down components to compute a continuous gaze point position on screen that hopefully matches the user's one. We have conducted an experiment to study and compare the performance of our method with a state-of-the-art approach. Our results are found significantly better with more than 100% of accuracy gained. This suggests that computing in realtime a gaze point in a 3D virtual environment is possible and is a valid approach as compared to object-based approaches.

Perception of biological motions is a key issue in order to evaluate the quality and the credibility of motions of virtual humans. This paper presents a perceptual study to evaluate if human beings are able to accurately distinguish differences in natural lifting motions with various masses in virtual environments (VE), which is not the case. However, they reached very close levels of accuracy when watching to computer animations compared to videos. Still, quotes of participants suggest that the discrimination process is easier in videos of real motions which included muscles contractions, more degrees of freedom, etc. These results can be used to help animators to design efficient physically-based animations.

The Walking-In-Place interaction technique was introduced to navigate infinitely in 3D virtual worlds by walking in place in the real world. The technique has been initially developed for users standing in immersive setups and was built upon sophisticated visual displays and tracking equipments.

In this paper, we present an experiment whose goal is to investigate the role of sound and vision in the recognition of different surface profiles in a walking scenario. Fifteen subjects participated to two within-subjects experiments where they were asked to interact with a desktop system simulating bumps, holes and flat surfaces by means of audio, visual and audio-visual cues. Results of the first experiment show that participants are able to successfully identify the surface profiles provided through the proposed audio-visual techniques. Results of a second experiment in which conflictual audiovisual stimuli were presented, reveal that for some of the proposed visual effects the visual feedback is dominant on the auditory one, while for the others the role of dominance is inverted.

Abstract This paper introduces the use of a visual attention model to improve the accuracy of gaze tracking systems. Visual attention models simulate the selective attention part of the human visual system. For instance, in a bottom‐up approach, a saliency map is defined for the image and gives an attention weight to every pixel of the image as a function of its colour, edge or intensity. Our algorithm uses an uncertainty window, defined by the gaze tracker accuracy, and located around the gaze point given by the tracker. Then, using a visual attention model, it searches for the most salient points, or objects, located inside this uncertainty window, and determines a novel, and hopefully, better gaze point. This combination of a gaze tracker together with a visual attention model is considered as the main contribution of the paper. We demonstrate the promising results of our method by presenting two experiments conducted in two different contexts: (1) a free exploration of a visually rich 3D virtual environment without a specific task, and (2) a video game based on gaze tracking involving a selection task. Our approach can be used to improve real‐time gaze tracking systems in many interactive 3D applications such as video games or virtual reality applications. The use of a visual attention model can be adapted to any gaze tracker and the visual attention model can also be adapted to the application in which it is used.

In this paper, we propose a model to predict the performance of selection techniques using Brain-Computer Interfaces based on P300 signals. This model is based on Markov theory and can compute both the time required to select a target and the number of visual flashes needed. We illustrate how to use this model with three different interaction techniques to select a target. A first experimental evaluation with three healthy participants shows a good match between the model and the experimental data.

In this paper we introduce a "spatialized haptic rendering" technique to enhance 6DOF haptic manipulation of virtual objects with impact position information using vibrations. This rendering technique uses our perceptive ability to determine the contact position by using the vibrations generated by the impact. In particular, the different vibrations generated by a beam are used to convey the impact position information. We present two experiments conducted to tune and evaluate our spatialized haptic rendering technique. The first experiment investigates the vibration parameters (amplitudes/frequencies) needed to enable an efficient discrimination of the force patterns used for spatialized haptic rendering. The second experiment is an evaluation of spatialized haptic rendering during 6DOF manipulation. Taken together, the results suggest that spatialized haptic rendering can be used to improve the haptic perception of impact position in complex 6DOF interactions.

This paper reports one experiment conducted to evaluate the influence of oscillating camera motions on the perception of traveled distances in virtual environments. In the experiment, participants viewed visual projections of translations along straight paths. They were then asked to reproduce the traveled distance during a navigation phase using keyboard keys. Each participant had to complete the task (1) with linear camera motion, and (2) with oscillating camera motion that simulates the visual flow generated by natural human walking. Taken together, our preliminary results suggest that oscillating camera motions allow a more accurate distance reproduction for short traveled distances.

In most virtual reality simulations the virtual world is larger than the real walking workspace. The workspace is often bounded by the tracking area or the display devices. This paper describes a novel interaction metaphor called the Magic Barrier Tape, which allows a user to navigate in a potentially infinite virtual scene while confined to a restricted walking workspace. The technique relies on the barrier tape metaphor and its do not cross implicit message by surrounding the walking workspace with a virtual barrier tape in the scene. Therefore, the technique informs the user about the boundaries of his walking workspace, providing an environment safe from collisions and tracking problems. It uses a hybrid position/rate control mechanism to enable real walking inside the workspace and rate control navigation to move beyond the boundaries by pushing on the virtual barrier tape. It provides an easy, intuitive and safe way of navigating in a virtual scene, without break of immersion. Two experiments were conducted in order to evaluate the Magic Barrier Tape by comparing it to two state-of-the-art navigation techniques. Results showed that the Magic Barrier Tape was faster and more appreciated than the compared techniques, while being more natural and less tiring. Considering it can be used in many different virtual reality systems, it is an interaction metaphor suitable for many different applications, from the entertainment field to training simulations scenarios.

In this paper we analyze and try to predict the gaze behavior of users navigating in virtual environments. We focus on first-person navigation in virtual environments which involves forward and backward motions on a ground-surface with turns toward the left or right. We found that gaze behavior in virtual reality, with input devices like mice and keyboards, is similar to the one observed in real life. Participants anticipated turns as in real life conditions, i.e. when they can actually move their body and head. We also found influences of visual occlusions and optic flow similar to the ones reported in existing literature on real navigations. Then, we propose three simple gaze prediction models taking as input: (1) the motion of the user as given by the rotation velocity of the camera on the yaw axis (considered here as the virtual heading direction), and/or (2) the optic flow on screen. These models were tested with data collected in various virtual environments. Results show that these models can significantly improve the prediction of gaze position on screen, especially when turning, in the virtual environment. The model based on rotation velocity of the camera seems to be the best trade-off between simplicity and efficiency. We suggest that these models could be used in several interactive applications using gaze point as input. They could also be used as a new top-down component in any existing visual attention model.

This paper deals with pattern rejection strategies for self-paced brain-computer interfaces (BCI). First, it introduces two pattern rejection strategies not used yet for self-paced BCI design: 1) the rejection class (RC) strategy and 2) thresholds on reliability functions (TRF) based on the automatic multiple-threshold learning algorithm. Second, it compares several rejection strategies using several classifiers, on motor imagery data, in order to identify their most desirable properties. Results showed that nonlinear classifiers led to the most efficient self-paced BCI. Concerning the reject option, RC outperformed a specialized reject classifier which outperformed TRF. Overall, the best results were obtained using the RC reject option and non-linear classifiers such as a Gaussian support vector machine, a fuzzy inference system or a radial basis function network.

This paper describes the use of user's focus point to improve some visual effects in virtual environments (VE). First, we describe how to retrieve user's focus point in the 3D VE using an eye-tracking system. Then, we propose the adaptation of two rendering techniques which aim at improving users' sensations during first-person navigation in VE using his/her focus point: (1) a camera motion which simulates eyes movement when walking, i.e., corresponding to vestibulo-ocular and vestibulocollic reflexes when the eyes compensate body and head movements in order to maintain gaze on a specific target, and (2) a depth-of-field (DoF) blur effect which simulates the fact that humans perceive sharp objects only within some range of distances around the focal distance. Second, we describe the results of an experiment conducted to study users' subjective preferences concerning these visual effects during first-person navigation in VE. It showed that participants globally preferred the use of these effects when they are dynamically adapted to the focus point in the VE. Taken together, our results suggest that the use of visual effects exploiting users' focus point could be used in several VR applications involving first- person navigation such as the visit of architectural site, training simulations, video games, etc.

Anaesthesia is a medical act which eliminates the feeling of pain as well as the motor reactions of a person, before performing a surgical operation. Loco-Regional Anaesthesia (LRA) concerns only a part of the body such as the front arm or leg. This practice is increasingly used today notably because the patient can remain conscious and can recover more rapidly. However, LRA still remains a risky procedure. In this paper, we introduce a novel medical simulator called SAILOR for the training for LRA with neurostimulation. SAILOR is based on desktop virtual reality, realistic 3-D rendering and interactive techniques with a classical mouse and keyboard. It simulates the various biological phenomena which can occur during an anaesthesia procedure. We also introduce a novel pseudo-haptic effect to enhance the palpation of the virtual patient's body and feel the inner organs. The first feedback from users of the commercialized DVD version of SAILOR as well as the results of pilot tests suggest that this simulator is a very promising tool for education and training for LRA.

This paper describes a virtual reality (VR) simulator, for the purpose of education, training and prototyping of telemanipulation of carbon nanotubes. Major challenges in interfacing a human operator with tasks of manipulating nanotubes via a haptic VR interface are outlined. After a review of previous efforts, we present the current state of our VR simulator for nanotube manipulation. The collision detection, interaction force modeling, deformation simulation and haptic rendering of nanotubes are then discussed. Results of virtual manipulation of carbon nanotubes are presented within an immersive VR set-up.

In previous work on so-called pseudo-haptic textures, we investigated the possibility of simulating sensations of texture without haptic devices by using the sole manipulation of the speed of a mouse cursor (a technique called speed technique). In this paper, we describe another technique (called Size technique) to enhance the speed technique and simulate texture sensations by varying the size of the cursor according to the local height of the texture displayed on the computer screen. With the size technique, the user would see an increase (decrease) in cursor size corresponding to a positive (negative) slope of the texture. We have conducted a series of experiments to study and compare the use of both the size and speed technique for simulating simple shapes like bumps and holes. In Experiment 1, our results showed that participants could successfully identify bumps and holes using the size technique alone. Performances obtained with the size technique reached a similar level of accuracy as found previously with the speed technique alone. In Experiment 2, we determined a point of subjective equality between bumps simulated by each technique separately, which suggests that the two techniques provide information that can be perceptually equivalent. In Experiment 3, using paradoxical situations of conflict between the two techniques, we have found that participants' answers were more influenced by the size technique, suggesting a dominance of the size over the speed technique. Furthermore, we have found a mutual reinforcement of the techniques, i.e., when the two techniques were consistently combined, the participants were more efficient in identifying the simulated shapes. In Experiment 4, we further observed the complex interactions between the information associated with the two techniques in the perception and in the decision process related to the accurate identification of bumps and holes. Taken together, our results promote the use of both techniques for the low-cost simulation of texture sensations in applications, such as videogames, internet, and graphical user interfaces.

This paper describes an experiment conducted to study the influence of visual appearance of user's avatar (or 3D cursor) on the manipulation of virtual objects in virtual environments (VE). Participants were asked to pick up a virtual cube and place it at a random location in a VE. We found that one visual property of the avatar (the presence or absence of a directional cue) could influence the way participants picked up the cube in the VE. When using an avatar or a 3D cursor with a strong directional cue (e.g., arrows pointing to the left or right), participants generally picked up the cube by a specific side (e.g., right or left side). When using 3D cursors with no main directional cue, participants more frequently picked up the virtual cube by its front or top

This paper studies the use of visual blur effects, i.e., blurring of parts of the image fed back to the user, for First-Person-Navigations in Virtual Environments (VE). First, we introduce a model of dynamic visual blur for VE which is based on two types of blur effect: (1) a Depth-of-Field blur (DoF blur) which simulates the blurring of objects located in front or back of the focus point of the eyes, and (2) a peripheral blur which simulates the blurring of objects located at the periphery of the field of vision. We introduce two new techniques to improve real-time DoF: (1) a paradigm to compute automatically the focal distance, and (2) a temporal filtering that simulates the accommodation phenomenon. Second, we describe the results of a pilot experiment conducted to study the influence of blur effects on the performance and preference of video gamers during multiplayer sessions. Interestingly, it seems that visual blur effects did not degrade performance of gamers and they were preferred and selected by nearly half of the participants to improve fun and game-play. Taken together, our results suggest that the use of visual blur effects could thus be suitable in videogames and in other virtual environments.

This paper decribes a general event-based approach to improve multimodal rendering of 6DOF (degree of freedom) contact between objects in interactive virtual object simulations. The contact events represent the different steps of two objects colliding with each other: (1) the state of free motion, (2) the impact event at the moment of collision (3) the friction state during the contact and (4) the detachment event at the end of the contact. The different events are used to improve the classical feedback by superimposing specific rendering techniques based on these events. First we propose a general method to generate these events based only on the objects' positions given by the simulation. Second, we describe a set of different types of multimodal feedback associated to the different events that we implemented in a complex virtual simulation dedicated to virtual assembly. For instance, we propose a visual rendering of impact, friction and detachment based on particle effects. We used the impact event to improve the 6DOF haptic rendering by superimposing a high frequency force pattern to the classical force feedback. We also implemented a realistic audio rendering using impact and friction sound on the corresponding events. All these first implementations can be easily extended with other event-based effects on various rigid body simulations thanks to our modular approach.

This paper describes a new interaction technique called haptic hybrid rotations to overcome the physical angular limitations of force-feedback devices when manipulating virtual objects. This technique is based on a hybrid control of the object manipulated with the device. When approaching the angular mechanical stops of the device, the control mode switches from angular position-control to rate-control. The forcefeedback of the device is used to simulate the use of an elastic device in the rate-control mode. An experiment was carried out to compare this technique with two other common alternatives that are used when manipulating virtual objects with force-feedback devices: rotations scaling, and the clutching technique. Our results showed that haptic hybrid rotations were both the fastest and the most appreciated technique for the proposed experiment.

This paper investigates the use of camera motions, in order to improve the sensation of walking in a Virtual Environment. A simple model of camera motion is first proposed. This model uses: (1) oscillating motions for the position of the camera, and (2) a compensation motion which changes the orientation of the camera and simulate oculomotor compensation to keep a constant focal point when walking. Then we describe two experiments which were conducted to study the characteristics of our model and the preference of the users in terms of sensation of walking. The first experiment compared the use of oscillating camera motions along the three directions of space. The oscillating motions were all preferred to the control condition (i.e. a linear motion, as if the user was driving a car). Furthermore, the participants preferred oscillating motions along the vertical axis, compared with the two other directions of space. The second experiment was focused on the use of a compensation motion. It showed that on average participants preferred a compensated motion during the walk, as compared with a motion with a constant orientation of the camera. These results are consistent with the way our body and eyes move naturally when walking in real life. Taken together, our results suggest that camera motions can considerably improve the sensation of walking in virtual environments. Camera motions could be further introduced in numerous applications of virtual reality in which the simulation of walking is important, such as: architectural visits, training simulations, or videogames.

This paper describes a set of visual cues of contact designed to improve the interactive manipulation of virtual objects in industrial assembly/maintenance simulations. These visual cues display information of proximity, contact and effort between virtual objects when the user manipulates a part inside a digital mock-up. The set of visual cues encloses the apparition of glyphs (arrow, disk, or sphere) when the manipulated object is close or in contact with another part of the virtual environment. Light sources can also be added at the level of contact points. A filtering technique is proposed to decrease the number of glyphs displayed at the same time. Various effects--such as change in color, change in size, and deformation of shape- can be applied to the glyphs as a function of proximity with other objects or amplitude of the contact forces. A preliminary evaluation was conducted to gather the subjective preference of a group of participants during the simulation of an automotive assembly operation. The collected questionnaires showed that participants globally appreciated our visual cues of contact. The changes in color appeared to be preferred concerning the display of distances and proximity information. Size changes and deformation effects appeared to be preferred in terms of perception of contact forces between the parts. Last, light sources were selected to focus the attention of the user on the contact areas.

This paper describes the use and characteristics of a virtual reality system called the Virtual Technical Trainer (VTT). This system is dedicated to vocational training courses on using and programming numerically-controlled milling machines. It aims at replacing the conventional mechanical milling machines which are currently used in such vocational training courses. VTT proposes an interactive manipulation of a cutter of a virtual milling machine, with visual, audio and haptic (force) feedback. VTT uses a haptic device which was specifically designed for the purpose of our pedagogical application. When the virtual cutter mills a piece of material, a plastic deformation algorithm is used and the material is progressively carved. Trainees can feel the cutting effort thanks to a force feedback which varies as a function of different simulation parameters (rotation speed of the tool, type of material to carve, etc). Realistic audio feedback (which was recorded in real situations) and additional visual assistances may be added, in order to increase the perception and understanding of the milling task. A preliminary evaluation of VTT showed that this simulator could be used by vocational trainers successfully. It could help them to teach the basic principles of machining at the first stages of vocational training courses on numerically-controlled milling machines. 1

This paper describes two psychophysical experiments which were conducted to evaluate the influence of the control/display (C/D) ratio on the perception of mass of manipulated objects in virtual environments (VE). In both experiments, a discrimination task was used in which participants were asked to identify the heavier object between two virtual balls. Participants could weigh each ball via a haptic interface and look at its synthetic display on a computer screen. Unknown to the participants, two parameters varied between each trial: the difference of mass between the balls and the C/D ratio used in the visual display when weighing the comparison ball. The data collected demonstrated that the C/D ratio significantly influenced the result of the mass discrimination task and sometimes even reversed it. The absence of gravity force largely increased this effect. These results suggest that if the visual motion of a manipulated virtual object is amplified when compared to the actual motion of the user's hand (i.e. if the C/D ratio used is smaller than 1), the user tends to feel that the mass of the object decreases. Thus, decreasing or amplifying the motions of the user in a VE can strongly modify the perception of haptic properties of objects that he/she manipulates. Designers of virtual environments could use these results for simplification considerations and also to avoid potential perceptual aberrations.

We present a new interaction technique to simulate textures in desktop applications without a haptic interface. The proposed technique consists in modifying the motion of the cursor on the computer screen - i.e. the Control/Display ratio. Assuming that the image displayed on the screen corresponds to a top view of the texture, an acceleration (or deceleration) of the cursor indicates a negative (or positive) slope of the texture. Experimental evaluations showed that participants could successfully identify macroscopic textures such as bumps and holes, by simply using the variations of the motion of the cursor. Furthermore, the participants were able to draw the different profiles of bumps and holes which were simulated, correctly. These results suggest that our technique enabled the participants to successfully conjure a mental image of the topography of the macroscopic textures. Applications for this technique are: the feeling of images (pictures, drawings) or GUI components (windows' edges, buttons), the improvement of navigation, or the visualization of scientific data.

The paper describes the HOMERE system: a multimodal system dedicated to visually impaired people to explore and navigate inside virtual environments. The system addresses three main applications: preparation for the visit of an existing site, training for the use of a blind cane, and ludic exploration of virtual worlds. The HOMERE system provides the user with different sensations when navigating inside a virtual world: a force feedback corresponding to the manipulation of a virtual blind cane, a thermal feedback corresponding to the simulation of a virtual sun, and an auditory feedback in spatialized conditions corresponding to the ambient atmosphere and specific events in the simulation. A visual feedback of the scene is also provided to enable sighted people to follow the navigation of the main user. HOMERE has been tested by several visually impaired people who were all confident about the potential of this prototype.

Describes a psychophysical experiment designed to study the phenomenon of illusion which occurs with pseudo-haptic feedback, and to identify the moment when this illusion occurs: the "boundary of illusion". The subjects were given the task of deciding which of two virtual springs is the stiffer, these springs being simulated with a PHANToM/sup TM/ force feedback device and displayed on a monoscopic computer screen. The first spring had a realistic behavior, since its visual and haptic displacements were identical. The second spring-the pseudo-haptic one-was stiffer on a haptic basis, but sometimes less stiff on a visual basis. The data collected allowed us to calculate the visual point of subjective equality (PSE) between the two springs, which represents the boundary of the sensory illusion. On average, a high value of PSE turned out to be -24%. This value increased monotonically when the haptic difference between the springs increased. This implies that more visual deformation is necessary to compensate for large haptic differences and qualifies the notion of visual dominance. However, this boundary varies greatly depending on the subjects and their sensory integration strategy. The subjects were sensitive to this illusion to varying degrees. They were divided into different populations-from those who were "haptically oriented" to those who were "visually oriented".

This paper considers whether a passive isometric input device, such as a Spaceball/sup TM/, used together with visual feedback, could provide the operator with a pseudo-haptic feedback. For this aim, two psychophysical experiments have been conducted. The first experiment consisted of a compliance discrimination, between two virtual springs hand-operated by means of the Spaceball/sup TM/. In this experiment, the stiffness (or compliance) JND turned out to be 6%. The second experiment assessed stiffness discrimination between a virtual spring and the equivalent spring in reality. In this case, the stiffness (or compliance) JND was found to be 13.4%. These results are consistent with previous outcomes on manual discrimination of compliance. Consequently, this consistency reveals that the passive apparatus that was used can, to some extent, simulate haptic information. In addition, a final test indicated that the proprioceptive sense of the subjects was blurred by visual feedback. This gave them the illusion of using a nonisometric device.