Immersion in virtual reality is a powerful sensory experience. It often leads to striking situations, such as experiencing virtual contacts on your avatar, despite the absence of physical contact on your real body. However, as of today, it is still unknown whether purely-virtual touch remains fictive/imaginative, or whether our brain processes it the same way as real touch. In this paper, we conducted an experiment comparing purely visual touch simulation (a virtual robot arm touching the avatar’s hand) with a congruent visuo-tactile simulation (visual feedback is coupled with a real robot arm touching the participant’s hand in the physical world). Using this setup, we investigated the neural responses to purely-virtual and visuo-tactile touch, and examined how these responses were modulated by cognitive workload induced by tasks within the virtual environment. The analysis of the electroencephalography (EEG recording) revealed a strong activation of the sensorimotor cortex in both conditions, although stronger and faster when physical touch occurs. Our results also revealed a decrease in evoked potentials under high cognitive workload, with the reduction being more pronounced for physical touch. Our results show that a purely-virtual touch is processed similarly to a physical touch, i.e, engaging sensorimotor regions despite the absence of direct somatosensory stimulation. This suggests that the perception of the body in virtual environments can extend beyond simple illusions, involving neural processes similar to those engaged during real physical experiences. Besides, in the age of metaverses becoming accessible to a wider audience, knowing that our brain perceives virtual contacts in VR similarly to physical contacts further underlines the need to design immersive environments that prioritize physical integrity and user safety.

Does seeing avatars during social interaction in virtual reality (VR) affect the neural synchrony of interacting people? To address this question, we assessed inter-brain synchronization (IBS), which reflects synchronization of brain activities among interacting people and is linked to interaction success. We designed a social interaction task in VR where participants' appearances were represented by virtual cursors only or additional human avatars on them. Participants performed a finger position tracking interaction under both conditions. We compared the IBS across interaction conditions by collecting electroencephalography (EEG), positional data, and virtual embodiment questionnaires. The results from 30 participants (15 pairs) show that seeing the avatars resulted in significantly higher virtual body ownership but comparable agency and positional synchronization compared to the virtual cursors-only condition. In the inter-brain analysis, we found a significant increase in IBS strength and an increasing trend in the number of IBS in the alpha band after interacting with avatar-embodied virtual cursors. Correlation analyses further revealed a significant association between alpha-band IBS and subjective body ownership, and a trend with social inclusion scores. While the influence of visual complexity cannot be ruled out, these findings suggest that avatar-induced embodiment may contribute to enhanced inter-brain synchrony during VR-mediated social interaction.

Humans experience social interactions partly through nonverbal communication, including proxemic behaviors and haptic sensations. Body language, facial expressions, personal spaces, and social touch are multiple factors influencing how a stranger's approach is experienced. Furthermore, the rise of virtual social platforms raises concerns about virtual harassment and the perception of personal space in VR: harassment is felt much more strongly in virtual spaces, and the psychological effects can be just as severe. While most virtual platforms have a 'personal bubble' feature that keeps strangers at a distance, it does not seem to suffice: personal space violations seem influenced by more than simply distance. With this paper, we aim to further clarify the variability of personal spaces. We focus on haptic stimulation, elaborating our hypotheses on the relationship between social touch and the perception of personal spaces. Users wore a haptic compression belt and were immersed in a virtual dark alley. Virtual agents approached them while exhibiting either neutral or threatening body language. In half of all trials, as the agent advanced, the compression belt tightened around the users' torsos with three different pressures. Participants could press a response button when uncomfortable with the agent's proximity. Peripersonal space violations occurred 31% earlier on average when the agent was visibly angry and the compression belt activated. A greater tightening pressure also slightly increased the personal sphere radius by up to 13%. Overall, our results are consistent with previous works on peripersonal spaces. They help further define our relationship to personal space boundaries and encourage using haptic devices during simulated social interactions in VR.

In this paper, we study how thermal haptic feedback can influence social interactions in virtual environments, with an emphasis on persuasion, focus, co-presence, and friendliness. Physical and social warmth have been repeatedly linked in psychological literature, which allows for speculations on the effect of thermal haptics on virtual social interactions. To that effect, we conducted a study on thermal feedback during simulated social interactions with a virtual agent. We tested three conditions: warm, cool, and neutral. Results showed that warm feedback positively influenced users’ perception of the agent and significantly enhanced persuasion and thermal comfort. Multiple users reported the agent feeling less ‘robotic’ and more ‘human’ during the warm condition. Moreover, multiple studies have previously shown the potential of vibrotactile feedback for social interactions. A second study thus evaluated the combination of warmth and vibrations for social interactions. The study included the same protocol and three similar conditions: warmth, vibrations, and warm vibrations. Warmth was perceived as more friendly, while warm vibrations heightened the agent’s virtual presence and persuasion. These results encourage the study of thermal haptics to support positive social interactions. Moreover, they suggest that some haptic feedback are more suited to certain types of social interactions and communication than others.

The understanding of multi-level spatial topologies is a difficult and frequent challenge in people with visual impairments daily life, impacting their independent mobility. Using the tools of the “maker” movement, and following an iterative co-design process with Orientation and Mobility instructors, we created an innovative tool (3D printed interactive model of a train station) for teaching complex spatial knowledge. Then, we did a comparative study with end users between the 3D interactive model that we designed and two 2D interactive tactile maps representing the same location. Our results show that the 3D interactive model is useful and usable, provides better satisfaction and is preferred to 2D tactile maps. In addition, complex spatial notions are better understood with the 3D model. Altogether, these results suggest that the “maker movement” may empower special education teachers with adapted and innovative tools.

In Virtual Reality (VR), a growing number of applications involve verbal communications with avatars, such as for teleconference, entertainment, virtual training, social networks, etc. In this context, our paper aims to investigate how tactile feedback consisting in vibrations synchronized with speech could influence aspects related to VR social interactions such as persuasion, co-presence and leadership. We conducted two experiments where participants embody a first-person avatar attending a virtual meeting in immersive VR. In the first experiment, participants were listening to two speaking virtual agents and the speech of one agent was augmented with vibrotactile feedback. Interestingly, the results show that such vibrotactile feedback could significantly improve the perceived co-presence but also the persuasiveness and leadership of the haptically-augmented agent. In the second experiment, the participants were asked to speak to two agents, and their own speech was augmented or not with vibrotactile feedback. The results show that vibrotactile feedback had again a positive effect on co-presence, and that participants perceive their speech as more persuasive in presence of haptic feedback. Taken together, our results demonstrate the strong potential of haptic feedback for supporting social interactions in VR, and pave the way to novel usages of vibrations in a wide range of applications in which verbal communication plays a prominent role.

Tangible User Interfaces (TUI) have been found to be relevant tools for collaborative learning by providing a shared workspace and enhancing joint visual attention. Researchers have explored the use of TUIs in a variety of curricular activities and found them particularly interesting for spatial exploration. However, very few studies have explored how TUIs could be used as a collaborative medium for people with visual impairments (VIs). In this study, we investigated the effect of tangible interaction (a small tangible robot) in a spatial collaborative task (a treasure hunt) involving two people with VIs. The aim was to evaluate the impact of the design of the TUI on the collaboration and the strategies used to perform the task. The experiment involved six dyads of people with VIs. The results showed that the collaboration was impacted by the interaction design and open interesting perspectives on the design of collaborative games for people with VIs.

The development of novel shape-changing or actuated tabletop tangible interfaces opens new perspectives for the design of physical and dynamic maps, especially for visually impaired (VI) users. Such maps would allow non-visual haptic exploration with advanced functions, such as panning and zooming. In this study, we designed an actuated tangible tabletop interface, called BotMap, allowing the exploration of geographic data through non-visual panning and zooming. In BotMap, small robots represent landmarks and move to their correct position whenever the map is refreshed. Users can interact with the robots to retrieve the names of the landmarks they represent. We designed two interfaces, named Keyboard and Sliders, which enable users to pan and zoom. Two evaluations were conducted with, respectively, ten blindfolded and eight VI participants. Results show that both interfaces were usable, with a slight advantage for the Keyboard interface in terms of navigation performance and map comprehension, and that, even when many panning and zooming operations were required, VI participants were able to understand the maps. Most participants managed to accurately reconstruct maps after exploration. Finally, we observed three VI people using the system and performing a classical task consisting in finding the more appropriate itinerary for a journey.

Maps are essential in everyday life, but inherently inaccessible to visually impaired users. They must be transcribed to non-editable tactile graphics, or rendered on very expensive shape changing displays. To tackle these issues, we developed a tangible tabletop interface that enables visually impaired users to build tangible maps on their own, using a new type of physical icon called Tangible Reels. Tangible Reels are composed of a sucker pad that ensures stability, with a retractable reel that renders digital lines tangible. In order to construct a map, audio instructions guide the user to precisely place Tangible Reels onto the table and create links between them. During subsequent exploration, the device provides the names of the points and lines that the user touches. A pre-study confirmed that Tangible Reels are stable and easy to manipulate, and that visually impaired users can understand maps that are built with them. A follow-up experiment validated that the designed system, including non-visual interactions, enables visually impaired participants to quickly build and explore maps of various complexities.

In the last few decades, a growing number of Electronic Orientation Aids (EOA) has been developed with the purpose of improving the autonomy of visually impaired people. However, the majority of those systems are not used by the blind due to limited usability. The main challenges to be addressed are about interaction and guidance. To address these issues, we designed a multimodal (input and output) Virtual Environment (VE) that simulates different interactions that could be used for space perception and guidance in an EOA. This platform subserves two goals: help designers to systematically test guidance strategies (i.e. for the development of new EOAs) and train blind people to use interactive EOAs, with an emphasis on cognitive mapping enhancement. In a multimodal VE, both objectives are assessed in a controlled, cost-effective, safe and flexible environment.

We have developed a real-time portable object recognition system based on a bio-inspired image analysis software to increase blind people autonomy by localizing and identifying surrounding objects. A working prototype of this system has been tested on the issue of currency bill recognition encountered by most of the blind people. Seven blind persons were involved in an experiment which demonstrated that the usability of the system was good enough for such a device to be used daily in real-life situations.