Immersion in virtual reality is a powerful sensory experience. It often leads to striking situations, such as experiencing virtual contacts on your avatar, despite the absence of physical contact on your real body. However, as of today, it is still unknown whether purely-virtual touch remains fictive/imaginative, or whether our brain processes it the same way as real touch. In this paper, we conducted an experiment comparing purely visual touch simulation (a virtual robot arm touching the avatar’s hand) with a congruent visuo-tactile simulation (visual feedback is coupled with a real robot arm touching the participant’s hand in the physical world). Using this setup, we investigated the neural responses to purely-virtual and visuo-tactile touch, and examined how these responses were modulated by cognitive workload induced by tasks within the virtual environment. The analysis of the electroencephalography (EEG recording) revealed a strong activation of the sensorimotor cortex in both conditions, although stronger and faster when physical touch occurs. Our results also revealed a decrease in evoked potentials under high cognitive workload, with the reduction being more pronounced for physical touch. Our results show that a purely-virtual touch is processed similarly to a physical touch, i.e, engaging sensorimotor regions despite the absence of direct somatosensory stimulation. This suggests that the perception of the body in virtual environments can extend beyond simple illusions, involving neural processes similar to those engaged during real physical experiences. Besides, in the age of metaverses becoming accessible to a wider audience, knowing that our brain perceives virtual contacts in VR similarly to physical contacts further underlines the need to design immersive environments that prioritize physical integrity and user safety.

Does seeing avatars during social interaction in virtual reality (VR) affect the neural synchrony of interacting people? To address this question, we assessed inter-brain synchronization (IBS), which reflects synchronization of brain activities among interacting people and is linked to interaction success. We designed a social interaction task in VR where participants' appearances were represented by virtual cursors only or additional human avatars on them. Participants performed a finger position tracking interaction under both conditions. We compared the IBS across interaction conditions by collecting electroencephalography (EEG), positional data, and virtual embodiment questionnaires. The results from 30 participants (15 pairs) show that seeing the avatars resulted in significantly higher virtual body ownership but comparable agency and positional synchronization compared to the virtual cursors-only condition. In the inter-brain analysis, we found a significant increase in IBS strength and an increasing trend in the number of IBS in the alpha band after interacting with avatar-embodied virtual cursors. Correlation analyses further revealed a significant association between alpha-band IBS and subjective body ownership, and a trend with social inclusion scores. While the influence of visual complexity cannot be ruled out, these findings suggest that avatar-induced embodiment may contribute to enhanced inter-brain synchrony during VR-mediated social interaction.

Identifying and minimizing physiological artifacts in EEG is challenging because these artifacts may corrupt the underlying brain activity severely. In this work, we proposed a hybrid approach to detect/reduce EEG artifacts by combining the MediaPipe face mesh model and artifact subspace reconstruction (ASR). Four types of artifacts, eye blinking, horizontal/vertical eye movements, and jaw movement during EEG measurement were generated to test our approach. We observed that real-time video-based artifact identification achieved over 95% accuracy in detecting eye blinking, horizontal eye movement, and jaw movement. Moreover, the targeted noise reduction was effective in analyzing the signal-to-noise ratio (SNR) for each specific artifact. This work may contribute to improving the reliability and accuracy of EEG data analysis in the real-world and online scenarios by providing a practical and effective approach to identifying and reducing physiological artifacts in real-time.

Motor imagery brain-computer interfaces (BCIs) can control external machines with neurophysiological signals during limb movement imagination without real movements. An electroencephalography (EEG)-based motor imagery task has notable characteristics of event-related (de) synchronization (ERD/ERS) in specific frequency bands, including alpha and beta rhythms in the sensorimotor area. Based on this phenomenon, motor imagery features are extracted typically using common spatial patterns (CSPs). However, some researchers have reported that ERD features have severe inter-subject variation. In this study, we investigated the correlation between various ERD features and classification accuracy during a motor imagery task. We found that ERDs may not be useful in estimating classification accuracy, although they are representative features of a motor imagery task.

Brain-computer interface (BCI) has been developed so that people can control computers or machines using their brain activity. Thus, BCI provides people who cannot move their bodies with new communication tools. The P300 speller is one of the applications used most commonly among the various types of BCI. Compared to other BCIs, the P300 speller achieves relatively high accuracy, but there remains room for improvement, such as reduction in calibration time and speeding up the speller by modulating the number of repetitions needed to enter a character. Hence, seeking correlates of the P300 speller performance may help improve the application. In this experiment, we investigated the correlation between a rapid serial visual presentation (RSVP) task and P300 speller performance to determine whether such a task is useful in predicting P300 speller performance. We found statistically significant correlations between the RSVP task ERP (P300) features and the P300 speller offline accuracy. Based on this observation, we expect that P300 speller performance can be predicted using a relatively simple cognitive task.