Empathy is central to social interaction, yet how it is externally expressed in virtual reality (VR) communication remains underexplored. In this study, we examined how directionality-aware cues of empathy, such as mimicry, eye contact, and body proximity, relate to cognitive and emotional empathy. We designed high- and low-empathy scenarios and recruited participants with acting experience to ensure clear emotional expressions. Our findings indicate that facial mimicry patterns differ by empathy type: cognitive empathy involves subtle, speech-related muscle movements, whereas emotional empathy is associated with more intense affective expressions. Interestingly, we also found that while facial expressions and lower-body mimicry tend to emerge unconsciously, upper-body mimicry occurs more consciously, suggesting distinct pathways of empathic embodiment. We also observed that vocal intensity mimicry and pitch variability serve as important indicators of empathy, and a consistent hand approach is closely linked to empathy. Additionally, emotional empathy fosters longer eye contact, whereas cognitive empathy stabilizes gaze and head movements. Finally, we constructed machine learning models to predict empathy from these external expressions. Our best classifier achieved an accuracy of 0.756 for cognitive empathy and 0.704 for emotional empathy, indicating the feasibility of objective assessment. These findings provide a deeper understanding of how empathy is manifested in VR communication and support the development of empathy-aware virtual agents and training systems.

The effect of sensory-specific guides on the virtual training of individuals with intellectual disabilities was investigated. A virtual vocational training system was developed to provide selective sensory guides, and participants were categorized into visual or auditory groups by identifying the sensory type of cues they utilized effectively. Video models and visible objects were employed as virtual interventions. Participants who received video model interventions showed a statistically significant improvement in task performance than the other group (Kruskal-Wallis H = 7.48, p = 0.006). The visual attention group that utilized video models improved by 46.72%, while the auditory attention group that used visible objects showed a lower improvement of 23.38% (Kruskal-Wallis H = 2.92, p = 0.026). Correlation analysis of virtual intervention utilization patterns revealed that these virtual interventions were beneficial for individuals with strong visual attention. Therefore, virtual interventions and teaching methods should be tailored to the sensory characteristics of individuals.

Representing emotion in collaborative Virtual Reality (VR) environments is an emerging topic, as VR can enable humans to express augmented emotions beyond their normal abilities. This research explores how emotion can be represented in collaborative VR beyond facial expressions. We developed a virtual system that communicates emotion using three sensory modalities (textual, auditory and visual) through two spatiotemporal representations (human-form avatar and superpower). We show real-time emotion through a natural avatar and objectify emotion states into superpower phenomena in in-situ environments for time periods. We incorporated subjective, physiological and behavioural measures to evaluate emotion in an asynchronous VR collaboration scenario. The results suggested that showing emotions through the avatar and superpower augmentations (audio-visual) provided the best immersive VR experience, where users were more aroused, and the positive emotion felt more dominating. We also found that understanding emotion requires easy and relatable visuals that people commonly acknowledge, whereas arousing emotion requires a change of environmental contexts to indicate different states. We provide design insights for using multisensory modalities in empathic VR systems to address the lack of a standardised representation of emotion in collaborative VR.

There are various social support services to help people with intellectual disabilities (ID) find jobs for their independent social and economic activities in life. With the advancement of virtual reality (VR) technologies, vocational training can take place in a virtual environment (VE) without temporal and spatial limitations. Therefore, opportunities for people with ID to receive professional vocational training in a VE continue to expand. Accordingly, this study proposes that virtual intervention (VI) technology can improve the effects of virtual training and learning transfer from the virtual to the real world. We defined VI as a supportive activity that assists trainees in a VE according to their individual ability and training situation through an instructor’s control in the real world. We designed the virtual intervention content (VIC) as intervening virtual objects and applied them to a virtual barista job training program. We developed the virtual intervention interface (VII) for smoothly supporting the interaction between the trainee in the VE and the instructor in the real world, especially for people with ID or beginner trainees. In this study, we derived statistically significant findings that demonstrated the effects of VI technology through two experiments conducted with 39 participants with ID. Firstly, VIC showed an effective intervention function that helped the participants solve problems during virtual vocational training. Also, we observed differences in the intervention effect according to the sensory perception characteristics of the participants. Secondly, trainees’ barista job performance rates improved by 37.43% after receiving virtual training. Moreover, the effects of training were largely maintained in the job performance assessment after three weeks in an actual cafe situation. This suggests that the effects of VI-based virtual training are meaningful not only within the VE but also in the real world for the transfer of learning. Furthermore, the VI method was significantly more effective and efficient than the conventional method using the human instructor’s verbal and physical intervention in terms of time, frequency, and success rate.

We present a systematic review of 111 papers that measure the impact of virtual experiences created through head-mounted displays (HMDs) on empathy. Our goal was to analyze the conditions and the extent to which virtual reality (VR) enhances empathy. To achieve this, we categorized the relevant literature according to measurement methods, correlated human factors, viewing experiences, topics, and participants. Meta-analysis was performed based on categorized themes, and under specified conditions, we found that VR can improve empathy. Emotional empathy increased temporarily after the VR experience and returned to its original level over time, whereas cognitive empathy remained enhanced. Furthermore, while VR did not surpass 2D video in improving emotional empathy, it did enhance cognitive empathy, which is associated with embodiment. Our results are consistent with existing research suggesting differentiation between cognitive empathy (influenced by environmental factors and learnable) and emotional empathy (highly heritable and less variable). Interactivity, target of empathy, and point of view were not found to significantly affect empathy, but participants' age and nationality were found to influence empathy levels. It can be concluded that VR enhances cognitive empathy by immersing individuals in the perspective of others and that storytelling and personal characteristics are more important than the composition of the VR scene. Our findings provide guiding information for creating empathy content in VR and designing experiments to measure empathy.

In VR, gaze information is useful for directly or indirectly analyzing a user’s interest. However, there are inconveniences in using the eye tracking in the VR device. To overcome the drawback, we propose a method of estimating an object of interest from user’s gesture instead of eye tracking. LightGBM model is trained by using distance and angle-based features that are extracted from 3d information of the object and the position and rotation of the VR device. We compared accuracy of each feature for VR device combinations and found out that it is more efficient to use all devices instead of individual devices and to use angle-based feature instead of distance-based feature with accuracy of 79.36%.

In this paper, we present a haptic touchscreen interface for mobile devices. A surface actuator composed of two parallel plates is mounted between a touch panel and a display module. It generates haptic feedback when a user input on a touch screen. The electrostatic force is generated when two parallel plates are charged and this phenomenon causes haptic feedback. When an input is detected on the touch screen, multimodal feedback that includes not only basic visual and auditory feedback but also haptic feedback occurs appropriately. Then, a user feels realistic physical feeling in the fingertips and it provides the feeing such as pressing a real keyboard. We have designed and implemented an actuator, thin and transparent, to provide haptic feedback and an interactive architecture to perform multimodal output.