Acquiring motor skills, such as dance, is considered a difficult task, as novice learners frequently encounter challenges in promptly comprehending and adjusting their movements in learning contexts. In this study, we present a virtual environment training system that incorporates haptic feedback and generative AI to support novices’ learning of body movements in dance. The system guides the user through haptic feedback in VR, generated from dance sequence data captured from professional choreographers. User studies with novice participants were conducted to 1) establish effective feedback for guidance and 2) evaluate the user’s training experiences using the system. We also conducted comprehensive interviews with participants and experts to confirm the design strategies and seek insights regarding the perception and utilization of virtual environments, along with haptic and visual feedback in dance learning processes. Based on the outcomes, we established the design guidelines for virtual environment training systems that facilitate learning bodily movement regulation, including in dance education.

Multi-sensory haptic experience is a key component in immersive media (IM), yet effective methods for authoring such sensory stimuli remain underexplored. In this paper, we introduce HAPhy, an AI-assisted co-creative authoring system that enables users to easily design multi-sensory haptic feedback. Leveraging large language models (LLMs), HAPhy first generates semantic descriptions of virtual objects and then infers suitable haptic rendering parameters and physical properties using both the descriptions and multi-view images. We conducted two user studies (N = 20 and N = 16) to evaluate the usability of HAPhy and the quality of the stimuli generated. Results show that HAPhy outperformed traditional manual authoring methods in usability, effectiveness, and time efficiency. A blinded comparison further showed that HAPhy-authored stimuli were rated comparably to manually authored stimuli and significantly better than AI-only outputs. The findings demonstrate that HAPhy significantly lowers the barrier to creating high-quality immersive media involving rich, diverse haptic feedback, offering a promising approach for advanced immersive media experiences.

Cybersickness is a major obstacle to immersive VR experiences, and various strategies have been proposed to overcome it. In particular, providing artificial sensory stimuli is considered a prominent approach, which improves presence and real-virtual congruence, thereby helping the situation. In this study, we evaluated the effectiveness of artificial sensory stimuli in inducing virtual walking sensations in mitigating cybersickness. We conducted two experiments involving a total of 80 participants: (1) a static user experiment ($\mathrm{N}=40$), in which participants used controller-based steering without physical movement, and (2) a dynamic user experiment ($\mathrm{N}=40$), in which participants walked on an omnidirectional treadmill, enabling virtual movement similar to real walking. In both experiments, participants were divided into two groups based on the presence or absence of vibrotactile feedback. We collected heart rate and subjective discomfort data during exposure to a VR environment and then evaluated cybersickness and presence. Across both experiments, the results indicated that virtual walking sensations induced through vibrotactile feedback significantly reduced cybersickness without negatively affecting the perceived task load or presence. Additionally, in the dynamic user locomotion experiment, a greater reduction in cybersickness was observed when vibrotactile feedback was present. These findings suggest that even if the real-virtual sensations are synchronized, vibrotactile feedback may induce cognitive distraction that further mitigates cybersickness.

This study investigated the effect of vibrotactile stimulus location on the balancing task in virtual reality (VR). Using a virtual snowboarding system with wearable haptic devices, we conducted a between-subject user study comparing the effectiveness of two different body locations–upper body (UB; torso vibrations) and lower body (LB; ankle vibrations). The real-time vibrotactile balance-correction feedback was generated by the Center of Pressure (CoP) calculated from the sensor array on insoles. The initial results showed that UB feedback is better than LB to improve users’ balance ability.

This paper compares force-feedback methods for sensorimotor skill transfer in a VR engraving art task. Using a 3-DoF force-feedback device, we implemented a VR engraving art system that provides two different haptic feedback methods of haptic guidance and disturbance. As haptic guidance, the force and profile we gathered from an art expert were presented as is, while the inverse direction of the profile was given as haptic disturbance. We evaluated the user’s task performance with two methods with a baseline of no haptic feedback and the results showed haptic disturbance led to slightly better performance than guidance.

Vibrotactile feedback has been actively utilized in many virtual reality applications to provide the sense of touch. In this preliminary work, we investigated the effects of vibrotactile feedback in the dart throwing task in a virtual environment. The user study compared the task performance, as well as observed the participants’ behavior in throwing tasks with vibrotactile feedback or not. The results showed that the participants made larger body movements during the task when vibrotactile feedback was on, while the feedback did not affect the task performance.

The need to generate convincing simulation of voices often arises in the context of avatar therapy, a treatment approach for disorders such as schizophrenia. This treatment involves patients interacting with simulations of the entity they imagine to be responsible for the voices they hear, for which there is often no external reference available. However, in such scenarios, there is little knowledge of how to design and reproduce these voices in a convincing manner. Existing voice manipulation interfaces are often complex to use, and highly limited in their ability to modify vocal characteristics beyond small adjustments. To address these challenges, we designed a framework that allows users to explore and select from a large set of voices, and thereafter manipulate the voice(s) to converge towards an effective match for one they have in mind. We demonstrated both the usability and superior performance of this system compared to existing voice manipulation interfaces.

The application of haptic technology in entertainment systems, such as Virtual Reality and 4D cinema, enables novel experiences for users and drives the demand for efficient haptic authoring systems. Here, we propose an automatic multimodal vibrotactile content creation pipeline that substantially improves the overall hapto-audiovisual (HAV) experience based on contextual audio and visual content from movies. Our algorithm is implemented on a low-cost system with nine actuators attached to a viewing chair and extracts significant features from video files to generate corresponding haptic stimuli. We implemented this pipeline and used the resulting system in a user study (n = 16), quantifying user experience according to the sense of immersion, preference, harmony, and discomfort. The results indicate that the haptic patterns generated by our algorithm complement the movie content and provide an immersive and enjoyable HAV user experience. This further suggests that the pipeline can facilitate the efficient creation of 4D effects and could therefore be applied to improve the viewing experience in home theatre environments.

This paper presents TouchPhoto, which provides visual-audio-tactile assistive features to enable visually-impaired users to take and understand photographs independently. A user can take photographs under auditory guidance and record audio tags to aid later recall of the photographs’ contents. For comprehension, the user can listen to audio tags embedded in a photograph while touching salient features, e.g., human faces, using an electrovibration display. We conducted two user studies with visually-impaired users, one for picture taking and the other for understanding and recall, in a two-month interval. They considered auditory assistance as very useful for taking and understanding photographs and tactile features as helpful but to a limited extent.

Haptic feedback has been finding increasing uses in affective interaction design. In this paper, we describe the emotional characteristics of multimodal tactile stimuli that combine vibrotactile and thermal stimuli. While varying three vibrotactile parameters of frequency, amplitude, and duration and one thermal parameter of constant temperature, we experimentally evaluated the valence and arousal of 1) thermal stimuli and 2) multimodal vibrotactile-thermal stimuli with 20 participants. Results indicated that the thermal and vibrotactile parameters have clear and somewhat independent effects on emotional responses. The findings of this study can contribute to expanding the design space of affective user interaction involving touch.

The rapid development of nano technology and mobile devices has created high demand for shape changing mobile devices in order to provide users with more functions in less space and improved usability. However, the fundamental incompatibility between flexible displays and rigid actuator components has brought a huge barrier for commercialization of this technology. This paper presents a flexible and bendable vibrotactile actuator that can be easily applied to shape changing mobile devices. The proposed vibrotactile actuator is made with an electro-conductive membrane based on polyurethane, a base membrane, and an airgap. Actuation of the proposed actuator is controlled by a polarity of both charged membranes and the actuator performance can be modulated by increasing level of biased electric potential. A user experiment showed that the proposed actuator can provide vibrations with sufficient strength for perception.

This paper is concerned with emotional responses of tactile icons. Using three sets of tactile icons in which four physical parameters-amplitude, frequency, duration, and envelope-were systematically varied, we estimated their valence and arousal scores in a perceptual experiment with 24 participants. Results showed that the four parameters have clear relationships to the emotional responses of tactile icons. Our tactile icons spanned to a large region in the valence-arousal space, but they did not elicit very positive-relaxing or very negative-relaxing emotional responses. These findings provide the design guidelines of tactile icons that have desired emotional properties.

Perceptual quality for media compression algorithms is traditionally evaluated through user studies. Such studies are time consuming, laborious and expensive, slowing down the development of new signal processing algorithms. To address this problem, a number of algorithmic quality prediction methodologies have been developed in the audio and video fields, something that is currently lacking in haptics research. In this paper, we present a novel method for predicting the perceptual quality degradation of compressed haptic texture signals. For this purpose, abstract perceptual features like Roughness, Brightness, etc. that capture the subjective experience of textures are exploited, in addition to low-level psychophysical models from the literature. As compared to the state-of-the-art, the presented prediction methodology shows an approximately 30% improvement in explaining the variance in the perceptual data.

Tactile icons, brief tactile stimuli conveying abstract information, have found their use in various applications, and their use with visual elements is increasing on touchscreen user interfaces. However, effective design guidelines of tactile icons for crossmodal use have not been established. This paper addresses this problem by investigating the congruence between visual and tactile icons based on the hypothesis that emotional agreement between the icons improves congruence. The validity of this hypothesis was examined in three experiments. In Exp. I, we selected common visual icons and estimated their emotional responses using the circumplex model of affect. Tactile icons to be used as a pair were designed in Exp. II by varying their amplitude, frequency, and envelope (rhythm). Their emotional responses were also evaluated. In Exp. III, the congruence of 192 crossmodal icons made by combining the visual icons (8) and the tactile icons (24) was evaluated, and these congruence scores were compared with the valence and arousal scores of the two unimodal icons obtained in Exp. I and II. Experimental results suggested that the congruence of a crossmodal icon highly depends on the agreement in the emotional responses between its visual and tactile icons. This finding provides feasibility to the development of general design guidelines and heuristics for crossmodal icons that rely on the relationship between the emotional responses from the individual modalities. Our approach is expected to advance the current practice that associates the physical parameters between the different senses with better intuitiveness and simplicity.