Parkinson’s disease (PD) commonly leads to gait disorders that necessitate long-term rehabilitation dependent on specialists and clinic-based interventions. To reduce dependence on clinicians and investigate how wearable technology can provide continuous guidance for rehabilitation training. We distilled key design principles from patient–clinician interviews and co-designed a gait training system. The system employs inertial measurement units (IMUs) to capture kinematic data, then delivers multimodal cueing (visual, auditory, and somatosensory) aligned with walking features. Two user studies (N = 16 PD patients) evaluated the effectiveness of multimodal cueing, examining strategies for information delivery and gait correction. Results indicated that visual and auditory cueing were more effective for process-oriented adjustments, whereas somatosensory stimulation better supported periodic cueing. Moreover, a dissociation between performance outcomes and user preferences was observed. These findings highlight the potential of wearable technology to provide continuous, daily training guidance for PD patients.

Motion sickness, in addition to its persistent long-term effects, also exhibits short-term effects characterized as transient physiological discomfort, which changes rapidly with variations in locomotion. However, such discomforts are challenging to assess using current subjective scales and objective physiological measurements. To tackle this issue, this paper suggests continuous measurement methods designed specifically for evaluating transient physiological discomfort during VR locomotion. Through a user-elicitation study, three preferred measurement methods—’squeezing ball’, ’sliding thumb’, and ’rubbing thigh’—were identified. These techniques were then evaluated for reliability, validity, attention, presence, and workload, with ’sliding thumb’ identified as the most effective option. The paper expands traditional measurement methods to capture users’ physiological experiences in VR interactions, offering practical choices for researchers in this field along with an in-depth discussion of design considerations, detailed implementation guidelines, and potential ways to optimize the VR experiences utilizing the measurement data.

Grasping objects typically relies on visual input to pre-shape the hand and plan movement trajectories, a process often disrupted in visually impaired (VI) individuals. ElectroGrasp is a wearable electro-tactile system that delivers anticipatory proprioceptive and tactile information through three complementary modalities: Grasping Orientation, Size, and Shape. This system dynamically conveys spatial features-thereby enhancing anticipatory grasp planning and control through tactile perception. Three experiments were conducted to evaluate ElectroGrasp. The first examined tactile pattern discriminability, size perception thresholds, and the reliability of orientation encoding. The second assessed learning time with ElectroGrasp and its effectiveness in supporting spatial representation, demonstrating accurate spatial perception of objects from electrotactile input. The third compared grasp aperture under audio versus electrotactile cues, revealing that ElectroGrasp reduced hand overshoot and regrasp corrections. Overall, the results demonstrate that ElectroGrasp provides efficient tactile information, enables improved anticipatory grasp planning comparable to visual cues, and offers a novel assistive solution for VI users.

In order to achieve early screening and assist clinical decision-making, automatic depression assessment based on multimodal data are highly anticipated. However, the existed methods often suffer from semantic gap and information redundancy due to heterogeneity among modalities. To address this challenge, this paper investigates a novel Feature Disentanglement and Fusion Network (FDFNet) for predicting depression severity from audio-visual cues. Firstly, we design the shared and private encoders to disentangle modality-shared and modalityprivate representations. The former representation that acquires joint information is subjected by similarity constraints between modalities to ensure their distributions as close as possible. The latter that can capture unique features of each modality is restrained by independence constraints for keeping their distributions distinct. The decoder is then developed to reconstruct unimodal representation with constraints to minimize information loss. Finally, an efficient fusion strategy through addition and concatenation is ultilized for aggregating information. Experimental results on four benchmark datasets demonstrate that the proposed FDFNet consistently outperforms several stateof-the-art methods, with the competitive MAE/RMSE values of 6.22/7.58 on AVEC2013, 5.21/6.49 on AVEC2014, 4.25/5.34 on DAIC-WOZ, and 4.41/5.10 on E-DAIC, indicating that multimodal deep learning based on audio-visual is an attractive solution for objectively evaluating the depression severity.

Generative content is increasingly applied in VR to provide immersive experiences, yet maintaining high generation quality remains challenging for audiovisual effects. Particularly in dynamic fluid phenomena, achieving realism and presence requires adherence to physical laws. To accomplish this objective, this work proposes an audiovisual synergy fluid animation generation framework, which enhances immersion by improving motion texture fidelity and audiovisual consistency. It comprises Detail-Enhanced Texture generator (DET) and Physics-Guided Audio generator (PGA). DET integrates Global-Local Physics guidance (GLP) and Temporal Texture Modeling (TTM) to produce video textures, explicitly optimizing dynamic details by leveraging local motion cues and assigned cumulative differences. PGA incorporates Visual Semantic Augmenter (VSA) and Rhythm Semantic Adapter (RSA) to synchronize audio by fusing static visual semantics with dynamic motion semantics to improve temporal coherence. By integrating DET and PGA, this framework strengthens audiovisual immersion in VR natural dynamic scenes from both visual and auditory perspectives. Quantitative and qualitative evaluations demonstrate that our approach surpasses most existing methods in terms of texture realism and audiovisual synchronization, offering new insights for advancing immersive experiences in dynamic VR phenomena.

Emotion regulation (ER) is a central topic in perception and cognition. With the development of virtual reality (VR) technology, its high levels of immersion, interactivity, and controllability have made it a promising tool for ER. Recent studies have demonstrated that VR-based interventions can enhance ER. However, the effectiveness of ER largely depends on the strategies employed. This critical factor has often been underexplored in VR-based ER research. To address this gap, this study investigates the effectiveness and neurophysiological characteristics of four ER strategies: distraction, emotional expression, cognitive reappraisal, and expressive suppression in VR, using both subjective ratings and objective neurophysiological data. A non-VR self-regulation strategy serves as the baseline. The results reveal that VR-based ER strategies significantly enhance individual emotional valence. For low-arousal negative emotions, distraction, emotional expression, and cognitive reappraisal show significant regulatory effects in VR, with emotional expression exhibiting the strongest impact. For high-arousal negative emotions, only cognitive reappraisal is effective. In contrast, expressive suppression does not produce significant effects for either type of negative emotion. These findings further support the role of VR in facilitating ER and provide empirical evidence for the informed selection of VR-based ER strategies. This research also contributes to the theoretical foundation of affective computing.

ABSTRACT Micro‐expressions (MEs) are spontaneous and transient facial movements that reflect real internal emotions and have been widely applied in various fields. Recent deep learning‐based methods have been rapidly developing in micro‐expression recognition (MER).Still, it is typical to focus on the one‐sided nature of MEs, covering only representational features or low‐ranking Action Unit (AU) features. The subtle changes in MEs characterize its feature representation weak and inconspicuous, making it tough to analyze MEs only from a single piece or a small amount of information to achieve a considerable recognition effect. In addition, the lower‐order information can only distinguish MEs from a single low‐dimensional perspective and neglects the potential of corresponding MEs and AU combinations to each other. To address these issues, we first explore how the higher‐order relations of different AU combinations correspond with MEs through statistical analysis. Afterward, based on this attribute, we propose an end‐to‐end multi‐stream model that integrates global feature learning and local muscle movement representation guided by AU semantic information. The comparative experiments were performed on benchmark datasets, with better performance than the state‐of‐art methods. Also, the ablation experiments demonstrate the necessity of our model to introduce the information of AU and its relationship to MER.

Foveated rendering reduces computational load by distributing resources based on the human visual system. This enables the implementation of ray tracing in virtual reality applications, where a high frame rate is essential to achieve visual immersion. However, traditional foveation methods based solely on eccentricity cannot adequately account for the complex behavior of visual attention. This is one of the main reasons that leads to lower perceived quality compared to non-foveated techniques. In this study, we introduce a novel rendering pipeline that incorporates ocular attention through the use of visual saliency. Based on foveation saliency, our approach facilitates the real-time production of high-quality images utilizing path tracing by distributing samples according to saliency metrics derived from geometric and historical data. To further augment image quality, an adaptive filtering process, aligned with the saliency metrics, is employed to reduce visible artifacts in non-foveal regions. Our experiments prove that this novel approach can demonstrate superior performance compared to previous methods, both in terms of quantitative metrics and perceived visual quality.

Multi-user locomotion in VR has grown increasingly common, posing numerous challenges. A key factor contributing to these challenges is the gaps in experience between active and passive users during co-locomotion. Yet, there remains a limited understanding of how and to what extent these experiential gaps manifest in diverse multi-user co-locomotion scenarios. This paper systematically explores the gaps in physiological and psychological experience indicators between active and passive users across various locomotion situations. Such situations include when active users walk, fly by joystick, or teleport, and passive users stand still or look around. We also assess the impact of factors such as sub-locomotion type, speed/teleport-interval, motion sickness susceptibility, etc. Accordingly, we delineate acceptability disparities between active and passive users, offering insights into leveraging notable experimental findings to mitigate discomfort during co-locomotion through avoidance or intervention.

Sketching in Virtual Reality (VR) is challenging mainly due to the absence of physical surface support and virtual depth perception cues, which induce high cognitive and sensorimotor load. This paper presents WieldingCanvas, an interactive VR sketching platform that integrates canvas manipulations to draw lines and curves in 3D. Informed by real-life examples of two-handed creative activities, WieldingCanvas interprets users’ spatial gestures to move, swing, rotate, transform, or fold a virtual canvas, whereby users simply draw primitive strokes on the canvas, which are turned into finer and more sophisticated shapes via the manipulation of the canvas. We evaluated the capability and user experience of WieldingCanvas with two studies where participants were asked to sketch target shapes. A set of freehand sketches of high aesthetic qualities were created, and the results demonstrated that WieldingCanvas can assist users with creating 3D sketches.

Automatic animation synthesis has attracted much attention from the community. As most existing methods take a small number of discrete expressions rather than continuous expressions, their integrity and reality of the facial expressions is often compromised. In addition, the easy manipulation with simple inputs and unsupervised processing, although being important to the automatic facial expression animation applications, is relatively less concerned. To address these issues, we propose an unsupervised continuous automatic facial expression animation approach through action units (AU) transfer in the latent space of generative adversarial networks. The expression descriptor which is depicted with AU vector is transferred into the input image without the need of labeled pairs of images and even without their expressions and further network training. We also propose a new approach to quickly generate input image's latent code and cluster the boundaries of different AU attributes with their latent codes. Two latent code operators, vector addition and continuous interpolation, are leveraged for facial expression animation simulating align with the boundaries in the latent space. Experiments have shown that the proposed approach is effective on facial expression translation and animation synthesis.

In this paper, a method, for automatically identifying and tracking garment related semantic focal feature points on 3D human model in motion sequence, is proposed. We consider the problem of automatic focal feature point identification and tracking when non-rigid shape deformation is occurred. The main contribution is that a novel method of tracking focal feature points when the human avatar move in front of depth camera. Firstly, we learn a regression analysis model that derives the relationship between sampled and focal feature points. Secondly, we build a model of correspondence maps to calculate the tracking results. The method can track garment-related feature points for different people in different motion and shape. We demonstrate on a wide variety of experiments that our approach leads to a significant identification and tracking result with input depth sequences.

The presentation of cloth bending properties plays a key role in cloth animation research. This paper proposed the approximate nonlinear bending model based on local geometric information. In the dynamic simulation, cloth was divided into several regions according to mean curvature of surface and bending force was updated according to the changes of the mean curvature in each region. The calculation of bending force was simple and accurate with the proposed model. Experimental results show that wrinkles and folds generated in a natural way and the efficiency of cloth simulation is improved.