Leveraging electrocardiogram (ECG) signals for emotion recognition represents a core challenge in affective computing, particularly in achieving robustness across diverse demographic groups (such as older adults with mild cognitive impairment). This challenge is rooted in three key issues: the complex multi-scale nature of ECG signals, high inter-individual physiological variability, and the need for computationally efficient temporal modeling for IoT applications. To address these issues systematically, this study proposes HCMNet, a novel, physiologically-inspired hybrid Convolutional-Mamba network. HCMNet’s architecture is problem-driven: a hierarchical scale-aware convolutional module captures multi-scale features analogous to HRV analysis; an innovative Non-Local Channel Convolutional Attention (NLCCA) mechanism mitigates inter-individual variability by learning to reshape the feature space; and a Mamba2-based Bidirectional State-Space Model (BiSSM) efficiently models temporal dynamics with linear complexity. Additionally, we validated the model on a self-built Wearable ECG emotion dataset comprising healthy elderly individuals and patients with mild cognitive impairment (MCI), as well as on public datasets WESAD and DREAMER. Experimental results demonstrate that our proposed HCMNet, through its synergistic hybrid architecture, effectively extracts robust emotional features. It not only achieves state-of-the-art performance on public benchmarks but also exhibits strong robustness for special populations. Furthermore, our in-depth adaptation analysis reveals that while a “one-model-fits-all” approach is infeasible for unseen subjects, HCMNet excels as a robust transferable base model that can be rapidly personalized, offering a practical paradigm for accurate and adaptable emotion recognition in real-world IoT settings. The source code is available at https://github.com/INSOCE/HCMNet.

Facial reactions convey crucial emotional information and coordinating interpersonal relationships in human dyadic interactions. While existing Multiple Appropriate Facial Reaction Generation (MAFRG) methods focus on generating multiple reasonable facial reactions, none of these approaches combines 2D and 3D facial behaviour information nor account for the influence of individuals’ facial identities, leading to inconsistencies in the generated facial reactions and limited capability in capturing subtle variations in facial depth and expression dynamics. This paper proposes a novel Hierarchical Multimodal Decoupling-Fusion (HMDF) framework that decouples 3D facial identity from expression behaviors, eliminating identity-based interference in the reaction generation process, which are integrated with audio-visual features through a cross-attention mechanism. Experiments show that our framework achieved the enhanced diversity and synchrony in the generated facial reactions.

The combination of wearable sensors and competitive sports provides quantitative information for scientific training, effectively assisting athletes in improving their athletic performance. This study presents a technical framework for athletic sports assessment in competitive swimming based on body-area sensor networks. In our approach, wearable inertial sensor nodes are placed on specific body parts of the athletes to capture motion data during different competitive swimming strokes. Multiwearable inertial sensor nodes are worn on specific body parts of athletes for real-time monitoring motion data during training sessions. A motion intensity detection-based error-state-Kalman-filter algorithm is proposed for multisensor data fusion. Additionally, through kinematic statistical analysis, the characteristics of joint motion during training are clearly explained. Furthermore, a deep learning network that fuses sensor time series and human skeleton graphs is proposed for different stroke phase segmentation, enabling quantitative measurement of motion phases, and several baseline classifiers are chosen for comparison to validate the robustness of our phase segmentation method. We also investigate the sensor combination selection issue during the phase segmentation process to determine the optimal sensor configuration. Our approach provides a scientific solution for the integration of wearable sensors and competitive sports, contributing to the high-quality development of the next generation of smart sports.

Early detection of mild cognitive impairment (MCI) is critical, as timely interventions during this transitional phase can slow or even prevent progression to Alzheimer’s disease. Eye-tracking (ET) signals recorded during virtual reality (VR)-based cognitive tasks present strong potential for MCI detection, as they integrate the immersive multisensory environment of VR with the rich temporal and behavioral information of ET signals. In this study, we built a VR-based system incorporating four Stroop-inspired tasks, proposed a corresponding ET signal dataset, and introduced a lightweight network for MCI detection. Using our system, a 38-subject MCI dataset was constructed, including 17 individuals with MCI and 21 healthy control individuals. However, the inherently non-stationary and redundant characteristics of ET signals often pose challenges for accurate classification. To overcome these issues, the proposed network, KWS-TA-CNN-PE, includes four key components: (1) Kymatio-based wavelet scattering transform (KWS) to calculate the wavelet scattering coefficients, generating time-robust features and minimizing memory requirements via a depth-first traversal strategy; (2) temporal attention (TA), which dynamically prioritizes the most informative time steps and suppresses noise in ET signals; (3) a one-dimensional convolutional neural network (CNN) that extracts localized temporal patterns; and (4) a probabilistic ensemble (PE) strategy that combines the outputs across the four tasks for final classification. Experimental results of our KWS-TA-CNN-PE network, using leave-one-subject-out cross-validation and a blind-test protocol, exhibit strong performance with accuracies of 0.8629 and 0.8621, respectively. The promising results highlight the clinical potential of the proposed system, dataset, and network.

Gait phase segmentation, pivotal for understanding lower limb motion, finds applications in diverse fields like medicine and sports. While existing method often struggle with accuracy and adaptability in real-world settings, this study presents a novel methodology employing particle filters for precise lower limb motion capture (MoCap) utilizing inertial sensors, which can be used in more everyday environments and in a wider range of applications over a longer period of time. The innovative approach adeptly tracks walking movements, labeling six gait phases via skeleton reconstruction facilitated by the MoCap algorithm. Subsequently, we propose a neural network architecture amalgamating temporal convolutional network (TCN), graph convolutional network (GCN), and long short-term memory (LSTM). This architecture integrates raw data from inertial sensors with joint angles derived from reconstructed motion, achieving accurate segmentation of the six gait phases. Experimental validation compares the MoCap algorithm against an optical motion capture system, and the neural network’s performance against state-of-the-art methods. Results demonstrate our method’s superior accuracy of 96.94%, highlighting its efficacy in addressing gait phase segmentation challenges and propelling advancements in gait analysis.

Horse riding, in its essence, is both an art and a captivating sport. In these events, riders showcase their refined skills and deep bond with their horses, most notably through changes in physical positioning. This research aims to accurately capture and analyze the spatial postures of riders in equestrian sports, exploring the dynamic differences between professional and amateur riders. To achieve this, we employed a multi-sensor data fusion approach based on human kinetics theories, enhanced by an extended Kalman filter. This method, combined with an optical tracking system, enabled us to intricately compare and analyze the 3D postures of riders during walking and trotting phases. Our research captured these nuanced postural shifts using an inertial sensor network, yielding nine-axis motion data of riders during their performance. The accuracy of our fusion technique was validated using the optical tracking system. By analyzing the motion data, we discerned posture variations among the riders of differing expertise levels, even when executing the same riding technique. The insights from this research offer a quantifiable metric for refining equestrian training and contribute to understanding the complex dynamics of rider–horse interaction.

In a closed environment lacking global positioning system (GPS) signals, how to achieve accurate navigation and positioning is a very challenging task. Zero velocity update (ZUPT) is a highly effective foot-mounted inertial pedestrian navigation systems in such environment. However, despite its effectiveness, the limitation of accurate detecting the zero-velocity-interval (ZVI) and heading drift are still the significant challenges of the ZUPT method. To address these issues, a deep learning method for adaptive ZVIs detection is established based solely on inertial sensors by comparing with the optical motion capture system. Additionally, an improved ZUPT-aided extend Kalman filter (EKF) divides the measurement updates of the ZVIs is established for multisensor data fusion, and the heading change with heuristic drift reduction (HDR) is also adopt as measurement, thereby yielding to limit the heading drift. Experimental results demonstrate that our method provides a better estimate of the heading angle, as well as more accurate ZVIs detection, leading to more precise dead-reckoning position estimates than other state-of-the-art methods.

The lower extremity exoskeleton, which can sense the neural motion state of the human body and then provide motion assistance, is gradually replacing the traditional wheelchairs and assistive devices, making many patients with disabilities or movement disorders able to regain the walking function. This survey provides a comprehensive review on recent technological advances in lower extremity neurorehabilitation exoskeleton from the perspectives of sensing, gait dynamics, and human–robot collaboration. For each technology category, a detailed comparison among state-of-the-art solutions is provided. The results show that the exoskeleton has been greatly improved in mechanical and learning ability. However, some issues, such as adaptability, safety, and efficiency still restrict the development of exoskeleton technology. To address these problems, the remaining open challenges and future directions to improve intelligence, sensing, gait analysis, trust, efficiency, generalization, and power consumption of exoskeleton are also presented and discussed.

The ability of a humanoid robot to imitate facial expressions with simultaneous head motions is crucial to natural human–robot interaction. This mirrored behavior from human beings to humanoid robots has high demands of similarity and real-time performance. To fulfill these needs, this article proposes a real-time robotic mirrored behavior of facial expressions and head motions based on lightweight networks. First, a humanoid robot that can change the state of its facial organs and neck through servo displacement is developed to achieve the mirrored behavior of facial expressions and head motions. Second, to overcome the high latency caused by deep learning models running in embedded devices, a lightweight deep learning network is constructed for detecting facial feature points, which can reduce model size and improve running speed without affecting the performance of the model. Finally, a mapping relationship of 68 facial feature points to optimal servo displacements is established to realize the mirrored behavior from human beings to humanoid robots. The experimental results show that the facial feature point recognition method based on the lightweight model performs better than other state-of-the-art methods, and our head motion tracking method can maintain high accuracy compared with the gold standard optical motion capture system NOKOV. Overall, our method ensures the accurate and real-time generation of robot mirrored behavior and has a certain reference value for the efficient and natural interaction between humans and robots.

Wearable inertial motion capture, a new type of motion capture technology, mainly estimates the human posture in 3-D space through multisensor data fusion. The available method for sensor fusion is usually aided by magnetometers to remove the drift error in yaw angle estimation, which in turn limits their application in the presence of a complex magnetic field environment. In this article, an extended Kalman filter (EKF) data fusion method is proposed to fuse the 9-axis sensor data. Meanwhile, the heuristic drift reduction (HDR) method is used to calibrate the accumulated error of a heading angle. In addition, the position in 3-D space is estimated by the foot-mounted zero-velocity-update (ZUPT) technique. Combining 3-D attitude and position, a biomechanical model of the human body is established to track the motion of a real human body. The EKF algorithm and position estimation methods are benchmarked against the golden standard, optical motion capture system, for various indoor experiments. In addition, various outdoor experiments are also conducted to verify the reliability of the proposed method. The results show that the proposed algorithm outperforms the available attitude estimation model in motion tracking and is feasible for 3-D human motion capture.

Horse-rider interaction analysis by wearable sensors is a promising tool for monitoring equestrian training. In this paper, a body sensor network (BSN) based equestrian motion analysis system is developed, which combines bespoke inertial measurement units (IMU) and MindWave electroencephalography (EEG) acquisition equipment. To fuse the mechanical and EEG signals collected from the system, emotional and attitude information can be obtained to analyze the interaction between the rider and horse in equestrian training. For motion data fusion, a novel method, exercise intensity extend kalman filter (EID-EKF), is proposed, which can also reconstruct the riders’ posture in different gaits by establishing a biomechanical model. The accuracy of our method is verified with the optical system Vicon to support the motion capture for four riding styles (walking, sitting troth, rising trot, canter). Finally, the emotion changes of the riders with different levels are quantified, and kinematic analysis is carried out by combining with inertial and emotional information. It is concluded from the experiment results that the estimation errors are well controlled, and motion patterns acquired according to the kinematic analysis are consistent with the actual situation.

Swimming is a worldwide popular sports whose performance is highly correlated to the posture.To analyze and recognize the posture in swimming, a monitoring system (SwimSense) for human swimming training based on wearable inertial sensors is established. In this paper, one inertial sensor node is arranged on the surface of lumbar, and the raw sensor data concerning four swimming styles was collected. Through data fusion method and statistical analysis, the features of posture and statistical information were extracted. Subsequently, we proposed an action recognition method based on HMM. According to the classification results of different swimming strokes, it can be concluded that our method has high recognition accuracy and certain reference values, which can be used in swimming training in the future.

Motion capture based on wearable inertial sensors is a promising technique for swimmers' training. To apply motion capture techniques properly, a swimming motion evaluation method based on inertial motion capture technology is proposed. Our proposed method uses a multisensor data fusion algorithm for swimmers’ attitude estimation, and the swimming posture is reconstructed in combination with a human biomechanical model. Furthermore, a comparative experiment between our proposed motion capture system and the NDI motion tracking system shows that our system performs reliably and accurately, and the estimation errors are well controlled. In addition, the accuracy of the orientation estimation algorithm ranges from$\text{1.65}^{\circ }$to$\text{3.66}^{\circ }$. The system can capture swimmers’ lumbar spine in four competitive swimming styles. A kinematic analysis of lumbar spine movement indicates that the patterns of swimmers’ lumbar spine movement can be used to evaluate training performance and provide quantitative data for swimmers.