With the continuous maturation of technology and the deepening of human-centered design philosophy, eye tracking technology has gradually been applied in research studies for the optimization of flight deck design in commercial aviation. While these studies have focused solely on Pilot Flying (PF), without considering the visual attention distribution of Pilot Monitoring (PM). The purpose of this study was to analyze and compare the visual attention distribution of PF and PM during different flight phases, providing more comprehensive guidance and optimization suggestion to the flight deck design in commercial aviation. This study was conducted in the A320 full-motion flight simulator, with eye tracking technology recording eye movements of PF and PM in the same flight crew simultaneously. The result showed significant differences in three eye movement metrics between PF and PM, not only throughout the whole common manual circuit but also within specific flight phases. Additionally, the results indicated that in certain flight phases, PF and PM exhibited statistically significant correlations in certain eye movement metrics for specific areas of the flight deck. However, in most other instances, no statistically significant correlations were observed between the eye movement metrics of PF and PM. Therefore, the future design of flight decks may benefit from considering differentiation between PF and PM sides in certain flight phases or areas. Additionally, it can be beneficial to make dynamic adjustments to the current flight deck display and control interface while maintaining the existing layout design.

Wearable sensing has gained considerable attention for its effectiveness in gesture recognition, offering convenient, natural, and efficient interaction in human-machine systems. However, practical deployment in resource-constrained settings remains challenging due to data-intensive acquisition demands, high training overhead, and limited computing capacity—factors that hinder large-scale applicability. To address these issues, we proposed a lightweight model for fine-grained gesture recognition under sparse-site sensing conditions. Experiments were conducted on a custom multimodal dataset comprising 15 static and 18 dynamic gestures. Modal assessments across diverse gesture types revealed the complementarity of wrist surface electromyography and kinematic signals, validating the effectiveness of multimodal fusion. In addition, we leveraged transfer learning to improve model generalization to unseen subjects while reducing data acquisition burden and computational overhead. To achieve the Accuracy-Cost trade-off, we formulated an objective function and validated its generalizability across diverse sensing configurations, hardware platforms, and standardization strategies. Experimental results showed that an average accuracy sacrifice of 3.8% can yield substantial resource savings, with an average reduction of 87.8% in data collection and fine-tuning time, facilitating resource-efficient allocation of gesture recognition systems in Internet of Things (IoT) environments. Finally, an online gesture recognition prototype was developed and evaluated, demonstrating the method’s robustness and practical utility in cross-domain transfer scenarios. These findings provide both theoretical and practical foundations for deploying gesture recognition-based rehabilitation systems within IoT settings.

Wearable sensing technology offers a natural and convenient means of human-computer interaction, particularly for gesture recognition, yet domain shifts in wrist-worn single-site sensing pose significant challenges for cross-domain gesture recognition. To address this, we proposed a generalized cross-domain framework for fine-grained gesture recognition using wrist-worn single-site sensing. Concretely, we presented a Multi-Branch Network, which combines feature-level multimodal fusion with enhanced inter-modal interaction to effectively capture fine-grained gestures. To this end, we constructed a multimodal dataset, which comprises fifteen static and eighteen dynamic gestures. Furthermore, we developed five fine-tuning strategies and evaluated them across the paradigms of cross-session, cross-subject, cross-gesture, and cross-modality. Through comprehensive analyses, this study provides valuable insights into the selection of optimal fine-tuning strategies and elucidates the internal mechanisms underlying multiple cross-domain paradigms. To investigate the intricate trade-off between recognition accuracy and computational cost, we applied nonlinear least squares to construct the Accuracy-Cost trade-off functions. Experimental findings indicated that the optimal transfer learning ratios for these cross-domain paradigms ranged from 6.1% to 9.0%, with most clustering around 9.0%, offering a valuable reference for determining optimal transfer learning ratios within diverse cross-domain scenarios. Additionally, we implemented a real-time online gesture recognition system, validating the feasibility of our approach through preliminary tests in real-world scenarios. In conclusion, this study serves as a preliminary investigation into the application of wrist-worn single-site sensing for fine-grained gesture recognition.

In this paper, we introduce an approach for visual tracking in videos that predicts the bounding box location of a target object at every frame. This tracking problem is formulated as a sequential decision-making process where both historical and current information are taken into account to decide the correct object location. We develop a deep reinforcement learning based strategy, via which the target object position is predicted and decided in a unified framework. Specifically, a RNN based prediction network is developed where local features and global features are fused together to predict object movement. Together with the predicted movement, some predefined possible offsets and detection results form into an action space. A decision network is trained in a reinforcement manner to learn to select the most reasonable tracking box from the action space, through which the target object is tracked at each frame. Experiments in an existing tracking benchmark demonstrate the effectiveness and robustness of our proposed strategy.