Pedestrian detection plays a critical role in intelligent perception systems in autonomous vehicles, which directly influences the reliability and safety of the overall system. Advanced in-vehicle sensor technology has enabled the continuous evolution of pedestrian detection systems by leveraging heterogeneous multimodal inputs such as RGB, infrared, depth, Light Detection And Ranging, and event data. Nevertheless, establishing a robust pedestrian detection system that is capable of integrating and processing such heterogeneous multimodal data effectively remains a significant challenge. At the same time, growing concerns about data privacy among automobile manufacturers have hindered further advances in detection model performance by restricting the sharing of private data within the industry. In this paper, a novel personalised federated learning framework, Kolmogorov-Arnold network-based Dual Expert Transformer Heterogeneous Personalized Federated Learning (KDET-HPFL), is proposed for multimodal pedestrian detection. To be specific, the KDET pedestrian detector is developed based on an expert feature selection module (which is designed to adaptively choose essential features from multimodal data) and a Group-Rational Kolmogorov-Arnold Network module, which enhances the feature extraction capabilities and improves the detection performance effectively. The HPFL framework is proposed for data privacy protection on heterogeneous multimodal data, where a cross-client aggregation (CCA) method is put forward by integrating different aggregation methods for certain layers in the KDET detector. With CCA, the HPFL framework achieves personalised feature retention of multimodal data pairs on multiple clients and improved model aggregation effect for each client. Experimental findings reveal that the proposed KDET-HPFL framework outperforms some existing personalised federated learning frameworks for pedestrian detection on four public datasets (i.e., LLVIP, STCrowd, InOutDoor, and EventPed) with mAP scores of 73.74%, 75.39%, 66.14%, and 79.57%, respectively.

Pedestrian trajectory prediction is an important research area with significant applications in autonomous driving and intelligent surveillance. However, existing studies on pedestrian trajectory prediction often suffer from a noticeable discrepancy between predicted and actual trajectories, due to incomplete extraction of pedestrian trajectory features and the randomness of the pedestrian walking process. The key objective of this article is to address this issue by proposing a method that can reasonably simulate the randomness of pedestrian walking and comprehensively extract pedestrian trajectory features. To achieve this, a novel social informer model built upon the informer model is proposed in this article. The social informer utilizes a transformer encoder-based interaction module to comprehensively extract pedestrian trajectory features, which are input into the informer model for further processing. Additionally, an adaptive variance mechanism is proposed to determine the optimal variance and accurately simulate the random nature of pedestrian walking. Finally, the proposed model is evaluated in a comparative experiment on ETH and UCY datasets, with results demonstrating that the proposed model outperforms other models, exhibiting improved accuracy and performance.

Health monitoring and fault diagnosis of high-speed train traction systems are essential for maintaining reliable operation. To effectively process multisensor signals and avoid overfitting, ensemble learning methods are employed, leveraging multiple base models to integrate data from various sensors and enhance fault diagnosis performance. However, conventional ensemble frameworks are often burdened by excessive model parameters, limiting their applicability on edge computing processors. To address these challenges, this study proposes a novel dual attention-guided ensemble framework. This framework incorporates multiple multiscale feature attention (MFA) modules and a decision fusion attention (DFA) module, designed to capture critical features from multisensor signals, optimize the capacity of prominent feature extraction, and simultaneously reducing trainable parameters. The proposed ensemble framework is validated on the hardware-in-the-loop (HIL) simulation platform for high-speed train traction control systems, with experimental results demonstrating its superior effectiveness over several recently published ensemble learning methods.

Electrocardiogram (ECG), as a crucial fine-grained cardiac feature, has been successfully recovered from radar signals in the literature, but the performance heavily relies on the high-quality radar signal and numerous radar-ECG pairs for training, restricting the applications in new scenarios due to data scarcity. Therefore, this work focuses on radar-based ECG recovery in new scenarios with limited data and proposes a cardio-focusing and-tracking (CFT) algorithm to precisely track the cardiac location to ensure an efficient acquisition of high quality radar signals. Furthermore, a transfer learning model (RFcardi) is proposed to extract cardio-related information from the radar signal without ECG ground truth based on the intrinsic sparsity of cardiac features, and only a few synchronous radar ECG pairs are required to fine-tune the pre-trained model for ECG recovery. The experimental results reveal that the proposed CFT can dynamically identify the cardiac location, and the RFcardi model can effectively generate faithful ECG recoveries after using a small number of radar-ECG pairs for training. The code and dataset will be made available after publication.

This paper introduces a novel deep-learning approach to predict true stress-strain curves of high-strength steels from small punch test (SPT) load-displacement data. The proposed approach uses Gramian Angular Field (GAF) to transform load-displacement sequences into images, capturing spatial-temporal features and employs a Sequence-to-Sequence (Seq2Seq) model with an LSTM-based encoder-decoder architecture, enhanced by multi-head cross-attention to improved accuracy. Experimental results demonstrate that the proposed approach achieves superior prediction accuracy, with minimum and maximum mean absolute errors of 0.15 MPa and 5.58 MPa, respectively. The proposed method offers a promising alternative to traditional experimental techniques in materials science, enhancing the accuracy and efficiency of true stress-strain relationship predictions.

Neuro-narrative games represent an emerging integration of brain-computer interface (BCI) and virtual reality (VR) technologies, leveraging imagery perspective and motor imagery (MI) task sequence as two critical design factors. However, the individual and combined influences of these factors on system usability remain underexplored, especially regarding the interaction between first- and third-person perspectives and the structuring of MI task sequences. This paper presents a unified evaluation framework that combines objective electroencephalogram (EEG)-based usability metrics with subjective user feedback to provide a comprehensive assessment in a custom-designed, user-centered BCI-VR narrative game system. Employing a within-subject experimental design, we systematically examine the main and interaction effects of perspective and sequence structure, revealing that immersive VR can mitigate perspective-induced usability gaps and that fixed MI sequences reduce cognitive workload and enhance user performance. Conversely, mixed task sequences boost user motivation but lead to increased mental workload. These findings contribute actionable design guidelines for expanding accessible, engaging, and user-centered BCI-VR game experiences to a broader range of users, ultimately supporting the development of more effective neuro-narrative systems. Importantly, our approach bridges objective neural data with subjective user experience, paving the way for holistic usability evaluation in future BCI-VR applications.

With the rapid development of high-speed train, health monitoring of high-speed train traction power system has gradually become a popular research topic. The traction asynchronous motor, as a key component in the traction power systems, greatly affects the reliability, stability, and safety of high-speed train operation. Normally, when faults occur, the train needs to immediately slow down or even stop to avoid unimaginable losses, resulting in limited fault data. Traditional data-driven fault diagnosis methods may face the local optimum problem during the optimization process when training samples are insufficient. In this study, a novel gossip strategy-based fault diagnosis method is proposed to prevent the local optimum problem, thus improving fault diagnosis performance. The proposed gossip strategy-based fault diagnosis method is validated on the hardware-in-the-loop high-speed train traction control system simulation platform, and the experimental results unequivocally show that the proposed method outperforms other well-known methods.

Pedestrian trajectory prediction is fundamental in various applications, such as autonomous driving, intelligent surveillance, and traffic management. Existing methods generally fall into two categories: model-driven approaches and data-driven approaches. However, both approaches have inherent limitations when applied to real-world scenarios, particularly in capturing the complex interactions between pedestrians and modeling the stochastic nature of human motion. Notably, there is a lack of research on integrating the strengths of model-driven and data-driven paradigms, which can better address these challenges. This paper aims to fill these limitations by proposing a novel model-data dual-driven approach, called Social Entropy Informer (SEI), for pedestrian trajectory prediction. SEI simultaneously models local and global pedestrian interactions while incorporating information entropy to capture human motion’s inherent randomness and uncertainty quantitatively, which provides a robust framework for predicting pedestrian trajectories. Furthermore, we propose a new loss function derived from information theory, which accounts for the stochasticity of pedestrian movement and enhances the model’s ability to generalize across diverse scenarios. The SEI framework integrates feature extraction, entropy-based stochastic modeling, and the new loss function, improving prediction accuracy and model interpretability. Experimental results demonstrate that SEI outperforms other benchmark methods in prediction accuracy.

Radar-based cardiac monitoring has become a popular research direction recently, but the fine-grained electrocardiogram (ECG) signal is still hard to reconstruct from millimeter-wave radar signal. The key obstacle is to decouple cardiac activities in the electrical domain (i.e., ECG) from that in the mechanical domain (i.e., heartbeat), and most existing research only uses purely data-driven methods to map such domain transformation as a black box. Therefore, this work first proposes a signal model that considers the fine-grained cardiac feature sensed by radar, and a novel deep learning framework called radarODE is designed to extract both temporal and morphological features for generating ECG. In addition, ordinary differential equations are embedded in radarODE as a decoder to provide morphological prior, helping the convergence of the model training and improving the robustness under body movements. After being validated on the dataset, the proposed radarODE achieves better performance compared with the benchmark in terms of missed detection rate, root mean square error, Pearson correlation coefficient with improvements of 9%, 16% and 19%, respectively. The validation results imply that radarODE is capable of recovering ECG signals from radar signals with high fidelity and can potentially be implemented in real-life scenarios

Zoning out, a form of cognitive disengagement, seriously challenges the effectiveness of virtual reality (VR) based upper limb rehabilitation. As therapy often involves repetitive tasks requiring sustained attention, undetected lapses in focus can reduce motor learning, engagement, and overall recovery outcomes. This research addresses this gap by proposing ZonAware, a novel strategy integrating real-time zoning out detection with adaptive intervention to enhance user engagement during VR rehabilitation. ZonAware identifies zoning out using five eye-tracking metrics: blink frequency, blink duration, pupil size, eye openness, and gaze duration. These signals are analysed through lightweight statistical models (Z-Score, Boxplot, and Modified Z-Score), with a hard voting mechanism producing binary classifications in real-time. Upon detection, a pattern changing intervention subtly modulates task difficulty by temporarily increasing, then decreasing it, to regain user focus without breaking immersion. Three user studies involving 70 healthy participants and 22 patients demonstrated the strategy's effectiveness. ZonAware achieved 98.24% detection accuracy with low latency (82-150 ms), reducing zoning out frequency by 53.57% and shortening disengagement duration from 18.1 to 4.8 seconds. The approach also improved user engagement, performance, and emotional motivation. ZonAware delivers one of the first real-time zoning out solutions for VR rehabilitation, offering an interpretable, theory-driven approach that enhances attention, engagement, and adaptability in human-computer interaction.

Integrating biofeedback technology, such as real-time eye-tracking, has revolutionized the landscape of virtual reality (VR) rehabilitation games, offering new opportunities for personalized therapy. Motivated to increase patient focus during rehabilitation, the Focus-Driven Augmented Feedback (FDAF) system was developed to enhance focus and maintain engagement during upper limb VR rehabilitation. This novel approach dynamically adjusts augmented visual feedback based on a patient's gaze, creating a personalised rehabilitation experience tailored to individual needs. This research aims to develop and comprehensively evaluate the FDAF system to enhance patient focus and maintain engagement in VR rehabilitation environments. The methodology involved three experimental studies, which tested varying levels of augmented feedback with 71 healthy participants and 17 patients requiring upper limb rehabilitation. The results demonstrated that a 30% augmented level was optimal for healthy participants, while a 20% was most effective for patients, ensuring sustained engagement without inducing discomfort. The research's findings highlight the potential of eye-tracking technology to dynamically customise feedback in VR rehabilitation, leading to more effective therapy and improved patient outcomes. This research contributes significant advancements in developing personalised VR rehabilitation techniques, offering valuable insights for future therapeutic applications.

Prior work has shown that frame rate significantly influences user behavior in fast-response tasks in 2D and 3D contexts. However, its impact on a steering task, which involves navigating an object along a path from the start to the end, remains relatively unexplored, especially in the context of virtual reality (VR). This task is considered a typical non-fast-response activity, as it does not demand rapid reactions within a limited time frame. Our work aims to understand and model users' steering behavior and predict movement time with different task complexities and frame rates in VR environments. We first conducted a user study to collect user behavior in a steering task with four factors: frame rate, path length, width, and radius of curvature. Based on the results, we then quantified the effects of frame rate and built two predictive models. Our models exhibited the best fit ($r^{2}> 0.957$r2>0.957) and over 17% improvement in prediction accuracy for movement time compared to existing models. Our models' robustness was further validated by applying them to predict steering performance with different VR tasks and frame rates. The two models keep the best predictability for both movement time and speed.

Compared with conventional virtual rehabilitation programs, the self-adaptive virtual rehabilitation system has the advantage of dynamically adjusting the training difficulty according to the users' real-time motion data collected, showing the potential to improve the rehabilitation experiences and assist the therapists with the flexibility provided. Movement enhancement, which visually amplifies the user's motion, exhibits significant promise in rehabilitation to improve the user's confidence and motivation. This study aims to propose a self-adaptive strategy in a virtual rehabilitation system based on movement enhancement and evaluate its effectiveness in improving user experience and performance. This study will be beneficial for the future development of virtual rehabilitation programs that combine self-adaptive systems and movement modulation, consequently helping individuals involved in virtual rehabilitation with improved user experience.

Hand-held augmented reality (AR) offers accessible, interactive rehabilitation options for patients with upper limb motor deficits. Incorporating hand-involved interactions (e.g., tangible and mid-air interactions) into hand-held AR provides patients with intuitive manners to perform rehabilitation exercises mimicking real-world activities. Previous work has shown the importance of user experience and motor performance in rehabilitation systems, but little was known in the literature regarding the impact of hand-involved interactions in hand-held AR on user experience and motor performance in rehabilitation exercises. Hence, this study aims to evaluate user experience and motor performance when using three types of hand-involved interactions in hand-held AR rehabilitation: (1) tangible cube (i.e., a space-multiplexed tangible interaction with a physical cube acting as a real proxy to manipulate a virtual object in the same form); (2) tangible controller (i.e., a time-multiplexed tangible interaction with a physical controller applied to manipulate a virtual object); and (3) hand motion (i.e., a form of mid-air interaction to move a virtual object with hands). Based on the findings from self-report, electroencephalography (EEG), and performance measures, this study reveals the advantages of the tangible cube over the tangible controller, both superior to the hand motion in hand-held AR rehabilitation regarding user experience and motor performance. This study offers new understanding of the advantages and disadvantages of various interaction techniques in hand-held AR rehabilitation, emphasizing crucial design considerations for these systems, with a focus on user experience and motor performance in upper limb rehabilitation.

Users’ motion representation in virtual reality (VR) can be modulated visually by introducing a mismatch with their real motion, which can bring benefits to exercise and rehabilitation and has great potential for exergame applications in VR. Users’ experience of control is a critical consideration for user experience in human–computer interaction and should be paid special attention when movement modulation is implemented in VR. However, how movement modulation affects users’ experience of control and motor performance has not been fully investigated in detail. This research included 49 participants and investigated how the experience of control is influenced by reaching movement modulation in two types: the enhancement and reduction modes. Different modulation modes were designed to study their influence on the explicit experience of control in self-ratings and the implicit measured experience of control in intentional binding and electroencephalography. Participants’ movement trajectory, velocity, and completion time were analyzed for motor performance. The results illustrate a significant effect of movement modulation on the users’ motor performance and experience of control in self-ratings and EEG. This study makes a major contribution through a comprehensive analysis of the experience of control with movement modulation and provides important and practical design considerations on movement modulation design in future exercise-based applications with positive controlling experiences in VR.

The use of Virtual Reality (VR) technology, especially in medical rehabilitation, has expanded to include tactile cues along with visual stimuli. For patients with upper limb hemiplegia, tangible handles with haptic stimuli could improve their ability to perform daily activities. Traditional VR controllers are unsuitable for patient rehabilitation in VR, necessitating the design of specialized tangible handles with integrated tracking devices. Besides, matching tactile stimulation with corresponding virtual visuals could strengthen users' embodiment (i.e., owning and controlling virtual bodies) in VR, which is crucial for patients' training with virtual hands. Haptic stimuli have been shown to amplify the embodiment in VR, whereas the effect of partial tactile stimulation from tangible handles on embodiment remains to be clarified. This research, including three experiments, aims to investigate how partial tactile feedback of tangible handles impacts users' embodiment, and we proposed a design concept called TouchMark for partial tactile stimuli that could help users quickly connect the physical and virtual worlds. To evaluate users' tactile and comfort perceptions when grasping tangible handles in a non-VR setting, various handles with three partial tactile factors were manipulated in Study 1. In Study 2, we explored the effects of partial feedback using three forms of TouchMark on the embodiment of healthy users in VR, with various tangible handles, while Study 3 focused on similar investigations with patients. These handles were utilized to complete virtual food preparation tasks. The tactile and comfort perceptions of tangible handles and users' embodiment were evaluated in this research using questionnaires and interviews. The results indicate that TouchMark with haptic line and ring forms over no stimulation would significantly enhance users' embodiment, especially for patients. The low-cost and innovative TouchMark approach may assist users, particularly those with limited VR experience, in achieving the embodiment and enhancing their virtual interactive experience.

Virtual reality (VR) has emerged as a powerful tool for rehabilitation. Many effective VR applications have been developed to support motor rehabilitation of people affected by motor issues. Movement reproduction, which transfers users' movements from the physical world to the virtual environment, is commonly used in VR rehabilitation applications. Three major components are required for movement reproduction in VR: (1) movement input, (2) movement representation, and (3) movement modulation. Until now, movement reproduction in virtual rehabilitation has not yet been systematically studied. This article aims to provide a state-of-the-art review on this subject by focusing on existing literature on immersive motor rehabilitation using VR. In this review, we provided in-depth discussions on the rehabilitation goals and outcomes, technology issues behind virtual rehabilitation, and user experience regarding movement reproduction. Similarly, we present good practices and highlight challenges and opportunities that can form constructive suggestions for the design and development of fit-for-purpose VR rehabilitation applications and can help frame future research directions for this emerging area that combines VR and health.

As players demand better-quality horror games, a study is conducted to find the more practical design choices to induce fear and tension in first-person PC horror games. This research investigates designs on enemy appearances and movement artificial intelligence logic and builds corresponding demos for experiments. The data is collected by questionnaires and experiments that record participants’ heart rates when playing horror games. The results show that using enemy designs with uncanny valley effect appearance and outflanking behaviour movement AI effectively induces fear and tension in first-person PC horror games. This study makes contribution to the design guidelines for game designers and developers in developing better-quality horror games.

Virtual avatars or hands in virtual reality connect users’ physical bodies and virtual worlds. Changes in the virtual hand representations (body continuity and hand realism) may affect user perceptions and task performance. However, there is no agreed conclusion on how they influence user perceptions (the sense of embodiment and presence) and limited evidence of task performance. Therefore, this paper investigates the impact of body continuity (connected and disconnected virtual hand) with three hand realism levels on user perceptions and task performance by self-report and objective performance data in virtual reality. The results revealed no significant results about body continuity on user perceptions, while a significant effect of hand realism levels on sense of embodiment and presence was found. Moreover, the abstract disconnected and connected hands reported lower task scores than those realistic hands from task performance data. Overall, this study provides new insights into further understanding user perceptions and task performance under the connected and disconnected hands, and it has practical reference value for exploring the later research on virtual hand representations.

Error-related potentials (ErrPs) are electrical signals of brain activity elicited by the perception of errors. They have been studied to understand the error processing function of humans and decoded to be used to detect erroneous output in brain-computer interface (BCI) systems. This study investigated the influence of task difficulty on ErrPs elicited by user-initiated outcome errors in an interactive game. The time-domain analysis showed the result that the highest peak of hard-level ErrPs had a longer latency than that of easy-level ErrPs. The time-frequency analysis showed that the power of α (8–13Hz) bands increases at about 200–600ms and the power of 1–5Hz increases starting at about 50ms for easy-level ErrPs, while the power of β (14–30Hz) bands increases at about 100–1000ms and the power of 1–5Hz increases start at about 300ms for hard-level ErrPs. The results provided preliminary evidence that the error processing takes more time for a task with higher difficulty levels, and the difference in spectral power distribution implies the differences in cognitive processes. The study helped explore the modulation mechanism of ErrPs and provided reference factors for the selection of ErrPs decoding algorithm in BCI systems to improve its efficiency and user experience.

Materials play an essential role in product design and affect many design aspects. Materials libraries are built by universities to provide resources and inspire design concept generation and decision-making. Augmented reality (AR) is a technology overlaying digital content onto the physical world and brings a new perspective on material library through increased engagement and interactivity. This work-in-progress paper aims to enhance materials education and foster disciplinary communication by establishing an AR materials library. The proposed library will contribute to design and engineering education by serving as a practical platform with material resources and a novel tool engaged in learning.

Deep transfer learning (DTL) has developed rapidly in the field of motor imagery (MI) on brain-computer interface (BCI) in recent years. DTL utilizes deep neural networks with strong generalization capabilities as the pre-training framework and automatically extracts richer and more expressive features during the training process. The goal of this paper is utilizing the DTL to classify MI electroencephalogram (EEG) signals on the premise of a small data set. The publicly available dataset III of the second BCI competition is applied in both the training part and testing part to evaluate the effectiveness of the proposed method. Firstly in the process, finite impulse response (FIR) filter and wavelet transform threshold denoising method are used to remove redundant signals and artifacts in EEG signals. Then, the continuous wavelet transform (CWT) is utilized to convert the one-dimensional EEG signal into a two-dimensional time-frequency amplitude representation as the input of the pre-trained convolutional neural network (CNN) for classifying two types of MI signals. Employing the input data of 140 trials for training, the final classification accuracy rate reaches 96.43%. Compared with the results of some superior machine learning models using the same data set, the accuracy and Kappa value of this DTL model are better. Therefore, the proposed scheme of MI EEG signal classification based on the DTL method offers preferably empirical performance.

Virtual object manipulation as a key feature has been studied in virtual reality (VR) environments. Previous studies highlighted user experience on three basic types of virtual object manipulation, translation, rotation and scaling. However, prior literature mainly studied task performance in manipulation modes with different degrees of freedom (DoF), and few studies assessed user experience by evaluating the psychological response, such as mental workload on these three basic manipulation types in virtual environments. This paper compared manipulation modes with 1DoF and 3DoF to assess users’ mental workload as a critical indicator of user experience by electroencephalogram (EEG) measurement and questionnaires in manipulation tasks on the webpage with VR effects (also known as WebVR). By applying signal processing and statistical methods to analyze EEG data from ten subjects, the results demonstrated that the participants generally perceive less mental workload by 1DoF manipulation modes than 3DoF on WebVR. Besides, this study also found some different results between objective and subjective data.

Tree-structured data are becoming ubiquitous nowadays and manipulating them based on similarity is essential for many applications. The generally accepted similarity measure for trees is the edit distance. Although similarity search has been extensively studied, searching for similar trees is still an open problem due to the high complexity of computing the tree edit distance. In this paper, we propose to transform tree-structured data into an approximate numerical multidimensional vector which encodes the original structure information. We prove that the L1 distance of the corresponding vectors, whose computational complexity is O(|T1| + |T2|), forms a lower bound for the edit distance between trees. Based on the theoretical analysis, we describe a novel algorithm which embeds the proposed distance into a filter-and-refine framework to process similarity search on tree-structured data. The experimental results show that our algorithm reduces dramatically the distance computation cost. Our method is especially suitable for accelerating similarity query processing on large trees in massive datasets.

Recently, techniques for supporting efficient similarity search over huge transaction datasets have emerged as an important research area. Several indexing schemes have been proposed towards this direction. Typically, these schemes provide a tradeoff between searching efficiency and indexing overhead in terms of space.