Acupuncture at pericardium 6 (PC6) has been clinically found to alleviate anxiety disorders and is thought to be related to regulatory effects on the autonomic nervous system, central nervous system, and default mode network. However, the relationship between the temporal characteristics of microstate changes and the related PC6 acupoints with twirling frequency manipulation remains unclear. In this study, resting-state electroencephalogram signals from 10 healthy participants were recorded at the PC6/sham acupoint with strong/weak twirling frequency manipulation. Electroencephalogram microstate analysis was conducted to obtain microstate maps and microstate parameters under different conditions. Our results showed the regulatory effects of PC6 on microstate classes C and D; the occurrence of class C was related to twirling frequency, and the duration of class D was related to acupuncture manipulation. Taken together, our findings indicate that the temporal characteristics of the microstates of classes C and D maybe essential for PC6 acupuncture with twirling frequency manipulation.

Functional magnetic resonance imaging (fMRI) allows the observation of brain functional connectivity patterns. Attention-based diagnostic models have been widely applied in fMRI data for brain disease diagnosis. However, the global attention mechanism of the Transformer faces challenges in adaptively identifying and focusing on significant brain regions and connections relevant to disease diagnosis while reducing attention to non-relevant regions and connections in fMRI data, as well as the degradation problem of the attention mechanism, thereby limiting the improvement in diagnostic accuracy. To address these problems, we propose a connection-mask-residual focused attention network (Trifocal Transformer) based on fMRI data for brain disease diagnosis. In the Trifocal Transformer, a Connection Focus Module is developed to simulate brain functional connectivity, thereby enhancing the attention mechanism's ability to focus on significant regions and connections relevant to disease diagnosis. To mitigate the potential negative impact of non-focused regions in the attention map, a learnable Mask Focus Module is designed to adaptively reduce attention to non-relevant regions and connections. To address the degradation of the attention mechanism's focusing ability, we establish Residual Focus Connections between the attention maps, which reinforce the focusing effect across layers and ensure stable attention to significant features. Comprehensive experimental results demonstrate that the Trifocal Transformer achieves superior diagnostic accuracies of 74.1% and 71.2% on ADHD-200 and ABIDE I datasets, respectively. Furthermore, our method reveals potentially disease-related regions of interest (ROIs), providing a new neuroimaging perspective for brain disease diagnosis and treatment.

Brain-controlled robotic arms (BCRAs) have emerged as promising end-effectors in Internet of Things (IoT) environments, enabling direct neural control of physical interactions in remote and complex scenarios. However, existing BCRA systems face challenges in achieving precise fine-grained control and adapting to diverse manipulation tasks. To address these limitations, this study proposes a novel BCRA system with Human-Centered Visual Evoked Potential (HC-VEP) paradigm. By leveraging vision-based spatial mapping, the system enables fine-grained, coordinate-level manipulation of the BCRA in complex 3D environments. To further enhance system performance, Foveal Attention Tracking (FAT) is integrated to rapidly estimate the user's intended grasp location, thereby improving interaction efficiency. Additionally, a Time-Frequency Domain Enhanced Network (TFDE-Net) is developed to improve electroencephalogram (EEG) decoding accuracy through advanced time-frequency feature extraction. Experimental results demonstrate the effectiveness of the proposed system. Offline evaluations show that TFDE-Net achieves a peak information transfer rate (ITR) of 111.81 bits/min, representing a 20.8% improvement over EEGNet. Online experiments in simulated environments demonstrate the efficiency of our paradigm. Specifically, HC-VEP with FAT reduces task completion time by 56.86% compared to HC-VEP without FAT, and by 68.17% compared to traditional SSVEP. Real-world validation experiments with physical robotic arms achieved an overall success rate of 90.0% in unshielded laboratory environments, demonstrating the system's robustness under realistic operating conditions. These findings validate the system's capability for flexible and efficient grasping of arbitrary objects in complex 3D environments, marking a important step toward in practical BCRA applications.

To enhance autonomous driving, innovative approaches have been proposed to generate simulated LiDAR data. However, these methods often face challenges in producing high-quality and controllable foreground objects. To cater to the needs of object-aware tasks in 3D perception, we introduce OLiDM, a novel framework capable of generating controllable and high-fidelity LiDAR data at both the object and scene levels. OLiDM consists of two pivotal components: the Object-Scene Progressive Generation (OPG) module and the Object Semantic Alignment (OSA) module. OPG adapts to user-specific prompts to generate desired foreground objects, which are subsequently employed as conditions in scene generation, ensuring controllable and diverse output at both the object and scene levels. This also facilitates the association of user-defined object-level annotations with the generated LiDAR scenes. Moreover, OSA aims to rectify the misalignment between foreground objects and background scenes, enhancing the overall quality of the generated objects. The broad efficacy of OLiDM is demonstrated across both unconditional and conditional LiDAR generation tasks, as well as 3D perception tasks. Specifically, on the KITTI-360 dataset, OLiDM surpasses prior state-of-the-art methods such as UltraLiDAR by 11.8 in FPD, producing data that closely mirrors real-world distributions. Additionally, in sparse-to-dense LiDAR completion, OLiDM achieves a significant improvement over LiDARGen, with a 57.47% increase in semantic IoU. Moreover, in 3D object detection, OLiDM enhances the performance of mainstream detectors by 2.4% in mAP and 1.9% in NDS, underscoring its potential in advancing 3D perception models.

Autonomous driving evaluation requires simulation environments that closely replicate actual road conditions, including real-world sensory data and responsive feedback loops. However, many existing simulations need to predict waypoints along fixed routes on public datasets or synthetic photorealistic data, i.e., open-loop simulation usually lacks the ability to assess dynamic decision-making. While the recent efforts of closed-loop simulation offer feedback-driven environments, they cannot process visual sensor inputs or produce outputs that differ from real-world data. To address these challenges, we propose DrivingSphere, a realistic and closed-loop simulation framework. Its core idea is to build 4D world representation and generate real-life and controllable driving scenarios. In specific, our framework includes a Dynamic Environment Composition module that constructs a detailed 4D driving world with a format of occupancy equipping with static backgrounds and dynamic objects, and a Visual Scene Synthesis module that transforms this data into high-fidelity, multi-view video outputs, ensuring spatial and temporal consistency. By providing a dynamic and realistic simulation environment, DrivingSphere enables comprehensive testing and validation of autonomous driving algorithms, ultimately advancing the development of more reliable autonomous cars. The benchmark will be publicly released.

Autonomous driving has seen remarkable advancements, largely driven by extensive real-world data collection. However, acquiring diverse and corner-case data remains costly and inefficient. Generative models have emerged as a promising solution by synthesizing realistic sensor data. However, existing approaches primarily focus on single-modality generation, leading to inefficiencies and misalignment in multimodal sensor data. To address these challenges, we propose OminiGen, which generates aligned multimodal sensor data in a unified framework. Our approach leverages a shared Bird's Eye View (BEV) space to unify multimodal features and designs a novel generalizable multimodal reconstruction method, UAE, to jointly decode LiDAR and multi-view camera data. UAE achieves multimodal sensor decoding through volume rendering, enabling accurate and flexible reconstruction. Furthermore, we incorporate a Diffusion Transformer (DiT) with a ControlNet branch to enable controllable multimodal sensor generation. Our comprehensive experiments demonstrate that OminiGen achieves desired performances in unified multimodal sensor data generation with multimodal consistency and flexible sensor adjustments.

Synthetic data is crucial for advancing autonomous driving (AD) systems, yet current state-of-the-art video generation models, despite their visual realism, suffer from subtle geometric distortions that limit their utility for downstream perception tasks. We identify and quantify this critical issue, demonstrating a significant performance gap in 3D object detection when using synthetic versus real data. To address this, we introduce Reinforcement Learning with Geometric Feedback (RLGF), RLGF uniquely refines video diffusion models by incorporating rewards from specialized latent-space AD perception models. Its core components include an efficient Latent-Space Windowing Optimization technique for targeted feedback during diffusion, and a Hierarchical Geometric Reward (HGR) system providing multi-level rewards for point-line-plane alignment, and scene occupancy coherence. To quantify these distortions, we propose GeoScores. Applied to models like DiVE on nuScenes, RLGF substantially reduces geometric errors (e.g., VP error by 21\%, Depth error by 57\%) and dramatically improves 3D object detection mAP by 12.7\%, narrowing the gap to real-data performance. RLGF offers a plug-and-play solution for generating geometrically sound and reliable synthetic videos for AD development.

Robotic arm operation based on human–machine collaboration in manufacturing scenarios for the Internet of Things (IoT) has become an important research direction, especially in three-dimensional (3-D) scenarios that require high precision and flexible operation. However, owing to the complexity of operating robotic arms in 3-D scenarios, it is challenging for humans to perform tasks in pilot mode, leading to unnatural human–machine interactions. In this study, an embodied visual evoked potential (E-VEP) paradigm is proposed that can be used to control robotic arms in manufacturing scenarios in embodier mode. In addition, an incremental self-learning intention decoding (ISLID) algorithm is established to address the temporal variability in electroencephalography (EEG) signals. A brain-controlled robotic arm system was developed on the basis of the E-VEP paradigm and the ISLID algorithm. Online free grasping experiments revealed that the task time cost, output delay, and intention output ratio of the proposed system were 89.04 s, 2.22 s, and 46.59%, respectively. Compared with those of brain-controlled robotic arm systems based on the dynamic visual evoked potential (D-VEP) and SSVEP paradigms, the system based on the E-VEP paradigm achieved reductions in the average task time cost of 13.44% and 24.54%, respectively, and reductions in the average intention output ratio of 17.01% and 26.65%, respectively. The proposed brain-controlled robotic arm system holds significant application value in intelligent manufacturing scenarios for the IoT, advancing the integration of brain–machine interfaces and IoT technologies. The video (https://youtu.be/WtRHew4WGyo) demonstrates the utilization process of the proposed brain-controlled robotic arm.

This paper presents a novel active force-controlled end-effector with passive structural compliance and intrinsic contact sensing capabilities. Slender elastic beams are introduced to provide the end-effector with structural compliance to accommodate inevitable fluctuations through deformation during operation. Meanwhile, flexible strain gauges are embedded into the slender elastic beams to endow the end-effector with intrinsic sensing capability. A kinetostatic model is established in closed-form to map the local deflections of strain gauges to the contact force with the environment. On this basis, model-based feedback control can be readily implemented to actively adjust the contact force between the end-effector and the workpiece. A prototype is developed, on which a variety of experiments are conducted to validate the effectiveness of the proposed end-effector. The results demonstrate that the end-effector can control the contact force accurately, with a maximum error of around 0.65N. Furthermore, a demonstration of robotic grinding using the developed prototype showcases its potential for practical applications.

Advancements in brain-machine interfaces (BMIs) have led to the development of novel rehabilitation training methods for people with impaired hand function. However, contemporary hand exoskeleton systems predominantly adopt passive control methods, leading to low system performance. In this work, an active brain-controlled hand exoskeleton system is proposed that uses a novel augmented reality-fused stimulus (AR-FS) paradigm as a human-machine interface, which enables users to actively control their fingers to move. Considering that the proposed AR-FS paradigm generates movement artifacts during hand movements, an enhanced decoding algorithm is designed to improve the decoding accuracy and robustness of the system. In online experiments, participants performed online control tasks using the proposed system, with an average task time cost of 16.27 s, an average output latency of 1.54 s, and an average correlation instantaneous rate (CIR) of 0.0321. The proposed system shows 35.37% better efficiency, 8.03% reduced system delay, and 35.28% better stability than the traditional system. This study not only provides an efficient rehabilitation solution for people with impaired hand function but also expands the application prospects of brain-control technology in areas such as human augmentation, patient monitoring, and remote robotic interaction. The video in Graphical Abstract Video demonstrates the user's process of operating the proposed brain-controlled hand exoskeleton system.

Transformers have been applied to medical image segmentation tasks owing to their excellent longrange modeling capability, compensating for the failure of Convolutional Neural Networks (CNNs) to extract global features. However, the standardized self-attention modules in Transformers, characterized by a uniform and inflexible pattern of attention distribution, frequently lead to unnecessary computational redundancy with high-dimensional data, consequently impeding the model's capacity for precise concentration on salient image regions. Additionally, achieving effective explicit interaction between the spatially detailed features captured by CNNs and the long-range contextual features provided by Transformers remains challenging. In this architecture, we propose a Hybrid Transformer and CNN architecture with Multi-scale Deformable Attention(HMDA), designed to address the aforementioned issues effectively. Specifically, we introduce a Multi-scale Spatially Adaptive Deformable Attention (MSADA) mechanism, which attends to a small set of key sampling points around a reference within the multi-scale features, to achieve better performance. In addition, we propose the Cross Attention Bridge (CAB) module, which integrates multi-scale transformer and local features through channelwise cross attention enriching feature synthesis. HMDA is validated on multiple datasets, and the results demonstrate the effectiveness of our approach, which achieves competitive results compared to the previous methods.

With the rapid advancement of technology, brain-controlled driving (BCD) has emerged as a contemporary focal point of research in academia and industry. BCD refers to the application of brain-machine interface (BMI) technology to driving, where control commands from the human brain are decoded by BMI technology and used to assist in the control of vehicles. However, existing BCD systems display inadequate performance in joint lateral and longitudinal control, and BCD systems in complex driving scenarios with other vehicles have not been studied. In this study, a nonlinear smooth controller is proposed, and a BCD system for complex driving scenarios is developed based on it. First, the BCD system is built from three modules, namely, the vehicle module, the BMI module and the controller module. Subsequently, the nonlinear smooth controller is developed based on the BMI controller, the proximal policy optimization (PPO) controller, and the self-adaptive collaborative (SAC) controller. The SAC controller is designed based on a sigmoid function to achieve nonlinear smoothness in the process of allocating control authority between the PPO controller and the BMI controller. The results of online driving experiments demonstrate that the proposed controller is better equipped to handle complex driving scenarios, exhibiting superior performance, heightened safety, and improved user experience compared to the PPO controller and BMI controller. This study holds significant value in advancing the practicality of BCD and providing a foundation for future research on BMI control.

With the rapid development of the internet of things (IoT) systems, the application potential of remote-oriented unmanned aerial vehicle (UAV) in IoT systems is becoming increasingly prominent. Brain-computer interface (BCI)-based remote-oriented UAV systems can not only leverage the natural advantages of the human brain in cognition and response, but also contribute to safer and more efficient operations in certain special environments. However, remote-oriented BCI systems still face challenges in spatial perception and control capabilities. In this study, a compressed-perceptual visual evoked potentials (CPVEP) paradigm and a human-machine closed-loop (HMCL) controller are proposed for a remote-oriented brain-controlled unmanned aerial vehicle (BCUAV). A BCVAV system for remote application scenarios is constructed based on the CPVEP paradigm and the HMCL controller. Online experiments demonstrates that all subjects have completed the navigation task by the proposed remote-oriented BCUAV system. Human-in-the-loop experiments show that the proposed system can significantly improve the system performance and adaptability of BCUAV to different environments, while significantly reducing the user’s workload. In the future, the proposed remote-oriented BCUAV system can be applied to various scenarios such as remote-controlled search and rescue, traffic monitoring and power line inspection.

Structural magnetic resonance imaging (sMRI) reveals the structural organization of the brain. Learning general brain representations from sMRI is an enduring topic in neuroscience. Previous deep learning models neglect that the brain, as the core of cognition, is distinct from other organs whose primary attribute is anatomy. Capturing the high-level representation associated with inter-individual cognitive variability is key to appropriately represent the brain. Given that this cognition-related information is subtle, mixed, and distributed in the brain structure, sMRI-based models need to both capture fine-grained details and understand how they relate to the overall global structure. Additionally, it is also necessary to explicitly express the cognitive information that implicitly embedded in local-global image features. Therefore, we propose MCPATS, a brain representation learning framework that combines Multi-task Collaborative Pre-training (MCP) and Adaptive Token Selection (ATS). First, we develop MCP, including mask-reconstruction to understand global context, distort-restoration to capture fine-grained local details, adversarial learning to integrate features at different granularities, and age-prediction, using age as a surrogate for cognition to explicitly encode cognition-related information from local-global image features. This co-training allows progressive learning of implicit and explicit cognition-related representations. Then, we develop ATS based on mutual attention for downstream use of the learned representation. During fine-tuning, the ATS highlights discriminative features and reduces the impact of irrelevant information. MCPATS was validated on three different public datasets for brain disease diagnosis, outperforming competing methods and achieving accurate diagnosis. Further, we performed detailed analysis to confirm that the MCPATS-learned representation captures cognition-related information.

Recurrent neural networks (RNNs) are powerful in the tasks oriented to sequential data, such as natural language processing and video recognition. However, because the modern RNNs have complex topologies and expensive space/computation complexity, compressing them becomes a hot and promising topic in recent years. Among plenty of compression methods, tensor decomposition, e.g., tensor train (TT), block term (BT), tensor ring (TR), and hierarchical Tucker (HT), appears to be the most amazing approach because a very high compression ratio might be obtained. Nevertheless, none of these tensor decomposition formats can provide both space and computation efficiency. In this article, we consider to compress RNNs based on a novel Kronecker CANDECOMP/PARAFAC (KCP) decomposition, which is derived from Kronecker tensor (KT) decomposition, by proposing two fast algorithms of multiplication between the input and the tensor-decomposed weight. According to our experiments based on UCF11, Youtube Celebrities Face, UCF50, TIMIT, TED-LIUM, and Spiking Heidelberg digits datasets, it can be verified that the proposed KCP-RNNs have a comparable performance of accuracy with those in other tensor-decomposed formats, and even 278 219× compression ratio could be obtained by the low-rank KCP. More importantly, KCP-RNNs are efficient in both space and computation complexity compared with other tensor-decomposed ones. Besides, we find KCP has the best potential of parallel computing to accelerate the calculations in neural networks.

OBJECTIVE: Brain-computer interfaces (BCIs) have been used in two-dimensional (2D) navigation robotic devices, such as brain-controlled wheelchairs and brain-controlled vehicles. However, contemporary BCI systems are driven by binary selective control. On the one hand, only directional information can be transferred from humans to machines, such as "turn left" or "turn right", which means that the quantified value, such as the radius of gyration, cannot be controlled. In this study, we proposed a spatial gradient BCI controller and corresponding environment coordinator, by which the quantified value of brain commands can be transferred in the form of a 2D vector, improving the flexibility, stability and efficiency of BCIs. METHODS: A horizontal array of steady-state visual stimulation was arranged to excite subject (EEG) signals. Covariance arrays between subjects' electroencephalogram (EEG) and stimulation features were mapped into quantified 2-dimensional vectors. The generated vectors were then inputted into the predictive controller and fused with virtual forces generated by the robot's predictive environment coordinator in the form of vector calculation. The resultant vector was then interpreted into the driving force for the robot, and real-time speed feedback was generated. RESULTS: The proposed SGC controller generated a faster (27.4 s vs. 34.9 s) response for the single-obstacle avoidance task than the selective control approach. In practical multiobstacle tasks, the proposed robot executed 39% faster in the target-reaching tasks than the selective controller and had better robustness in multiobstacle avoidance tasks (average failures significantly dropped from 27% to 4%). SIGNIFICANCE: This research proposes a new form of brain-machine shared control strategy that quantifies brain commands in the form of a 2-D control vector stream rather than selective constant values. Combined with a predictive environment coordinator, the brain-controlled strategy of the robot is optimized and provided with higher flexibility. The proposed controller can be used in brain-controlled 2D navigation devices, such as brain-controlled wheelchairs and vehicles.