Advancements in artificial intelligence have propelled affective computing toward unprecedented accuracy and real-world impact. By leveraging the unique strengths of brain signals and ocular dynamics, we introduce a novel multimodal framework that integrates EEG and eye-movement (EM) features synergistically to achieve more reliable emotion recognition. First, our EEG Feature Encoder (EFE) uses a convolutional architecture inspired by the human visual cortex's eccentricity-receptive-field mapping, enabling the extraction of highly discriminative neural patterns. Second, our EM Feature Encoder (EMFE) employs a Kolmogorov-Arnold Network (KAN) to overcome the sparse sampling and dimensional mismatch inherent in EM data; through a tailored multilayer design and interpolation alignment, it generates rich, modality-compatible representations. Finally, the core Multimodal Iterative Attentional Feature Fusion (MIAFF) module unites these streams: alternating global and local attention via a Hierarchical Channel Attention Module (HCAM) to iteratively refine and integrate features. Comprehensive evaluations on SEED (3-class) and SEED-IV (4-class) benchmarks show that our method reaches leading-edge accuracy. However, our experiments are limited by small homogeneous datasets, untested cross-cultural robustness, and potential degradation in noisy or edge-deployment settings. Nevertheless, this work not only underscores the power of biomimetic encoding and iterative attention but also paves the way for next-generation brain-computer interface applications in affective health, adaptive gaming, and beyond.

Spiking neural networks (SNNs) can be operated in an event-driven manner to save energy consumption of artificial neural networks (ANNs), which has attracted enormous research interests for their high biological plausibility and powerful spatiotemporal information processing. However, representative studies only evaluated SNNs on static temporal tasks or short sequence tasks, which could not fully demonstrate the advantages of SNNs in spatiotemporal learning. In addition, we point out that the existing directly trained SNNs to face the problems of long-term memory, network degeneration, gradient saturation, and heterogeneity learning, these limit the performance of SNNs. In this article, we propose channelwise regional integrate and multiple firing (CRIMF) neuron to improve the spatiotemporal learning of SNNs. First, CRIMF neuron contains a new internal state of regional current that enhances the memory of spiking neurons and facilitates the learning of temporal information over long time steps. Second, CRIMF neuron is implemented with the multiple firing mechanisms; it is able to adjust the distribution of membrane potential and membrane potential gradient in the single firing mechanism, thus mitigating the underactivation and gradient saturation. Third, CRIMF neuron is trained with the channelwise learning strategy for the targeted learning of different types of temporal features, and an index of differentiation degree is proposed to visualize the effectiveness of the channelwise learning strategy. We also introduce the regional current reset equation and normalize the input of postsynaptic neurons in spatiotemporal dimension to avoid network degeneration. Finally, we select two emotion electroencephalogram (EEG) datasets and perform the evaluations based on manual features and raw signals. Experimental results show that CRIMF-based SNNs outperform the state-of-the-art methods in static temporal task, and CRIMF neurons are superior to the advanced spiking neurons and recurrent units of ANNs in dynamic temporal task, using low energy consumption.

Since digital spiking signals can carry rich information and propagate with low computational consumption, spiking neural networks (SNNs) have received great attention from neuroscientists and are regarded as the future development object of neural networks. However, generating the appropriate spiking signals remains challenging, which is related to the dynamics property of neurons. Most existing studies imitate the biological neurons based on the correlation of synaptic input and output, but these models have only one time constant, thus ignoring the structural differentiation and versatility in biological neurons. In this article, we propose the reconstruction of adaptive leaky integrate-and-fire (R-ALIF) neuron to perform complex behaviors similar to real neurons. First, a synaptic cleft time constant is introduced into the membrane voltage charging equation to distinguish the leakage degree between the neuron membrane and the synaptic cleft, which can expand the representation space of spiking neurons to facilitate SNNs to obtain better information expression way. Second, R-ALIF constructs a voltage threshold adjustment equation to balance the firing rate of output signals. Third, three time constants are transformed into learnable parameters, enabling the adaptive adjustment of dynamics equation and enhancing the information expression ability of SNNs. Fourth, the computational graph of R-ALIF is optimized to improve the performance of SNNs. Moreover, we adopt a temporal dropout (TemDrop) method to solve the overfitting problem in SNNs and propose a data augmentation method for neuromorphic datasets. Finally, we evaluate our method on CIFAR10-DVS, ASL-DVS, and CIFAR-100, and achieve top1 accuracy of 81.0%, 99.8%, and 67.83%, respectively, with few time steps. We believe that our method will further promote the development of SNNs trained by spatiotemporal backpropagation (STBP).

Although electromyography (EMG) signals and interaction force have been widely used in patient cooperative or interactive training, the conventional EMG based control usually breaks the process into a patient-driven phase and a separate passive phase, which is not desirable. In this research, an active interaction controller based on motion recognition and adaptive impedance control is proposed and implemented on a six-DOFs parallel robot for lower limb rehabilitation. The root mean square (RMS) features of EMG signals integrating with the support vector machine (SVM) classifier were used to online predict the lower limb intention in advance and to trigger the robot assistance. The impedance control strategy was adopted to directly influence the robot assistance velocity and allow the exercise to follow a physiological trajectory. Moreover, an adaptive scheme learned the muscle activity level in real time and adapted the robot impedance in accordance with patient's voluntary participation efforts. Experimental results on several healthy subjects demonstrated that the lower limb motion intention can be precisely predicted in advance, and the robot assistance mode was also adjustable based on human-robot interaction and muscle activity level of subjects. Comparing with the conventional EMG-triggered assistance methods, such a strategy can increase patient's motivation because the subject's movement intention, active efforts as well as the muscle activity level changes can be directly reflected in the trajectory pattern and the robot assistance speeds.