Motor Imagery (MI) decoding technology based on electroencephalogram (EEG) has broad application prospects in brain-computer interface and other related fields. However, existing methods suffer from two main limitations: (1) Processing the raw EEG signal holistically neglects the distinct information carried by different intrinsic frequency sub-bands, limiting the extraction of fine-grained, mode-specific features; (2) Most methods' reliance on standard convolutional networks inadequately captures the complex spatial correlations among electrode channels, failing to fully model the underlying functional brain connectivity. To overcome these limitations, we propose a Graph-Augmented Multimodal Dual-Path Network named GMDNet, which achieves a comprehensive fusion of spatio-temporal features. The first path, Local Modal Analysis, employs Multi-variable Variational Mode Decomposition (MVMD) to deconstruct the signal into multiple frequency bands, capturing detailed modal dynamics. The second path, Global Structural Embedding, utilizes a Graph Convolutional Network (GCN) to explicitly model the spatial topology of the electrode array. By adaptively learning the weights between different EEG electrode channels, this global path captures deep correlations across different brain regions. Evaluated on the BCI Competition IV 2a dataset show that the model achieves an accuracy of 80.79% in the four-category cross-session MI task, outperforming other leading methods. Moreover, the spatial features extracted by the model are highly consistent with the functional connectivity patterns of the brain during MI tasks. Our work demonstrates that GMDNet provides a powerful and robust framework for EEG decoding, holding significant promise for both BCI applications and broader neurophysiological analysis.

The inherent scarcity of high-quality electroencephalography (EEG) datasets critically constrains the development of robust brain-computer interface (BCI). Data augmentation has thus emerged as a crucial strategy for artificially enlarging the dataset. However, existing augmentation frameworks often struggle to generate highfidelity signals. In this paper, we propose a novel EEG data augmentation framework based on the Graph-Embedded Conditional Diffusion model for generating artificial EEG (GCDE) to augment dataset and improve the performance of EEG decoder. Unlike Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN), GCDE employs an iterative denoising process to generate realistic signals. We integrate Graph Embedding within a U-Net architecture to learn the spatial topological relationships among multiple EEG electrodes, thereby capturing the complex temporal and neuroscientific significance inherent EEG signals. Additionally, we incorporate label embedding to enable conditional, classspecific generation. We evaluate GCDE on the BCI Competition IV 2a dataset and investigate the optimal ratio that maximizes performance improvements. GCDE achieves significant improvements of 6.95 % in EEGNet and 3.36% in the ST-GF model when the generated data constitutes 75 % of the real data. To validate its generalizability, we further conduct experiments on the SEED (emotion recognition) and Fatigue (fatigue detection) datasets, where accuracy with EEGNet increase to$98.87 \%(+2.64 \%)$and$93.81 \%(+3.60 \%)$, respectively. These comprehensive results demonstrate that GCDE is a powerful and generalizable framework for generating high-quality EEG signals, advancing data augmentation in BCI and other EEGbased domains. Our code will be released upon acceptance.

Recent studies have integrated convolutions into transformers to introduce inductive bias and improve generalization performance. However, the static nature of conventional convolution prevents it from dynamically adapting to input variations, resulting in a representation discrepancy between convolution and self-attention as self-attention calculates attention matrices dynamically. Furthermore, when stacking token mixers that consist of convolution and self-attention to form a deep network, the static nature of convolution hinders the fusion of features previously generated by self-attention into convolution kernels. These two limitations result in a suboptimal representation capacity of the constructed networks. To find a solution, we propose a lightweight dual dynamic token mixer (D-Mixer) to simultaneously learn global and local dynamics, that is, mechanisms that compute weights for aggregating global contexts and local details in an input-dependent manner. D-Mixer works by applying an efficient global attention module and an input-dependent depthwise convolution separately on evenly split feature segments, endowing the network with strong inductive bias and an enlarged effective receptive field. We use D-Mixer as the basic building block to design TransXNet, a novel hybrid CNN-transformer vision backbone network that delivers compelling performance. In the ImageNet-1K image classification task, TransXNet-T surpasses Swin-T by 0.3% in top-1 accuracy while requiring less than half of the computational cost. Furthermore, TransXNet-S and TransXNet-B exhibit excellent model scalability, achieving top-1 accuracy of 83.8% and 84.6%, respectively, with reasonable computational costs. In addition, our proposed network architecture demonstrates strong generalization capabilities in various dense prediction tasks, outperforming other state-of-the-art networks while having lower computational costs. Code is publicly available at https://github.com/LMMMEng/TransXNet.

The Motor Imagery (MI) decoding based on electroencephalogram (EEG), has promising applications. However, most current methods face two main issues: (1) They usually rely on convolutional neural networks to extract temporal features of MI signals without fully considering the brain’s functional connectivity during MI tasks. (2) They lack analysis and recognition of MI features slices and non-tasks slices within EEG signals, leading to poor generalization and robustness. To address these problems, we propose a novel deep learning model based on graph neural network to learn spatial features between multiple electrode channels and integrate the brain’s functional connectivity features. Additionally, it restructures time slices features segmented by the sliding time window algorithm to enhance MI temporal features in EEG signal. Therefor our model achieves the fusion of spatial and temporal features. To enhance the convergence effect of the model, we introduce electrode channel spatial positions as prior knowledge to initialize the parameters of the graph convolutional network parameters. Experimental evaluations on the publicly available EEG MI dataset from BCI Competition IV 2a show that our model achieves a four-class cross-session classification accuracy of 82.38%. Compared with other methods, our model yields the best results, demonstrating its superiority. Furthermore, the results indicate that the spatial feature obtained through our model bears resemblance to the brain functional connectivity patterns identified during MI tasks. To conclude, the fusion of spatial and temporal features with graph model shows the great application potential for EEG MI signals decoding and other EEG analysis.

In the above article[1], there are errors on pages 2045 and 2046. Section METHOD.D and Section METHOD.E should be the subsections of Section METHOD.C, i.e., METHOD.C: 1) Relation Graph Construction; 2) Message Passing via Relation Graph; and 3) Mapping Disease Relation to Regions.

Detecting lesions from computed tomography (CT) scans relies on two aspects of the input: intra-slice texture information from the key slice and inter-slice structural context information from the adjacent slices. However, most existing methods ignore the correlation and complementarity between texture and structural information resulting in unexpected loss of performance. In this paper, a novel Composite Context Fusion Network (CCF-Net) is proposed to jointly model intra-slice and inter-slice features so as to prove the effectiveness of the two-steam framework. To extract both texture and structural information, two streams of 2D and 3D convolutional modules are employed in each stage. Moreover, a Composite Fusion architecture equipped with Inter-slice Correlative Fusion (ICF) modules is proposed to achieve stage-by-stage feature fusion in order to excavate and exchange information between texture-aware and context-aware features. Extensive experiments show that the proposed CCF-Net is able to achieve state-of-the-art detection performance on the multi-disease CT lesion detection task and significantly surpass the baseline methods.1

Decision response and feedback in gambling are interrelated. Different decisions lead to different ranges of feedback, which in turn influences subsequent decisions. However, the mechanism underlying the continuous decision-feedback process is still left unveiled. To fulfill this gap, we applied the hidden Markov model (HMM) to the gambling electroencephalogram (EEG) data to characterize the dynamics of this process. Furthermore, we explored the differences between distinct decision responses (i.e. choose large or small bets) or distinct feedback (i.e. win or loss outcomes) in corresponding phases. We demonstrated that the processing stages in decision-feedback process including strategy adjustment and visual information processing can be characterized by distinct brain networks. Moreover, time-varying networks showed, after decision response, large bet recruited more resources from right frontal and right center cortices while small bet was more related to the activation of the left frontal lobe. Concerning feedback, networks of win feedback showed a strong right frontal and right center pattern, while an information flow originating from the left frontal lobe to the middle frontal lobe was observed in loss feedback. Taken together, these findings shed light on general principles of natural decision-feedback and may contribute to the design of biologically inspired, participant-independent decision-feedback systems.

Instance level detection and segmentation of thoracic diseases or abnormalities are crucial for automatic diagnosis in chest X-ray images. Leveraging on constant structure and disease relations extracted from domain knowledge, we propose a structure-aware relation network (SAR-Net) extending Mask R-CNN. The SAR-Net consists of three relation modules: 1. the anatomical structure relation module encoding spatial relations between diseases and anatomical parts. 2. the contextual relation module aggregating clues based on query-key pair of disease RoI and lung fields. 3. the disease relation module propagating co-occurrence and causal relations into disease proposals. Towards making a practical system, we also provide ChestX-Det, a chest X-Ray dataset with instance-level annotations (boxes and masks). ChestX-Det is a subset of the public dataset NIH ChestX-ray14. It contains ~3500 images of 13 common disease categories labeled by three board-certified radiologists. We evaluate our SAR-Net on it and another dataset DR-Private. Experimental results show that it can enhance the strong baseline of Mask R-CNN with significant improvements. The ChestX-Det is released at https://github.com/Deepwise-AILab/ChestX-Det-Dataset.

In this paper, we propose a novel end-to-end trainable Video Question Answering (VideoQA) framework with three major components: 1) a new heterogeneous memory which can effectively learn global context information from appearance and motion features; 2) a redesigned question memory which helps understand the complex semantics of question and highlights queried subjects; and 3) a new multimodal fusion layer which performs multi-step reasoning by attending to relevant visual and textual hints with self-updated attention. Our VideoQA model firstly generates the global context-aware visual and textual features respectively by interacting current inputs with memory contents. After that, it makes the attentional fusion of the multimodal visual and textual representations to infer the correct answer. Multiple cycles of reasoning can be made to iteratively refine attention weights of the multimodal data and improve the final representation of the QA pair. Experimental results demonstrate our approach achieves state-of-the-art performance on four VideoQA benchmark datasets.

The fundamental challenge of small inter-person variation requires Person Re-Identification (Re-ID) models to capture sufficient fine-grained information. This paper proposes to discover diverse discriminative visual cues without extra assistance, e.g., pose estimation, human parsing. Specifically, a Class Activation Maps (CAM) augmentation model is proposed to expand the activation scope of baseline Re-ID model to explore rich visual cues, where the backbone network is extended by a series of ordered branches which share the same input but output complementary CAM. A novel Overlapped Activation Penalty is proposed to force the new branch to pay more attention to the image regions less activated by the old ones, such that spatial diverse visual features can be discovered. The proposed model achieves state-of-the-art results on three person Re-ID benchmarks. Moreover, a visualization approach termed ranking activation map (RAM) is proposed to explicitly interpret the ranking results in the test stage, which gives qualitative validations of the proposed method.

With the growing number of audio excerpts through various media and distribution channels, advanced audio analysis approaches have received significant interest in the multimedia field. However, current audio analysis approaches are still far from satisfactory due to the semantic gaps between the low-level acoustic features and high-level semantics perceived by human brain. In order to alleviate the problem, this paper propose a novel computational framework to bridge acoustic features with high-level semantic features derived from functional magnetic resonance imaging (fMRI) signals which record the brain's response during free listening to music/speech excerpts, and to explore the brain auditory network composition of acoustic features for different types of music/speech excerpts. Specifically, we identify meaningful brain networks and corresponding brain activities representing high-level semantic features via a novel group-wise sparse representation of whole brain fMRI signals. Then we associate the brain activities with specific low-level acoustic features and analyze the auditory network composition of acoustic features for different types of music/speech excerpts. Experimental results demonstrate that multiple acoustic features are involved in the brain auditory networks during free listening to music/speech excerpts. Meanwhile, there is considerable variability of auditory network composition of acoustic features for different types of music/speech. Our results provide new insights of how to narrow the semantic gaps in audio content analysis.

Nowadays, there are many movie trailers publicly available on social media website such as YouTube, and many thousands of users have independently indicated whether they like or dislike those trailers. Although it is understandable that there are multiple factors that could influence viewers' like or dislike of the trailer, we aim to address a preference question in this work: Can subjective multimedia features be developed to predict the viewer's preference presented by like (by thumbs-up) or dislike (by thumbs-down) during and after watching movie trailers? We designed and implemented a computational framework that is composed of low-level multimedia feature extraction, feature screening and selection, and classification, and applied it to a collection of 725 movie trailers. Experimental results demonstrated that, among dozens of multimedia features, the single low-level multimedia feature of shot length variance is highly predictive of a viewer's “like/dislike” for a large portion of movie trailers. We interpret these findings such that variable shot lengths in a trailer tend to produce a rhythm that is likely to stimulate a viewer's positive preference. This conclusion was also proved by the repeatability experiments results using another 600 trailer videos and it was further interpreted by viewers'eye-tracking data.