Multimodal fusion has become an important research technique in neuroscience that completes downstream tasks by extracting complementary information from multiple modalities. Existing multimodal research on brain networks mainly focuses on two modalities, structural connectivity (SC) and functional connectivity (FC). Recently, extensive literature has shown that the relationship between SC and FC is complex and not a simple one-to-one mapping. The coupling of structure and function at the regional level is heterogeneous. However, all previous studies have neglected the modal regional heterogeneity between SC and FC and fused their representations via "simple patterns", which are inefficient ways of multimodal fusion and affect the overall performance of the model. In this paper, to alleviate the issue of regional heterogeneity of multimodal brain networks, we propose a novel Regional Heterogeneous multimodal Brain networks Fusion Strategy (RH-BrainFS). Briefly, we introduce a brain subgraph networks module to extract regional characteristics of brain networks, and further use a new transformer-based fusion bottleneck module to alleviate the issue of regional heterogeneity between SC and FC. To the best of our knowledge, this is the first paper to explicitly state the issue of structural-functional modal regional heterogeneity and to propose a solution. Extensive experiments demonstrate that the proposed method outperforms several state-of-the-art methods in a variety of neuroscience tasks.

Abstract Most of the shape classification methods are based on a single closed contour. However, practical shapes always have complex contours, for example, a combination of multiple open contours. How to accurately identify complex shapes is an unsolved problem. In this research, a novel method is proposed to classify complex shapes. The proposed method firstly encodes a complex shape to an angle code and a sparsity code, then input these codes to a 1‐D CNN for extracting features and classification. Experiments on two datasets show this novel method is superior in terms of classification accuracy. These two datasets are practical shape dataset collected by this paper on internet and MPEG‐7 CE‐1 Part B. The proposed method achieves higher classification accuracy than compared methods. In order to show the performance of the proposed method on each class, the accuracy on each class is analyzed. Ablation experiment is conducted to show the contribution of each module in the network. The result shows that each module is meaningful in the network, because without any module the accuracy drops.