Accurate segmentation of gliomas in multi-modal MRI is a critical prerequisite for clinical diagnosis and treatment. Deep learning has made significant progress in automated glioma segmentation. However, existing methods still face challenges in multi-scale feature extraction and multi-modal feature fusion. This paper proposes a Global-Local Collaboration and Task-driven Fusion Network (GLC- TFNet) for glioma segmentation in multi-modal MRIs. To enhance multi-scale feature extraction, a global-local collaboration encoder is proposed, which utilizes a dual-path mechanism to synergistically integrate global contextual information with local details. For multi-modal feature fusion, a task-driven fusion module is designed, which incorporates clinical diagnostic priors as constraints to optimize the contributions of multi-modal features in critical regions. Comprehensive experiments conducted on two publicly released datasets, BraTS2020 and BraTS2021, demon-strate that GLC- TFNet outperforms current state-of-the-art methods. Specifically, it achieves Dice scores of 94.82%,90.08%, and 88.09% for the whole tumor, tumor core, and enhancing tumor on the BraTS2020 dataset, and 92.88%, 90.61 %, and 87.06% on the BraTS2021 dataset, respectively.

Physicians typically combine multi-modal data to make a graded diagnosis of breast tumors. However, most existing breast tumor grading methods rely solely on image information, resulting in limited accuracy in grading. This paper proposes a Multi-information Selection Aggregation Graph Convolutional Networks (MSA-GCN) for breast tumor grading. Firstly, to fully utilize phenotypic data reflecting the clinical and pathological characteristics of tumors, an automatic combination screening and weight encoder is proposed for phenotypic data, which can construct a population graph with improved structural information. Then, a graph structure is designed through similarity learning to reflect the correlation between patient image features. Finally, a multi-information selection aggregation mechanism is employed in the graph convolution model to extract the effective features of multi-modal data and enhance the classification performance of the model. The proposed method is evaluated on different clinical datasets from the Digital Database for Screening Mammography (DDSM) and INbreast. The average classification accuracies are 90.74% and 85.35%, respectively, surpassing the performance of existing methods. In conclusion, our method effectively fuses image and non-image information, leading to a significant improvement in the accuracy of breast tumor grading.

Traditional image fusion focuses on selecting an effective decomposition approach to extract representative features from the source image and attempts to find appropriate fusion rules to merge extracted features respectively. However, the existing image decomposition tools are mostly based on kernels or global energy-optimized functions limiting the performance of the wide range of image contents. This paper proposes a novel infrared and visible image fusion method based on deep decomposition network and saliency analysis (named DDNSA). First, the modified residual dense network (MRDN) is trained with a publicly available dataset to learn the decomposition process. Second, the structure and texture features of source images are separated by the trained decomposition network. Then, according to the characteristics of the above features, we construct the combination of local and global saliency maps by using stacked sparse autoencoder and visual saliency mechanism to fuse the structural features. Besides, we propose a bi-direction edge-strength fusion strategy for merging the texture features. Finally, the resultant image is reconstructed by combining the fused structure and texture features. The experimental results confirm that our proposed method outperforms the state-of-the-art methods in both visual perception and objective evaluation.

Recently, Graph Convolutional Networks (GCNs) have proven to be a powerful mean for Computer Aided Diagnosis (CADx). This approach requires building a population graph to aggregate structural information, where the graph adjacency matrix represents the relationship between nodes. Until now, this adjacency matrix is usually defined manually based on phenotypic information. In this paper, we propose an encoder that automatically selects the appropriate phenotypic measures according to their spatial distribution, and uses the text similarity awareness mechanism to calculate the edge weights between nodes. The encoder can automatically construct the population graph using phenotypic measures which have a positive impact on the final results, and further realizes the fusion of multimodal information. In addition, a novel graph convolution network architecture using multi-layer aggregation mechanism is proposed. The structure can obtain deep structure information while suppressing over-smooth, and increase the similarity between the same type of nodes. Experimental results on two databases show that our method can significantly improve the diagnostic accuracy for Autism spectrum disorder and breast cancer, indicating its universality in leveraging multimodal data for disease prediction.

In order to extract cerebrovascular vessels in Digital Subtraction Angiography sequence and improve the diagnosis accuracy, we propose a method of precise segmentation of blood vessels based on DSA sequence. Firstly, we used the piece-wise linear transformation for adjusting the grayscale and the multi-scale Hessian matrix for enhancing the overall image. Secondly, we design an algorithm using the gradient of the vascular edge to strengthen the edge. Third, we use morphological methods to eliminate the noise around the blood vessels. Finally, we segment the blood vessels and fuse the blood vessels of each frame with the weighted method to get a more comprehensive blood vessel architecture. The main contribution is to accurately segment the single-frame blood vessel information through the enhancement of blood vessel edges and noise removal, and to fuse every frame information to obtain more comprehensive blood vessels. Experimental results show that the proposed method can segment blood vessels more accurately. The result has a good visual diagnostic quality.