Multimodal imaging has become an essential tool in clinical ophthalmology, offering complementary perspectives for disease diagnosis. However, current automated diagnostic approaches often fail to fully exploit the rich, complementary information provided by different imaging modalities. In this paper, to advance automated ophthalmic disease diagnosis through effective multimodal data integration, we propose the OphFusionNet, a novel multimodal learning framework based on uncertainty-driven multi-scale multimodal feature fusion. Drawing inspiration from clinical observations that ophthalmic lesions often appear at multiple spatial scales, we design a multi-scale feature fusion module with sparse self-attention (MSFF-SSA). This module captures hierarchical representations while suppressing redundancy, thereby enhancing both the expressiveness and efficiency of the extracted features. To further improve multimodal fusion, we introduce an uncertainty-aware multimodal fusion module with a game-theoretic selection strategy (UMF-GTSS). This component estimates the uncertainty associated with different features and adaptively weights them based on their relative reliability, yielding more robust and trustworthy diagnostic outcomes. To mitigate the tendency to over-rely on dominant modalities and underutilize the informative potential of subordinate ones, we propose a modality distillation strategy (MDS), which leverages multimodal features to guide and refine the learning of single-modal representations. This strategy enhances generalization and boosts the discriminative capacity of each individual modality. The OphFusionNet was evaluated on four publicly available ophthalmic datasets. Extensive experiments demonstrate that our approach achieves superior multimodal integration, resulting in state-of-the-art (SOTA) performance in multimodal ophthalmic disease diagnosis. The code will be available at: https://github.com/wb66715/OphFusionNet.

Segmentation of prostate cancer on ultrasound (US) images is challenging because of the low-intensity contrast around the region of interest (ROI) contour caused by the intestinal gas. To the best of our knowledge, our work is one of the extremely rare groups to segment prostate cancer on US images, which designs a coarse-to-refinement segmentation framework with four merits: 1) it fuses the advantages of the deep learning- and principal curve-based algorithms to locate the ROI and fit the data center automatically, respectively; 2) a principal curve-based polygon searching method is designed by newly adding some constraints so that the performance of our method could be enhanced; 3) the quantum characteristics are added into evolution network, while newly adding numerous- operator scheme and global optimum search schemes; 4) an interpretable mathematical mapping model is developed to denote the ROI contour. Segmentation results show that our network outperforms other medical image segmentation models.

Ultrasound kidney image segmentation presents significant challenges due to missing or ambiguous boundaries. In this study, we introduce a coarse-to-refinement approach incorporating four novel aspects. Firstly, we leverage the properties of a principal curve (PC) to automatically fine-tune the curve shape and employ a neural network's learning ability to reduce model error. Secondly, a deep fusion learning network is utilized for the coarse segmentation step, incorporating a parallel architecture to enhance deep-learning performance. Thirdly, addressing the limitation of standard PC-based methods in determining the number of vertices automatically, we propose an automatic searching polygon tracking method using a mean shift clustering-based approach to replace the projection and vertex extension step in standard PC-based methods. Lastly, we develop an explainable mathematical map function for the kidney contour, as denoted by the neural network output (i.e., optimized vertices), which aligns well with the ground truth contour. We conducted various experiments to evaluate our method's performance, demonstrating its effectiveness in ultrasound kidney image segmentation.

Ultrasound image segmentation is crucial for early disease detection and treatment planning but remains a challenging task due to the low contrast of organ boundaries and varying image quality. Current methods often require manual intervention or have limited accuracy. In this paper, we propose a novel hybrid framework that combines an automatic option polygon segment (AOPS) algorithm and a distributed- and memory-based evolution (DME) algorithm for precise ultrasound organ segmentation. Our pipeline consists of two cascaded stages: (1) a coarse segmentation step using the AOPS algorithm, which determines the number of vertices/clusters without human intervention, and (2) a refinement step using the DME algorithm to hunt for the optimal neural network, which is then used to represent a smooth, explainable mathematical expression of the organ boundary. We employ the fractional backpropagation learning network with L2 regularization (FBLN) for training and use the scaled exponential linear unit (SELU) activation function to address the vanishing gradient problem. This is a new attempt such a hybrid framework is applied to ultrasound organ segmentation tasks, and it demonstrates significant contributions in terms of accuracy, smoothness, and computational efficiency.

Accurate segmentation of the prostate is important to image-guided prostate biopsy and brachytherapy treatment planning. However, the incompleteness of prostate boundary increases the challenges in the automatic ultrasound prostate segmentation task. In this work, an automatic coarse-to-fine framework for prostate segmentation was developed and tested. Our framework has four metrics: first, it combines the ability of deep learning model to automatically locate the prostate and integrates the characteristics of principal curve that can automatically fit the data center for refinement. Second, to well balance the accuracy and efficiency of our method, we proposed an intelligent determination of the data radius algorithm-based modified polygon tracking method. Third, we modified the traditional quantum evolution network by adding the numerous-operator scheme and global optimum search scheme for ensuring population diversity and achieving the optimal model parameters. Fourth, we found a suitable mathematical function expressed by the parameters of the machine learning model to smooth the contour of the prostate. Results on the multiple datasets demonstrate that our method has good segmentation performance.

Abstract Due to the varying appearance in the upper clavicle bone region, sharp corner at the costophrenic angle, the presence of strong edges at the rib cage and clavicle and the lack of a consistent anatomical shape among different individuals, accurate segmentation of lung on chest radiographs remains challenging. In this work, we propose a novel segmentation method for lung segmentation, containing two subnetworks, where few manually delineated points are used as the approximate initialization. The first one is a preprocessing subnetwork based on a deep learning model (i.e. Deep Belief Network and K-Nearest Neighbor). The second one is a refinement subnetwork, designed to make the preprocessed result to be optimized by combining an improved principal curve method and a machine learning method. To prove the performance of the proposed method, several public datasets were evaluated with Dice Similarity Coefficient (DSC), overlap score (Ω), Sensitivity (Sen), Positive Predictive Value (PPV), global Error (E) and execution time (t). Compared with state-of-the-art methods, our method reaches superior segmentation performance.

Among all image features, the contour is one of the most critical features for displaying the shape of the object intuitively. Due to unseen/missing regions of transrectal ultrasound images caused by imaging artifacts and limited field of view, accurate and robust ultrasound prostate contour extraction is challenging. Hence, we propose a triple cascaded framework for ultrasound prostate contour extraction using a few existing points as the prior. The proposed scheme contains two types of data mining: principal curve-based and machine learning-based methods. The first stage is using an improved polygonal segment method to obtain a contour composed of line segments connected by sorted vertices, where only a few radiologist-defined seed points are used as the prior. The second stage is to achieve an optimal machine learning-based approach based on an improved differential evolution-based method. The third stage is to find a map function (realized by the machine learning-based method) to generate the smooth contour represented by the output of neural network (i.e., optimized vertices) to match the ground truth contour. Our results demonstrated that the performance of the proposed method outperformed several other state-of-the-art methods.