Accurate brain tumor segmentation plays an important role in clinical diagnosis, treatment planning, and therapeutic response monitoring. Multi-modal MRI provides complementary structural and functional information, but existing methods remain limited by their inadequate exploitation of cross-modal complementarity and their inability to effectively handle modality-specific disparities and redundant information. To address these challenges, this paper proposes a novel hierarchical teacher-student learning framework with adaptive cross-modal fusion. MRI modalities are grouped into teacher modalities (Flair and T1c) and student modalities (T2 and T1) based on their intrinsic tumor-related characteristics. Central to this framework is the Modality Guidance Module (MGM), which consists of two key components designed to achieve multi-modal feature distillation. Within MGM, the Modality Enhancement Module (MEM) extracts highly discriminative features from teacher modalities. While the Modality Fusion Module (MFM) leverages these features to guide and refine the learning of student modalities. To further capture inter-modal dependencies, a Cross-Modal Fusion Module (CMFM) is introduced to adaptively integrate complementary information across all modalities. Extensive experiments on the BraTS 2018, 2019 and 2020 datasets demonstrate that the proposed method achieves superior performance compared with state-of-the-art approaches. Beyond brain tumor segmentation, the hierarchical teacher-student paradigm and adaptive fusion strategy also hold potential for broader multi-modal image analysis tasks.

Multimodal survival analysis aims to combine heterogeneous data sources to improve the prediction quality of survival outcomes. However, this task is particularly challenging due to high heterogeneity and noise across data sources. Additionally, the exact survival time is often censored (partially known) due to incomplete event observation. To address the above challenges, we propose a novel interpretable evidential multimodal survival fusion model, EsurvFusion. This model is designed to combine multimodal data at the decision level using Epistemic Random Fuzzy Sets that jointly handle both data and model uncertainty while incorporating modality-level reliability. Specifically, EsurvFusion first models unimodal data with newly introduced Gaussian random fuzzy numbers, producing possible unimodal survival predictions along with corresponding aleatory and epistemic uncertainty. It then estimates modality-level reliability through a reliability discounting layer to correct the misleading impact of noisy data modalities. Finally, a multimodal evidence fusion layer is introduced to combine the discounted predictions, revealing modality-level influence based on the learned reliability coefficients. Extensive experiments on four multimodal cancer survival datasets demonstrate the effectiveness of our model in handling highly heterogeneous data, establishing a new state-of-the-art performance on several benchmarks.

Modelling patient trajectories from longitudinal electronic health records (EHRs) is crucial for early chronic disease prediction. Foundation models (FMs), benefiting from the computational power and generalization abilities, offer a promising direction towards understanding patient health progression. However, key challenges of adopting FMs in clinical decisions remain in (1) incorporating multi-modal EHR data into an FM effectively for unified patient representations and (2) ensuring model generalizability across various clinical domains with distribution shifts. To address these challenges, we propose MsHeCare, an FM-based, two-stage learning framework integrating multi-modal representation learning and multi-source domain adaptation (MSDA). In the pre-training stage, MsHeCare performs self-supervised contrastive learning and masked language modelling tasks to mitigate semantic biases across text-based diagnostic and treatment sequences. A cross-attention mechanism further fuses these temporal features with structured static demographic information, enhancing personalized patient representations. In the fine-tuning stage, MsHeCare incorporates an MSDA framework with a novel source importance estimation method, facilitating adaptive knowledge transfer across domains and improving model generalizability. Experiments on two real-world EHR datasets demonstrate that MsHeCare significantly outperforms single-domain baselines and state-of-the-art MSDA methods. Furthermore, we validate the robustness of MsHeCare across varying target-domain data sizes and demonstrate its alignment with clinical practices through case studies, underscoring its potential for multi-modal, domain-adaptive predictive healthcare systems.

Qika Lin, Tianzhe Zhao, Kai He, Zhen Peng, Fangzhi Xu, Ling Huang, Jingying Ma, Mengling Feng. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Time-to-event analysis provides insights into clinical prognosis and treatment recommendations. However, this task is more challenging than standard regression problems due to the presence of censored observations. Additionally, the lack of confidence assessment, model robustness, and prediction calibration raises concerns about the reliability of predictions. To address these challenges, we propose an evidential regression model specifically designed for time-to-event prediction. Our approach computes a degree of belief for the event time occurring within a time interval, without any strict distribution assumption. Meanwhile, the proposed model quantifies both epistemic and aleatory uncertainties using Gaussian Random Fuzzy Numbers and belief functions, providing clinicians with uncertainty-aware survival time predictions. Experimental evaluations using simulated and real-world survival datasets highlight the potential of our approach for enhancing clinical decision-making in survival analysis. • Calculate survival time without strict distribution assumptions. • Quantifies epistemic and aleatory uncertainties via GRFN and belief functions. • Provides uncertainty-aware survival predictions for clinicians. • Validated on simulated & real-world survival datasets.

Medical image segmentation is critical for disease diagnosis and treatment assessment. However, concerns regarding the reliability of segmentation regions persist among clinicians, mainly attributed to the absence of confidence assessment, robustness, and calibration to accuracy. To address this, we introduce deep evidential segmentation model (DEviS), an easily implementable foundational model that seamlessly integrates into various medical image segmentation networks. DEviS not only enhances the calibration and robustness of baseline segmentation accuracy but also provides high-efficiency uncertainty estimation for reliable predictions. By leveraging subjective logic theory, we explicitly model probability and uncertainty for medical image segmentation. Here, the Dirichlet distribution parameterizes the distribution of probabilities for different classes of the segmentation results. To generate calibrated predictions and uncertainty, we develop a trainable calibrated uncertainty penalty. Furthermore, DEviS incorporates an uncertainty-aware filtering (UAF) module, which designs the metric of uncertainty-calibrated error to filter out-of-distribution (OOD) data. We conducted validation studies on publicly available datasets, including ISIC2018, KiTS2021, LiTS2017, and BraTS2019, to assess the accuracy and robustness of different backbone segmentation models enhanced by DEviS, as well as the efficiency and reliability of uncertainty estimation. Additionally, two potential clinical trials were conducted using the UAF module. The clinical application conducted on the Johns Hopkins OCT and Duke OCT-DME datasets demonstrated the effectiveness of the model in filtering OOD data. The second trial evaluated its efficacy in filtering high-quality data on the FIVES datasets. At last, the proposed DEviS method was extended to semi-supervised medical image segmentation, where it exhibited strong robustness under noisy conditions. Our code has been released in https://github.com/Cocofeat/DEviS.

The comprehensive integration of machine learning healthcare models within clinical practice remains suboptimal, notwithstanding the proliferation of high-performing solutions reported in the literature. A predominant factor hindering widespread adoption pertains to an insufficiency of evidence affirming the reliability of the aforementioned models. Recently, uncertainty quantification methods have been proposed as a potential solution to quantify the reliability of machine learning models and thus increase the interpretability and acceptability of the results. In this review, we offer a comprehensive overview of the prevailing methods proposed to quantify the uncertainty inherent in machine learning models developed for various medical image tasks. Contrary to earlier reviews that exclusively focused on probabilistic methods, this review also explores non-probabilistic approaches, thereby furnishing a more holistic survey of research pertaining to uncertainty quantification for machine learning models. Analysis of medical images with the summary and discussion on medical applications and the corresponding uncertainty evaluation protocols are presented, which focus on the specific challenges of uncertainty in medical image analysis. We also highlight some potential future research work at the end. Generally, this review aims to allow researchers from both clinical and technical backgrounds to gain a quick and yet in-depth understanding of the research in uncertainty quantification for medical image analysis machine learning models.

With the wide use of visual sensors in the Internet of Things (IoT) in the past decades, huge amounts of images are captured in people's daily lives, which poses challenges to traditional deep-learning-based image retrieval frameworks. Most such frameworks need a large amount of annotated training data, which are expensive. Moreover, machines still lack human intelligence, as illustrated by the fact that they pay less attention to the interesting regions that humans generally focus on when searching for images. Hence, this paper proposes a novel unsupervised framework that focuses on the instance object in the image and integrates human intelligence into the deep-learning-based image retrieval. This framework is called adversarial instance-level image retrieval (AILIR). We incorporate adversarial training and an attention mechanism into this framework that considers human intelligence with artificial intelligence. The generator and discriminator are redesigned to guarantee that the generator retrieves similar images while the discriminator selects unmatched images and creates an adversarial reward for the generator. A minimax game is conducted by the adversarial reward retrieval mechanism until the discriminator is unable to judge whether the image sequence retrieved matches the query. Comparison and ablation experiments on four benchmark datasets prove that the proposed adversarial training framework indeed improves instance retrieval and outperforms the state-of-the-art methods focused on instance retrieval.