Detecting early brain anomalies is crucial for patient prognosis and recovery, but obtaining expert-annotated data is challenging, especially for clinically silent early brain anomalies. Unsupervised brain anomaly detection, which identifies anomalous regions by modeling normal brain patterns, has gained interest for its label efficiency. However, the inherent variability in normal brains and subtle anomalies that closely resemble normal tissue pose challenges for traditional autoencoders in distinguishing anomalies. Denoising AutoEncoder (DAE) methods have been explored to enhance the model's ability, while their success hinges on effective noise generation strategies. In this paper, we introduce a novel, structure-preserving noise generation scheme based on cross-modal CutMix, aiming to enhance the diversity of noise patterns while preserving the anatomical structure of the brain. To enhance the robustness of DAE learning, we propose an ensemble approach featuring dual experts, each incorporating distinct scale of noise. This dual-expert scheme effectively amplifies reconstruction errors in anomalous regions and suppresses false alarms in healthy areas. Additionally, we propose an anatomically-aware bidirectional consistency loss to ensure high-fidelity reconstruction at the regional level, using superpixels for anatomy perception and bidirectional distillation for reliable knowledge transfer. Extensive experiments across two different settings demonstrate the effectiveness and generalization ability of our proposed method.

Malignant brain tumors have become an aggressive and dangerous disease that leads to death worldwide. Multi-modal MRI data is crucial for accurate brain tumor segmentation, but missing modalities common in clinical practice can severely degrade the segmentation performance. While incomplete multi-modal learning methods attempt to address this, learning robust and discriminative features from arbitrary missing modalities remains challenging. To address this challenge, we propose a novel Semantic-guided Masked Mutual Learning (SMML) approach to distill robust and discriminative knowledge across diverse missing modality scenarios. Specifically, we propose a novel dual-branch masked mutual learning scheme guided by Hierarchical Consistency Constraints (HCC) to ensure multi-level consistency, thereby enhancing mutual learning in incomplete multi-modal scenarios. The HCC framework comprises a pixel-level constraint that selects and exchanges reliable knowledge to guide the mutual learning process. Additionally, it includes a feature-level constraint that uncovers robust inter-sample and inter-class relational knowledge within the latent feature space. To further enhance multi-modal learning from missing modality data, we integrate a refinement network into each student branch. This network leverages semantic priors from the Segment Anything Model (SAM) to provide supplementary information, effectively complementing the masked mutual learning strategy in capturing auxiliary discriminative knowledge. Extensive experiments on three challenging brain tumor segmentation datasets demonstrate that our method significantly improves performance over state-of-the-art methods in diverse missing modality settings.

ABSTRACT Multi‐modal diagnosis of ophthalmic disease is becoming increasingly important because combining multi‐modal data allows for more accurate diagnosis. Color fundus photograph (CFP) and optical coherence tomography (OCT) are commonly used as two non‐invasive modalities for ophthalmic examination. However, the diagnosis of each modality is not entirely accurate. Compounding the challenge is the difficulty in acquiring multi‐modal data, with existing datasets frequently lacking paired multi‐modal data. To solve these problems, we propose multi‐modal distribution fusion diagnostic algorithm and cross‐modal generation algorithm. The multi‐modal distribution fusion diagnostic algorithm first calculates the mean and variance separately for each modality, and then generates multi‐modal diagnostic results in a distribution fusion manner. In order to generate the absent modality (mainly OCT data), three sub‐networks are designed in the cross‐modal generation algorithm: cross‐modal alignment network, conditional deformable autoencoder and latent consistency diffusion model (LCDM). Finally, we propose multi‐task collaboration strategy where diagnosis and generation tasks are mutually reinforcing to achieve optimal performance. Experimental results demonstrate that our proposed method yield superior results compared to state‐of‐the‐arts.

Detecting early ischemic lesions (EIL) in computed tomography (CT) images is crucial for reducing diagnostic time and minimizing neuron loss due to oxygen deprivation. This paper introduces DCTP-Net, a dual-branch network for segmenting acute ischemic stroke lesions in CT images, consisting of a segmentation branch and a prompt-aware branch. The segmentation branch uses an encoder-decoder network as the backbone to identify lesions, where the encoder fuses CT image features with prompt features from the prompt-aware branch. To enhance semantic feature extraction and reduce the impact of cerebral structural details, we introduce a cross-collaboration dynamic connection (CCDC) module to link the encoder and decoder. The prompt-aware branch includes a learnable prompt (LP) block to incorporate cerebral prior knowledge, and the prompt-aware encoder (PAE) combines the LP block with multi-level features from the segmentation branch for more precise representation. Additionally, we propose a CLIP-enhance textual prompt (CETP) module that utilizes the CLIP text encoder to generate specialized convolutional parameters for the segmentation head. These parameters are tailored to the unique characteristics of each input image, improving segmentation performance. Qualitative and quantitative studies reveal that DCTP-Net outperforms the current state-of-the-art, IS-Net, with Dice score increases of 3.9% on AISD and 3.8% on ISLES2018, demonstrating its superiority in EIL segmentation.