Marine Animal Segmentation (MAS) aims at identifying and segmenting marine animals from complex marine environments. Most of previous deep learning-based MAS methods struggle with the long-distance modeling issue. Recently, Segment Anything Model (SAM) has gained popularity in general image segmentation. However, it lacks of perceiving fine-grained details and frequency information. To this end, we propose a novel learning framework, named Hierarchical Frequency Prompted SAM (HFP-SAM) for high-performance MAS. First, we design a Frequency Guided Adapter (FGA) to efficiently inject marine scene information into the frozen SAM backbone through frequency domain prior masks. Additionally, we introduce a Frequency-aware Point Selection (FPS) to generate highlighted regions through frequency analysis. These regions are combined with the coarse predictions of SAM to generate point prompts and integrate into SAM's decoder for fine predictions. Finally, to obtain comprehensive segmentation masks, we introduce a Full-View Mamba (FVM) to efficiently extract spatial and channel contextual information with linear computational complexity. Extensive experiments on four public datasets demonstrate the superior performance of our approach. We will make our code publicly available upon the acceptance.

Multi-Modal Image Fusion (MMIF) aims to combine images from different modalities to produce fused images, retaining texture details and preserving significant information. Recently, some MMIF methods incorporate frequency domain information to enhance spatial features. However, these methods typically rely on simple serial or parallel spatial-frequency fusion without interaction. In this paper, we propose a novel Interactive Spatial-Frequency Fusion Mamba (ISFM) framework for MMIF. Specifically, we begin with a Modality-Specific Extractor (MSE) to extract features from different modalities. It models long-range dependencies across the image with linear computational complexity. To effectively leverage frequency information, we then propose a Multi-scale Frequency Fusion (MFF). It adaptively integrates low-frequency and high-frequency components across multiple scales, enabling robust representations of frequency features. More importantly, we further propose an Interactive Spatial-Frequency Fusion (ISF). It incorporates frequency features to guide spatial features across modalities, enhancing complementary representations. Extensive experiments are conducted on six MMIF datasets. The experimental results demonstrate that our ISFM can achieve better performances than other state-of-the-art methods. The source code is available at https://github.com/Namn23/ISFM.

Semi-supervised polyp segmentation (SSPS) is crucial for the computer-aided diagnosis of colorectal cancer, as it reduces the reliance on extensive labeled data. Although previous SSPS methods have achieved notable success, further investigation into critical issues remains necessary. In this paper, we focus on two key challenges in polyp segmentation: First, how can models extract discriminative features effectively? Second, how can we design a mechanism to suppress cognitive bias in SSPS? To address these, we propose a Feature Selection and Dual Perturbation Network integrating a feature selection and interaction module (FSIM) and a dual perturbation mechanism. The FSIM selects texture-rich information and aggregates discriminative features. Inspired by the immune response, the dual perturbation mechanism employs non-shared parameters to independently handle input and network perturbations. Moreover, a constrained loss function encourages effective collaboration among network components, enhancing robustness and reducing cognitive bias. Extensive experiments on multiple polyp datasets demonstrate our method consistently outperforms state-of-the-art SSPS approaches. Remarkably, even when using only 50 % of the labeled data, our approach surpasses several advanced fully supervised models.

Recently, methods based on Diffusion Probability Models (DPM) have achieved notable success in the field of medical image segmentation. However, most of these methods do not perform well in segmenting ambiguous areas when dealing with prostate segmentation tasks due to the low distinguishability of prostate images and the high overlap of its boundary with adjacent organs. To address this issue, this paper introduces a diffusion-based framework named ProSegDiff, ProSegDiff employs an Adapter to dynamically adjust features from the conditional network to align with the denoising process of the denoising network. Furthermore, the denoising process is conducted in the latent space to minimize the consumption of computational resources, and a proposed selection strategy is employed to identify the better results from multiple inferences. Extensive comparative experiments on four benchmark datasets demonstrate the effectiveness of this method, which achieves superior performance across four evaluation metrics.

Automatic and precise medical image segmentation (MIS) is of vital importance for clinical diagnosis and analysis. Current MIS methods mainly rely on the convolutional neural network (CNN) or self-attention mechanism (Transformer) for feature modeling. However, CNN-based methods suffer from the inaccurate localization owing to the limited global dependency while Transformer-based methods always present the coarse boundary for the lack of local emphasis. Although some CNN-Transformer hybrid methods are designed to synthesize the complementary local and global information for better performance, the combination of CNN and Transformer introduces numerous parameters and increases the computation cost. To this end, this paper proposes a CNN-Transformer rectified collaborative learning (CTRCL) framework to learn stronger CNN-based and Transformer-based models for MIS tasks via the bi-directional knowledge transfer between them. Specifically, we propose a rectified logit-wise collaborative learning (RLCL) strategy which introduces the ground truth to adaptively select and rectify the wrong regions in student soft labels for accurate knowledge transfer in the logit space. We also propose a class-aware feature-wise collaborative learning (CFCL) strategy to achieve effective knowledge transfer between CNN-based and Transformer-based models in the feature space by granting their intermediate features the similar capability of category perception. Extensive experiments on three popular MIS benchmarks demonstrate that our CTRCL outperforms most state-of-the-art collaborative learning methods under different evaluation metrics. The source code will be publicly available athttps://github.com/LanhooNg/CTRCL.

Multi-Modal Image Fusion (MMIF) aims to integrate complementary image information from different modalities to produce informative images. Previous deep learning-based MMIF methods generally adopt Convolutional Neural Networks (CNNs) or Transformers for feature extraction. However, these methods deliver unsatisfactory performances due to the limited receptive field of CNNs and the high computational cost of Transformers. Recently, Mamba has demonstrated a powerful potential for modeling long-range dependencies with linear complexity, providing a promising solution to MMIF. Unfortunately, Mamba lacks full spatial and frequency perceptions, which are very important for MMIF. Moreover, employing Image Reconstruction (IR) as an auxiliary task has been proven beneficial for MMIF. However, a primary challenge is how to leverage IR efficiently and effectively. To address the above issues, we propose a novel framework named Spatial-Frequency Enhanced Mamba Fusion (SFMFusion) for MMIF. More specifically, we first propose a three-branch structure to couple MMIF and IR, which can retain complete contents from source images. Then, we propose the Spatial-Frequency Enhanced Mamba Block (SFMB), which can enhance Mamba in both spatial and frequency domains for comprehensive feature extraction. Finally, we propose the Dynamic Fusion Mamba Block (DFMB), which can be deployed across different branches for dynamic feature fusion. Extensive experiments show that our method achieves better results than most state-of-the-art methods on six MMIF datasets. The source code is available at https://github.com/SunHui1216/SFMFusion.

Recently, the Diffusion Probabilistic Model (DPM)-based methods have achieved substantial success in the field of medical image segmentation. However, most of these methods are not effective in addressing the issue of blurred edges in prostate segmentation tasks. To address this issue, This paper proposes a framework based on the diffusion model, named BGDiff, which is based on Boundary Guided Injection Module(BGIM) and Adaptive Boundary Loss for prostate segmentation. The BGIM can establish connections between the denoising processes of adjacent steps, thereby providing stable guidance for boundary areas as the denoising progresses step by step, while the Adaptive Boundary Loss adjusts the loss weights for more challenging boundaries based on the model’s active feedback. Extensive experiments on four benchmark datasets demonstrate the effectiveness of the proposed method and achieve state-of-the-art performance on four evaluation metrics. The source code will be publicly available at https://github.com/zjlGO/BGDiff