Consistency learning combined with pseudo label training is a mainstream paradigm in semi-supervised medical image segmentation (SSMIS), yet it faces two challenges: 1)The one-sided search and handling of uncertainty regions make it difficult to address complex uncertainty situations caused by multiple networks. 2)The premature reaching of consensus in traditional SSMIS frameworks hinders the information complementarity between sub-networks. To address these, we propose a multi-view interactive interference training framework based on cross-network uncertainty (MIICU). Specifically, we design a cross-network uncertainty region searching (CUS) module, which combines the predictions from two networks to provide more reasonable uncertainty regions. Then, we design a dual-decision complementary displacement (DCD) strategy that performs displacement operations on cross-network uncertainty regions in different scenarios, so as to facilitate the learning of these regions. We further propose a multi-view interactive interference training strategy by expanding the training environment with three different concatenation forms, encouraging information complementarity between sub-networks. Evaluation on the TN3K and ACDC datasets shows that our approach outperforms existing SSMIS methods and is comparable to fully supervised methods. Code has been released at GitHub

Federated learning enables a server to train a global model by leveraging data distributed across multiple clients. However, its inherently distributed and iterative nature introduces significant communication overhead as well as potential privacy concerns. To mitigate these issues, one-shot federated learning restricts communication between the server and clients to a single round. Nevertheless, this constraint often results in reduced accuracy. In the field of medical diagnosis, data security is particularly critical and has become a major research focus. In this paper, we propose a one-shot federated learning framework and conduct experiments on three popular datasets and a melanoma image dataset collected by ourselves. To further enhance performance, we introduce One-shot Federated Learning framework for Melanoma Diagnosis, OFL-MD, which incorporates a pre-trained diffusion model CLIP to assist clients in generating synthetic datasets and employs differential privacy to further enhance the privacy. The synthetic datasets are then transmitted to the global model for lightweight fine-tuning, thereby improving accuracy. We evaluate our approach on both widely used benchmark datasets and our own melanoma dataset. The results validate the effectiveness of our multimodal one-shot federated learning framework, which not only preserves data privacy and achieves high global model accuracy but also shows promise for future deployment in assisting dermatologists with melanoma detection of patients.

Zero-shot medical image diagnosis is driven by vision-language pretraining on large-scale image-text pairs. However, aligning chest X-rays with reports remains challenging, particularly when multiple abnormalities with overlapping visual features lead to ambiguous correspondences. To address this challenge, we propose MedCMCL-VLP, a novel vision-language pretraining framework that simulates the hierarchical learning process of medical professionals. The framework consists of two key components: (1) a new enhanced Vision Mamba encoder with a spatial adaptor to capture long-range spatial dependencies and small-scale abnormalities in high-resolution chest X-rays and (2) a knowledge-guided text encoder that integrates medical domain knowledge with large language models (LLMs) to capture fine-grained semantic information from radiology reports. More importantly, MedCMCL-VLP employs a two-phase training paradigm that guides the model from novice to expert levels, effectively improving diagnostic accuracy. We evaluate MedCMCL-VLP on five public chest X-ray datasets. The experimental results show that our framework achieves state-of-the-art performance in both zero-shot and fine-tuned classification tasks.

Super-resolution plays a crucial role in enhancing the diagnostic utility of ultrasound images by recovering fine anatomical structures. Due to their remarkable performance in natural image restoration and generation, diffusion probabilistic models have become attractive for ultrasound image superresolution that highly requires complex textures and structural details modelling. However, existing diffusion-based superresolution methods typically apply uniform noise perturbations and reconstruction strategies across the entire image. Such global treatment neglects the varying reconstruction difficulty and clinical importance of different regions, often resulting in blurry lesion boundaries and suboptimal recovery of diagnostically significant structures. To address this issue, we propose UGDT-SR, a structure-aware ultrasound image super-resolution framework that integrates Bayesian uncertainty modeling and Transformer-guided structural representation into the diffusion process. Specifically, we introduce a Bayesian uncertainty network to estimate pixel-wise reconstruction difficulty, generating structural uncertainty masks that adaptively modulate local noise injection during forward diffusion. Furthermore, we inject these uncertainty masks into the conditional encoder to guide the model's attention toward clinically relevant regions throughout the reconstruction. Extensive experiments on both public and clinical ultrasound datasets demonstrate that UGDT-SR consistently outperformed CNN-, GAN-, and diffusion-based baselines, achieving superior perceptual quality and structural fidelity, especially in lesion areas.

Weakly supervised methods typically guided the pixel-wise training by comparing the predictions to single-level labels containing diverse segmentation-related information at once, but struggled to represent subtle feature differences between nodule and background regions and confused incorrect information, resulting in under-fitting or over-fitting in the segmentation predictions. This work proposes a weakly supervised network that generates multi-level labels from four-point annotation to refine diverse constraints for delicate nodule segmentation. The Distance-Similarity Fusion Prior referring to the points annotations filters out information irrelevant to nodules. The bounding box and pure foreground/background labels, generated from the point annotation, guarantee the rationality of the prediction in the arrangement of target localization and the spatial distribution of target/background regions, respectively. Our proposed network outperforms existing weakly supervised methods on two public datasets with respect to accuracy and robustness, improving the applicability of deep-learning-based segmentation in the clinical practice of thyroid nodule diagnosis.

Exploring complementary information between RGB and thermal/depth modalities is crucial for bi-modal salient object detection (BSOD). However, the distinct characteristics of different modalities often lead to large differences in information distributions. Existing models, which rely on convolutional operations or plug-and-play attention mechanisms, struggle to address this issue. To overcome this challenge, we rethink the relationship between information complementarity and long-range relevance, and propose a uniform broad-view Twins Transformer Network (TwinsTNet) for accurate BSOD. Specifically, to efficiently fuse bi-modal information, we first design the Cross-Modal Federated Attention (CMFA), which mines complementary cues across modalities through element-wise global dependency. Second, to ensure accurate modality fusion, we propose the Semantic Consistency Attention Loss, which supervises the co-attention feature in CMFA using the ground-truth-generated attention map. Additionally, existing BSOD models lack the exploration of inter-layer interactions, for which we propose the Cross-Scale Retracing Attention (CSRA), which retrieves query-relevant information from stacked features of all previous layers, enabling flexible cross-layer interactions. The cooperation between CMFA and CSRA mitigates inductive bias in both modality and layer dimensions, enhancing TwinsTNet's representational capability. Extensive experiments demonstrate that TwinsTNet outperforms twenty-two existing state-of-the-art models on ten BSOD benchmark datasets. The code is available at: https://github.com/JoshuaLPF/TwinsTNet.

Contrast-enhanced ultrasound (CEUS) has been extensively employed as an imaging modality in thyroid nodule diagnosis due to its capacity to visualise the distribution and circulation of micro-vessels in organs and lesions in a non-invasive manner. However, current CEUS-based thyroid nodule diagnosis methods suffered from: 1) the blurred spatial boundaries between nodules and other anatomies in CEUS videos, and 2) the insufficient representations of the local structural information of nodule tissues by the features extracted only from CEUS videos. In this paper, we propose a novel dual-branch network with a cross-modality-attention mechanism for thyroid nodule diagnosis by integrating the information from tow related modalities, i.e., CEUS videos and ultrasound image. The mechanism has two parts: US-attention-from-CEUS transformer (UAC-T) and CEUS-attention-from-US transformer (CAU-T). As such, this network imitates the manner of human radiologists by decomposing the diagnosis into two correlated tasks: 1) the spatio-temporal features extracted from CEUS are hierarchically embedded into the spatial features extracted from US with UAC-T for the nodule segmentation; 2) the US spatial features are used to guide the extraction of the CEUS spatio-temporal features with CAU-T for the nodule classification. The two tasks are intertwined in the dual-branch end-to-end network and optimized with the multi-task learning (MTL) strategy. The proposed method is evaluated on our collected thyroid US-CEUS dataset. Experimental results show that our method achieves the classification accuracy of 86.92%, specificity of 66.41%, and sensitivity of 97.01%, outperforming the state-of-the-art methods. As a general contribution in the field of multi-modality diagnosis of diseases, the proposed method has provided an effective way to combine static information with its related dynamic information, improving the quality of deep learning based diagnosis with an additional benefit of explainability.

Weakly-supervised learning methods have become increasingly attractive for medical image segmentation, but suffered from a high dependence on quantifying the pixel-wise affinities of low-level features, which are easily corrupted in thyroid ultrasound images, resulting in segmentation over-fitting to weakly annotated regions without precise delineation of target boundaries. We propose a dual-branch weakly-supervised learning framework to optimize the backbone segmentation network by calibrating semantic features into rational spatial distribution under the indirect, coarse guidance of the bounding box mask. Specifically, in the spatial arrangement consistency branch, the maximum activations sampled from the preliminary segmentation prediction and the bounding box mask along the horizontal and vertical dimensions are compared to measure the rationality of the approximate target localization. In the hierarchical prediction consistency branch, the target and background prototypes are encapsulated from the semantic features under the combined guidance of the preliminary segmentation prediction and the bounding box mask. The secondary segmentation prediction induced from the prototypes is compared with the preliminary prediction to quantify the rationality of the elaborated target and background semantic feature perception. Experiments on three thyroid datasets illustrate that our model outperforms existing weakly-supervised methods for thyroid gland and nodule segmentation and is comparable to the performance of fully-supervised methods with reduced annotation time. The proposed method has provided a weakly-supervised segmentation strategy by simultaneously considering the target's location and the rationality of target and background semantic features distribution. It can improve the applicability of deep learning based segmentation in the clinical practice.

The anatomies in low-dose computer tomography (LDCT) are usually distorted during the zooming-in observation process due to the small amount of quantum. Super-resolution (SR) methods have been proposed to enhance qualities of LDCT images as post-processing approaches without increasing radiation damage to patients, but suffered from incorrect prediction of degradation information and incomplete leverage of internal connections within the 3D CT volume, resulting in the imbalance between noise removal and detail sharpening in the super-resolution results. In this paper, we propose a novel LDCT SR network where the degradation information self-parsed from the LDCT slice and the 3D anatomical information captured from the LDCT volume are integrated to guide the backbone network. The prior degradation estimator (PDE) is proposed following the contrastive learning strategy to estimate the degradation features in the LDCT images without paired low-normal dose CT images. The self-guidance fusion module (SGFM) is designed to capture anatomical features with internal 3D consistencies between the squashed images along the coronal, sagittal, and axial views of the CT volume. Finally, the features representing degradation and anatomical structures are integrated to recover the CT images with higher resolutions. We apply the proposed method to the 2016 NIH-AAPM Mayo Clinic LDCT Grand Challenge dataset and our collected LDCT dataset to evaluate its ability to recover LDCT images. Experimental results illustrate the superiority of our network concerning quantitative metrics and qualitative observations, demonstrating its potential in recovering detail-sharp and noise-free CT images with higher resolutions from the practical LDCT images.

Visible and infrared image fusion(VIF) provides more comprehensive understanding of a scene and can facilitate subsequent processing. Although frequency domain contains valuable global information in low frequency and rapid pixel intensity variation data in high frequency of images, existing fusion methods mainly focus on spatial domain. To close this gap, a novel VIF method in frequency domain is proposed. First, a frequency-domain feature extraction module is developed for source images. Then, a frequency-domain transformer fusion method is designed to merge the extracted features. Finally, a residual reconstruction module is introduced to obtain final fused images. To the best of our knowledge, it is the first time that image fusion study is conducted from frequency domain perspective. Comprehensive experiments on three datasets, i.e., MSRS, TNO, and Roadscene, demonstrate that the proposed approach obtains superior fusion performance over several state-of-the-art fusion methods, indicating its great potential as a generic backbone for VIF tasks.

Accurately predicting the response to neoadjuvant chemotherapy (NCT) is crucial for gastric cancer treatment planning. Multimodal diagnostic methods enhance prediction accuracy by integrating images and clinical features, but they often overlook intra-class differences, causing feature entanglement, and single models struggle to capture dataset diversity. To address these issues, we propose Clustering Multimodal Ensemble Learning (CMEL) framework, which aligns and fuses image features from different domains with clinical features and performs decision fusion using an ensemble model. Specifically, we design the clustering process using Siamese network and memory bank to partition images in feature space and introduce contrastive clustering loss to enhance clustering properties and reduce feature entanglement across domains. We design Hierarchical Feature Alignment Fusion (HFAF) module to generate multi-level image features at different depths and then separately align and fuse them with clinical features, providing diverse features to cover variations in the dataset. We design Dynamic Multi-Classifier Decision Fusion (DMDF) module to predict on fused features using a gating mechanism supervised by pseudo-labels for dynamic weights, enhancing prediction across different domains. On our collected the Gastric Cancer Chemotherapy Response (GCCR) dataset, CMEL achieves an AUC of 76.09%, accuracy of 72.00%, PPV of 86.02%. These results demonstrate the feasibility of joint image and clinical data in predicting NCT response and provide valuable insights for gastric cancer treatment. The code is available at https://github.com/WHX0259/CMEL.

Existing enhancement methods are empirically expected to help the high-level end computer vision task: however, that is observed to not always be the case in practice. We focus on object or face detection in poor visibility enhancements caused by bad weathers (haze, rain) and low light conditions. To provide a more thorough examination and fair comparison, we introduce three benchmark sets collected in real-world hazy, rainy, and low-light conditions, respectively, with annotated objects/faces. We launched the UG2+ challenge Track 2 competition in IEEE CVPR 2019, aiming to evoke a comprehensive discussion and exploration about whether and how low-level vision techniques can benefit the high-level automatic visual recognition in various scenarios. To our best knowledge, this is the first and currently largest effort of its kind. Baseline results by cascading existing enhancement and detection models are reported, indicating the highly challenging nature of our new data as well as the large room for further technical innovations. Thanks to a large participation from the research community, we are able to analyze representative team solutions, striving to better identify the strengths and limitations of existing mindsets as well as the future directions.

This paper proposes a novel colour image enhancement method which uses wavelet-based texture characteristic morphological component analysis (WT-TC-MCA) to enhance the textural differences in the luminance channel of the colour image. The image enhancement method is intended to be the preprocessing method prior to the use of the colour image segmentation. The input colour image is firstly transformed to CIELab colour space to separate the luminance channel from the chromatic channels. Then only the luminance channel is enhanced by the WT-TC-MCA method to enhance the textural differences between different textures. Therefore, the colour image is enhanced with more differentiate textures while preserving the chromatic information. The experimental results show that the proposed method can enhance different colour image segmentation algorithms more than the state-of-the-art colour image enhancement method.

This paper proposes a new texture enhancement method which uses an image decomposition that allows different visual characteristics of textures to be represented by separate components in contrast with previous methods which either enhance texture indirectly or represent all texture information using a single image component. Our method is intended to be used as a preprocessing step prior to the use of texture-based image segmentation algorithms. Our method uses a modification of morphological component analysis (MCA) which allows texture to be separated into multiple morphological components each representing a different visual characteristic of texture. We select four such texture characteristics and propose new dictionaries to extract these components using MCA. We then propose procedures for modifying each texture component and recombining them to produce a texture-enhanced image. We applied our method as a preprocessing step prior to a number of texture-based segmentation methods and compared the accuracy of the results, finding that our method produced results superior to comparator methods for all segmentation algorithms tested. We also demonstrate by example the main mechanism by which our method produces superior results, namely that it causes the clusters of local texture features of each distinct image texture to mutually diverge within the multidimensional feature space to a vastly superior degree versus the comparator enhancement methods.