In utilizing deep learning techniques for medical image segmentation, two types of imbalance issues are observed: inter-class imbalance between majority and minority classes and intra-class imbalance between easy and hard samples. However, existing loss functions typically confuse these issues, leading to enhancements that cater to only one aspect. Moreover, loss functions optimized for specific tasks often exhibit limited generalizability. To address these issues, we propose Inter-class and Intra-class Balance loss, as well as a unified loss termed Balance loss. The Inter-class Balance loss controls the extent of hard sample mining for majority class samples by considering the frequency of minority classes present in each input image. This approach requires no manual adjustment weights and adapts automatically to different datasets. The Intra-class Balance loss enhances the network's ability to learn from hard samples by performing mining on hard samples within each class. We evaluate our loss functions on five segmentation tasks with varying degrees of class imbalance. The experimental results show that our proposed Balance loss enhances segmentation performance compared with the current loss functions and exhibits superior robustness.

Radiology report generation (RRG), intended to automatically generate a coherent free-text report describing the clinical observations of a radiograph, has been attracting increasing attention from researchers. In recent years, the Transformer-based encoder-decoder architecture has been adopted by most existing methods. However, they neglect the structural rationality issue when applying this single-modal architecture to the multi-modal RRG task, where information can only flow from visual features to textual features, but not in the opposite direction. This information asymmetry results in visual features having no knowledge of the textual features, sending out all visual information, including a large amount of heterogeneous noise. Consequently, this introduces significant resistance to the downstream decoder, which substantially limits or even harms the generation process. To tackle this problem, we present a method where a cross-counter-repeat attention is developed to integrate useful information from two separate modalities, and a memory-driven visual semantics enhancing module is designed to reinforce the visual features with strong time-ordered semantic information. Experimental results on the widely-used IU-Xray dataset show that our approach achieves the state-of-the-art performance, with a remarkable 6.9% improvement in BLEU-4 score. Further analyses also demonstrate that our method can generate sufficiently comprehensive reports to assist radiologists in their clinical decision-making.

In the context of medical image segmentation, complementary information among multi-modality images can improve segmentation performance. However, acquiring the complete multi-modality data in clinical settings is difficult. To tackle this problem, we propose a novel multi-modality knowledge distillation segmentation framework, which allows the inference performance of single-modality closer to that of multi-modality. In order to facilitate the extraction of valuable information from the multi-modality teacher network, we first introduce a subtask named patch-selection to distill the patch-level knowledge and improve the generalization capacity of networks simultaneously. Moreover, we employ contrastive learning distillation by defining patch-level positive and negative pairs in embedding, which can encourage the student network to extract more potential information from single-modality input and better understand the similarities and differences with the teacher network in representations. The evaluation process on the BraTS 2018 dataset shows the state-of-the-art performance of our method.

Although medical image segmentation has achieved remarkable results with supervised learning, obtaining labeled data remains challenging and costly. To counteract this, we present the MTSPSeg, a multi-task self-supervised learning framework. We establish the dynamic gradient learning rate (DGLR) strategy and the dynamic multi-task loss weight adjustment method (DWL) to relieve potential early convergence and diverse learning difficulties of self-supervised learning. Our method deeply integrates the prior knowledge inherent in medical images, facilitates collaborative learning among tasks, and delves further into the latent information within the images. Experimental results demonstrate that regardless of the proportion of labeled data, our method outperforms baseline models on the KiTS19 and LiTS datasets, achieving state-of-the-art performance on the KiTS19 dataset.

Medical image segmentation is a critical task for clinical diagnosis and research. However, dealing with highly imbalanced data remains a significant challenge in this domain, where the region of interest (ROI) may exhibit substantial variations across different slices. This presents a significant hurdle to medical image segmentation, as conventional segmentation methods may either overlook the minority class or overly emphasize the majority class, ultimately leading to a decrease in the overall generalization ability of the segmentation results. To overcome this, we propose a novel approach based on multi-step reinforcement learning, which integrates prior knowledge of medical images and pixel-wise segmentation difficulty into the reward function. Our method treats each pixel as an individual agent, utilizing diverse actions to evaluate its relevance for segmentation. To validate the effectiveness of our approach, we conduct experiments on four imbalanced medical datasets, and the results show that our approach surpasses other state-of-the-art methods in highly imbalanced scenarios. These findings hold substantial implications for clinical diagnosis and research.