Unsupervised Domain Adaptation (UDA) for semantic segmentation has emerged as a key research focus due to its ability to enhance model generalizability without additional annotations. Existing UDA methods often utilize adversarial or self-training for domain adaptation but frequently overlook areas in images with similar or slightly blurred textures, resulting in significant performance discrepancies across different scale categories. To address this issue, we propose Unstable Category Feature Enhancement (UCFE), which dynamically separates stable and unstable categories during training using the Dynamic Unstable category Decoupling Module (DUDM). This allows for category decoupling in input images. Subsequently, the Multi-Unstable-category Enhancement Module (MUEM) learns complex relationships between categories, enabling precise feature extraction for unstable categories. Experimental results demonstrate that UCFE can accurately segment regions with similar textures and slight blurs, significantly outperforming state-of-the-art methods on benchmark datasets.

Lesions in medical imaging exhibit considerable variability in location and size, while image quality is frequently compromised by noise and artifacts. These complex disturbance patterns undermine the stability of feature extraction and significantly complicate precise segmentation. To address these challenges, we propose the Disturbance-Aware Lesion Segmentation Network (DASNet), a segmentation framework based on probabilistic modeling, designed to achieve robust feature representation under diverse disturbing conditions. DASNet introduces a dual-encoder architecture to separately capture observable and latent disturbances: the spatial adaptive encoder is employed to extract visible deformation features of lesions (positional offset and area proportion), while the Gaussian distribution encoder models latent uncertainties in the feature space, regularized by posterior probability supervision to align learned distributions with true lesion feature distributions. The representations from both encoders are integrated during the decoding phase, guiding the generation of reliable features. Extensive experiments conducted on ultrasound, dermoscopy, and colonoscopy datasets demonstrate that DASNet consistently achieves superior segmentation accuracy and exhibits strong generalization across multiple imaging modalities.

Deep learning-based semantic segmentation technology has become a critical tool in assisting doctors with automatic lesion segmentation in medical images. However, the high cost of acquiring large-scale, pixel-level annotations poses a significant challenge, limiting the scalability and application of fully supervised semantic segmentation models. To address this, weakly supervised learning-based semantic segmentation models have emerged as a promising solution. These models can accurately segment lesion regions using only weak annotations, such as image-level or frame-level labels, significantly reducing the annotation burden. This approach has gained substantial attention in current research.Among various medical imaging modalities, ultrasound imaging stands out as a primary diagnostic tool due to its rapid imaging speed, ease of use, and accessibility. This paper focuses on the study of thyroid ultrasound imaging, aiming to achieve accurate classification of nodule regions. The goal is to provide clinicians with more precise diagnostic information, improving decision-making in thyroid disease diagnosis.

Self-training is a strong baseline for semi-supervised domain adaptive semantic segmentation. However, it inevitably introduces biased links between features and concepts in the prediction of certain "hard pixels", which may mislead the generalization of models. We consider these hard pixels to come from two aspects: First, the labels are severely imbalanced and distributed across domains and classes, which may lead to features biased towards source domains and majority classes. Second, the naive threshold filtering pseudo-label methods limit the supervision of hard pixels. To address the above problems, we propose a novel co-training framework with progressive distribution alignment and uncertainty-interactive relabeling strategies. More concretely, a progressive distribution alignment strategy is proposed to match distribution across domains while providing additional supervision for tail class pixels. Additionally, an uncertainty-interactive relabeling strategy is proposed to retain more supervisory information for hard pixels and reduce the overall uncertainty of the pseudo-labels. Experiments on two widely-used benchmarks demonstrate the effectiveness of the proposed PDAUR, achieving state-of-the-art results.

Refining the seed region to obtain finely annotated pseudo masks for training a segmentation model is a crucial step in the multi-stage weakly supervised semantic segmentation (WSSS) framework. One of the most popular refinement methods, IRN, extends seed regions towards the edges in the image. However, we observed that, due to the lack of guidance from semantic information, IRN’s refinement may lead the generation of partially erroneous refinement directions. To address this issue, we leverage prototypes to recover the overlooked category semantic information in the refinement stage. We propose a prototype-based pseudo mask reclassification post-processing (PtReCl) to correct misclassified pixels in the pseudo masks, generating refined pseudo masks with more accurate coverage. Experimental evaluations demonstrate that our post-processing approach brings improvements in both pseudo mask quality and segmentation results on PASCAL VOC and MS COCO datasets, achieving state-of-the-art performance on VOC.

Since ultrasound imaging technique is convenient and real-time, it plays a crucial role in diagnosing thyroid nodules. With the development of deep learning, computer-aided diagnosis models have been widely applied to diagnose thyroid nodules, in which thyroid nodule segmentation is a basic yet essential task. Semi-supervised learning is a popular topic for thyroid ultrasound images segmentation under limited annotations. However, current mainstream semi-supervised methods (e.g., for nature image scenes) commonly produce poor segmentation results when facing thyroid ultrasound images. We analyze that there are two main reasons: First, since these methods lack the targeted learning for the ambiguous regions of images that may contain complementary clues for segmentation, the model will likely be over-fitting in the regions that are easy to predict, leading to fail to make full use of the unlabeled data. Second, these methods lack the shape constraint on thyroid nodules, resulting in incomplete segmentation shape on the nodule boundary. To address the above issues, we propose a novel Uncertainty-guided Dual Task Framework (UDTF). Concretely, we propose an Uncertainty Region Selection Module (URSM) to guide the segmentation task to learn from the ambiguous regions calculated by prototype under the constraint of consistency regularization. Additionally, to further keep the integrity of the nodules, we propose a Dual Task Module (DTM) to impose the shape constraint on thyroid nodules by exploring the task-level consistency between the segmentation and auxiliary reconstruction task. Extensive experiments are conducted on two thyroid ultrasound image datasets, including a private Philips Thyroid Ultrasound dataset and a public TN3K dataset. The results show that UDTF achieves superior performance compared to several other state-of-the-art methods.

The class imbalance problem, which is prevalent in medical image datasets, seriously affects the diagnostic effectiveness of deep learning-based network models. Recently, the method based on two-stage learning has produced promising results in solving class imbalance. In two-stage learning, the learning of unbiased classifiers has been well studied, but the representation of imbalanced data is still being explored. In this paper, we focus on the representation learning stage of class-imbalanced and propose a novel balanced and discriminative contrastive learning (BDCL) method. Compared with supervised contrastive learning, BDCL has two improvements: temperature dynamic learning, which balances the gradient contribution of negative samples from different classes; hard example prototypes, which can better learn the feature differences between tail classes. A variety of experimental results on the imbalanced medical image datasets show that BDCL enables the network models to learn representations with desirable balancedness and discriminativeness, effectively solving the class imbalance problem.

The resolution of a whole slide image (WSI) is too large to process directly, but WSI can be segmented into patches and be classified through multiple instance learning (MIL). Some patches have either close distances or similar pathological morphology, indicating that there are at least two types of relationships between patches. However, the existing MIL methods often deal with instance relationships simplistically. To solve this problem, we propose a new model named DualGCN-MIL. It analyzes the two types of relationships in WSI by constructing two different graphs in high-dimensional feature space and physical coordinate space and uses graph convolutional networks with unshared parameters for learning. Various experiments conducted on the Camelyon dataset have shown that compared to state-of-the-art methods, we can achieve better performance in multiple indicators, with accuracy of 88.89% and 85.11% on the 16 and 17 datasets, respectively, which proves that DualGCN-MIL has a higher accuracy in WSI classification. Implementation can be obtained from https://github.com/UnmatchedKatana/DualGCN-MIL.

Semi-supervised domain generalization (SSDG) aims to build a domain-generalized model using partially labeled data from source domains. Mainstream SSDG methods follow the augmentation consistency in FixMatch. However, the extraction of domain-invariant features may be challenging due to the absence of feature-based operations, further leading to overfitting of the classifier. To this end, we propose Multi-level Augmentation Consistency Learning (MACMatch), which improves the generalization of feature extractor and classifier through feature-based augmentation consistency. On the other hand, existing methods assume labeled data are class-balanced and domain-balanced, which is easily violated in practice. Based on this, we introduce Representativity and Diversity-based Sample Selection (RDSS), which models data as graphs to evaluate reasonable samples for labeling, relaxing the assumption for labeled data. Experiments on PACS and OfficeHome demonstrate that MACMatch outperforms state-of-the-art SSDG methods. Furthermore, MAC-Match with RDSS achieves competitive results without domain and class priori assumptions. Code is available at https://github.com/Y-J-Zhang/MACMatch-RDSS.

Generating class activation maps (CAM) as seed regions is a crucial step in weakly supervised semantic segmentation (WSSS). During the training of classification models, instance area information is incorporated into feature vectors and learned by the network. However, during CAM generation, pixel-level prediction scores are solely dependent on features and classifier weights, neglecting the area factor. Such area information becomes a source of bias between training and CAM generation, leading to a significant under-activation (false negatives) of foreground regions. To address this issue, we propose a class area intervention strategy to fine-tune the classification model and rectify this bias. Additionally, we employ a partition activation strategy that can simultaneously consider both under activation and over-activation to generate more precise AICAMs. AICAM is applicable to various WSSS methods based on CAM or its variants. The evaluations on both the PASCAL VOC and MS COCO datasets demonstrate that our method enhances multiple WSSS techniques.

Unsupervised Domain Adaptation (UDA) methods have become essential for computer-aided diagnostic analysis on medical images due to the advantage of improving the model generalization ability with fewer annotations. The disentanglement-based UDA methods improve the segmentation performance significantly by disentangling the features into domain-specific and domain-invariant components. However, such methods neglect detailed texture in ultrasound images, resulting in inaccurate segmentations of nodules. To address the above problem, we propose a Task-Oriented Feature Disentanglement (TOFD) method to achieve better UDA segmentation on images from different ultrasound machines. TOFD achieves finer-grained disentanglement by the Feature Constraint Module based on maximum entropy. Furthermore, the Domain Label Encoding Module can improve the stability of features by integrating the latent feature of domain bias. Extensive experiments demonstrate that TOFD can effectively enhance the model’s generalization capability and confirm the performance of TOFD on ultrasound images exceeds that of state-of-the-art UDA semantic segmentation.

The class imbalance problem, which is prevalent in medical image datasets, seriously affects the diagnostic effectiveness of deep learning-based network models. To alleviate this problem, data re-sampling and loss re-weighting techniques are often used to reshape the decision boundary of the classifier. However, these techniques still lead to biased decision boundary due to the lack of sufficient and diversified samples in the tail classes of medical image datasets. In this paper, we propose an implicit feature augmentation with feature transfer (FT-IFA) method which solves the class imbalance problem by expanding the feature space of tail classes to reshape the decision boundary of the classifier. FT-IFA utilizes prototype similarity to transfer the rich transformation information from the head classes to tail classes on the basis of the balanced feature space, enriching the intra-class diversity of tail classes. Experimental results on two class-imbalanced medical image datasets show that FT-IFA outperforms the current state-of-the-art methods and effectively solves the class imbalance problem.

It is well known that weakly supervised semantic segmentation requires only image-level labels for training, which greatly reduces the annotation cost. In recent years, prototype-based approaches, which prove to substantially improve the segmentation performance, have been favored by a wide range of researchers. However, we are surprised to find that there are semantic gaps between different regions within the same object, hindering the optimization of prototypes, so the traditional prototypes can not adequately represent the entire object. Therefore, we propose region-specific prototypes to adaptively describe the regions themselves, which alleviate the effect of semantic gap by separately obtaining prototypes for different regions of an object. In addition, to obtain more representative region-specific prototypes, a plug-and-play Spatially Fused Attention Module is proposed for combining the spatial correlation and the scale correlation of hierarchical features. Extensive experiments are conducted on PASCAL VOC 2012 and MS COCO 2014, and the results show that our method achieves state-of-the-art performance using only image-level labels.

Acoustic-to-articulatory inversion (AAI) aims to estimate the parameters of articulators from speech audio. There are two common challenges in AAI, which are the limited data and the unsatisfactory performance in speaker independent scenario. Most current works focus on extracting features directly from speech and ignoring the importance of phoneme information which may limit the performance of AAI. To this end, we propose a novel network called SPN that uses two different streams to carry out the AAI task. Firstly, to improve the performance of speaker-independent experiment, we propose a new phoneme stream network to estimate the articulatory parameters as the phoneme features. To the best of our knowledge, this is the first work that extracts the speaker-independent features from phonemes to improve the performance of AAI. Secondly, in order to better represent the speech information, we train a speech stream network to combine the local features and the global features. Compared with state-of-the-art (SOTA), the proposed method reduces 0.18mm on RMSE and increases 6.0% on Pearson correlation coefficient in the speaker-independent experiment. The code has been released at https://github.com/liujinyu123/AAINetwork-SPN.

The generation-based data augmentation method can overcome the challenge caused by the imbalance of medical image data to a certain extent. However, most of the current research focus on images with unified structure which are easy to learn. What is different is that ultrasound images are structurally inadequate, making it difficult for the structure to be captured by the generative network, resulting in the generated image lacks structural legitimacy. Therefore, a Progressive Generative Adversarial Method for Structurally Inadequate Medical Image Data Augmentation is proposed in this paper, including a network and a strategy. Our Progressive Texture Generative Adversarial Network alleviates the adverse effect of completely truncating the reconstruction of structure and texture during the generation process and enhances the implicit association between structure and texture. The Image Data Augmentation Strategy based on Mask-Reconstruction overcomes data imbalance from a novel perspective, maintains the legitimacy of the structure in the generated data, as well as increases the diversity of disease data interpretably. The experiments prove the effectiveness of our method on data augmentation and image reconstruction on Structurally Inadequate Medical Image both qualitatively and quantitatively. Finally, the weakly supervised segmentation of the lesion is the additional contribution of our method.

Lesion detection is one of the most important issues in the field of ultrasound image analysis. Recent studies exploit convolutional neural networks to either classify the nodules or locate them with bounding boxes (detection-based algorithms). However, the training of these methods demand considerable and laborious object-level annotations labeled by ultrasound experts and may lack the explicit border information for accurate localization. To reduce the annotation cost and improve the borders feature of proposal box, we propose a novel Border Sensitive Network (BSNet) that extract border features from the extreme point of the borders in proposal box to enhance the single point feature and use a vector to represent arbitrary distribution of border locations. Experimental results demonstrate that BSNet achieves state-of-the-art performance compared with other weakly supervised methods in Thyroid Ultrasound Images dataset, and further analysis reflects their value in clinical practice.

With the maturity of artificial intelligence, AI-aided diagnosis technology is gradually widely applied in clinical medicine. However, for the same pathological tissue, medical images produced by different types of instruments usually possess different data distributions. Because of the domain shift phenomenon, AI-aided diagnosis cannot accurately diagnose medical images in other domains, which is a waste of precious medical images. This paper proposes a Multi-Scale Self-Attention Unsupervised Domain Adaptive framework (MSDAN), which consists of three modules. First, the multi-scale framework constrains the source domain features and target domain features by optimizing adversarial losses with different level features. Second, the mix-up discriminator extracts latent spatial features by mixing up source domain and target domain features. Finally, MSDAN learns the geometric information of the pathological tissues in medical images through the self-attention module, thereby improving the transfer effect of the semantic information in medical images. Extensive experiments prove that the proposed approach can achieve superior performance on tasks with various degrees of domain shift and data complexity, especially for thyroid ultrasound images.

MRI technology is usually used to distinguish the grade of the tumor in the patient. Due to technical limitations, the classification of tumors (high-grade gliomas and metastases) on MRI images has become a problem for doctors. At present, the widely used neural network is gradually applied to the tumor classification of MRI images, which not only reduces the burden of human resources, but also shows good classification accuracy. Although different good experimental data are obtained under various neural networks, there is still a problem in using these neural networks for tumor classification: the semantic information expressed on the image by the deep features of the neural network is too scattered, and it is difficult to concentrate the lesion area. In this article, we propose a new strategy that combines the approximate symmetry properties of the MRI image with neural network, then uses a dual-branch network instead of the basic network for feature extraction, and adds complementary learning to the network, different features fusion and attention mechanism to enrich detailed information. Our method performs multiple comparison and ablation experiments on the dataset of glioma and metastasis, which proves that the proposed method is effective for tumor classification assisted by MRI.

Complicated medical image analysis often requires a combination of disease classification, lesion detection and lesion segmentation. However, models designed for different tasks produce inconsistent or non-corresponding predictions and ignore the implicit connections between tasks. We propose a novel framework, which makes full use of the fact that segmentation and detection are mutually beneficial, boosts these three tasks in a unified framework. The proposed Information Enhancement Module uses classification information as a beneficial supplement to locate lesion quickly for segmentation. To further achieve fine segmentation with clear boundaries, we propose a Boundary-aware Loss, which dynamically adjusts supervised signal, so that our model pays more attention to boundary in later stages of training. Through experiments conducted on Thyroid Ultrasound dataset, we have demonstrated the good performance of the proposed method in joint segmentation and detection.

Link prediction is the main task of knowledge graph completion, predicting missing relations between entities based the existing links among the entities. The problem of knowledge graph completion can be framed as a third-order binary tensor completion problem. In this case, tensor decomposition seems like a natural solution. And many previous studies have shown that tensor decomposition methods are superior to Trans-based methods in link prediction experiments. Typical tensor decomposition methods are Canonical Polyadic (CP) decomposition and Tucker decomposition. In this paper, we propose Block term decomposition Embedding model (BTDE) for link prediction based on Block term decomposition (which can be seen as a combination of CP decomposition and Tucker decomposition) of the binary tensor representation of knowledge graph triples. The embeddings learned through BTDE is interpretable. In addition, we prove BTDE is fully expressive and derive the bound on its entity and relation embedding dimensionality for full expressivity which is the same as TuckER and smaller than the bound of previous start-of-the-art models ComplEx and SimplE. We show empirically that BTDE outperforms most previous state-of-the-art models across five standard link prediction datasets.

Thyroid ultrasound image is an important basis for artificial intelligence assisted treatment of thyroid-related diseases, but existing images usually contain special cross symbols which represent the location of nodules marked by doctors, thus affecting the features and diagnostic results extracted by the deep learning algorithm. We propose Pyramid GAN(Py-GAN) for blind image inpainting to remove cross symbols. Py-GAN contains a generator with pyramid structure and a global discriminator. The global discriminator improves the authenticity of the corrupted regions and image consistency. The generator uses the joint context loss to get clear image restoration, which prevents the information loss of non-completion area. The inpainting results of the proposed Py-GAN not only maintains the texture and structural information of the original image, but also has the greatest advantage that there are no artifacts in the corrupted regions, achieving pixel-level realism. Both qualitative and quantitative comparisons are superior to existing learning/non-learning image inpainting works.

Image segmentation based on deep learning has greatly promoted the development of the field of computer-aided diagnosis. However, the large scale medical annotation of ground truth is so difficult that it directly affects the performance of existing segmentation models. In this work, an Attention based Semi-supervised Neural Network is proposed, which can complete end-to-end segmentation task of thyroid ultrasound image with weakly annotated classification data and a small amount of fully annotated segmentation data. Two kinds of attention modules are proposed to improve network performance through the trainable feedforward structure of bottom-up and top-down so as to suppress or activate the feature channels and image regions respectively. The experimental results show that when there is only 13% of fully annotated data, the Jaccard similarity coefficient of thyroid nodule segmentation is 74.91%, 4.97% higher than VGG-based semi-supervised model. The classification accuracy of benign and malignant is increased from 91.67% to 95.00%. Equally important, with the same number of fully annotated data, our model has better generalization than that of the supervised segmentation models.