Unsupervised Domain Adaptation (UDA) for semantic segmentation has emerged as a key research focus due to its ability to enhance model generalizability without additional annotations. Existing UDA methods often utilize adversarial or self-training for domain adaptation but frequently overlook areas in images with similar or slightly blurred textures, resulting in significant performance discrepancies across different scale categories. To address this issue, we propose Unstable Category Feature Enhancement (UCFE), which dynamically separates stable and unstable categories during training using the Dynamic Unstable category Decoupling Module (DUDM). This allows for category decoupling in input images. Subsequently, the Multi-Unstable-category Enhancement Module (MUEM) learns complex relationships between categories, enabling precise feature extraction for unstable categories. Experimental results demonstrate that UCFE can accurately segment regions with similar textures and slight blurs, significantly outperforming state-of-the-art methods on benchmark datasets.

Lesions in medical imaging exhibit considerable variability in location and size, while image quality is frequently compromised by noise and artifacts. These complex disturbance patterns undermine the stability of feature extraction and significantly complicate precise segmentation. To address these challenges, we propose the Disturbance-Aware Lesion Segmentation Network (DASNet), a segmentation framework based on probabilistic modeling, designed to achieve robust feature representation under diverse disturbing conditions. DASNet introduces a dual-encoder architecture to separately capture observable and latent disturbances: the spatial adaptive encoder is employed to extract visible deformation features of lesions (positional offset and area proportion), while the Gaussian distribution encoder models latent uncertainties in the feature space, regularized by posterior probability supervision to align learned distributions with true lesion feature distributions. The representations from both encoders are integrated during the decoding phase, guiding the generation of reliable features. Extensive experiments conducted on ultrasound, dermoscopy, and colonoscopy datasets demonstrate that DASNet consistently achieves superior segmentation accuracy and exhibits strong generalization across multiple imaging modalities.

Accurate wind speed forecasting is essential for mitigating the challenges of wind power grid integration. However, existing wind speed prediction models overlook the distributional shift problem within wind speed series, and this time-varying distribution can significantly impact wind prediction accuracy. In this paper, we propose the Distribution Shift and Feature Decoupling Network (DSFD-Net), which addresses the issue of distributional shifts occurring both within the input series and between the input and predicted series through a distribution matching model and distribution mapping module, respectively. Additionally, we introduce a feature decoupling module to mitigate the feature loss encountered in our work. We conduct extensive experiments on two datasets, and comprehensive experimental results demonstrate that DSFD-Net achieves at least a 4.1% reduction in error metrics compared to other wind speed forecasting models, indicating superior performance.

Deep learning-based semantic segmentation technology has become a critical tool in assisting doctors with automatic lesion segmentation in medical images. However, the high cost of acquiring large-scale, pixel-level annotations poses a significant challenge, limiting the scalability and application of fully supervised semantic segmentation models. To address this, weakly supervised learning-based semantic segmentation models have emerged as a promising solution. These models can accurately segment lesion regions using only weak annotations, such as image-level or frame-level labels, significantly reducing the annotation burden. This approach has gained substantial attention in current research.Among various medical imaging modalities, ultrasound imaging stands out as a primary diagnostic tool due to its rapid imaging speed, ease of use, and accessibility. This paper focuses on the study of thyroid ultrasound imaging, aiming to achieve accurate classification of nodule regions. The goal is to provide clinicians with more precise diagnostic information, improving decision-making in thyroid disease diagnosis.

Sequential recommender systems aim to learn user preferences through historical interaction sequences. User interactions are driven both by popular trends and personal interests, introducing two types of noise: popular choices triggered by conformist behavior and irrelevant terms that do not reflect the user’s true interests. These noises inhibit the model’s ability to learn user representations. However, the lack of explicit noise labels makes the sequence denoising problem extremely challenging. We propose a new sequence denoising model (PISD), which extracts both popularity signal and interest signal to accurately identify and remove noisy items while maximizing the retention of personalized preference information. This approach generates clean subsequences for sequential recommendation tasks. Extensive experiments on three publicly available datasets show that our model is compatible with most sequential recommendations and significantly outperforms state-of-the-art denoising methods.

Self-training is a strong baseline for semi-supervised domain adaptive semantic segmentation. However, it inevitably introduces biased links between features and concepts in the prediction of certain "hard pixels", which may mislead the generalization of models. We consider these hard pixels to come from two aspects: First, the labels are severely imbalanced and distributed across domains and classes, which may lead to features biased towards source domains and majority classes. Second, the naive threshold filtering pseudo-label methods limit the supervision of hard pixels. To address the above problems, we propose a novel co-training framework with progressive distribution alignment and uncertainty-interactive relabeling strategies. More concretely, a progressive distribution alignment strategy is proposed to match distribution across domains while providing additional supervision for tail class pixels. Additionally, an uncertainty-interactive relabeling strategy is proposed to retain more supervisory information for hard pixels and reduce the overall uncertainty of the pseudo-labels. Experiments on two widely-used benchmarks demonstrate the effectiveness of the proposed PDAUR, achieving state-of-the-art results.

Refining the seed region to obtain finely annotated pseudo masks for training a segmentation model is a crucial step in the multi-stage weakly supervised semantic segmentation (WSSS) framework. One of the most popular refinement methods, IRN, extends seed regions towards the edges in the image. However, we observed that, due to the lack of guidance from semantic information, IRN’s refinement may lead the generation of partially erroneous refinement directions. To address this issue, we leverage prototypes to recover the overlooked category semantic information in the refinement stage. We propose a prototype-based pseudo mask reclassification post-processing (PtReCl) to correct misclassified pixels in the pseudo masks, generating refined pseudo masks with more accurate coverage. Experimental evaluations demonstrate that our post-processing approach brings improvements in both pseudo mask quality and segmentation results on PASCAL VOC and MS COCO datasets, achieving state-of-the-art performance on VOC.

Since ultrasound imaging technique is convenient and real-time, it plays a crucial role in diagnosing thyroid nodules. With the development of deep learning, computer-aided diagnosis models have been widely applied to diagnose thyroid nodules, in which thyroid nodule segmentation is a basic yet essential task. Semi-supervised learning is a popular topic for thyroid ultrasound images segmentation under limited annotations. However, current mainstream semi-supervised methods (e.g., for nature image scenes) commonly produce poor segmentation results when facing thyroid ultrasound images. We analyze that there are two main reasons: First, since these methods lack the targeted learning for the ambiguous regions of images that may contain complementary clues for segmentation, the model will likely be over-fitting in the regions that are easy to predict, leading to fail to make full use of the unlabeled data. Second, these methods lack the shape constraint on thyroid nodules, resulting in incomplete segmentation shape on the nodule boundary. To address the above issues, we propose a novel Uncertainty-guided Dual Task Framework (UDTF). Concretely, we propose an Uncertainty Region Selection Module (URSM) to guide the segmentation task to learn from the ambiguous regions calculated by prototype under the constraint of consistency regularization. Additionally, to further keep the integrity of the nodules, we propose a Dual Task Module (DTM) to impose the shape constraint on thyroid nodules by exploring the task-level consistency between the segmentation and auxiliary reconstruction task. Extensive experiments are conducted on two thyroid ultrasound image datasets, including a private Philips Thyroid Ultrasound dataset and a public TN3K dataset. The results show that UDTF achieves superior performance compared to several other state-of-the-art methods.

The class imbalance problem, which is prevalent in medical image datasets, seriously affects the diagnostic effectiveness of deep learning-based network models. Recently, the method based on two-stage learning has produced promising results in solving class imbalance. In two-stage learning, the learning of unbiased classifiers has been well studied, but the representation of imbalanced data is still being explored. In this paper, we focus on the representation learning stage of class-imbalanced and propose a novel balanced and discriminative contrastive learning (BDCL) method. Compared with supervised contrastive learning, BDCL has two improvements: temperature dynamic learning, which balances the gradient contribution of negative samples from different classes; hard example prototypes, which can better learn the feature differences between tail classes. A variety of experimental results on the imbalanced medical image datasets show that BDCL enables the network models to learn representations with desirable balancedness and discriminativeness, effectively solving the class imbalance problem.

Recommender systems based on Matrix Factorization are widely used. However, they can easily suffer from the problem of overrecommendation of popular items, i.e., popularity bias. To mitigate popularity bias, current methods often uniformly model interactions' popularity bias degree considering user activity and interacted item's popularity, and then force the recommenders to focus more on less biased interactions. However, users' popularity preference for candidate items also plays an important role in estimating popularity bias, which is ignored by current methods. Therefore, their uniform modeling of bias degree results in sub-optimal debiasing performance. To address this issue, our core idea is to estimate personalized bias degrees to perform user-specific debiasing. We first derive a predefined bias degree obtained by items' popularity, then scale it considering users' candidate items' popularity. Extensive experiments conducted on two classic MF-based recommenders and three real-world datasets demonstrate that our approach outperforms state-of-the-art methods for popularity debiasing.

The resolution of a whole slide image (WSI) is too large to process directly, but WSI can be segmented into patches and be classified through multiple instance learning (MIL). Some patches have either close distances or similar pathological morphology, indicating that there are at least two types of relationships between patches. However, the existing MIL methods often deal with instance relationships simplistically. To solve this problem, we propose a new model named DualGCN-MIL. It analyzes the two types of relationships in WSI by constructing two different graphs in high-dimensional feature space and physical coordinate space and uses graph convolutional networks with unshared parameters for learning. Various experiments conducted on the Camelyon dataset have shown that compared to state-of-the-art methods, we can achieve better performance in multiple indicators, with accuracy of 88.89% and 85.11% on the 16 and 17 datasets, respectively, which proves that DualGCN-MIL has a higher accuracy in WSI classification. Implementation can be obtained from https://github.com/UnmatchedKatana/DualGCN-MIL.

Semi-supervised domain generalization (SSDG) aims to build a domain-generalized model using partially labeled data from source domains. Mainstream SSDG methods follow the augmentation consistency in FixMatch. However, the extraction of domain-invariant features may be challenging due to the absence of feature-based operations, further leading to overfitting of the classifier. To this end, we propose Multi-level Augmentation Consistency Learning (MACMatch), which improves the generalization of feature extractor and classifier through feature-based augmentation consistency. On the other hand, existing methods assume labeled data are class-balanced and domain-balanced, which is easily violated in practice. Based on this, we introduce Representativity and Diversity-based Sample Selection (RDSS), which models data as graphs to evaluate reasonable samples for labeling, relaxing the assumption for labeled data. Experiments on PACS and OfficeHome demonstrate that MACMatch outperforms state-of-the-art SSDG methods. Furthermore, MAC-Match with RDSS achieves competitive results without domain and class priori assumptions. Code is available at https://github.com/Y-J-Zhang/MACMatch-RDSS.

Generating class activation maps (CAM) as seed regions is a crucial step in weakly supervised semantic segmentation (WSSS). During the training of classification models, instance area information is incorporated into feature vectors and learned by the network. However, during CAM generation, pixel-level prediction scores are solely dependent on features and classifier weights, neglecting the area factor. Such area information becomes a source of bias between training and CAM generation, leading to a significant under-activation (false negatives) of foreground regions. To address this issue, we propose a class area intervention strategy to fine-tune the classification model and rectify this bias. Additionally, we employ a partition activation strategy that can simultaneously consider both under activation and over-activation to generate more precise AICAMs. AICAM is applicable to various WSSS methods based on CAM or its variants. The evaluations on both the PASCAL VOC and MS COCO datasets demonstrate that our method enhances multiple WSSS techniques.

Unsupervised Domain Adaptation (UDA) methods have become essential for computer-aided diagnostic analysis on medical images due to the advantage of improving the model generalization ability with fewer annotations. The disentanglement-based UDA methods improve the segmentation performance significantly by disentangling the features into domain-specific and domain-invariant components. However, such methods neglect detailed texture in ultrasound images, resulting in inaccurate segmentations of nodules. To address the above problem, we propose a Task-Oriented Feature Disentanglement (TOFD) method to achieve better UDA segmentation on images from different ultrasound machines. TOFD achieves finer-grained disentanglement by the Feature Constraint Module based on maximum entropy. Furthermore, the Domain Label Encoding Module can improve the stability of features by integrating the latent feature of domain bias. Extensive experiments demonstrate that TOFD can effectively enhance the model’s generalization capability and confirm the performance of TOFD on ultrasound images exceeds that of state-of-the-art UDA semantic segmentation.

Current methods for classifying IoT malware predominantly utilize binary and family classifications. However, these outcomes lack the detailed granularity to describe malicious behavior comprehensively. This limitation poses challenges for security analysts, failing to support further analysis and timely preventive actions. To achieve fine-grained malicious behavior identification in the lurking stage of IoT malware, we propose MaGraMal. This approach, leveraging masked graph representation, supplements traditional classification methodology, empowering analysts with critical insights for rapid responses. Through the empirical study, which took three person-months, we identify and summarize four fine-grained malicious behaviors during the lurking stage, constructing an annotated dataset. Our evaluation of 224 algorithm combinations results in an optimized model for IoT malware, achieving an accuracy of 75.83%. The maximum improvement brought by the hybrid features and graph masking achieves 5% and 4.16%, respectively. The runtime overhead analysis showcases MaGraMal’s superiority over the existing dynamic analysis-based detection tool (12x faster). This pioneering work combines machine learning and static features for malicious behavior profiling.

Personalized news recommendations are challenging due to the huge number of daily articles. While deep learning has achieved success in news recommendations, methods in the past often overlook the diversity of users' preferences. Recent works have explored multi-interest models to address this limitation. However, interests have different effects on click behavior, and directly modeling the matching between interests and candidates leads to the issue of spurious correlations. Specifically, when highly correlated interests obscure the true motivation for clicking, the model is unable to distinguish the interest that actually caused the click. To address this problem, this paper re-models the relationship between interests and click behavior from a causal perspective. Our proposed Counterfactual Weighted method for user M ulti-Interest modeling (CWMI) consists of a disentangled multi-interest extractor and an interest re-weighting module. Specifically, we first model the effect of interest on click behavior from a causal perspective. Then, we learn the disentangled user interests that only incorporate information from the currently clustered news. Finally, in the counterfactual world, we intervene with the current interest and re-weight it by comparing the changes in the ranking of candidates. We learned about the evolution of interest over time additionally. Experimental results on real-world news datasets demonstrate the effectiveness of the proposed methods, including disentangling interests and identifying the real interest that motivates clicks.

Multi-behavior sequential recommendation (MBSR) predicts a user's next item of interest based on their interaction history across different behavior types. Although existing studies have proposed capturing the correlation between different types of behavior, two important challenges have not been explored: i) Dealing with heterogeneous item transitions (both global and local perspectives). ii) Mitigating the issue of noise that arises from the incorporation of auxiliary behaviors. To address these issues, we propose a novel solution, Global Heterogeneous Graph and Target Interest Denoising for Multi-behavior Sequential Recommendation (GHTID). In particular, we view the transitions between behavior types of items as different relationships and propose two heterogeneous graphs. By considering the relationship between items under different behavioral types of transformations, we propose two heterogeneous graph convolution modules and explicitly learn heterogeneous item transitions. Moreover, we utilize two attention networks to integrate long-term and short-term interests associated with the target behavior to alleviate the noisy interference of auxiliary behaviors. Extensive experiments on four real-world datasets demonstrate that our method outperforms other state-of-the-art methods.

Sequential recommendation aims to capture users’ dynamic preferences. Due to the limited information in the sequence and the uncertain user behavior, data sparsity has always been a key problem. Although data augmentation methods can alleviate this issue, unreliable data can affect the performance of such models. To solve the above problems, we propose a new framework, namelyReliableData AugmentedContrastive LearningRecommender (RDCRec). Specifically, in order to generate more high-quality reliable items for data augmentation, we design a multi-attributes oriented sequence generator. It moves auxiliary information from the input layer to the attention layer for learning a better attention distribution. Then, we replace a percentage of items in the original sequence with reliable items generated by the generator as the augmented sequence, for creating a high-quality view for contrastive learning. In this way, RDCRec can extract more meaningful user patterns by using the self-supervised signals of the reliable items, thereby improving recommendation performance. Finally, we train a discriminator to identify unreplaced items in the augmented sequence thus we can update item embeddings selectively in order to increase the exposure of more reliable items and improve the accuracy of recommendation results. The discriminator, as an auxiliary model, is jointly trained with the generative task and the contrastive learning task. Large experiments on four popular datasets that are commonly used demonstrate the effectiveness of our new method for sequential recommendation.

The task of sequential recommendation aims to predict a user’s preference by analyzing the user’s historical behaviours. Existing methods model item transitions through leveraging sequential patterns. However, they mainly consider the target user’s behaviours and dynamic characteristics, while often ignoring high-order collaborative connections when modelling user preferences. Some recent works try to use graph-based methods to introduce high-order collaborative signals for sequential recommendation. However, these methods are flawed by two problems: the sequential patterns cannot be effectively mined and their way of introducing high-order collaborative signals is not suitable for sequential recommendation. To address these problems, we propose to fully exploit sequence features and model high-order collaborative signals for sequential recommendation. We propose a N eighbor user I ntention-based S equential Rec ommender (NISRec), which utilizes the intentions of high-order connected neighbor users as high-order collaborative signals in order to improve recommendation performance for the target user. The NISRec contains two main modules: the neighbor user intention embedding module (NIE) and the fusion module. The NIE module describes both the long-term and short-term intentions of neighbor users and aggregates them separately. The fusion module uses these two types of aggregated intentions to model high-order collaborative signals in both the embedding process and user preference modelling phase for recommendations of the target user. Experimental results show that our new approach outperforms the state-of-the-art methods on both sparse and dense datasets. Extensive studies further show the effectiveness of the diverse neighbor intentions introduced by the NISRec.

The class imbalance problem, which is prevalent in medical image datasets, seriously affects the diagnostic effectiveness of deep learning-based network models. To alleviate this problem, data re-sampling and loss re-weighting techniques are often used to reshape the decision boundary of the classifier. However, these techniques still lead to biased decision boundary due to the lack of sufficient and diversified samples in the tail classes of medical image datasets. In this paper, we propose an implicit feature augmentation with feature transfer (FT-IFA) method which solves the class imbalance problem by expanding the feature space of tail classes to reshape the decision boundary of the classifier. FT-IFA utilizes prototype similarity to transfer the rich transformation information from the head classes to tail classes on the basis of the balanced feature space, enriching the intra-class diversity of tail classes. Experimental results on two class-imbalanced medical image datasets show that FT-IFA outperforms the current state-of-the-art methods and effectively solves the class imbalance problem.

It is well known that weakly supervised semantic segmentation requires only image-level labels for training, which greatly reduces the annotation cost. In recent years, prototype-based approaches, which prove to substantially improve the segmentation performance, have been favored by a wide range of researchers. However, we are surprised to find that there are semantic gaps between different regions within the same object, hindering the optimization of prototypes, so the traditional prototypes can not adequately represent the entire object. Therefore, we propose region-specific prototypes to adaptively describe the regions themselves, which alleviate the effect of semantic gap by separately obtaining prototypes for different regions of an object. In addition, to obtain more representative region-specific prototypes, a plug-and-play Spatially Fused Attention Module is proposed for combining the spatial correlation and the scale correlation of hierarchical features. Extensive experiments are conducted on PASCAL VOC 2012 and MS COCO 2014, and the results show that our method achieves state-of-the-art performance using only image-level labels.

Acoustic-to-articulatory inversion (AAI) aims to estimate the parameters of articulators from speech audio. There are two common challenges in AAI, which are the limited data and the unsatisfactory performance in speaker independent scenario. Most current works focus on extracting features directly from speech and ignoring the importance of phoneme information which may limit the performance of AAI. To this end, we propose a novel network called SPN that uses two different streams to carry out the AAI task. Firstly, to improve the performance of speaker-independent experiment, we propose a new phoneme stream network to estimate the articulatory parameters as the phoneme features. To the best of our knowledge, this is the first work that extracts the speaker-independent features from phonemes to improve the performance of AAI. Secondly, in order to better represent the speech information, we train a speech stream network to combine the local features and the global features. Compared with state-of-the-art (SOTA), the proposed method reduces 0.18mm on RMSE and increases 6.0% on Pearson correlation coefficient in the speaker-independent experiment. The code has been released at https://github.com/liujinyu123/AAINetwork-SPN.

In recent years, the combination of knowledge graph (KG) and recommendation algorithms has achieved significant success. KG contains rich information to improve users and items representations as side information. Existing KG-aware recommendation methods have shown effectiveness in some cases. However, we contend these methods mainly focus on extracting structural semantic information from KG, while not fully exploring the collaborative semantic information (semantically similar information that may not be directly reflected in the graph structure). Considering the heterogeneity of KG, we design a novel method under the contrastive learning paradigm, named Knowledge Graph Recommendation based on Structural Semantics and Collaborative Semantics (KGSC). In this paper, we focus on exploring the integration of collaborative semantic information and studying the complementary relationship between structural semantics and collaborative semantics. Specifically, we assume that users (items) with similar representations exist in the same latent space. By mapping all entities representations into a collaborative semantic space, we are able to capture the underlying "commonality" features among entities. Then, we include them into contrastive pairs, to better mine users (items) features and relationships. Extensive experiments on three real-world public datasets show that our KGSC consistently outperforms state-of-the-art techniques.

In recent years, computer-aided diagnosis technology has made breakthroughs in clinical medicine. However, due to the different models and configurations of ultrasound instruments, the thyroid ultrasound images collected by different medical centers have different visual characteristics. It leads to domain shift in multi-center data and the lower generalization of computer-aided diagnosis models. We consider this limitation may attribute to the minor inter-class differences in thyroid ultrasound images, resulting in confusion space before and after domain feature alignment. Therefore, we propose an adversarial domain adaptation network with enhanced feature discriminability method. Among them, the discriminative feature learning module strengthens the discriminability of the learned features, so that the class distribution in each domain presents a high cohesion and low coupling effect; the class alignment module reduces the distance between the same class samples across domains to achieve class alignment and further strengthen the discriminability of each class. Experiments show that our method outperforms other state-of-the-art algorithms on the internal thyroid ultrasound image dataset. The method can effectively improve the model's generalization ability on multi-center thyroid ultrasound images.

Sequential recommendation aims to capture users' dynamic preferences, in which data sparsity is a key problem. Most contrastive learning models leverage data augmentation to address this problem, but they amplify noises in original sequences. Contrastive learning has the assumption that two views (positive pairs) obtained from the same user behavior sequence must be similar. However, noises typically disturb the user's main intention, which results in the dissimilarity of two views.

Previous works derive personalized speech features by training the model on a large dataset composed of his/her audio sounds. It was reported that face information has a strong link with the speech sound. Thus in this work, we innovatively extract personalized speech features from human faces to synthesize personalized speech using neural vocoder. A Face-based Residual Personalized Speech Synthesis Model (FR-PSS) containing a speech encoder, a speech synthesizer and a face encoder is designed for PSS. In this model, by designing two speech priors, a residual-guided strategy is introduced to guide the face feature to approach the true speech feature in the training. Moreover, considering the error of feature’s absolute values and their directional bias, we formulate a novel tri-item loss function for face encoder. Experimental results show that the speech synthesized by our model is comparable to the personalized speech synthesized by training a large amount of audio data in previous works.

The generation-based data augmentation method can overcome the challenge caused by the imbalance of medical image data to a certain extent. However, most of the current research focus on images with unified structure which are easy to learn. What is different is that ultrasound images are structurally inadequate, making it difficult for the structure to be captured by the generative network, resulting in the generated image lacks structural legitimacy. Therefore, a Progressive Generative Adversarial Method for Structurally Inadequate Medical Image Data Augmentation is proposed in this paper, including a network and a strategy. Our Progressive Texture Generative Adversarial Network alleviates the adverse effect of completely truncating the reconstruction of structure and texture during the generation process and enhances the implicit association between structure and texture. The Image Data Augmentation Strategy based on Mask-Reconstruction overcomes data imbalance from a novel perspective, maintains the legitimacy of the structure in the generated data, as well as increases the diversity of disease data interpretably. The experiments prove the effectiveness of our method on data augmentation and image reconstruction on Structurally Inadequate Medical Image both qualitatively and quantitatively. Finally, the weakly supervised segmentation of the lesion is the additional contribution of our method.

Lesion detection is one of the most important issues in the field of ultrasound image analysis. Recent studies exploit convolutional neural networks to either classify the nodules or locate them with bounding boxes (detection-based algorithms). However, the training of these methods demand considerable and laborious object-level annotations labeled by ultrasound experts and may lack the explicit border information for accurate localization. To reduce the annotation cost and improve the borders feature of proposal box, we propose a novel Border Sensitive Network (BSNet) that extract border features from the extreme point of the borders in proposal box to enhance the single point feature and use a vector to represent arbitrary distribution of border locations. Experimental results demonstrate that BSNet achieves state-of-the-art performance compared with other weakly supervised methods in Thyroid Ultrasound Images dataset, and further analysis reflects their value in clinical practice.

Weakly supervised segmentation techniques based on medical images ease the reliance of models on pixel-level annotation while advancing the field of computer-aided diagnosis. However, the differences in nodule size in thyroid ultrasound images and the limitations of class activation maps in weakly supervised segmentation methods lead to under- and over-segmentation problems in prediction. To alleviate this problem, we propose a novel weakly supervised segmentation network. This method is based on a dual branch soft erase module that expands the foreground response region while constraining the erroneous expansion of the foreground region by the enhancement of background features. In addition, the sensitivity of the network to the nodule scale size is enhanced by the scale feature adaptation module, which in turn generates integral and high-quality segmentation masks. The results of experiments performed on the thyroid ultrasound image dataset showed that our model outperformed existing weakly supervised semantic segmentation methods with Jaccard and Dice coefficients of 50.1% and 64.5%, respectively.

In self-supervised monocular depth estimation, the depth discontinuity and motion objects' artifacts are still challenging problems. Existing self-supervised methods usually utilize two views to train the depth estimation network and use one single view to make predictions. Compared with static views, abundant dynamic properties between video frames are beneficial to refining depth estimation, especially for dynamic objects. In this work, we improve the self-supervised learning framework for depth estimation using consecutive frames from monocular and stereo videos. The main idea is to exploit an implicit depth cue extractor which leverages dynamic and static cues to generate useful depth proposals. These cues can predict distinguishable motion contours and geometric scene structures. Moreover, a new high-dimensional attention module is proposed to extract a clear global transformation, which effectively suppresses the uncertainty of local descriptors in high-dimensional space, resulting in a more reliable optimization in the learning framework. Experiments demonstrate that the proposed framework outperforms the state-of-the-art on KITTI and Make3D datasets.

It is difficult and expensive to obtain labels for image semantic segmentation tasks. This has led to more and more researches focusing on weakly supervised semantic segmentation (WSSS) with image-level labels and the class activation maps (CAM) is often used to locate objects. The performance of WSSS methods largely depends on the accuracy of the generated CAMs, but current methods can only perform rough locali/ation, especially cannot fit CAM to the object’s edge well. Therefore, we propose the Edge Enhancement Network (EEN), which uses the shallow features of the network to enhance edge information. This allows us to obtain more accurate CAMs that fit the edges only through image-level labels. It greatly improves the accuracy of pseudo ground truth and easily reaches the level of current mainstream methods. After many experiments, it has reached 67.0% mIoU on the Pascal VOC2012 validation set, which exceeds the latest method under the same training settings.

Cued Speech (CS) is a communication system for deaf people or hearing impaired people, in which a speaker uses it to aid a lipreader in phonetic level by clarifying potentially ambiguous mouth movements with hand shape and positions.Feature extraction of multi-modal CS is a key step in CS recognition.Recent supervised deep learning based methods suffer from noisy CS data annotations especially for hand shape modality.In this work, we first propose a self-supervised contrastive learning method to learn the feature representation of image without using labels.Secondly, a small amount of manually annotated CS data are used to fine-tune the first module.Thirdly, we present a module, which combines Bi-LSTM and self-attention networks to further learn sequential features with temporal and contextual information.Besides, to enlarge the volume and the diversity of the current limited CS datasets, we build a new British English dataset containing 5 native CS speakers.Evaluation results on both French and British English datasets show that our model achieves over 90% accuracy in hand shape recognition.Significant improvements of 8.75% (for French) and 10.09% (for British English) are achieved in CS phoneme recognition correctness compared with the state-of-the-art.

Cued Speech (CS) is a visual communication system for the deaf or hearing impaired people. It combines lip movements with hand cues to obtain a complete phonetic repertoire. Current deep learning based methods on automatic CS recognition suffer from a common problem, which is the data scarcity. Until now, there are only two public single speaker datasets for French (238 sentences) and British English (97 sentences). In this work, we propose a cross-modal knowledge distillation method with teacher-student structure, which transfers audio speech information to CS to overcome the limited data problem. Firstly, we pretrain a teacher model for CS recognition with a large amount of open source audio speech data, and simultaneously pretrain the feature extractors for lips and hands using CS data. Then, we distill the knowledge from teacher model to the student model with frame-level and sequence-level distillation strategies. Importantly, for frame-level, we exploit multi-task learning to weigh losses automatically, to obtain the balance coefficient. Besides, we establish a five-speaker British English CS dataset for the first time. The proposed method is evaluated on French and British English CS datasets, showing superior CS recognition performance to the state-of-the-art (SOTA) by a large margin.

With the maturity of artificial intelligence, AI-aided diagnosis technology is gradually widely applied in clinical medicine. However, for the same pathological tissue, medical images produced by different types of instruments usually possess different data distributions. Because of the domain shift phenomenon, AI-aided diagnosis cannot accurately diagnose medical images in other domains, which is a waste of precious medical images. This paper proposes a Multi-Scale Self-Attention Unsupervised Domain Adaptive framework (MSDAN), which consists of three modules. First, the multi-scale framework constrains the source domain features and target domain features by optimizing adversarial losses with different level features. Second, the mix-up discriminator extracts latent spatial features by mixing up source domain and target domain features. Finally, MSDAN learns the geometric information of the pathological tissues in medical images through the self-attention module, thereby improving the transfer effect of the semantic information in medical images. Extensive experiments prove that the proposed approach can achieve superior performance on tasks with various degrees of domain shift and data complexity, especially for thyroid ultrasound images.

MRI technology is usually used to distinguish the grade of the tumor in the patient. Due to technical limitations, the classification of tumors (high-grade gliomas and metastases) on MRI images has become a problem for doctors. At present, the widely used neural network is gradually applied to the tumor classification of MRI images, which not only reduces the burden of human resources, but also shows good classification accuracy. Although different good experimental data are obtained under various neural networks, there is still a problem in using these neural networks for tumor classification: the semantic information expressed on the image by the deep features of the neural network is too scattered, and it is difficult to concentrate the lesion area. In this article, we propose a new strategy that combines the approximate symmetry properties of the MRI image with neural network, then uses a dual-branch network instead of the basic network for feature extraction, and adds complementary learning to the network, different features fusion and attention mechanism to enrich detailed information. Our method performs multiple comparison and ablation experiments on the dataset of glioma and metastasis, which proves that the proposed method is effective for tumor classification assisted by MRI.

Complicated medical image analysis often requires a combination of disease classification, lesion detection and lesion segmentation. However, models designed for different tasks produce inconsistent or non-corresponding predictions and ignore the implicit connections between tasks. We propose a novel framework, which makes full use of the fact that segmentation and detection are mutually beneficial, boosts these three tasks in a unified framework. The proposed Information Enhancement Module uses classification information as a beneficial supplement to locate lesion quickly for segmentation. To further achieve fine segmentation with clear boundaries, we propose a Boundary-aware Loss, which dynamically adjusts supervised signal, so that our model pays more attention to boundary in later stages of training. Through experiments conducted on Thyroid Ultrasound dataset, we have demonstrated the good performance of the proposed method in joint segmentation and detection.

Link prediction is the main task of knowledge graph completion, predicting missing relations between entities based the existing links among the entities. The problem of knowledge graph completion can be framed as a third-order binary tensor completion problem. In this case, tensor decomposition seems like a natural solution. And many previous studies have shown that tensor decomposition methods are superior to Trans-based methods in link prediction experiments. Typical tensor decomposition methods are Canonical Polyadic (CP) decomposition and Tucker decomposition. In this paper, we propose Block term decomposition Embedding model (BTDE) for link prediction based on Block term decomposition (which can be seen as a combination of CP decomposition and Tucker decomposition) of the binary tensor representation of knowledge graph triples. The embeddings learned through BTDE is interpretable. In addition, we prove BTDE is fully expressive and derive the bound on its entity and relation embedding dimensionality for full expressivity which is the same as TuckER and smaller than the bound of previous start-of-the-art models ComplEx and SimplE. We show empirically that BTDE outperforms most previous state-of-the-art models across five standard link prediction datasets.

Thyroid ultrasound image is an important basis for artificial intelligence assisted treatment of thyroid-related diseases, but existing images usually contain special cross symbols which represent the location of nodules marked by doctors, thus affecting the features and diagnostic results extracted by the deep learning algorithm. We propose Pyramid GAN(Py-GAN) for blind image inpainting to remove cross symbols. Py-GAN contains a generator with pyramid structure and a global discriminator. The global discriminator improves the authenticity of the corrupted regions and image consistency. The generator uses the joint context loss to get clear image restoration, which prevents the information loss of non-completion area. The inpainting results of the proposed Py-GAN not only maintains the texture and structural information of the original image, but also has the greatest advantage that there are no artifacts in the corrupted regions, achieving pixel-level realism. Both qualitative and quantitative comparisons are superior to existing learning/non-learning image inpainting works.

Image segmentation based on deep learning has greatly promoted the development of the field of computer-aided diagnosis. However, the large scale medical annotation of ground truth is so difficult that it directly affects the performance of existing segmentation models. In this work, an Attention based Semi-supervised Neural Network is proposed, which can complete end-to-end segmentation task of thyroid ultrasound image with weakly annotated classification data and a small amount of fully annotated segmentation data. Two kinds of attention modules are proposed to improve network performance through the trainable feedforward structure of bottom-up and top-down so as to suppress or activate the feature channels and image regions respectively. The experimental results show that when there is only 13% of fully annotated data, the Jaccard similarity coefficient of thyroid nodule segmentation is 74.91%, 4.97% higher than VGG-based semi-supervised model. The classification accuracy of benign and malignant is increased from 91.67% to 95.00%. Equally important, with the same number of fully annotated data, our model has better generalization than that of the supervised segmentation models.

Effectively analyzing and mining large-scale heterogeneous information networks (HINs) by adopting network representation learning (NRL) approaches have received increasing attention. The abundant semantic and structural information contained in HINs not only facilitates network analysis and downstream tasks, but also poses special challenges to well capture that rich information. With the intention to preserve such rich yet potential information during HIN embedding, we first discuss the latent dependence existed in indirect neighbors, then study the different abilities of forward layer and backward layer of bidirectional recurrent neural network to remain semantic of HINs. And finally, we propose a novel representation learning model for HIN, namely RL4HIN. RL4HIN utilizes a skip-dependence strategy for enhancing the latent dependence between farther neighbors, and then develops a proposed weighted loss function in order to balance such difference between forward and backward layer. Extensive experiments, including node classification and visualization, have been conducted on two large- scale and real-world HINs. The experimental results show that RL4HIN significantly outperforms several state-of-the-art NRL approaches.

The incumbent sharding schemes usually assign the nodes to different committees randomly to meet the demands of security and efficiency at the same time. For example, Elastico protocol obtains a random value by letting the node perform proof of work, and then uses this value for sharding. However, the strategy of random sharding ignores the objective differences between nodes, causing performance gaps between different committees in blockchain. This creates a bottleneck in the transaction throughput of the blockchain. In the paper, we propose a node rating based sharding scheme for blockchain system called NRSS. The key idea of NRSS is to evaluate nodes in the network by both the speeds and results of transactions verification before, and then assign them into different committees by balancing the score to reduce the performance gap between committees and increase the speed of transaction process. We implement NRSS in a local blockchain system, and the experiment results show that NRSS can increase the sharding effect of a blockchain, with an average throughput increase of 32.2% in the simulation environment where the node performance difference is up to 75%, depending on the number of nodes in the committee that are preset in the blockchain.

Knowledge representation learning (KRL), which transforms both the entities and relations into continuous low dimensional continuous vector space, has attracted considerable research. Most of existing knowledge graph (KG) completion models only considers the structural representation of triples, but do not consider the important text information about entity descriptions in the knowledge base. We propose a text-enhanced KG model based on gated convolution network (GConvTE), which can learn entity descriptions and symbol triples jointly by feature fusion. Specifically, each triple (head entity, relation, tail entity) is represented as a 3-column structural embedding matrix, a 3-column textual embedding matrix and a 3-column joint embedding matrix where each column vector represents a triple element. Textual embeddings are obtained by bidirectional gated recurrent unit with attention (A-BGRU) encoding entity descriptions and joint embeddings are obtained by the combination of textual embeddings and structural embeddings. Extending feature dimension in embedding layer, these three matrixs are concatenated into 3-channel feature block to be fed into convolution layer, where the gated unit is added to selectively output the joint features maps. These feature maps are concatenated and then multiplied with a weight vector via a dot product to return a score. The experimental results show that our model GConvTE achieves better link performance than previous state-of-art embedding models on two benchmark datasets.