Pre-trained language models (PrLMs) demonstrate impressive performance on the sentiment analysis task. However, the large number of trainable parameters brings about heavy computational costs, which become more serious in multi-domain scenarios. In this paper, we propose to extract multi-layer features from the PrLM for efficient training since the training process is independent to its large backbone. Meanwhile, compared with the conventional feature extraction, we leverage prompts to induce PrLM for generating sentiment-aware features which lead to significant improvement on the sentiment analysis. In addition, most previous methods adopted a domain alignment paradigm for multi-domain learning, which becomes cumbersome when the number of domains is large. Therefore, we propose a novel prompt-augmented cross-domain contrastive learning for generalizable performance, which clusters samples with the same label under different prompts or domains. Our method is evaluated on two public multi-domain sentiment analysis benchmarks, which significantly outperforms recent state-of-the-art methods. Extensive ablation studies also verify the effectiveness of each proposed component.

The performance of pedestrian detectors typically relies on sufficient labeled data, and semi-supervised learning is a promising way to address the deficiency in manual annotations by utilizing sufficient unlabeled images. In this work, we design a Structure-Controllable Pedestrian Instance Generation approach (SCPIG), which is tailored to semi-supervised pedestrian detection. Specifically, we adopt a mask encoder to transform mask images into the embeddings encapsulating structure knowledge. In addition, we incorporate a mapping network to transform random latent code and a conditional generation network to synthesize diverse pedestrian instances, where the transformed code and mask embedding control pedestrian appearance and structure, respectively. The synthesized pedestrian instances are used to construct high-quality pseudo-labeled images for training pedestrian detectors. Extensive experiments validate the effectiveness of SCPIG in controllable pedestrian instance synthesizing and semi-supervised pedestrian detection.

Multi-view clustering (MVC) has shown that leveraging both consistency and complementary information across views enhances clustering performance. However, most existing methods focus on aligning features into the same dimension, often neglecting cross-view heterogeneity and introducing discrepancies. To address this, we propose a novel multi-view clustering framework that combines cross-view neighborhood contrastive learning with a cross-attention view-mixup feature learning mechanism. Specifically, the cross-attention view-mixup module learns view-invariant feature representations by capturing complementary and consistent information, while the neighborhood contrastive learning module uncovers semantic structures across views based on the learned mixup features. By implicitly performing feature mixup across views and effectively integrating cross-view neighborhood contrastive learning, our method alleviates cross-view discrepancies and enables more effective integration of complementary and consistent information, ultimately enhancing clustering performance. Experiments conducted on several real datasets demonstrate the effectiveness of our proposed method in comparision with several representative MVC approaches.

Graph-based multiview clustering (MVC) approaches have demonstrated impressive performance by leveraging the consistency properties of multiview data in an unsupervised manner. However, existing methods for graph learning heavily rely on either Euclidean structures or the manifold topological structures derived from fixed view-specific graphs. Unfortunately, these approaches may not accurately reflect the consensus topological structure in a multiview setting. To address this limitation and enhance the intrinsic graph learning process, an adaptive exploration of a more appropriate consistency topological structure is required. Toward this end, we propose a novel approach called collaborative topological graph learning (CTGL) for MVC. The key idea is to adaptively discover the consistent topological structure to guide intrinsic graph learning. We achieve this by introducing an auxiliary consistency graph that formulates the topological relevance learning function. However, estimating the auxiliary consistency graph is not straightforward, as it is based on the learned view-specific graphs and requires prior availability. To overcome this challenge, we develop a collaborative learning strategy that simultaneously learns both the auxiliary consistency graph and view-specific graphs using tensor learning techniques. This strategy enables the adaptive exploration of the consistency topological structure during graph learning, resulting in more accurate clustering outcomes. Extensive experiments are provided to show the effectiveness of the proposed method. The source code can be found at https://github.com/CLiu272/CTGL.

Recently, fine-tuning the large pre-trained language models on the labeled sentiment dataset achieves appealing performance. However, the obtained model may not generalize well to the other domains due to the domain shift, and it is expensive to update the entire parameters within the large models. Although some existing domain matching methods are proposed to alleviate the above issues, there are multiple relevant source domains in practice which makes the whole training more costly and complicated. To this end, we focus on the efficient unsupervised multi-source sentiment adaptation task which is more challenging and beneficial for real-world applications. Specifically, we propose to extract multi-layer features from the large pre-trained model, and design a dynamic parameters fusion module to exploit these features for both efficient and adaptive tuning. Furthermore, we propose a novel feature structure matching constraint, which enforces similar feature-wise correlations across different domains. Compared with the traditional domain matching methods which tend to pull all feature instances close, we show that the proposed feature structure matching is more robust and generalizable in the multi-source scenario. Extensive experiments on several multi-source sentiment analysis benchmarks demonstrate the effectiveness and superiority of our proposed framework.

With the advent of generative models and vision-language pre-training, significant improvement has been made in text-driven face manipulation. The text embedding can be used as target supervision for expression control. However, it is non-trivial to associate with its 3D attributes, i.e., pose and illumination. To address these issues, we propose a Text-conditional Attribute aLignment approach for 3D controllable face image synthesis, and our model is referred to as TcALign. Specifically, since the 3D rendered image can be precisely controlled with the 3D face representation, we first propose a Text-conditional 3D Editor to produce the target face representation to realize text-driven manipulation in the 3D space. An attribute embedding space spanned by the target-related attributes embeddings is also introduced to infer the disentangled task-specific direction. Next, we train a cross-modal latent mapping network conditioned on the derived difference of 3D representation to infer a correct vector in the latent space of Style-GAN. This correction vector learning design can accurately transfer the attribute manipulation on 3D images to 2D images. We show that the proposed method delivers more precise text-driven multi-attribute manipulation for 3D controllable face image synthesis. Extensive qualitative and quantitative experiments verify the effectiveness and superiority of our method over the other competing methods.

Facial makeup transfer aims to replicate reference makeup on target face, and the existing methods are mainly based on a generic adversarial training process. In this work, we design a Reference-conditional Makeup-aware Discrimination approach (RcMD) to facilitate makeup transfer. Specifically, we perform region-wise semantic feature extraction from a reference makeup image and a source image without makeup. A generator learns to capture and render the reference makeup by modulating the region-wise intermediate features. To ensure precise makeup on target face, we incorporate a reference-conditional discrimination network, which learns to measure the regional makeup consistency between reference and synthesized images. Considering the discrepancy between reference and target faces, an alignment module is trained to fuse the extracted features, conditioned on the reference style. Based on the feature statistics, we perform regional real-synthesized makeup discrimination to ensure precise makeup rendering. Extensive experiments are performed to demonstrate the effectiveness of our designed modules and the superior performance of RcMD in transferring diverse real-world facial makeup.

Partially View-aligned Clustering (PVC) presents a challenge as it requires a comprehensive exploration of complementary and consistent information in the presence of partial alignment of view data. Existing PVC methods typically learn view correspondence based on latent features that are expected to contain common semantic information. However, latent features obtained from heterogeneous spaces, along with the enforcement of alignment into the same feature dimension, can introduce cross-view discrepancies. In particular, partially view-aligned data lacks sufficient shared correspondences for the critical common semantic feature learning, resulting in inaccuracies in establishing meaningful correspondences between latent features across different views. While feature representations may differ across views, instance relationships within each view could potentially encode consistent common semantics across views. Motivated by this, our aim is to learn view correspondence based on graph distribution metrics that capture semantic view-invariant instance relationships. To achieve this, we utilize similarity graphs to depict instance relationships and learn view correspondence by aligning semantic similarity graphs through optimal transport with graph distribution. This facilitates the precise learning of view alignments, even in the presence of heterogeneous view-specific feature distortions. Furthermore, leveraging well-established cross-view correspondence, we introduce a cross-view contrastive learning to learn semantic features by exploiting consistency information. The resulting meaningful semantic features effectively isolate shared latent patterns, avoiding the inclusion of irrelevant private information. We conduct extensive experiments on several real datasets, demonstrating the effectiveness of our proposed method for the PVC task.

Clustering of the single-cell RNA-seq (scRNA-seq) transcriptome profiles is able to identify cell types, which is beneficial to improve the understanding of disease progression. However, in practice, the single-cell expression data often contains a significant number of missing values as a result of technical variability. Missing data is a critical challenge in scRNA-seq clustering analysis since the unknown value does not reflect the underlying true expression level and makes it difficult to discovering cell types by applying clustering algorithms directly. Various approaches have been developed to overcome missing data issue in scRNA-seq clustering. Most of them recover missing expression values by borrowing observed data from similar cells or synthesizing data via generative adversarial networks. Such that the biologically meaningful cluster structure has not been sufficiently exploited. In this work, we introduce ColImpute, a collaborative structure-preserved missing data imputation approach for the scRNA-seq clustering. Specifically, a cluster structure-preserved imputation module and a subspace clustering module, which respectively perform missing data imputation and cell subtypes identification, are integrated into a unified optimization framework to train the two networks in a collaborative manner. Consequently, the clustering module effectively contributes cluster-structure information to guide the trainning process of the missing data imputation module. Simultaneously, the cluster structure-preserved imputation module reciprocally enhances the performance of the clustering module by generating more precise recovered samples. Promising experimental results show that the proposed method is effective for both the data imputation and the cell types identification.

Incomplete multi-view clustering (IMVC) presents a significant challenge due to the need for effectively exploring complementary and consistent information within the context of missing views. One promising strategy to tackle this challenge is to recover missing views by inferring the missing samples. However, such approaches often fail to fully utilize discriminative structural information or adequately address consistency, as it requires such information to be known or learnable in advance, which contradicts the incomplete data setting. In this study, we propose a novel approach calledLatentStructure-Aware view recovery (LaSA) for the IMVC task. Our objective is to recover missing views through discriminative latent representations by leveraging structural information. Specifically, our method offers a unified closed-form formulation that simultaneously performs missing data inference and latent representation learning, using a learned intrinsic graph as structural information. This formulation, incorporating graph structure information, enhances the inference of missing data while facilitating discriminative feature learning. Even when intrinsic graph is initially unknown due to incomplete data, our formulation allows for effective view recovery and intrinsic graph learning through an iterative optimization process. To further enhance performance, we introduce an iterative consistency diffusion process, which effectively leverages the consistency and complementary information across multiple views. Extensive experiments demonstrate the effectiveness of the proposed method compared to state-of-the-art approaches.

In this work, we study a more realistic challenging scenario in multiview clustering (MVC), referred to as incomplete MVC (IMVC) where some instances in certain views are missing. The key to IMVC is how to adequately exploit complementary and consistency information under the incompleteness of data. However, most existing methods address the incompleteness problem at the instance level and they require sufficient information to perform data recovery. In this work, we develop a new approach to facilitate IMVC based on the graph propagation perspective. Specifically, a partial graph is used to describe the similarity of samples for incomplete views, such that the issue of missing instances can be translated into the missing entries of the partial graph. In this way, a common graph can be adaptively learned to self-guide the propagation process by exploiting the consistency information, and the propagated graph of each view is in turn used to refine the common self-guided graph in an iterative manner. Thus, the associated missing entries can be inferred through graph propagation by exploiting the consistency information across all views. On the other hand, existing approaches focus on the consistency structure only, and the complementary information has not been sufficiently exploited due to the data incompleteness issue. By contrast, under the proposed graph propagation framework, an exclusive regularization term can be naturally adopted to exploit the complementary information in our method. Extensive experiments demonstrate the effectiveness of the proposed method in comparison with state-of-the-art methods. The source code of our method is available at the https://github.com/CLiu272/TNNLS-PGP.

Incomplete multi-view clustering (IMVC) is challenging, as it requires adequately exploring complementary and consistency information under the incompleteness of data. Most existing approaches attempt to overcome the incompleteness at instance-level. In this work, we develop a new approach to facilitate IMVC from a new perspective. Specifically, we transfer the issue of missing instances to a similarity graph completion problem for incomplete views, and propose a self-supervised multi-view graph completion algorithm to infer the associated missing entries. Further, by incorporating constrained feature learning, the inferred graph can be naturally leveraged in representation learning. We theoretically show that our feature learning process performs an Auto-Regressive filter function by encoding the learned similarity graph, which could yield discriminative representation for a clustering task. Extensive experiments demonstrate the effectiveness of the proposed method in comparison with state-of-the-art methods.

Recently, fine-tuning the pre-trained language model (PrLM) on labeled sentiment datasets demonstrates impressive performance. However, collecting labeled sentiment dataset is time-consuming, and fine-tuning the whole PrLM brings about much computation cost. To this end, we focus on multi-source unsupervised sentiment adaptation problem with the pre-trained features, which is more practical and challenging. We first design a dynamic feature network to fully exploit the extracted pre-trained features for efficient domain adaptation. Meanwhile, with the difference of the traditional source-target domain alignment methods, we propose a novel asymmetric mutual learning strategy, which can robustly estimate the pseudo-labels of the target domain with the knowledge from all the other source models. Experiments on multiple sentiment benchmarks show that our method outperforms the recent state-of-the-art approaches, and we also conduct extensive ablation studies to verify the effectiveness of each the proposed module.

Deep subspace clustering has achieved remarkable performance in unsupervised clustering tasks. The self-supervised approach is further introduced to learn more discriminative representation for enhancing clustering performance. Despite the significant improvement of clustering performance by exploiting self-supervision information, these approaches heavily depend on the high quality of pseudo-label from the current clustering result and this will inevitably degrade the clustering performance when the obtained pseudo-labels are incorrect. To solve this issue, we develop a robust self-supervised deep subspace clustering approach by exploiting the reliable self-supervised information during training. The proposed method is involved in two key steps: a diffusion processing step is developed to improve self-expressiveness matrix such that more accurate clustering result (pseudo-labels) can be obtained. More importantly, we further propose to estimate and exploit the reliability of the assigned pseudo-label for each sample to alleviate the negative impact of incorrect pseudo-labels, such that the unreliable self-supervision can be further alleviated. Experimental studies on several benchmark datasets validate the effectiveness of our approach in terms of refining the self-supervised information. The source code of the proposed method is available at the https://github.com/stuljj/RSDSC.git.

In this work, we propose an unsupervised ensemble learning method via network generation, referred to as UELNG. Specifically, we first generate weights of clustering ensemble models by adopting HyperGAN, and obtain diverse partitions for data. With these partitions, we can easily identify high-confident pseudo-labels as supervised information to predict low-entropy labels for unlabeled augmented data, thereby enhancing the quality of pseudo-labels and clustering accuracy. We conduct experiments on multiple data sets. Experimental results indicate that our method outperforms state-of-the-art methods by 0.3%, 1.8%, 5.7%, 3.2% and 2.4% on MNIST, STL-10, CIFAR-10, Reuters and 20News, respectively, which demonstrates the effectiveness of our proposed UELNG.

In this paper, we investigate a challenging unsupervised domain adaptation setting --- unsupervised model adaptation. We aim to explore how to rely only on unlabeled target data to improve performance of an existing source prediction model on the target domain, since labeled source data may not be available in some real-world scenarios due to data privacy issues. For this purpose, we propose a new framework, which is referred to as collaborative class conditional generative adversarial net to bypass the dependence on the source data. Specifically, the prediction model is to be improved through generated target-style data, which provides more accurate guidance for the generator. As a result, the generator and the prediction model can collaborate with each other without source data. Furthermore, due to the lack of supervision from source data, we propose a weight constraint that encourages similarity to the source model. A clustering-based regularization is also introduced to produce more discriminative features in the target domain. Compared to conventional domain adaptation methods, our model achieves superior performance on multiple adaptation tasks with only unlabeled target data, which verifies its effectiveness in this challenging setting.

Deep learning demonstrates its impressive success across various machine learning problems. However, its performance often suffers in the case where the training and test data sets follow different distributions, due to the domain shift. Most current domain adaptation methods minimize the discrepancy between the source and target domains by enforcing the alignment of their marginal distributions without considering the class-level matching. Consequently, data from different classes may become close together after mapping. To address this issue, we propose an unsupervised domain adaptation method by generating image-label pairs in the target domain, in which the model is augmented with the generated target pairs and achieve class-level transfer. Specifically, we integrate generative adversarial networks (GAN) into the model predictor, where the generator fed with labels aims to produce corresponding target domain images with a well-designed semantic loss. Meanwhile, compared to previous methods which focus on discrepancy reduction across domains, i.e., image to image translation, our model focuses on semantic preservation during image generation. Our model is straightforward yet effective for unsupervised domain adaptation problems. Without any labels in the target domain in all the experiments, we demonstrate the validity of our approach by presenting the plausible generated target image-label pairs. In addition, our proposed method achieves the best or comparable performance on multiple unsupervised domain adaptation datasets which include image classification and semantic segmentation.

In this paper, we explore how to leverage readily available unlabeled data to improve semi-supervised human detection performance. For this purpose, we specifically modify the region proposal network (RPN) for learning on a partially labeled dataset. Based on commonly observed false positive types, a verification module is developed to assess foreground human objects in the candidate regions to provide an important cue for filtering the RPN's proposals. The remaining proposals with high confidence scores are then used as pseudo annotations for re-training our detection model. To reduce the risk of error propagation in the training process, we adopt a self-paced training strategy to progressively include more pseudo annotations generated by the previous model over multiple training rounds. The resulting detector re-trained on the augmented data can be expected to have better detection performance. The effectiveness of the main components of this framework is verified through extensive experiments, and the proposed approach achieves state-of-the-art detection results on multiple scene-specific human detection benchmarks in the semi-supervised setting.

For unsupervised domain adaptation, the process of learning domain-invariant representations could be dominated by the labeled source data, such that the specific characteristics of the target domain may be ignored. In order to improve the performance in inferring target labels, we propose a targetspecific network which is capable of learning collaboratively with a domain adaptation network, instead of directly minimizing domain discrepancy. A clustering regularization is also utilized to improve the generalization capability of the target-specific network by forcing target data points to be close to accumulated class centers. As this network learns and specializes to the target domain, its performance in inferring target labels improves, which in turn facilitates the learning process of the adaptation network. Therefore, there is a mutually beneficial relationship between these two networks. We perform extensive experiments on multiple digit and object datasets, and the effectiveness and superiority of the proposed approach is presented and verified on multiple visual adaptation benchmarks, e.g., we improve the state-ofthe-art on the task of MNIST→SVHN from 76.5% to 84.9% without specific augmentation.

Learning class-conditional data distributions is crucial for Generative Adversarial Networks (GAN) in semi-supervised learning. To improve both instance synthesis and classification in this setting, we propose an enhanced TripleGAN (EnhancedTGAN) model in this work. We follow the adversarial training scheme of the original TripleGAN, but completely re-design the training targets of the generator and classifier. Specifically, we adopt feature-semantics matching to enhance the generator in learning class-conditional distributions from both the aspects of statistics in the latent space and semantics consistency with respect to the generator and classifier. Since a limited amount of labeled data is not sufficient to determine satisfactory decision boundaries, we include two classifiers, and incorporate collaborative learning into our model to provide better guidance for generator training. The synthesized high-fidelity data can in turn be used for improving classifier training. In the experiments, the superior performance of our approach on multiple benchmark datasets demonstrates the effectiveness of the mutual reinforcement between the generator and classifiers in facilitating semi-supervised instance synthesis and classification.

Existing domain adaptation approaches generalize models trained on the labeled source domain data to the unlabeled target domain data by forcing feature distributions of two domains closer. However, these approaches are likely to ignore the semantic information during the feature alignment between source and target domain. In this paper, we propose a new unsupervised domain adaptation framework to learn the cross-domain features and disentangle the semantic information concurrently. Specifically, we firstly combine the task-specific classification and domain adversarial learning to obtain the cross-domain features by mapping the data of both domains with the shared feature extractor. Secondly, we integrate the domain adversarial learning and the within-domain reconstruction to disentangle the semantic information from the domain information. Thirdly, we include a cross-domain transformation to further refine the feature extractor, which in turn improves the performances of the task classifier. We compare our proposed model to previous state-of-the-art methods on domain adaptation digit classification tasks. Experimental results show that our model achieves better performances than the other counterparts, which demonstrates the superiority and effectiveness of our model.