Source-free domain adaptation (SFDA) aims to transfer knowledge from a source domain to an unlabeled target domain without requiring access to source data. Although previous works have focused on clustering target domain samples from continuous training, there are still some challenges: i) More source domain knowledge is forgotten with more training epochs. ii) Achieving better learning results often requires increased computational resources. To solve these problems, we propose a novel Marginal Exploration for Source-Free Domain Adaptation (ME-SFDA) method, which is a multi-scale information fusion learning based on our designed Pyramidal Atkinson-Shiffrin memory. Specifically, we design a two-step module to split samples into clustered cores and response scatters by sensory memory. Then, a novel technique is proposed for clustering samples in a hierarchical way, utilizing long-term memory to cluster cores derived from splitting the samples earlier and guide response scatters. To effectively divide samples of different classes, we propose a method that encourages unambiguous cluster assignments for the samples using multi-scale fusion information. To verify the generality of our approach, we not only discuss the UDA and SFDA tasks but also apply it to the semi-supervised domain adaptation (SSDA), which utilizes a few labeled target samples based on UDA. Extensive experiments on all utilized standard benchmarks indicate that our approach outperforms previous SOTA methods.

Just recognizable distortion (JRD) has emerged as a promising paradigm for machine-centric video coding. However, existing JRD-guided coding methods are limited by coarse annotation granularity and high computational cost, which hinder their deployment. In this paper, we first investigate the impact of different JRD annotation strategies on downstream task performance. By incorporating both instance-level and contextual information, we construct a new JRD dataset with fine-grained annotations compatible with object detection and instance segmentation tasks. To enhance quantization parameter (QP) map prediction while maintaining computational efficiency, we propose a novel spiking neural network (SNN)-based framework that decomposes video frames into spatial structures, channel interactions, and temporal patterns. Furthermore, we introduce a spiking attention mechanism to aggregate task-relevant features and employ adaptive scaling vectors to suppress machine-perceived redundancy, enabling targeted bitrate allocation aligned with task-critical content. Extensive experiments on multiple datasets and backbones demonstrate that our approach consistently outperforms state-of-the-art codec-based and JRD-guided methods in maintaining task performance at ultra-low bitrates, while significantly reducing computational overhead.

As a specialized paradigm of domain adaptation, blended-target domain adaptation (BTDA) transfers knowledge from a source domain to a blended target domain. In this paper, we propose an Energy-Driven Explicit Alignment Network (EDEAN) framework that innovatively applies energy-based models (EBMs) to address BTDA problems. We observe that EBMs display free energy biases when the source domain and the target domain data originate from different distributions. Therefore, we use these biases as a measure of the discrepancies between the source domain and the target domain and align them by minimizing these biases via the free energy alignment (FEA) module. We further propose the balanced weight distribution (BWD) module, which comprehensively considers the complementary information between the linear and semantic pseudo-labels and obtains the corresponding complementary information by mixing both label types. Moreover, we propose the normalized free energy (NFE) module, which assigns higher weights to high free energy samples and dynamically corrects the pseudo-labels by continuously updating these weights. We also conducted experiments on four widely used BTDA databases and achieved substantial improvements over the latest BTDA methods.

Blended-target domain adaptation (BTDA) leverages learned source knowledge to adapt the model to a blended-target domain that is composed of multiple unlabeled sub-target domains with distinct statistical characteristics. The existing BTDA methods usually overlook semantic correlation information across multiple domains and domain shifts among sub-target domains, resulting in suboptimal adaptation performance. To fully harness semantic knowledge and alleviate domain shifts in hybrid data distribution, we propose a collaborative semantic consistency alignment (CSCA) method for BTDA. Specifically, we achieve distribution alignment by minimizing the sliced Wasserstein distance between the source and target feature distributions. To alleviate complex domain shifts among all sub-target domains in the hybrid feature space, we design graph networks to propagate and share semantic knowledge across domains, which reduces semantic discrepancies among multiple domains. Additionally, we propose a double consistency regularization method to reduce the susceptibility of the model to domain-specific information, further facilitating semantic alignment and alleviating domain shifts. Extensive experiments on several datasets show that CSCA achieves promising classification performance.

Multi-source domain adaptation (MSDA), which utilizes multiple source domains to align the distribution of a single target domain, is a popular and challenging setting in domain adaptation (DA). However, existing MSDA approaches are difficult to obtain sufficient target domain knowledge, which serve as the transfer object. Furthermore, the target distributions are confused in the real world, i.e., the model cannot obtain the domain labels of target domains. To tackle these problems, we consider a more realistic DA setting Multi-Source Blended-Target Domain Adaptation (MBDA) and propose an Invertible Projection and Conditional Alignment (IPCA) method. Specifically, to reduce the impact of the distribution discrepancy, we construct an invertible projection for the source and blended-target domains. Then, we adopt a projection consistency regularization to our model, which makes the model more robust on the domain-specific parts. In addition, because the labels of the blended-target domain are unseen, we introduce conditional discrepancy to obtain the domain-level discriminative information and guide the classifier to serve as the discriminator, which is suitable for MBDA settings. Extensive experiment results on the ImageCLEF-DA, Office-Home, and DomainNet datasets validate the effectiveness of our method.

Nighttime semantic segmentation is an indispensable component in practical applications, such as automated vehicles. However, it is often hindered by the lack of annotations due to interference caused by inadequate lighting or exposure. To overcome these difficulties, we propose a Dual-Path Consistency (DPC) unsupervised domain adaptation (UDA) approach. One path is Image Darkening Path (IDP), in which feature representations of original images and darkened images extracted from the feature encoder are leveraged to maintain cross-domain style consistency. Another path is Image Masking Path (IMP), in which the masked images are reconstructed under the guidance of pseudo-labels, aiming to maintain content consistency in an entirely identical scenario. Extensive experiments on four bench-marks demonstrate the superior performance of the proposed DPC for nighttime semantic segmentation.

Active domain adaptation (ADA) queries the labels of a limited number of selected target samples to help transfer knowledge from a source domain to a target domain. However, most ADA methods are still mainly oriented to a single target domain and not applicable for the blended target domain. To tackle these issues, we propose a concise and effective approach named Context-Guided Active Domain Adaptation (CGDA) to achieve active blended-target domain adaptation (BTDA). CGDA captures spatial context and local context to effectively use and understand context information to significantly improve the performance of active BTDA. First, we capture the spatial context relations of the target data through a masked image aware (MIA) module and then adapt the model to the inputs of a blended target domain through a blended feature augment (BFA) module. Furthermore, we utilize the local inconsistency of model predictions and uncertainty to design a selection criterion for selecting samples with more abundant local context information. Experiments on several BTDA datasets show that the performance of CGDA is significantly better than existing BTDA methods.

Large vision-language models (VLMs) like CLIP have exhibited strong image classification capabilities in addressing the unsupervised domain adaptation (UDA) problem. However, previous research has faced significant challenges, as UDA was primarily designed for single-target domain scenarios and does not effectively address more realistic situations. Therefore, we turn the perspective of VLMs to the more practical task, namely blended-target domain adaptation (BTDA). In this work, we propose a Multi-Prompt Coordination Learning (MPCL) approach that can effectively address the BTDA problem. In contrast to prior prompt learning works, we propose a multi-prompt learning module for BTDA, enabling VLMs to adapt more rapidly to downstream tasks. Furthermore, we propose a visual-textual alignment mechanism by maximizing the mutual information between global and local predictions. Finally, we improve the diversity of predictions by employing a regularization term that maximizes the batch nuclear norm. Extensive experiments demonstrate that MPCL significantly outperforms state-of-the-art methods.

Leveraging pre-trained Vision-Language Models (VLMs) for downstream tasks has gained significant attention recently, particularly in Multi-Source Domain Adaptation (MSDA). However, most existing VLMs-based MSDA approaches rely on domain-specific text prompts, which struggle to capture domain-invariant representations. In addition, multiple domains in MSDA introduce significant distribution discrepancies, complicating the design of effective text prompts. To address these challenges, we propose a Domain-aware Visual Context Prompt (DVCP) method, which leverages domain-level features to bridge the domain gaps. Specifically, we design domain-aware text prompts (DTP) module that maps global visual information into the textual prompt embedding space, creating trainable text prompts that incorporate domain-level visual information. Then, we construct a domain-aware visual tuning (DVT) module that collaboratively leverages domain-level and instance-level features to align distributions across multiple domains. Extensive experiments conducted on four popular MSDA benchmarks including Office31, ImageCLEF-DA, Office-Home, and DomainNet, demonstrate the superiority of the proposed method.

Black-box Unsupervised Domain Adaptation (BUDA) aims to transfer source domain knowledge to an unlabeled target domain, without accessing the source data or trained source model. Recent diffusion models have significantly advanced the ability to generate images from texts. While they can produce realistic visuals across diverse prompts and demonstrate impressive compositional generalization, these diffusion-based domain adaptation methods focus solely on composition, overlooking their sensitivity to textual nuances. In this work, we propose a novel diffusion-based method, called Rectifying-reasoning Errors of Diffusion (RrED) for BUDA. RrED is a two-stage learning strategy under diffusion supervision to effectively enhance the target model via the decomposed text and visual encoders from the diffusion model. Specifically, RrED consists of two stages: Diffusion-Target model Rectification (DTR) and Self-rectifying Reasoning Model (SRM). In DTR, we decouple the image and text encoders within the diffusion model: the visual encoder integrates our proposed feature-sensitive module to generate inferentially-enhanced visuals, while the text encoder enables multi-modal joint fine-tuning. In SRM, we prioritize the BUDA task itself, leveraging the target model's differential reasoning capability to rectify errors during learning. Extensive experiments confirm that RrED significantly outperforms other methods on four benchmark datasets, demonstrating its effectiveness in enhancing reasoning and generalization abilities.

Domain Adaptation of Black-box Predictors (DABP) transfers knowledge from a labeled source domain to an unlabeled target domain, without requiring access to either source data or source model. Common practices of DABP leverage reliable samples to suppress negative information about unreliable samples. However, there are still some problems: i) Excessive attention to reliable sample aggregation leads to premature overfitting; ii) Valuable information in unreliable samples is often overlooked. To address them, we propose a novel spatial learning approach, called Controlled Visual Hallucination via Thalamus-driven Decoupling Network (CVH-TDN). Specifically, CVH-TDN is the first work that introduces the thalamus-driven decoupling network in the visual task, relying on its connection with hallucination to control the direction of sample generation in feature space. CVH-TDN is composed of Hallucination Generation (HG), Hallucination Alignment (HA), and Hallucination Calibration (HC), aiming to explore the spatial relationship information between samples and hallucinations. Extensive experiments confirm that CVH-TDN achieves SOTA performance on four standard benchmarks.

Video anomaly detection (VAD) is of great importance for a variety of real-time applications in video surveillance. Most deep learning-based anomaly detection algorithms adopt a one-class learning scheme to train a classifier using only normal data to distinguish between normal and abnormal events during the test phase. However, these methods, whether they are reconstruction or prediction models, commonly face the challenge of the model’s overly strong representation capability, which leads to excessive fitting of abnormal events and thus limits the performance of the model in diverse scenarios. To address these challenges, this work develops a novel residual autoencoder and generative adversarial network-based 3-D convolutional network, called READ3D-net, for anomaly detection. An adaptive multimodal pseudoanomaly generator is developed to simulate and generate diverse pseudoanomalies, aiming to enhance the model’s ability to extract the features with regard to “abnormal” behaviors while reducing the interference of background on the detection performance. In addition, residual structures are incorporated into the design of a reconstruction-based autoencoder model to enhance its feature extraction and discriminative capabilities. To further improve the model’s reconstruction ability on normal data, a dynamic generative adversarial strategy is proposed for effective feature learning. Extensive experiments conducted on public benchmark datasets demonstrate that the proposed model is more competitive than the state-of-the-art methods in video anomaly detection tasks, fully validating the effectiveness and practicality of the proposed approach.

Unsupervised domain adaptation is mainly focused on the tasks of transferring knowledge from a fully-labeled source domain to an unlabeled target domain. However, in some scenarios, the labeled data are expensive to collect, which cause an insufficient label issue in the source domain. To tackle this issue, some works have focused on few-shot unsupervised domain adaptation (FUDA), which transfers predictive models to an unlabeled target domain through a source domain that only contains a few labeled samples. Yet the relationship between labeled and unlabeled source domains are not well exploited in generating pseudo-labels. Additionally, the few-shot setting further prevents the transfer tasks as an excessive domain gap is introduced between the source and target domains. To address these issues, we newly proposed an adaptive dispersal and collaborative clustering (ADCC) method for FUDA. Specifically, for the shortage of the labeled source data, a collaborative clustering algorithm is constructed that expands the labeled source data to obtain more distribution information. Furthermore, to alleviate the negative impact of domain-irrelevant information, we construct an adaptive dispersal strategy that introduces an intermediate domain and pushes both the source and target domains to this intermediate domain. Extensive experiments on the Office31, Office-Home, miniDomainNet, and VisDA-2017 datasets showcase the superior performance of ADCC compared to the state-of-the-art FUDA methods.

Multi-source domain adaptation (MSDA) has garnered significant attention due to its emphasis on transferring knowledge from multiple labeled source domains to a single unlabeled target domain. MSDA requires sufficient labeled data from multiple source domains, but in practice, massive unlabeled data exist instead of well-labeled data. Multiple target domains also provide plenty of information, which is useful for domain adaptation. However, most MSDA studies overlook the critical scenario of multi-source and multi-target domain adaptation (MMDA). To address these problems, we propose a Multiple Adaptation Network (MAN) approach for MMDA, which utilizes multiple alignment strategies for each source-target domain pair-group to align relevant specific feature spaces. MAN also aligns multiple classifiers for the relevant feature spaces to optimize the decision boundaries of multiple target domains. Moreover, to consider the task relations of multiple classifiers, we minimize the semantic differences between the target-conditioned classifiers and utilize a weight learning category to optimize this process. To fully utilize the information from multiple target domains, we transfer the style information of the target data to the source data, aiding in the training of multiple classifiers. Extensive experiments in challenge domain adaptation benchmarks, including the ImageCLEF-DA, Office-Home, DomainNet, and RGB-to-thermal datasets, demonstrate the superiority of our method over the state-of-the-art approaches.

The popularity of blended-target domain adaptation (BTDA) is growing since target data in the real world often come from multiple domains with different data distributions. Most BTDA studies adapt directly from the source domain to the target domains without considering which kinds of semantic information embedded in images should be explored. Therefore, some irrelevant semantic information is inevitably used, which leads to negative transfer. To address these issues, we propose a semantic dual-adversarial network (SDN) method for BTDA. Specifically, to suppress irrelevant semantic information, we adopt a min-max game strategy between the classifier and the feature extractor. The classifier tries to maximize the prediction distribution discrepancy, whereas the extractor endeavors to minimize this discrepancy. In this process, irrelevant semantic information is suppressed and the principal semantic information is emphasized. To align the categorical distributions, we train a category-aware domain discriminator and a feature extractor with category labels. In addition, we introduce a random ratio-based feature fusion scheme to augment the source domain, which can decrease domain gaps. At last, we propose a weighted negative self-supervised learning method to enhance the model's generalization. Extensive experiments on multiple benchmarks showcase that our method significantly outperforms the prior state-of-the-art methods in BTDA.

Heterogeneous domain adaptation seeks to learn an effective classifier or regression model for unlabeled target samples by using the well-labeled source samples but residing in different feature spaces and lying different distributions. Most recent works have concentrated on learning domain-invariant feature representations to minimize the distribution divergence via target pseudo-labels. However, two critical issues need to be further explored: 1) new feature representations should be not only domain-invariant but also category-correlative and discriminative and 2) alleviating the negative transfer caused by the incorrect pseudo-labeling target samples could boost the adaptation performance during the iterative learning process. To address these issues, in this paper, we put forward a novel heterogeneous domain adaptation method to learn category-correlative and discriminative representations, referred to as correlative and discriminative feature learning (CDFL). Specifically, CDFL aims to learn a feature space where class-specific feature correlations between the source and target domains are maximized, the divergences of marginal and conditional distribution between the source and target domains are minimized, and the distances of inter-class distribution are forced to be maximized to ensure the discriminative ability. Meanwhile, a selective pseudo-labeling procedure based on the correlation coefficient and classifier prediction is introduced to boost class-specific feature correlation and discriminative distribution alignment in an iteration way. Extensive experiments certify that CDFL outperforms the State-of-the-Art algorithms on five standard benchmarks.

Semisupervised domain adaptation (SSDA) utilizes a few labeled target samples to learn the pseudolabels for the test target samples in classification tasks. However, the learned pseudolabels inevitably produce mistakes, and utilizing these incorrect pseudolabels in each training step causes negative effects to accumulate, degrading the performance of the models. To address these issues, in this paper, we propose a novel approach, pseudolabel distillation with adversarial contrastive learning (PDACL), for SSDA. Specifically, we utilize image alignment technique to learn uncertainty knowledge between global and local features and knowledge distillation to explore more useful knowledge to ensure the quality of the pseudolabels. A margin cosine loss is introduced to align different domains. We design an image joint discriminator and utilize multifilter techniques to retain the accuracy of the pseudolabels. Extensive experiments were conducted on several data benchmarks, and the results demonstrate that our proposed approach achieves advanced performance in SSDA.

Blended-target domain adaptation (BTDA), which implicitly mixes multiple sub-target domains into a fine domain, has attracted more attention in recent years. Most previously developed BTDA approaches focus on utilizing a single source domain, which makes it difficult to obtain sufficient feature information for learning domain-invariant representations. Furthermore, different feature distributions derived from different domains may increase the uncertainty of models. To overcome these issues, we propose a style adaptation and uncertainty estimation (SAUE) approach for multi-source blended-target domain adaptation (MBDA). Specifically, we exploit the extra knowledge acquired from the blended-target domain, where a similarity factor is adopted to select more useful target style information for augmenting the source features. \!Then, to mitigate the negative impact of the domain-specific attributes, we devise a function to estimate and mitigate uncertainty in category prediction. Finally, we construct a simple and lightweight adversarial learning strategy for MBDA, effectively aligning multi-source and blended-target domains without the requirements of domain labels of the target domains. Extensive experiments conducted on several challenging DA benchmarks, including the ImageCLEF-DA, Office-Home, VisDA 2017, and DomainNet datasets, demonstrate the superiority of our method over the state-of-the-art (SOTA) approaches.

As a main branch of domain adaptation (DA), multi-source DA (MSDA) has attracted increasing attention for exploiting information from multi-source domain data. However, how to effectively explore useful information from each source domain for target tasks is still a key problem. In this paper, to fully explore multiple information of different domain data, we propose a graph correlated discriminant embedding (GCDE) method for MSDA. In GCDE, the category-discriminative information, manifold structure, and correlation learning are fully considered. Specifically, GCDE encodes the within- and between- class information of each domain data, preserves the local and global structure information of the data, and extracts the maximization correlative features from different domains by designing a novel correlative learning scheme. We also extend GCDE to a nonlinear case and obtain kernel GCDE (KGCDE). We have conducted extensive experiments on four public data benchmarks to verify the performance of GCDE and KGCDE. The promising performance on the databases prove the efficiency of our methods with the comparison of the advanced approaches.

As a branch of domain adaptation (DA), multi-source DA (MSDA) is a challenging issue that aims to transfer knowledge from multiple well-labeled source domains to a target domain for target tasks. However, most existing related works focus on single-target domain adaptation, and multiple target domain adaptation is not accounted for. We believe that multiple target domains provide valuable knowledge. Meanwhile, in multi-source and multi-target adaptation scenarios, feature generators with static parameters have difficulty generating deep features of each individual domain. In this paper, we propose a Dynamic Generator With Attention (DGWA) method for multi-source and multi-target domain adaptation to adapt domain-agnostic deep features in a multi-source and multi-target domain scenario. The feature generator with dynamic parameters can dynamically change its parameters with data input from different domains, which greatly improves the generalization of the feature pools. An attention mechanism is used in our DGWA to learn more transferable information from different domains. To demonstrate the performance of DGWA, we conduct extensive experiments on several popular domain adaptation datasets, including the digits, Office+Caltech10, Office-Home, and ImageCLEF-DA datasets. The experimental results demonstrate that our method performs better than state-of-the-art methods.

In unsupervised domain adaptation (UDA), negative transfer is one of the most challenging problems. Due to complex environments, the used domain data are always corrupted by noise or outliers in many applications. If the noisy data are directly used for domain adaptation, the disturbances and negative influence of the noise are also shifted for the target tasks. Thus, preventing disturbances and negative effects caused by noise are key problems in UDA that need to be addressed. In this article, a low-rank correlation learning (LRCL) method is proposed for UDA. In LRCL, the noisy domain data are recovered by low-rank learning; then both domain data are cleaned. Hence, the disturbances and negative effects of the noise are prevented. The maximized correlated features of the clean data from the source and target domains are learned by a novel correlation regularization term in a latent common space. LRCL also reduces the distribution difference of the learned clean source and target data by constructing a reconstruction term, in which the clean target data are linearly represented by the clean source data. To explore the temporal and structural information of the data, we further extend LRCL into a graph case and propose graph LRCL (GLRCL). Extensive experiments have been conducted on several public data benchmarks, and the experimental results demonstrate that our methods can effectively prevent negative transfer and obtain better classification outcomes than other compared approaches.

In this paper, we present a novel approach named correlation-guided distribution and geometry alignments (CDGA) for heterogeneous domain adaptation. Unlike existing methods that typically combine feature alignment and domain alignment into a single objective function, our proposed CDGA separates the two alignments into distinct steps. The two adaptation steps are: paired canonical correlation analysis (PCCA) and distribution and geometry alignments (DGA). In the PCCA step, CDGA focuses on maximizing the within-category correlation between source and target samples to produce the dimension-aligned feature representations for the next adaptation step. In the DGA step, CDGA is responsible for learning a classifier that incorporates both distribution and geometry alignments. Furthermore, during this step, the highly confident pseudo labeled samples are carefully selected for the next iteration of PCCA, establishing a beneficial coupling between PCCA and DGA to improve the adaptation performance in an iterative manner. Experimental results on various visual cross-domain benchmarks demonstrate that CDGA achieves remarkable performance compared to the existing shallow heterogeneous domain adaptation methods and even exhibits superiority over the state-of-the-art neural network-based approaches.

Multi-center analysis has attracted increasing attention in brain disease diagnosis, because it provides effective approaches to improve disease diagnostic performance by making use of the information from different centers. However, in practical multi-center applications, data uncertainty is more common than that in single center, which brings challenge to robust modeling of diagnosis. In this article, we proposed a multi-discriminator active adversarial network (MDAAN) to alleviate the uncertainties at the center, feature, and label levels for multi-center brain disease diagnosis. First, we extract the latent invariant representation of the source center and target center to reduce domain shift by adversarial learning strategy. Second, the proposed method adaptively evaluates the contribution of different source centers in fusion by measuring data distribution difference between source and target center. Moreover, only the hard learning samples in target center are identified to label with low sample annotation cost. Finally, we treat the selected samples as the auxiliary domain to alleviate the negative transfer and improve the robustness of the multi-center model. We extensively compare the proposed approach with several state-of-the-art multi-center methods on the five-center schizophrenia dataset, and the results demonstrate that our method is superior to the previous methods in identifying brain disease.

As a branch of transfer learning, domain adaptation leverages useful knowledge from a source domain to a target domain for solving target tasks. Most of the existing domain adaptation methods focus on how to diminish the conditional distribution shift and learn invariant features between different domains. However, two important factors are overlooked by most existing methods: 1) the transferred features should be not only domain invariant but also discriminative and correlated, and 2) negative transfer should be avoided as much as possible for the target tasks. To fully consider these factors in domain adaptation, we propose a guided discrimination and correlation subspace learning (GDCSL) method for cross-domain image classification. GDCSL considers the domain-invariant, category-discriminative, and correlation learning of data. Specifically, GDCSL introduces the discriminative information associated with the source and target data by minimizing the intraclass scatter and maximizing the interclass distance. By designing a new correlation term, GDCSL extracts the most correlated features from the source and target domains for image classification. The global structure of the data can be preserved in GDCSL because the target samples are represented by the source samples. To avoid negative transfer issues, we use a sample reweighting method to detect target samples with different confidence levels. A semi-supervised extension of GDCSL (Semi-GDCSL) is also proposed, and a novel label selection scheme is introduced to ensure the correction of the target pseudo-labels. Comprehensive and extensive experiments are conducted on several cross-domain data benchmarks. The experimental results verify the effectiveness of the proposed methods over state-of-the-art domain adaptation methods.

As a multivariate data analysis tool, canonical correlation analysis (CCA) has been widely used in computer vision and pattern recognition. However, CCA uses Euclidean distance as a metric, which is sensitive to noise or outliers in the data. Furthermore, CCA demands that the two training sets must have the same number of training samples, which limits the performance of CCA-based methods. To overcome these limitations of CCA, two novel canonical correlation learning methods based on low-rank learning are proposed in this paper for image representation, named robust canonical correlation analysis (robust-CCA) and low-rank representation canonical correlation analysis (LRR-CCA). By introducing two regular matrices, the training sample numbers of the two training datasets can be set as any values without any limitation in the two proposed methods. Specifically, robust-CCA uses low-rank learning to remove the noise in the data and extracts the maximization correlation features from the two learned clean data matrices. The nuclear norm andL1-norm are used as constraints for the learned clean matrices and noise matrices, respectively. LRR-CCA introduces low-rank representation into CCA to ensure that the correlative features can be obtained in low-rank representation. To verify the performance of the proposed methods, five publicly image databases are used to conduct extensive experiments. The experimental results demonstrate the proposed methods outperform state-of-the-art CCA-based and low-rank learning methods.

Domain adaptation leverages rich knowledge from a related source domain so that it can be used to perform tasks in a target domain. For more knowledge to be obtained under relaxed conditions, domain adaptation methods have been widely used in pattern recognition and image classification. However, most of the existing domain adaptation methods only consider how to minimize different distributions of the source and target domains, which neglects what should be transferred for a specific task and suffers negative transfer by distribution outliers. To address these problems, in this paper, we propose a novel domain adaptation method called weighted correlation embedding learning (WCEL) for image classification. In the WCEL approach, we seamlessly integrated correlation learning, graph embedding, and sample reweighting into a unified learning model. Specifically, we extracted the maximum correlated features from the source and target domains for image classification tasks. In addition, two graphs were designed to preserve the discriminant information from interclass samples and neighborhood relations in intraclass samples. Furthermore, to prevent the negative transfer problem, we developed an efficient sample reweighting strategy to predict the target with different confidence levels. To verify the performance of the proposed method in image classification, extensive experiments were conducted with several benchmark databases, verifying the superiority of the WCEL method over other state-of-the-art domain adaptation algorithms.

As one of the most prevalent branches of transfer learning, domain adaptation is dedicated to generalizing the knowledge of a source domain to a target domain to perform machine learning tasks. In domain adaptation, the key strategy is to overcome the shift between different domains and learn shared features with domain invariance. However, most existing methods focus on extracting the common features of the source and target domains, and do not consider the shift problem of class center in the target domain caused by this process. Specifically, when we align the domain distributions, we often ignore the inherent feature attributes of the data, or under the guidance of false pseudo-labels, cause the target domain data to be far away from the class center after projection. This is not conducive to classification task. To address these problems, in this study, we propose a novel domain adaptation method, referred to as discriminative invariant alignment (DIA), for image representation. DIA enriches the knowledge matrix by combining the class discriminative information of the source domain and local data structure information of the target domain into a new framework. By introducing the maximum margin criterion of the source domain, the classification boundaries are expanded. To verify the performance of the proposed method, we compared DIA with several state-of-the-art methods on five benchmark databases. The experimental results show that DIA is superior to the state-of-the-art methods.

Unsupervised domain adaptation (UDA) aims to transfer knowledge between different domains. Most of the existing UDA methods try to align the conditional distribution between the source and target domains by utilizing the information of pseudo labels induced from the target domain. To tackle the negative transfer caused by inaccurate pseudo labels, we propose a novel UDA method named progressive distribution alignment based on label correction (PDALC). Specifically, PDALC uses the class discriminative information to perform subspace learning to obtain the domain invariance subspace. Furthermore, a new mechanism of pseudo label correction is introduced to measure the reliability of pseudo labels and utmostly correct the inaccurate pseudo labels. By combining subspace learning with label correction, the performance of PDALC can be continuously improved, which in turn reduces the generation of inaccurate pseudo labels. The experimental results show that the proposed method outperforms the state-of-the-art UDA methods.

In transfer learning, how to effectively transfer useful information from the source domain to the target domain is crucial. In this paper, we propose a novel transfer learning method for image classification, named manifold transfer learning via discriminant regression analysis (MTL-DRA), to transfer the local geometry structure information from the source domain to the target domain and ensure that the transform matrix is robust or sparse so that samples from different domains can be well combined. In MTL-DRA, we encode discriminant information of the source domain to the target domain by introducing between- and within-class graphs to preserve within-class similarity and reduce between-class similarity. With different norms as constraints, MTL-DRA overcomes the disturbance of noise and avoids negative transfer learning. To improve the robustness of MTL-DRA, we encode a nuclear norm constraint and propose robust MTL-DRA (RMTL-DRA). We analyzed the convergence and complexity of the two proposed methods. To verify the performance of the proposed methods, we conducted extensive experiments on five public image benchmarks. The experimental results show that the proposed methods outperform state-of-the-art transfer learning methods.

Neighborhood preserving embedding (NPE) has been proposed to encode overall geometry manifold embedding information. However, the class-special structure of the data is destroyed by noise or outliers existing in the data. To address this problem, in this article, we propose a novel embedding approach called robust flexible preserving embedding (RFPE). First, RFPE recovers the noisy data by low-rank learning and obtains clean data. Then, the clean data are used to learn the projection matrix. In this way, the projective learning is totally unaffected by noise or outliers. By encoding a flexible regularization term, RFPE can keep the property of the data points with a nonlinear manifold and be more flexible. RFPE searches the optimal projective subspace for feature extraction. In addition, we also extend the proposed RFPE to a kernel case and propose kernel RFPE (KRFPE). Extensive experiments on six public image databases show the superiority of the proposed methods over other state-of-the-art methods.

Preserving projection-based methods are good for finding the manifold structure embedded in data. As they use the Euclidean distance as a metric, which is sensitive to noise and outliers in data, nuclear norm-based 2D locality preserving projection (NN-2DLPP) is thus proposed to improve the robustness of 2DLPP. However, NN-2DLPP does not consider the discriminant ability of data. In order to improve the discriminant ability of preserving projection methods, in this paper, we use preserving projection learning with structurally incoherence of data and propose structurally incoherent low-rank 2DLPP (SILR-2DLPP) for image classification. This approach provides a discriminative representation of preserving projection learning by recovering the distinct different classes of the data. SILR-2DLPP searches the optimal subspace and low-rank representation simultaneously. We further extend SILR-2DLPP to a kernel case and propose kernel SILR-2DLPP (KSILR-2DLPP) to obtain a nonlinear representation. The theoretical analysis including the convergence and computational complexity of SILR-2DLPP are presented. To verify the performance of SILR-2DLPP and KSILR-2DLPP, six well-known image databases were used in the experiments. The experimental results show that the proposed methods are superior to the previous preserving projection methods for image classification.

2-D linear discriminant analysis (2DLDA) has been widely used in pattern recognition and image classification. 2DLDA selects discriminative features from the up and left corner of images. However, 2DLDA uses the Frobenius norm (F-norm), which is sensitive to noise or outliers in data, as a metric. In this paper, we propose a novel framework, called horizontal and vertical nuclear norm-based 2DLDA (HVNN-2DLDA) for image representation. In the proposed framework, HVNN-2DLDA methods (i.e., HNN-2DLDA and VNN-2DLDA) are proposed, and both use the nuclear norm as a criterion. The nuclear norm can provide more structure and global information for the reconstruction of noisy images. HNN-2DLDA and VNN-2DLDA represent images in the row and column directions, respectively. In addition, by combining the row and column directions, we propose a bilateral nuclear norm-based 2DLDA method called BNN-2DLDA. The advantage of BNN-2DLDA over HNN-2DLDA and VNN-2DLDA is that an image sample can be represented by both the row and the column directions instead of only the row or column direction. HVNN-2DLDA learns a set of local optimal projection vectors by maximizing the ratio of the nuclear norm of the between-class scatter matrix and the nuclear norm of the within-class scatter matrix. To verify the robustness and recognition performance in image classification of HVNN-2DLDA, six public image databases are used for experiments. The experimental results demonstrate the effectiveness and the feasibility of the proposed framework.

2-D neighborhood preserving projection (2DNPP) uses 2-D images as feature input instead of 1-D vectors used by neighborhood preserving projection (NPP). 2DNPP requires less computation time than NPP. However, both NPP and 2DNPP use the L2norm as a metric, which is sensitive to noise in data. In this paper, we proposed a novel NPP method called low-rank 2DNPP (LR-2DNPP). This method divided the input data into a component part that encoded low-rank features, and an error part that ensured the noise was sparse. Then, a nearest neighbor graph was learned from the clean data using the same procedure as 2DNPP. To ensure that the features learned by LR-2DNPP were optimal for classification, we combined the structurally incoherent learning and low-rank learning with NPP to form a unified model called discriminative LR-2DNPP (DLR2DNPP). By encoding the structural incoherence of the learned clean data, DLR-2DNPP could enhance the discriminative ability for feature extraction. Theoretical analyses on the convergence and computational complexity of LR-2DNPP and DLR-2DNPP were presented in details. We used seven public image databases to verify the performance of the proposed methods. The experimental results showed the effectiveness of our methods for robust image representation.

Nonnegative matrix factorization (NMF), known as a famous matrix factorization technique, has been widely used in pattern recognition and computer vision. NMF represents the input data matrix as a product of two nonnegative factors. As NMF is based on the Euclidean distance, which is sensitive to noise or errors in the data, some robust NMF methods are proposed. Mainly focusing on parts-based representation, these robust NMF methods often neglect global representation of data. In fact, the global geometry information of data is more robust than the local information about the noisy data in terms of image classification. In order to effectively improve the robustness of NMF and learn part-based and global representation of the data, a novel method low-rank nonnegative factorization (LRNF) is proposed in this paper. First, we assume that the data are grossly corrupted, and the$L_{1} $norm is used as a sparse constraint on the assumed noise matrix. Then, LRNF learns a low-rank matrix with the global representation ability. Finally, we make a nonnegative factorization of the learned low-rank matrix. We can obtain a base matrix, which preserves locality and globality properties of the data in the meantime. Extensive experiments have been conducted on nine real-world image databases to verify the performance of the proposed LRNF method by comparing with the state-of-the-art algorithms on robust dimensionality reduction.

As a popular dimensionality reduction method, nonnegative matrix factorization (NMF) has been widely used in image classification. However, the NMF does not consider discriminant information from the data themselves. In addition, most NMF-based methods use the Euclidean distance as a metric, which is sensitive to noise or outliers in data. To solve these problems, in this paper, we introduce structural incoherence and low-rank to NMF and propose a novel nonnegative factorization method, called structurally incoherent low-rank NMF (SILR-NMF), in which we jointly consider structural incoherence and low-rank properties of data for image classification. For the corrupted data, we use the norm as a constraint to ensure the noise is sparse. SILR-NMF learns a clean data matrix from the noisy data by low-rank learning. As a result, the SILR-NMF can capture the global structure information of the data, which is more robust than local information to noise. By introducing the structural incoherence of the learned clean data, SILR-NMF ensures the clean data points from different classes are as independent as possible. To verify the performance of the proposed method, extensive experiments are conducted on six image databases. The experimental results demonstrate that our proposed method has substantial gain over existing NMF approaches.

Nonnegative matrix factorization (NMF), which aims at obtaining the nonnegative low-dimensional representation of data, has received wide attention. To obtain more effective nonnegative discriminant bases from the original NMF, in this paper, a novel method called nonnegative discriminant matrix factorization (NDMF) is proposed for image classification. NDMF integrates the nonnegative constraint, orthogonality, and discriminant information in the objective function. NDMF considers the incoherent information of both factors in standard NMF and is proposed to enhance the discriminant ability of the learned base matrix. NDMF projects the low-dimensional representation of the subspace of the base matrix to regularize the NMF for discriminant subspace learning. Based on the Euclidean distance metric and the generalized Kullback-Leibler (KL) divergence, two kinds of iterative algorithms are presented to solve the optimization problem. The between- and within-class scatter matrices are divided into positive and negative parts for the update rules and the proofs of the convergence are also presented. Extensive experimental results demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art discriminant NMF algorithms.

Robustness to noises, outliers, and corruptions is an important issue in linear dimensionality reduction. Since the sample-specific corruptions and outliers exist, the class-special structure or the local geometric structure is destroyed, and thus, many existing methods, including the popular manifold learning- based linear dimensionality methods, fail to achieve good performance in recognition tasks. In this paper, we focus on the unsupervised robust linear dimensionality reduction on corrupted data by introducing the robust low-rank representation (LRR). Thus, a robust linear dimensionality reduction technique termed low-rank embedding (LRE) is proposed in this paper, which provides a robust image representation to uncover the potential relationship among the images to reduce the negative influence from the occlusion and corruption so as to enhance the algorithm's robustness in image feature extraction. LRE searches the optimal LRR and optimal subspace simultaneously. The model of LRE can be solved by alternatively iterating the argument Lagrangian multiplier method and the eigendecomposition. The theoretical analysis, including convergence analysis and computational complexity, of the algorithms is presented. Experiments on some well-known databases with different corruptions show that LRE is superior to the previous methods of feature extraction, and therefore, it indicates the robustness of the proposed method. The code of this paper can be downloaded from http://www.scholat.com/laizhihui.

Two-dimensional locality preserving projections (2DLPP) that use 2D image representation in preserving projection learning can preserve the intrinsic manifold structure and local information of data. However, 2DLPP is based on the Euclidean distance, which is sensitive to noise and outliers in data. In this paper, we propose a novel locality preserving projection method called nuclear norm-based two-dimensional locality preserving projections (NN-2DLPP). First, NN-2DLPP recovers the noisy data matrix through low-rank learning. Second, noise in data is removed and the learned clean data points are projected on a new subspace. Without the disturbance of noise, data points belonging to the same class are kept as close to each other as possible in the new projective subspace. Experimental results on six public image databases with face recognition, object classification, and handwritten digit recognition tasks demonstrated the effectiveness of the proposed method.

As one of the most popular dimensionality reduction techniques, locality preserving projections (LPP) has been widely used in computer vision and pattern recognition. However, in practical applications, data is always corrupted by noises. For the corrupted data, samples from the same class may not be distributed in the nearest area, thus LPP may lose its effectiveness. In this paper, it is assumed that data is grossly corrupted and the noise matrix is sparse. Based on these assumptions, we propose a novel dimensionality reduction method, named low-rank preserving projections (LRPP) for image classification. LRPP learns a low-rank weight matrix by projecting the data on a low-dimensional subspace. We use the L21 norm as a sparse constraint on the noise matrix and the nuclear norm as a low-rank constraint on the weight matrix. LRPP keeps the global structure of the data during the dimensionality reduction procedure and the learned low rank weight matrix can reduce the disturbance of noises in the data. LRPP can learn a robust subspace from the corrupted data. To verify the performance of LRPP in image dimensionality reduction and classification, we compare LRPP with the state-of-the-art dimensionality reduction methods. The experimental results show the effectiveness and the feasibility of the proposed method with encouraging results.

Sparse representation for classification (SRC) has attracted much attention in recent years. It usually performs well under the following assumptions. The first assumption is that each class has sufficient training samples. In other words, SRC is not good at dealing with the undersampled problem, i.e., each class has few training samples, even single sample. The second one is that the sample vectors belonging to different classes should not distribute on the same vector direction. However, the above two assumptions are not always satisfied in real-world problems. In this paper, we propose a novel SRC based algorithm, i.e., kernel sparse representation based classifier for undersampled problem (KSRC-UP) to perform classification. It does not need the above assumptions in principle. KSRC-UP can deal well with the small scale and high dimensional real world data sets. Experiments on the popular face databases show that our KSRC-UP method can perform better than other SRC methods.

In different color spaces, the three color channels might have different relationship, but most of color face recognition methods exploit the color information in a simple way. In this paper, we propose a novel hybrid fusion scheme for color face recognition, which first uses two-phase test sample representation (TPTSR) to obtain matching scores of each color channel of the test sample and then uses the hybrid fusion scheme to combine these three kinds of matching scores for classification of the test sample. The hybrid fusion scheme exploits low- and high-order components of three kinds of matching scores based on the sum and product rule. Scores from each color channel generated from TPTSR includes both little correlated and very correlated scores, to extract low- and high-order components of these scores will allow them to be well integrated and used for classification. For evaluating the proposed method, we not only make a comparison of our method with some global and local methods such as principal component analysis (PCA), linear discriminant analysis (LDA), kernel PCA (KPCA), kernel LDA (KLDA), locality preserving projection (LPP) and TPTSR. We also make a comparison of our method with some recently proposed local feature based methods, such as color local Gabor wavelets (CLGW), color local binary pattern (CLBP) and tensor discriminant color space (TDCS).