Medical vision-language pre-training (VLP) offers significant potential for advancing medical image understanding by leveraging paired image-report data. However, existing methods are limited by False Negatives (FaNe) induced by semantically similar texts and insufficient fine-grained cross-modal alignment. To address these limitations, we propose FaNe, a semantic-enhanced VLP framework. To mitigate false negatives, we introduce a semantic-aware positive pair mining strategy based on text-text similarity with adaptive normalization. Furthermore, we design a text-conditioned sparse attention pooling module to enable fine-grained image-text alignment through localized visual representations guided by textual cues. To strengthen intra-modal discrimination, we develop a hard-negative aware contrastive loss that adaptively reweights semantically similar negatives. Extensive experiments on five downstream medical imaging benchmarks demonstrate that FaNe achieves state-of-the-art performance across image classification, object detection, and semantic segmentation, validating the effectiveness of our framework.

Spectral clustering is a prevalent clustering method in which an affinity matrix is constructed based on all samples (the number is n ), leading to high computational complexity and making it infeasible for dealing with large-scale data directly. In this study, we introduce an Approximate Spectral Embedding Representation method (ASER). By employing an anchor-based strategy, the spectral embedding representation of the selected anchors (the number is m , m ≪ n ) is used to approximate the spectral embedding representation of the original samples. Unlike available methods that approximate the similarity matrix based on an anchor graph, we directly implement the approximation in the spectral embedding space. Moreover, the properties of the formed anchor graph are inherited from the original space to the spectral embedding space. The time complexity of conducting spectral clustering is significantly reduced from O ( n 3 ) to be linear with respect to n , without relying on any acceleration operations for eigenvalue decomposition. Experimental results on toy examples and benchmark datasets with large sizes demonstrate the effectiveness and efficiency of the proposed model.

As a specialized paradigm of domain adaptation, blended-target domain adaptation (BTDA) transfers knowledge from a source domain to a blended target domain. In this paper, we propose an Energy-Driven Explicit Alignment Network (EDEAN) framework that innovatively applies energy-based models (EBMs) to address BTDA problems. We observe that EBMs display free energy biases when the source domain and the target domain data originate from different distributions. Therefore, we use these biases as a measure of the discrepancies between the source domain and the target domain and align them by minimizing these biases via the free energy alignment (FEA) module. We further propose the balanced weight distribution (BWD) module, which comprehensively considers the complementary information between the linear and semantic pseudo-labels and obtains the corresponding complementary information by mixing both label types. Moreover, we propose the normalized free energy (NFE) module, which assigns higher weights to high free energy samples and dynamically corrects the pseudo-labels by continuously updating these weights. We also conducted experiments on four widely used BTDA databases and achieved substantial improvements over the latest BTDA methods.

Low-light image enhancement (LLIE) aims to improve illumination while preserving high-quality color and texture. However, existing methods often fail to extract reliable feature representations due to severely degraded pixel-level information under low-light conditions, resulting in poor texture restoration, color inconsistency, and artifact. To address these challenges, we propose LightQANet, a novel framework that introduces quantized and adaptive feature learning for low-light enhancement, aiming to achieve consistent and robust image quality across diverse lighting conditions. From the static modeling perspective, we design a Light Quantization Module (LQM) to explicitly extract and quantify illumination-related factors from image features. By enforcing structured light factor learning, LQM enhances the extraction of light-invariant representations and mitigates feature inconsistency across varying illumination levels. From the dynamic adaptation perspective, we introduce a Light-Aware Prompt Module (LAPM), which encodes illumination priors into learnable prompts to dynamically guide the feature learning process. LAPM enables the model to flexibly adapt to complex and continuously changing lighting conditions, further improving image enhancement. Extensive experiments on multiple low-light datasets demonstrate that our method achieves state-of-the-art performance, delivering superior qualitative and quantitative results across various challenging lighting scenarios.

Diabetic retinopathy (DR), with its large patient population, has become a formidable threat to human visual health. In the clinical diagnosis of DR, multi-view fundus images are considered to be more suitable for DR diagnosis because of the wide coverage of the field of view. Therefore, different from most of the previous single-view DR grading methods, we design a dynamic selection-driven multi-view DR grading method to fit clinical scenarios better. Since lesion information plays a key role in DR diagnosis, previous methods usually boost the model performance by enhancing the lesion feature. However, during the actual diagnosis, ophthalmologists not only focus on the crucial parts, but also exclude irrelevant features to ensure the accuracy of judgment. To this end, we introduce the idea of dynamic selection and design a series of selection mechanisms from fine granularity to coarse granularity. In this work, we first introduce an Ophthalmic Image Reader (OIR) agent to provide the model with pixel-level prompts of suspected lesion areas. Moreover, a Multi-View Token Selection Module (MVTSM) is designed to prune redundant feature tokens and realize dynamic selection of key information. In the final decision stage, we dynamically fuse multi-view features through the novel Multi-View Mixture of Experts Module (MVMoEM), to enhance key views and reduce the impact of conflicting views. Extensive experiments on a large multi-view fundus image dataset with 34,452 images demonstrate that our method performs favorably against state-of-the-art models.

Confidence-based pseudo-label selection usually generates overly confident yet incorrect predictions, due to the early misleadingness of model and overfitting inaccurate pseudo-labels in the learning process, which heavily degrades the performance of semi-supervised contrastive learning. Moreover, segmentation and classification tasks are treated independently and the affinity fails to be fully explored. To address these issues, we propose a novel semi-supervised dual-threshold contrastive learning strategy for ultrasound image classification and segmentation, named Hermes. This strategy combines the strengths of contrastive learning with semi-supervised learning, where the pseudo-labels assist contrastive learning by providing additional guidance. Specifically, an inter-task attention and saliency module is also developed to facilitate information sharing between the segmentation and classification tasks. Furthermore, an inter-task consistency learning strategy is designed to align tumor features across both tasks, avoiding negative transfer for reducing features discrepancy. To solve the lack of publicly available ultrasound datasets, we have collected the SZ-TUS dataset, a thyroid ultrasound image dataset. Extensive experiments on two public ultrasound datasets and one private dataset demonstrate that Hermes consistently outperforms several state-of-the-art methods across various semi-supervised settings. The code is available at https://github.com/Aventador8/Hermes.

In order to obtain good medical image segmentation results, existing studies usually extract multi-scale features or design special attention to obtain global and local information of images. However, the above methods are very cumbersome or have a high computational burden. Theoretically, global and local contexts are used to locate objects and refine their contours respectively. Therefore, from a novel points prediction perspective, this work adopts a convolutional model and employs the deep supervision method to assist segmentation by predicting points of the lesion location and boundary. Specifically, we use two points prediction branches to generate location and boundary heatmaps, which implicitly locate lesions and refine contours. Furthermore, in order to train the multi-task model effectively, this work proposes a simple supervision signals design principle to guide deep supervision, so that the training of the auxiliary branch does not conflict with the main task branch. It is worth noting that our simple and end-to-end approach achieves the state-of-the-art results without the need for post-processing. Extensive experiments on three challenging medical image segmentation benchmarks demonstrate the superior performance.

Kernel methods are powerful tools for nonlinear learning with well-established theory. The scalability issue has been their long-standing challenge. Despite the existing success, there are two limitations in large-scale kernel methods: (i) The memory overhead is too high for users to afford; (ii) existing efforts mainly focus on kernel ridge regression (KRR), while other models lack study. In this paper, we propose Joker, a joint optimization framework for diverse kernel models, including KRR, logistic regression, and support vector machines. We design a dual block coordinate descent method with trust region (DBCD-TR) and adopt kernel approximation with randomized features, leading to low memory costs and high efficiency in large-scale learning. Experiments show that Joker saves up to 90% memory but achieves comparable training time and performance (or even better) than the state-of-the-art methods.

Low-light image enhancement (LLIE) aims to improve low-illumination images. However, existing methods face two challenges: (1) uncertainty in restoration from diverse brightness degradations; (2) loss of texture and color information caused by noise suppression and light enhancement. In this paper, we propose a novel enhancement approach, CodeEnhance, by leveraging discrete codebook priors and image refinement to address these challenges. In particular, we reframe LLIE as learning an image-to-code mapping from low-light images to discrete codebook, which has been learned from high-quality images. To enhance this process, a Semantic Embedding Module (SEM) is introduced to integrate semantic information with low-level features, and a Codebook Shift (CS) mechanism, designed to adapt the pre-learned codebook to better suit the distinct characteristics of our low-light dataset. Additionally, we present an Interactive Feature Transformation (IFT) module to refine texture and color information during image reconstruction, allowing for interactive enhancement based on user preferences. Extensive experiments on both real-world and synthetic benchmarks demonstrate that the incorporation of prior knowledge and controllable information transfer significantly enhances LLIE performance in terms of quality and fidelity. The proposed CodeEnhance exhibits superior robustness to various degradations, including uneven illumination, noise, and color distortion. The code can be obtained from https://github.com/csxuwu/CodeEnhance or https://www.scholat.com/laizhihui.cn .

Abstract With the advantage of exploiting complementary and consensus information across multiple views, techniques for Multi‐view Clustering have attracted increasing attention in recent years. However, it is common that data on some views is not completed in real‐world applications, which brings the challenge of partial mapping between the views. To explore the information hidden in the local geometric structure and recover missing instances through mining the information hidden in existing instances, a self‐inferring incomplete multi‐view clustering algorithm is proposed. Firstly, the incomplete multi‐view data is replenished directly and exploited as variables for inferring the missing instances. And then, a feature graph constraint is united in consensus learning. Besides, a similarity graph learning method is imposed to preserve the local manifold structure. At last, the inferred instances are filled in the missing instances for learning better consensus representation in the iterative process. Extensive experiment results show that this method can improve the clustering performance compared with the state‐of‐the‐art methods.

Subspace learning and Support Vector Machine (SVM) are two critical techniques in pattern recognition, playing pivotal roles in feature extraction and classification. However, how to learn the optimal subspace such that the SVM classifier can perform the best is still a challenging problem due to the difficulty in optimization, computation, and algorithm convergence. To address these problems, this paper develops a novel method named Optimal Discriminant Support Vector Machine (ODSVM), which integrates support vector classification with discriminative subspace learning in a seamless framework. As a result, the most discriminative subspace and the corresponding optimal SVM are obtained simultaneously to pursue the best classification performance. The efficient optimization framework is designed for binary and multi-class ODSVM. Moreover, a fast sequential minimization optimization (SMO) algorithm with pruning is proposed to accelerate the computation in multi-class ODSVM. Unlike other related methods, ODSVM has a strong theoretical guarantee of global convergence, highlighting its superiority and stability. Numerical experiments are conducted on thirteen datasets and the results demonstrate that ODSVM outperforms existing methods with statistical significance.

Node classification tasks aim to assign labels or categories to entire graphs based on their structural properties or node attributes. It can be adopted for various types of graph systems, including but not limited to network traffic, biological networks, knowledge graphs, etc., especially to social networks. This problem is well-studied, and solutions have demonstrated significant success in numerous real-world applications. However, in the situation where emerging categories are scarce or even have no labeled data, classical methods perform poorly on the whole, which has attracted growing attention. Based on this, in this article, we divide researches for node classification in social networks into two broad categories: traditional methods and novel strategies (few-shot/zero-shot learning). In traditional node classification methods, we summarize some classical methods for both homogeneous and heterogeneous networks, which includes unsupervised classifier, matrix factorization techniques, supervised methods, random-walk, and meta-path. Meanwhile, we introduce novel methods in few-shot or zero-shot learning. The article outlines the technical principles of various methods and analyzes their performance across different classes. It further summarizes the benchmark datasets used for evaluating node classification tasks. Finally, the major opportunities, challenges, and future research directions in few-shot and zero-shot learning for node classification in graph scenarios are discussed.

Video anomaly detection (VAD) is of great importance for a variety of real-time applications in video surveillance. Most deep learning-based anomaly detection algorithms adopt a one-class learning scheme to train a classifier using only normal data to distinguish between normal and abnormal events during the test phase. However, these methods, whether they are reconstruction or prediction models, commonly face the challenge of the model’s overly strong representation capability, which leads to excessive fitting of abnormal events and thus limits the performance of the model in diverse scenarios. To address these challenges, this work develops a novel residual autoencoder and generative adversarial network-based 3-D convolutional network, called READ3D-net, for anomaly detection. An adaptive multimodal pseudoanomaly generator is developed to simulate and generate diverse pseudoanomalies, aiming to enhance the model’s ability to extract the features with regard to “abnormal” behaviors while reducing the interference of background on the detection performance. In addition, residual structures are incorporated into the design of a reconstruction-based autoencoder model to enhance its feature extraction and discriminative capabilities. To further improve the model’s reconstruction ability on normal data, a dynamic generative adversarial strategy is proposed for effective feature learning. Extensive experiments conducted on public benchmark datasets demonstrate that the proposed model is more competitive than the state-of-the-art methods in video anomaly detection tasks, fully validating the effectiveness and practicality of the proposed approach.

Unsupervised domain adaptation is mainly focused on the tasks of transferring knowledge from a fully-labeled source domain to an unlabeled target domain. However, in some scenarios, the labeled data are expensive to collect, which cause an insufficient label issue in the source domain. To tackle this issue, some works have focused on few-shot unsupervised domain adaptation (FUDA), which transfers predictive models to an unlabeled target domain through a source domain that only contains a few labeled samples. Yet the relationship between labeled and unlabeled source domains are not well exploited in generating pseudo-labels. Additionally, the few-shot setting further prevents the transfer tasks as an excessive domain gap is introduced between the source and target domains. To address these issues, we newly proposed an adaptive dispersal and collaborative clustering (ADCC) method for FUDA. Specifically, for the shortage of the labeled source data, a collaborative clustering algorithm is constructed that expands the labeled source data to obtain more distribution information. Furthermore, to alleviate the negative impact of domain-irrelevant information, we construct an adaptive dispersal strategy that introduces an intermediate domain and pushes both the source and target domains to this intermediate domain. Extensive experiments on the Office31, Office-Home, miniDomainNet, and VisDA-2017 datasets showcase the superior performance of ADCC compared to the state-of-the-art FUDA methods.

Multi-source domain adaptation (MSDA) has garnered significant attention due to its emphasis on transferring knowledge from multiple labeled source domains to a single unlabeled target domain. MSDA requires sufficient labeled data from multiple source domains, but in practice, massive unlabeled data exist instead of well-labeled data. Multiple target domains also provide plenty of information, which is useful for domain adaptation. However, most MSDA studies overlook the critical scenario of multi-source and multi-target domain adaptation (MMDA). To address these problems, we propose a Multiple Adaptation Network (MAN) approach for MMDA, which utilizes multiple alignment strategies for each source-target domain pair-group to align relevant specific feature spaces. MAN also aligns multiple classifiers for the relevant feature spaces to optimize the decision boundaries of multiple target domains. Moreover, to consider the task relations of multiple classifiers, we minimize the semantic differences between the target-conditioned classifiers and utilize a weight learning category to optimize this process. To fully utilize the information from multiple target domains, we transfer the style information of the target data to the source data, aiding in the training of multiple classifiers. Extensive experiments in challenge domain adaptation benchmarks, including the ImageCLEF-DA, Office-Home, DomainNet, and RGB-to-thermal datasets, demonstrate the superiority of our method over the state-of-the-art approaches.

The popularity of blended-target domain adaptation (BTDA) is growing since target data in the real world often come from multiple domains with different data distributions. Most BTDA studies adapt directly from the source domain to the target domains without considering which kinds of semantic information embedded in images should be explored. Therefore, some irrelevant semantic information is inevitably used, which leads to negative transfer. To address these issues, we propose a semantic dual-adversarial network (SDN) method for BTDA. Specifically, to suppress irrelevant semantic information, we adopt a min-max game strategy between the classifier and the feature extractor. The classifier tries to maximize the prediction distribution discrepancy, whereas the extractor endeavors to minimize this discrepancy. In this process, irrelevant semantic information is suppressed and the principal semantic information is emphasized. To align the categorical distributions, we train a category-aware domain discriminator and a feature extractor with category labels. In addition, we introduce a random ratio-based feature fusion scheme to augment the source domain, which can decrease domain gaps. At last, we propose a weighted negative self-supervised learning method to enhance the model's generalization. Extensive experiments on multiple benchmarks showcase that our method significantly outperforms the prior state-of-the-art methods in BTDA.

Medical image segmentation is a challenging task especially when dealing with unlabeled data. It has been proven that the use of complementary information for co-training is effective for medical image segmentation. We find that high-confidence regions are easy to segment, but low-confidence regions are difficult to correctly segment, which results in poor segmentation performance. Therefore, accurate segmentation in low-confidence regions is the effective way to enhance the performance of medical image segmentation. To achieve this goal, we propose a novel co-training strategy named low-confidence heterogeneous pixel-prototype contrastive learning with uncertainty-guide cross supervision for semi-supervised medical image segmentation. Specifically, an uncertainty-guided cross supervision module is firstly designed to estimate the confidence score of predictions from multiple outputs and then perform cross supervision between high-confidence regions. Moreover, we design a low-confidence heterogeneous prototype contrastive learning, which builds the relationship between low-confidence pixel and heterogeneous prototype to mine discriminative information in low-confidence regions so as to improve the segmentation performance. Extensive experiments on medical image datasets demonstrate that our method outperforms state-of-the-art methods in segmentation results.

Hash learning aims to map multimedia data into Hamming space, in which the data point is represented by low-dimensional binary codes and the similarity relationships are preserved. Despite existing hash learning methods have been effectively used in data retrieval tasks for its merits of low memory cost and high computational efficiency, there still remain two major technical challenges. Firstly, due to the discrete constraints of hash codes, traditional hash methods typically use relaxation strategy to learn real-value features and then quantize them into binary codes through a sign function, resulting in significant quantization errors. Secondly, hash codes are usually low-dimensional, which would be inadequate to preserve either the information of each data point or the relationship between two. These two challenges would greatly limit the retrieval performance of learned hash codes. To solve these problems, we introduce a novel quantization method called scaling factor quantization to enhance hash learning. Unlike traditional hashing methods, we propose to map the data into two parts, i.e., hash codes and scaling factors, to learn the representative codes for the use of retrieval. Specifically, we design a multi-output branch network structure, i.e., Deep Scaling factor Quantization Network (DSQN) and an iterative training strategy for DSQN to learn the two parts of mapping. Comprehensive experiments conducted on three benchmark datasets demonstrate that the hash codes and scaling factors learned by DSQN significantly improve retrieval accuracy compared to existing hash learning methods.

As a main branch of domain adaptation (DA), multi-source DA (MSDA) has attracted increasing attention for exploiting information from multi-source domain data. However, how to effectively explore useful information from each source domain for target tasks is still a key problem. In this paper, to fully explore multiple information of different domain data, we propose a graph correlated discriminant embedding (GCDE) method for MSDA. In GCDE, the category-discriminative information, manifold structure, and correlation learning are fully considered. Specifically, GCDE encodes the within- and between- class information of each domain data, preserves the local and global structure information of the data, and extracts the maximization correlative features from different domains by designing a novel correlative learning scheme. We also extend GCDE to a nonlinear case and obtain kernel GCDE (KGCDE). We have conducted extensive experiments on four public data benchmarks to verify the performance of GCDE and KGCDE. The promising performance on the databases prove the efficiency of our methods with the comparison of the advanced approaches.

As a branch of domain adaptation (DA), multi-source DA (MSDA) is a challenging issue that aims to transfer knowledge from multiple well-labeled source domains to a target domain for target tasks. However, most existing related works focus on single-target domain adaptation, and multiple target domain adaptation is not accounted for. We believe that multiple target domains provide valuable knowledge. Meanwhile, in multi-source and multi-target adaptation scenarios, feature generators with static parameters have difficulty generating deep features of each individual domain. In this paper, we propose a Dynamic Generator With Attention (DGWA) method for multi-source and multi-target domain adaptation to adapt domain-agnostic deep features in a multi-source and multi-target domain scenario. The feature generator with dynamic parameters can dynamically change its parameters with data input from different domains, which greatly improves the generalization of the feature pools. An attention mechanism is used in our DGWA to learn more transferable information from different domains. To demonstrate the performance of DGWA, we conduct extensive experiments on several popular domain adaptation datasets, including the digits, Office+Caltech10, Office-Home, and ImageCLEF-DA datasets. The experimental results demonstrate that our method performs better than state-of-the-art methods.

In unsupervised domain adaptation (UDA), negative transfer is one of the most challenging problems. Due to complex environments, the used domain data are always corrupted by noise or outliers in many applications. If the noisy data are directly used for domain adaptation, the disturbances and negative influence of the noise are also shifted for the target tasks. Thus, preventing disturbances and negative effects caused by noise are key problems in UDA that need to be addressed. In this article, a low-rank correlation learning (LRCL) method is proposed for UDA. In LRCL, the noisy domain data are recovered by low-rank learning; then both domain data are cleaned. Hence, the disturbances and negative effects of the noise are prevented. The maximized correlated features of the clean data from the source and target domains are learned by a novel correlation regularization term in a latent common space. LRCL also reduces the distribution difference of the learned clean source and target data by constructing a reconstruction term, in which the clean target data are linearly represented by the clean source data. To explore the temporal and structural information of the data, we further extend LRCL into a graph case and propose graph LRCL (GLRCL). Extensive experiments have been conducted on several public data benchmarks, and the experimental results demonstrate that our methods can effectively prevent negative transfer and obtain better classification outcomes than other compared approaches.

In this paper, we present a novel approach named correlation-guided distribution and geometry alignments (CDGA) for heterogeneous domain adaptation. Unlike existing methods that typically combine feature alignment and domain alignment into a single objective function, our proposed CDGA separates the two alignments into distinct steps. The two adaptation steps are: paired canonical correlation analysis (PCCA) and distribution and geometry alignments (DGA). In the PCCA step, CDGA focuses on maximizing the within-category correlation between source and target samples to produce the dimension-aligned feature representations for the next adaptation step. In the DGA step, CDGA is responsible for learning a classifier that incorporates both distribution and geometry alignments. Furthermore, during this step, the highly confident pseudo labeled samples are carefully selected for the next iteration of PCCA, establishing a beneficial coupling between PCCA and DGA to improve the adaptation performance in an iterative manner. Experimental results on various visual cross-domain benchmarks demonstrate that CDGA achieves remarkable performance compared to the existing shallow heterogeneous domain adaptation methods and even exhibits superiority over the state-of-the-art neural network-based approaches.

Graph-based clustering approaches, especially the family of spectral clustering, have been widely used in machine learning areas. The alternatives usually engage a similarity matrix that is constructed in advance or learned from a probabilistic perspective. However, unreasonable similarity matrix construction inevitably leads to performance degradation, and the sum-to-one probability constraints may make the approaches sensitive to noisy scenarios. To address these issues, the notion of typicality-aware adaptive similarity matrix learning is presented in this study. The typicality (possibility) rather than the probability of each sample being a neighbor of other samples is measured and adaptively learned. By introducing a robust balance term, the similarity between any pairs of samples is only related to the distance between them, yet it is not affected by other samples. Therefore, the impact caused by the noisy data or outliers can be alleviated, and meanwhile, the neighborhood structures can be well captured according to the joint distance between samples and their spectral embeddings. Moreover, the generated similarity matrix has block diagonal properties that are beneficial to correct clustering. Interestingly, the results optimized by the typicality-aware adaptive similarity matrix learning share the common essence with the Gaussian kernel function, and the latter can be directly derived from the former. Extensive experiments on synthetic and well-known benchmark datasets demonstrate the superiority of the proposed idea when comparing with some state-of-the-art methods.

Image-to-Image synthesis paradigms have been widely used for facial expression synthesis. However, current generators are apt to either produce artifacts for largely posed and non-aligned faces or unduly change the identity information like AdaIN-based generator. In this work, we suggest to use image style feature to surrogate the expression cues in the generator, and propose a multi-task learning paradigm to explore this style information via the supervision learning and feature disentanglement. While the supervision learning can make the encoded style specifically represent the expression cues and enable the generator to produce correct expression, the feature disentanglement of content and style cues enables the generator to better preserve the identity information in expression synthesis. Experimental results show that the proposed algorithm can well reduce the artifacts for the synthesis of posed and non-aligned expressions, and achieves competitive performances in terms of FID, PNSR and classification accuracy, compared with four publicly available GANs. The code and pre-trained models are available at https://github.com/lumanxi236/MTSS.

The task of domain generalization is to learn a classification model from multiple source domains and generalize it to unknown target domains. The key to domain generalization is learning discriminative domain-invariant features. Invariant representations are achieved using adversarial domain generalization as one of the primary techniques. For example, generative adversarial networks have been widely used, but suffer from the problem of low intra-class diversity, which can lead to poor generalization ability. To address this issue, we propose a new method called auxiliary classifier in adversarial domain generalization (CloCls). CloCls improve the diversity of the source domain by introducing auxiliary classifier. Combining typical task-related losses, e.g., cross-entropy loss for classification and adversarial loss for domain discrimination, our overall goal is to guarantee the learning of condition-invariant features for all source domains while increasing the diversity of source domains. Further, inspired by smoothing optima have improved generalization for supervised learning tasks like classification. We leverage that converging to a smooth minima with respect task loss stabilizes the adversarial training leading to better performance on unseen target domain which can effectively enhances the performance of domain adversarial methods. We have conducted extensive image classification experiments on benchmark datasets in domain generalization, and our model exhibits sufficient generalization ability and outperforms state-of-the-art DG methods.

High-dimensional small sample size data, which may lead to singularity in computation, are becoming increasingly common in the field of pattern recognition. Moreover, it is still an open problem how to extract the most suitable low-dimensional features for the support vector machine (SVM) and simultaneously avoid singularity so as to enhance the SVM's performance. To address these problems, this article designs a novel framework that integrates the discriminative feature extraction and sparse feature selection into the support vector framework to make full use of the classifiers' characteristics to find the optimal/maximal classification margin. As such, the extracted low-dimensional features from high-dimensional data are more suitable for SVM to obtain good performance. Thus, a novel algorithm, called the maximal margin SVM (MSVM), is proposed to achieve this goal. An alternatively iterative learning strategy is adopted in MSVM to learn the optimal discriminative sparse subspace and the corresponding support vectors. The mechanism and the essence of the designed MSVM are revealed. The computational complexity and convergence are also analyzed and validated. Experimental results on some well-known databases (including breastmnist, pneumoniamnist, colon-cancer, etc.) show the great potential of MSVM against classical discriminant analysis methods and SVM-related methods, and the codes can be available on https://www.scholat.com/laizhihui.

Current fully supervised facial landmark detection methods have progressed rapidly and achieved remarkable performance. However, they still suffer when coping with faces under large poses and heavy occlusions for inaccurate facial shape constraints and insufficient labeled training samples. In this article, we propose a semisupervised framework, that is, a self-calibrated pose attention network (SCPAN) to achieve more robust and precise facial landmark detection in challenging scenarios. To be specific, a boundary-aware landmark intensity (BALI) field is proposed to model more effective facial shape constraints by fusing boundary and landmark intensity field information. Moreover, a self-calibrated pose attention (SCPA) model is designed to provide a self-learned objective function that enforces intermediate supervision without label information by introducing a self-calibrated mechanism and a pose attention mask. We show that by integrating the BALI fields and SCPA model into a novel SCPAN, more facial prior knowledge can be learned and the detection accuracy and robustness of our method for faces with large poses and heavy occlusions have been improved. The experimental results obtained for challenging benchmark datasets demonstrate that our approach outperforms state-of-the-art methods in the literature.

Support vector machine (SVM), as a supervised learning method, has different kinds of varieties with significant performance. In recent years, more research focused on nonparallel SVM, where twin SVM (TWSVM) is the typical one. In order to reduce the influence of outliers, more robust distance measurements are considered in these methods, but the discriminability of the models is neglected. In this article, we propose robust manifold twin bounded SVM (RMTBSVM), which considers both robustness and discriminability. Specifically, a novel norm, that is, capped$L_{1}$-norm, is used as the distance metric for robustness, and a robust manifold regularization is added to further improve the robustness and classification performance. In addition, we also use the kernel method to extend the proposed RMTBSVM for nonlinear classification. We introduce the optimization problems of the proposed model. Subsequently, effective algorithms for both linear and nonlinear cases are proposed and proved to be convergent. Moreover, the experiments are conducted to verify the effectiveness of our model. Compared with other methods under the SVM framework, the proposed RMTBSVM shows better classification accuracy and robustness.

As a branch of transfer learning, domain adaptation leverages useful knowledge from a source domain to a target domain for solving target tasks. Most of the existing domain adaptation methods focus on how to diminish the conditional distribution shift and learn invariant features between different domains. However, two important factors are overlooked by most existing methods: 1) the transferred features should be not only domain invariant but also discriminative and correlated, and 2) negative transfer should be avoided as much as possible for the target tasks. To fully consider these factors in domain adaptation, we propose a guided discrimination and correlation subspace learning (GDCSL) method for cross-domain image classification. GDCSL considers the domain-invariant, category-discriminative, and correlation learning of data. Specifically, GDCSL introduces the discriminative information associated with the source and target data by minimizing the intraclass scatter and maximizing the interclass distance. By designing a new correlation term, GDCSL extracts the most correlated features from the source and target domains for image classification. The global structure of the data can be preserved in GDCSL because the target samples are represented by the source samples. To avoid negative transfer issues, we use a sample reweighting method to detect target samples with different confidence levels. A semi-supervised extension of GDCSL (Semi-GDCSL) is also proposed, and a novel label selection scheme is introduced to ensure the correction of the target pseudo-labels. Comprehensive and extensive experiments are conducted on several cross-domain data benchmarks. The experimental results verify the effectiveness of the proposed methods over state-of-the-art domain adaptation methods.

Most facial landmark detection methods predict landmarks by mapping the input facial appearance features to landmark heatmaps and have achieved promising results. However, when the face image is suffering from large poses, heavy occlusions and complicated illuminations, they cannot learn discriminative feature representations and effective facial shape constraints, nor can they accurately predict the value of each element in the landmark heatmap, limiting their detection accuracy. To address this problem, we propose a novel Reference Heatmap Transformer (RHT) by introducing reference heatmap information for more precise facial landmark detection. The proposed RHT consists of a Soft Transformation Module (STM) and a Hard Transformation Module (HTM), which can cooperate with each other to encourage the accurate transformation of the reference heatmap information and facial shape constraints. Then, a Multi-Scale Feature Fusion Module (MSFFM) is proposed to fuse the transformed heatmap features and the semantic features learned from the original face images to enhance feature representations for producing more accurate target heatmaps. To the best of our knowledge, this is the first study to explore how to enhance facial landmark detection by transforming the reference heatmap information. The experimental results from challenging benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in the literature.

The performance of k-means clustering is often degenerate when dealing with high-dimensional and noisy scenarios. In this study, an end-to-end robust clustering method with low-rank linear embedding techniques (RCLR) is presented in conjunction with k-means. Sparse coefficients and a space projection matrix can be simultaneously learned. The global structures and local neighborhood properties are well captured in the learning procedures. Both the processes of clustering and dimensionality reduction are realized at the same time. The notions of clustering, dimensionality reduction, low-rank representation, and local property preservation are seamlessly integrated into a unified model. The limitation of error accumulation encountered in the previous two-stage clustering framework involving low-rank representation can be alleviated. This is the first attempt to introduce both the global and local geometrical structures into k-means directly, as well L2,1-norm is used as a basic metric instead of the conventional F-norm to further improve the robustness and interpretation of the model. The superiority of the proposed RCLR method is demonstrated by extensive experiments completed on various well-known benchmark datasets.

Mammogram mass detection is a difficult task due to the mass character of the tiny area, fuzzy boundary, and occlusion. To address these problems, this paper proposes a novel detection network for mammogram mass detection. Firstly, we propose a novel feature fusion structure and Small Target Attention Module (STAM) to improve the model's ability to detect small masses. Secondly, Results-oriented Loss (ROL) is adopted to obtain better model performance. Finally, Incremental Positive Selection (IPS) is used to divide positive and negative anchors. The scarcity of breast mammogram images for training aggravates the difficulty of mass detection. Thus, we open our collected dataset, which contains 1456 mammogram images from 400 patients. Since the model includes a double feature fusion structure, the proposed network is named Dual Fusion Mass Detector (DFMD). Experiment results show that DFMD is robust to various variations on scale, blurry and occlusion.

Breast disease is one of the primary diseases endangering women's health. Accurate segmentation of breast lesions can help doctors diagnose breast diseases. However, the size and morphology of breast lesions are different, and the intensity of breast tissue is uneven. Thus, it is challenging to segment the lesion area accurately. In this paper, we propose Dual-scale Feature Fusion (DSFF) module and Edgeloss to segment breast lesions. The DSFF module aims to integrate two-scale features and design another effective skip connection scheme to reduce false positive regions. To solve the problem of unclear segmentation boundary, we design Edgeloss for additional supervision on the boundary region to obtain a finer segmentation boundary. The experiment results show that the proposed DL-UNet with the DSFF module and new Edgeloss performs best in several classic networks.

Recently, contrastive learning has shown great potential in medical image segmentation. Due to the lack of expert annotations, however, it is challenging to apply contrastive learning in semi-supervised scenes. To solve this problem, we propose a novel uncertainty-guided pixel contrastive learning method for semi-supervised medical image segmentation. Specifically, we construct an uncertainty map for each unlabeled image and then remove the uncertainty region in the uncertainty map to reduce the possibility of noise sampling. The uncertainty map is determined by a well-designed consistency learning mechanism, which generates comprehensive predictions for unlabeled data by encouraging consistent network outputs from two different decoders. In addition, we suggest that the effective global representations learned by an image encoder should be equivariant to different geometric transformations. To this end, we construct an equivariant contrastive loss to strengthen global representation learning ability of the encoder. Extensive experiments conducted on popular medical image benchmarks demonstrate that the proposed method achieves better segmentation performance than the state-of-the-art methods.

Supervised discrete hashing (SDH) has widely attracted attention because it can directly obtain hash codes rather than relaxing the discrete constraint, which reduces information loss during the quantization process. However, SDH and its extensions tend to ignore the underlying geometry structure of data, while in many cases, data is lying on a low-dimensional manifold. Therefore, based on SDH, we propose a novel manifold learning-based supervised discrete hashing method in this paper to further improve the performance. Our goal is to learn compact hash codes by discovering latent information and using the local structure of data. To this end, we utilize the anchor graph to capture local neighborhood relationship embedding in data for manifold learning. Besides, since the previous methods use a strict binary label matrix for classification, we jointly optimize a relaxed label matrix to increase the discriminative ability of the proposed model. As such, the learned hash codes are more suitable for classification. Compared with SDH and other popular hashing methods, the experimental results on four large-scale datasets show that the proposed hashing method has better performance.

Fuzzy clustering techniques, especially fuzzy C-means (FCM) and its weighted variants, are typical partitive clustering models that are widely used for revealing possible hidden structures in data. Although they can quantitatively depict the overlapping areas with a partition matrix, their performances deteriorate when dealing with high-dimensional data because the distance computations may be negatively impacted by the irrelevant features, and then the concentration effect may arise. Moreover, they are sensitive to noisy environments. To tackle these obstacles, a robust jointly sparse fuzzy clustering method (RJSFC) is proposed in this study. The representative prototypes, sparse membership grades, and an orthogonal projection matrix are simultaneously learnt when optimizing RJSFC. The obtained low-dimensional embeddings can preserve the local neighborhood structure, and the clustering is conducted in the transformed lower dimensional space rather than the original space, which improves the capability of fuzzy clustering for dealing with high-dimensional scenarios. Furthermore,${L_{2,1}}$-norm is exploited as the basic metric for both loss and regularization parts in RJSFC, the robustness of the model and the interpretability of the extracted features are enhanced. The notions of fuzzy clustering, neighborhood structure preservation, and feature extraction are seamlessly integrated into a unified model. The limitation of the previous two-stage clustering framework when dealing with high-dimensional data entailing dimensionality reduction and clustering procedures separately can be effectively addressed. Extensive experimental results on various well-known datasets demonstrate the usefulness of RJSFC when comparing with some state-of-the-art methods.

As a multivariate data analysis tool, canonical correlation analysis (CCA) has been widely used in computer vision and pattern recognition. However, CCA uses Euclidean distance as a metric, which is sensitive to noise or outliers in the data. Furthermore, CCA demands that the two training sets must have the same number of training samples, which limits the performance of CCA-based methods. To overcome these limitations of CCA, two novel canonical correlation learning methods based on low-rank learning are proposed in this paper for image representation, named robust canonical correlation analysis (robust-CCA) and low-rank representation canonical correlation analysis (LRR-CCA). By introducing two regular matrices, the training sample numbers of the two training datasets can be set as any values without any limitation in the two proposed methods. Specifically, robust-CCA uses low-rank learning to remove the noise in the data and extracts the maximization correlation features from the two learned clean data matrices. The nuclear norm andL1-norm are used as constraints for the learned clean matrices and noise matrices, respectively. LRR-CCA introduces low-rank representation into CCA to ensure that the correlative features can be obtained in low-rank representation. To verify the performance of the proposed methods, five publicly image databases are used to conduct extensive experiments. The experimental results demonstrate the proposed methods outperform state-of-the-art CCA-based and low-rank learning methods.

Domain adaptation leverages rich knowledge from a related source domain so that it can be used to perform tasks in a target domain. For more knowledge to be obtained under relaxed conditions, domain adaptation methods have been widely used in pattern recognition and image classification. However, most of the existing domain adaptation methods only consider how to minimize different distributions of the source and target domains, which neglects what should be transferred for a specific task and suffers negative transfer by distribution outliers. To address these problems, in this paper, we propose a novel domain adaptation method called weighted correlation embedding learning (WCEL) for image classification. In the WCEL approach, we seamlessly integrated correlation learning, graph embedding, and sample reweighting into a unified learning model. Specifically, we extracted the maximum correlated features from the source and target domains for image classification tasks. In addition, two graphs were designed to preserve the discriminant information from interclass samples and neighborhood relations in intraclass samples. Furthermore, to prevent the negative transfer problem, we developed an efficient sample reweighting strategy to predict the target with different confidence levels. To verify the performance of the proposed method in image classification, extensive experiments were conducted with several benchmark databases, verifying the superiority of the WCEL method over other state-of-the-art domain adaptation algorithms.

As one of the most prevalent branches of transfer learning, domain adaptation is dedicated to generalizing the knowledge of a source domain to a target domain to perform machine learning tasks. In domain adaptation, the key strategy is to overcome the shift between different domains and learn shared features with domain invariance. However, most existing methods focus on extracting the common features of the source and target domains, and do not consider the shift problem of class center in the target domain caused by this process. Specifically, when we align the domain distributions, we often ignore the inherent feature attributes of the data, or under the guidance of false pseudo-labels, cause the target domain data to be far away from the class center after projection. This is not conducive to classification task. To address these problems, in this study, we propose a novel domain adaptation method, referred to as discriminative invariant alignment (DIA), for image representation. DIA enriches the knowledge matrix by combining the class discriminative information of the source domain and local data structure information of the target domain into a new framework. By introducing the maximum margin criterion of the source domain, the classification boundaries are expanded. To verify the performance of the proposed method, we compared DIA with several state-of-the-art methods on five benchmark databases. The experimental results show that DIA is superior to the state-of-the-art methods.

Sparse discriminative projection learning has attracted much attention due to its good performance in recognition tasks. In this article, a framework called generalized embedding regression (GER) is proposed, which can simultaneously perform low-dimensional embedding and sparse projection learning in a joint objective function with a generalized orthogonal constraint. Moreover, the label information is integrated into the model to preserve the global structure of data, and a rank constraint is imposed on the regression matrix to explore the underlying correlation structure of classes. Theoretical analysis shows that GER can obtain the same or approximate solution as some related methods with special settings. By utilizing this framework as a general platform, we design a novel supervised feature extraction approach called jointly sparse embedding regression (JSER). In JSER, we construct an intrinsic graph to characterize the intraclass similarity and a penalty graph to indicate the interclass separability. Then, the penalty graph Laplacian is used as the constraint matrix in the generalized orthogonal constraint to deal with interclass marginal points. Moreover, the$L_{2,1}$-norm is imposed on the regression terms for robustness to outliers and data’s variations and the regularization term for jointly sparse projection learning, leading to interesting semantic interpretability. An effective iterative algorithm is elaborately designed to solve the optimization problem of JSER. Theoretically, we prove that the subproblem of JSER is essentially an unbalanced Procrustes problem and can be solved iteratively. The convergence of the designed algorithm is also proved. Experimental results on six well-known data sets indicate the competitive performance and latent properties of JSER.

Recently, heatmap regression has been widely explored in facial landmark detection and obtained remarkable performance. However, most of the existing heatmap regression-based facial landmark detection methods neglect to explore the high-order feature correlations, which is very important to learn more representative features and enhance shape constraints. Moreover, no explicit global shape constraints have been added to the final predicted landmarks, which leads to a reduction in accuracy. To address these issues, in this article, we propose a multiorder multiconstraint deep network (MMDN) for more powerful feature correlations and shape constraints' learning. Especially, an implicit multiorder correlating geometry-aware (IMCG) model is proposed to introduce the multiorder spatial correlations and multiorder channel correlations for more discriminative representations. Furthermore, an explicit probability-based boundary-adaptive regression (EPBR) method is developed to enhance the global shape constraints and further search the semantically consistent landmarks in the predicted boundary for robust facial landmark detection. It is interesting to show that the proposed MMDN can generate more accurate boundary-adaptive landmark heatmaps and effectively enhance shape constraints to the predicted landmarks for faces with large pose variations and heavy occlusions. Experimental results on challenging benchmark data sets demonstrate the superiority of our MMDN over state-of-the-art facial landmark detection methods.

Finger vein has attracted substantial attention due to its good security. However, the variability of the finger vein data will be caused by the illumination, environment temperature, acquisition equipment, and so on, which is a great challenge for finger vein recognition. To address this problem, we propose a novel method to design an endto-end deep Convolutional Neural Network (CNN) for robust finger vein recognition. The approach mainly includes an Intrinsic Feature Learning (IFL) module using an auto-encoder network and an Extrinsic Feature Learning (EFL) module based on a Siamese network. The IFL module is designed to estimate the expectation of intra-class finger vein images with various offsets and rotation, while the EFL module is constructed to learn the inter-class feature representation. Then, robust verification is finally achieved by considering the distances of both intrinsic and extrinsic features. We conduct experiments on two public datasets (i.e. SDUMLA-HMT and MMCBNU_6000) and an in-house dataset (MultiView-FV) with more deformation finger vein images, and the equal error rate (EER) is 0.47%, 0.1%, and 1.69% respectively. The comparison against baseline and existing algorithms shows the effectiveness of our proposed method.

Although current face alignment algorithms have obtained pretty good performances at predicting the location of facial landmarks, huge challenges remain for faces with severe occlusion and large pose variations, etc. On the contrary, semantic location of facial boundary is more likely to be reserved and estimated on these scenes. Therefore, we study a two-stage but end-to-end approach for exploring the relationship between the facial boundary and landmarks to get boundary-aware landmark predictions, which consists of two modules: the self-calibrated boundary estimation (SCBE) module and the boundary-aware landmark transform (BALT) module. In the SCBE module, we modify the stem layers and employ intermediate supervision to help generate high-quality facial boundary heatmaps. Boundary-aware features inherited from the SCBE module are integrated into the BALT module in a multi-scale fusion framework to better model the transformation from boundary to landmark heatmap. Experimental results conducted on the challenging benchmark datasets demonstrate that our approach outperforms state-of-the-art methods in the literature. Code will be available soon.

Recently, many methods based on low-rank representation have been proposed for fabric defect detection. Most of them relax the low-rank decomposition problem to a nuclear norm minimization (NNM) problem to pursue the convexity of the objective function. When solving the standard NNM problem, matrix singular values have to be treated equally. This, however, would be impractical in the scenario of fabric defect detection as the matrix singular values have clear physical meanings, and thus, they should be treated differently. In this article, we propose a weighted double-low-rank decomposition method (WDLRD) to treat the matrix singular values differently by assigning different weights. Thus, the most important/distinguishing characteristics of a fabric image can be preserved. Another difference between WDLRD and the other existing low-rank-based methods is that WDLRD considers a defective fabric image being decomposed to two low-rank matrices, i.e., low-rank defect-free matrix and low-rank defect matrix, as the defect-free and defective regions are usually composed of homogeneous objects that have a high correlation. Besides, WDLRD is more robust for defect detection in various situations by adding a noise term to avoid noise or other interference on the fabric surface. In addition, a defect prior is incorporated into the objective function of WDLRD to guide locating the defective regions. The proposed optimization problem can be easily solved by an iterative algorithm based on augmented Lagrange multipliers. Experimental results on TILDA, periodically patterned fabric, and Textile & Apparel Artificial Intelligence databases show that the proposed WDLRD obtains better performance than state-of-the-art methods in locating the defective regions on fabric images.Note to Practitioners—This article is motivated by the problem that the performance of fabric defect detection in the textile industry is poor. It is necessary to develop an effective method to improve the defect detection accuracy and reduce overall manufacturing cost. Existing automatic defect detection approaches usually contain two stages: first, capture fabric images from the weaving machine and then use a defect detection algorithm in a host computer to conduct a real-time inspection and give an alarm if defects occur. This article focuses on locating defects for given defective images after the procedure of binary classification (which determines an image as defective or defect free). The article proposes an objective function to mathematically interpret the optimization problem between fabric images and predictive defects. The optimal solution can be obtained by employing the alternating direction method of multipliers (ADMMs). The proposed method is described as a new defect detection algorithm. Extensive experiments were conducted to evaluate the algorithm, and the experimental results indicate that the proposed method is superior to many existing fabric defect detection methods. Preliminary experiments suggest that this method is feasible but has not yet been really used in production. In future research, we will collect more fabric images from the textile industry and develop large-scale databases for verifying the proposed method for real-life applications.

As the extended version of conventional Ridge Regression, L2,1-norm based ridge regression learning methods have been widely used in subspace learning since they are more robust than Frobenius norm based regression and meanwhile guarantee joint sparsity. However, conventional L2,1-norm regression methods encounter the small-class problem and meanwhile ignore the local geometric structures, which degrade their performances. To address these problems, we propose a novel regression method called Locality Preserving Robust Regression (LPRR). In addition to using the L2,1-norm for jointly sparse regression, we also utilize capped L2-norm in loss function to further enhance the robustness of the proposed algorithm. Moreover, to make use of local structure information, we also integrate the property of locality preservation into our model since it is of great importance in dimensionality reduction. The convergence analysis and computational complexity of the proposed iterative algorithm are presented. Experimental results on four datasets indicate that the proposed LPRR performs better than some famous subspace learning methods in classification tasks.

Facial action units (AUs) analysis plays an important role in facial expression recognition (FER). Existing deep spectral convolutional networks (DSCNs) have made encouraging performance for FER based on a set of facial local regions and a predefined graph structure. However, these regions do not have close relationships to AUs, and DSCNs cannot model the dynamic spatial dependencies of these regions for estimating different facial expressions. To tackle these issues, we propose a novel double dynamic relationships graph convolutional network (DDRGCN) to learn the strength of the edges in the facial graph by a trainable weighted adjacency matrix. We construct facial graph data by 20 regions of interest (ROIs) guided by different facial AUs. Furthermore, we devise an efficient graph convolutional network in which the inherent dependencies of vertices in the facial graph can be learned automatically during network training. Notably, the proposed model only has 110K parameters and 0.48MB model size, which is significantly less than most existing methods. Experiments on four widely-used FER datasets demonstrate that the proposed dynamic relationships graph network achieves superior results compared to existing light-weight networks, not just in terms of accuracy but also model size and speed.

Though widely used in image classification, convolutional neural networks (CNNs) are prone to noise interruptions, i.e. the CNN output can be drastically changed by small image noise. To improve the noise robustness, we try to integrate CNNs with wavelet by replacing the common down-sampling (max-pooling, strided-convolution, and average pooling) with discrete wavelet transform (DWT). We firstly propose general DWT and inverse DWT (IDWT) layers applicable to various orthogonal and biorthogonal discrete wavelets like Haar, Daubechies, and Cohen, etc., and then design wavelet integrated CNNs (WaveCNets) by integrating DWT into the commonly used CNNs (VGG, ResNets, and DenseNet). During the down-sampling, WaveCNets apply DWT to decompose the feature maps into the low-frequency and high-frequency components. Containing the main information including the basic object structures, the low-frequency component is transmitted into the following layers to generate robust high-level features. The high-frequency components are dropped to remove most of the data noises. The experimental results show that WaveCNets achieve higher accuracy on ImageNet than various vanilla CNNs. We have also tested the performance of WaveCNets on the noisy version of ImageNet, ImageNet-C and six adversarial attacks, the results suggest that the proposed DWT/IDWT layers could provide better noise-robustness and adversarial robustness. When applying WaveCNets as backbones, the performance of object detectors (i.e., faster R-CNN and RetinaNet) on COCO detection dataset are consistently improved. We believe that suppression of aliasing effect, i.e. separation of low frequency and high frequency information, is the main advantages of our approach. The code of our DWT/IDWT layer and different WaveCNets are available at https://github.com/CVI-SZU/WaveCNet.

Heatmap regression (HR) has become one of the mainstream approaches for face alignment and has obtained promising results under constrained environments. However, when a face image suffers from large pose variations, heavy occlusions and complicated illuminations, the performances of HR methods degrade greatly due to the low resolutions of the generated landmark heatmaps and the exclusion of important high-order information that can be used to learn more discriminative features. To address the alignment problem for faces with extremely large poses and heavy occlusions, this paper proposes a heatmap subpixel regression (HSR) method and a multi-order cross geometry-aware (MCG) model, which are seamlessly integrated into a novel multi-order high-precision hourglass network (MHHN). The HSR method is proposed to achieve high-precision landmark detection by a well-designed subpixel detection loss (SDL) and subpixel detection technology (SDT). At the same time, the MCG model is able to use the proposed multi-order cross information to learn more discriminative representations for enhancing facial geometric constraints and context information. To the best of our knowledge, this is the first study to explore heatmap subpixel regression for robust and high-precision face alignment. The experimental results from challenging benchmark datasets demonstrate that our approach outperforms state-of-the-art methods in the literature.

Person Re-identification (ReID) aims to retrieve the pedestrian with the same identity across different views. Existing studies mainly focus on improving accuracy, while ignoring their efficiency. Recently, several hash based methods have been proposed. Despite their improvement in efficiency, there still exists an unacceptable gap in accuracy between these methods and real-valued ones. Besides, few attempts have been made to simultaneously explicitly reduce redundancy and improve discrimination of hash codes, especially for short ones. Integrating Mutual learning may be a possible solution to reach this goal. However, it fails to utilize the complementary effect of teacher and student models. Additionally, it will degrade the performance of teacher models by treating two models equally. To address these issues, we propose a salience-guided iterative asymmetric mutual hashing (SIAMH) to achieve high-quality hash code generation and fast feature extraction. Specifically, a salience-guided self-distillation branch (SSB) is proposed to enable SIAMH to generate hash codes based on salience regions, thus explicitly reducing the redundancy between codes. Moreover, a novel iterative asymmetric mutual training strategy (IAMT) is proposed to alleviate drawbacks of common mutual learning, which can continuously refine the discriminative regions for SSB and extract regularized dark knowledge for two models as well. Extensive experiment results on five widely used datasets demonstrate the superiority of the proposed method in efficiency and accuracy when compared with existing state-of-the-art hashing and real-valued approaches. The code is released at https://github.com/Vill-Lab/SIAMH.

In transfer learning, how to effectively transfer useful information from the source domain to the target domain is crucial. In this paper, we propose a novel transfer learning method for image classification, named manifold transfer learning via discriminant regression analysis (MTL-DRA), to transfer the local geometry structure information from the source domain to the target domain and ensure that the transform matrix is robust or sparse so that samples from different domains can be well combined. In MTL-DRA, we encode discriminant information of the source domain to the target domain by introducing between- and within-class graphs to preserve within-class similarity and reduce between-class similarity. With different norms as constraints, MTL-DRA overcomes the disturbance of noise and avoids negative transfer learning. To improve the robustness of MTL-DRA, we encode a nuclear norm constraint and propose robust MTL-DRA (RMTL-DRA). We analyzed the convergence and complexity of the two proposed methods. To verify the performance of the proposed methods, we conducted extensive experiments on five public image benchmarks. The experimental results show that the proposed methods outperform state-of-the-art transfer learning methods.

Convolutional Neural Networks (CNNs) are generally prone to noise interruptions, i.e., small image noise can cause drastic changes in the output. To suppress the noise effect to the final predication, we enhance CNNs by replacing max-pooling, strided-convolution, and average-pooling with Discrete Wavelet Transform (DWT). We present general DWT and Inverse DWT (IDWT) layers applicable to various wavelets like Haar, Daubechies, and Cohen, etc., and design wavelet integrated CNNs (WaveCNets) using these layers for image classification. In WaveCNets, feature maps are decomposed into the low-frequency and high-frequency components during the down-sampling. The low-frequency component stores main information including the basic object structures, which is transmitted into the subsequent layers to extract robust high-level features. The high-frequency components, containing most of the data noise, are dropped during inference to improve the noise-robustness of the WaveCNets. Our experimental results on ImageNet and ImageNet-C (the noisy version of ImageNet) show that WaveCNets, the wavelet integrated versions of VGG, ResNets, and DenseNet, achieve higher accuracy and better noise-robustness than their vanilla versions.

Hashing has attracted widespread attention in image retrieval because of its fast retrieval speed and low storage cost. Compared with supervised methods, unsupervised hashing methods are more reasonable and suitable for large-scale image retrieval since it is always difficult and expensive to collect true labels of the massive data. Without label information, however, unsupervised hashing methods can not guarantee the quality of learned binary codes. To resolve this dilemma, this paper proposes a novel unsupervised hashing method called Label Self-Adaption Hashing (LSAH), which contains effective hashing function learning part and self-adaption label generation part. In the first part, we utilize anchor graph to keep the local structure of the data and introduce joint sparsity into the model to extract effective features for high-quality binary code learning. In the second part, a self-adaptive cluster label matrix is learned from the data under the assumption that the nearest neighbor points should have a large probability to be in the same cluster. Therefore, the proposed LSAH can make full use of the potential discriminative information of data to guide the learning of binary codes. It is worth noting that LSAH can learn effective binary codes, hashing function and cluster labels simultaneously in a unified optimization framework. To solve the resulting optimization problem, an Augmented Lagrange Multiplier based iterative algorithm is elaborately designed. Extensive experiments on three large-scale data sets indicate the promising performance of the proposed LSAH.

Hashing has been recognized as an efficient representation learning method to effectively handle big data due to its low computational complexity and memory cost. Most of the existing hashing methods focus on learning the low-dimensional vectorized binary features based on the high-dimensional raw vectorized features. However, the studies on how to obtain preferable binary codes from the original 2D image features for retrieval is very limited. This paper proposes a bilinear supervised discrete hashing (BSDH) method based on 2D image features which utilizes bilinear projections to binarize the image matrix features such that the intrinsic characteristics in the 2D image space are preserved in the learned binary codes. Meanwhile, the bilinear projection approximation and vectorization binary codes regression are seamlessly integrated together to formulate the final robust learning framework. Furthermore, a discrete optimization strategy is developed to alternatively update each variable for obtaining the high-quality binary codes. In addition, two 2D image features, traditional SURF-based FVLAD feature, and CNN-based AlexConv5 feature are designed for further improving the performance of the proposed BSDH method. The results of extensive experiments conducted on four benchmark datasets show that the proposed BSDH method almost outperforms all competing hashing methods with different input features by different evaluation protocols.

This paper proposes a novel discriminative regression method, called adaptive locality preserving regression (ALPR) for classification. In particular, ALPR aims to learn a more flexible and discriminative projection that not only preserves the intrinsic structure of data, but also possesses the properties of feature selection and interpretability. To this end, we introduce a target learning technique to adaptively learn a more discriminative and flexible target matrix rather than the pre-defined strict zero-one label matrix for regression. Then, a locality preserving constraint regularized by the adaptive learned weights is further introduced to guide the projection learning, which is beneficial to learn a more discriminative projection and avoid overfitting. Moreover, we replace the conventional `Frobenius norm' with the special l2,1norm to constrain the projection, which enables the method to adaptively select the most important features from the original high-dimensional data for feature extraction. In this way, the negative influence of the redundant features and noises residing in the original data can be greatly eliminated. Besides, the proposed method has good interpretability for features owing to the row-sparsity property of the l2,1norm. Extensive experiments conducted on the synthetic database with manifold structure and many real-world databases prove the effectiveness of the proposed method.

Neighborhood preserving embedding (NPE) has been proposed to encode overall geometry manifold embedding information. However, the class-special structure of the data is destroyed by noise or outliers existing in the data. To address this problem, in this article, we propose a novel embedding approach called robust flexible preserving embedding (RFPE). First, RFPE recovers the noisy data by low-rank learning and obtains clean data. Then, the clean data are used to learn the projection matrix. In this way, the projective learning is totally unaffected by noise or outliers. By encoding a flexible regularization term, RFPE can keep the property of the data points with a nonlinear manifold and be more flexible. RFPE searches the optimal projective subspace for feature extraction. In addition, we also extend the proposed RFPE to a kernel case and propose kernel RFPE (KRFPE). Extensive experiments on six public image databases show the superiority of the proposed methods over other state-of-the-art methods.

Hashing is an important topic in image processing, as it can help save a considerable amount of storage and computational cost. Recently, inspired by 2D strategies employed in other areas of image processing, such as feature extraction, some 2D-based hashing methods were proposed. Related papers have shown that these methods may have better image retrieval performance in terms of both effectiveness and efficiency. However, the difference in the retrieval performances of hashing methods resulting from different forms of input (1D or 2D) has not been previously studied. Whether the widely used bilinear strategy in 2D-based hashing can truly help improve the retrieval precision has not been investigated in existing research. In this paper, we conduct a comparison study on 1D and 2D feature-based hashing methods and attempt to theoretically and experimentally analyse the differences in using 1D and 2D features in hashing. Furthermore, two new hashing methods are proposed for conducting the comparison experiments. Through a comprehensive study, we obtain three main conclusions in this paper: 1) Linear projection on 1D features and bilinear projection on 2D features are essentially the same. 2) 2D-based hashing methods are obviously more efficient than 1D-based methods for analysing high-dimensional input features. 3) 2D-based hashing methods show generally better performance for solving small sample size problems.

This paper proposes a novel method called Jointly Sparse Locality Regression (JSLR) for feature extraction and selection. JSLR utilizes joint L2,1-norm minimization on regularization term, and also introduces the locality to characterize the local geometric structure of the data. There are three main contributions in JSLR for face recognition. Firstly, it eliminates the drawback in ridge regression and Linear Discriminant Analysis (LDA) that when the number of the classes is too small, not enough projections can be obtained for feature extraction. Secondly, by using the local geometric structure as the regularization term, JSLR is able to preserve local information and find an embedding subspace which can detect the most essential data manifold structure. Moreover, since the L2,1-norm based loss function is robust to outliers in data points, JSLR provides the joint sparsity for robust feature selection. The theoretical connections of the proposed method and the previous regression methods are explored and the convergence of the proposed algorithm is also proved. Experimental evaluation on several well-known data sets shows the merits of the proposed method on feature selection and classification.

Tensor completion arouses much attention in signal processing and machine learning. The tensor train (TT) decomposition has shown better performances than the Tucker decomposition in image and video inpainting. In this paper, we propose a novel tensor completion model based on a newly defined latent Schatten TT norm. Then, the statistical performance is analyzed by establishing a non-asymptotic upper bound on the estimation error. Further, a scalable algorithm is developed to efficiently solve the model. Experimental results of color image inpainting demonstrate that the proposed norm has promising performances compared to other variants of Schatten norm.

Due to the superiority in exploiting the ubiquitous "spatial-shifting" property in modern multi-way data, the recently proposed low-tubal-rank model has been successfully applied for tensor recovery in signal processing and computer vision. In this paper, we define the generalized tensor Dantzig selector to recover a low-tubal-rank tensor from noisy linear measurements. Algorithmically, we develop an efficient algorithm based on the ADMM framework. Statistically, we establish non-asymptotic upper bounds on the estimation error for the problems of tensor completion and compressive sensing. Numerical experiments illustrate that our bounds can predict the scaling behavior of the estimation error. Experiments on realword datasets show the effectiveness of the proposed model.

Real multi-way data may suffer from missing entries, noise and outliers simultaneously. The recently proposed tubal nuclear norm (TNN) has shown its superiority in tensor completion. However, statistical analysis of TNN based models is still deficient. This paper aims to robustly recover a polluted incomplete tensor with rigorous statistical guarantee. Specifically, an estimator based on a weighed variant of TNN is proposed to complete a low-tubal-rank tensor corrupted by element sparse errors or slice sparse sample outliers from partial noisy observations. Non-asymptotic upper bounds on the estimation error are established and further proved to be minimax optimal up to a log factor. Sharpness of the upper bounds is verified on synthetic datasets and superiority of the proposed estimator is demonstrated through robust video inpainting.

Recently, a series of deep supervised hashing methods were proposed for binary code learning. However, due to the high computation cost and the limited hardware's memory, these methods will first select a subset from the training set, and then form a mini-batch data to update the network in each iteration. Therefore, the remaining labeled data cannot be fully utilized and the model cannot directly obtain the binary codes of the entire training set for retrieval. To address these problems, this paper proposes an interesting regularized deep model to seamlessly integrate the advantages of deep hashing and efficient binary code learning by using the anchor graph. As such, the deep features and label matrix can be jointly used to optimize the binary codes, and the network can obtain more discriminative feedback from the linear combinations of the learned bits. Moreover, we also reveal the algorithm mechanism and its computation essence. Experiments on three large-scale datasets indicate that the proposed method achieves better retrieval performance with less training time compared to previous deep hashing methods.

Majority of face recognition systems can only retrieve protected ID photos from the government agency in China. These facial images covered with mesh-like curves are termed meshface. Meshface could significantly affect the performance of face recognition systems. Although some GAN based methods are proposed to address this issue by translating meshface images to clean face images, their capacities are limited. In this paper, we introduce the introspective modules to encourage the generator to reconstruct face images and accelerate the learning process of the discriminator. Both a private meshface dataset and the public LFW dataset are used for experiments. The quantitative evaluation on both datasets proves that the introspective GAN can recover face image with better quality. Additionally, face recognition performance is also significantly improved.

Deep part-based methods in recent literature have revealed the great potential of learning local part-level representation for pedestrian image in the task of person re-identification. However, global features that capture discriminative holistic information of human body are usually ignored or not well exploited. This motivates us to investigate joint learning global and local features from pedestrian images. Specifically, in this work, we propose a novel framework termed tree branch network (TBN) for person re-identification. Given a pedestrain image, the feature maps generated by the backbone CNN, are partitioned recursively into several pieces, each of which is followed by a bottleneck structure that learns finer-grained features for each level in the hierarchical tree-like framework. In this way, representations are learned in a coarse-to-fine manner and finally assembled to produce more discriminative image descriptions. Experimental results demonstrate the effectiveness of the global and local feature learning method in the proposed TBN framework. We also show significant improvement in performance over state-of-the-art methods on three public benchmarks: Market-1501, CUHK-03 and DukeMTMC.

Shadowed sets provide a meaningful description of information granules by abstracting the corresponding fuzzy sets into three categories: full acceptance, full rejection, and uncertain (represented by shadows). One of the main motivating points to derive shadowed sets from fuzzy sets is the determination and explanation of the separation thresholds based on a specific optimization mechanism. The available optimization objective functions are mainly discussed on semantic interpretations and their mathematical properties; constructive algorithms for optimal solutions have rarely been reported. In this paper, the continuous and convex properties of Pedrycz's optimization objective function to construct shadowed sets, as well as the existence and uniqueness of solution points, are analyzed in detail. It is demonstrated that different approximation region partitions would be generated even under the same optimization model, which requires further criteria to make the constructed shadowed sets well-defined. To address this limitation, the notions of passive and active constrained shadowed sets are introduced. A fast algorithm to obtain the proposed constrained shadowed sets is also designed based on the analyzed mathematical properties. Its performance is then illustrated by some typical fuzzy sets and some real data from the UCI repository.

Ridge regression is widely used in multiple variable data analysis. However, in very high-dimensional cases such as image feature extraction and recognition, conventional ridge regression or its extensions have the small-class problem, that is, the number of the projections obtained by ridge regression is limited by the number of the classes. In this paper, we proposed a novel method called generalized robust regression (GRR) for jointly sparse subspace learning which can address the problem. GRR not only imposes L2,1-norm penalty on both loss function and regularization term to guarantee the joint sparsity and the robustness to outliers for effective feature selection, but also utilizes L2,1-norm as the measurement to take the intrinsic local geometric structure of the data into consideration to improve the performance. Moreover, by incorporating the elastic factor on the loss function, GRR can enhance the robustness to obtain more projections for feature selection or classification. To obtain the optimal solution of GRR, an iterative algorithm was proposed and the convergence was also proved. Experiments on six wellknown data sets demonstrate the merits of the proposed method. The result indicates that GRR is a robust and efficient regression method for face recognition.

Preserving projection-based methods are good for finding the manifold structure embedded in data. As they use the Euclidean distance as a metric, which is sensitive to noise and outliers in data, nuclear norm-based 2D locality preserving projection (NN-2DLPP) is thus proposed to improve the robustness of 2DLPP. However, NN-2DLPP does not consider the discriminant ability of data. In order to improve the discriminant ability of preserving projection methods, in this paper, we use preserving projection learning with structurally incoherence of data and propose structurally incoherent low-rank 2DLPP (SILR-2DLPP) for image classification. This approach provides a discriminative representation of preserving projection learning by recovering the distinct different classes of the data. SILR-2DLPP searches the optimal subspace and low-rank representation simultaneously. We further extend SILR-2DLPP to a kernel case and propose kernel SILR-2DLPP (KSILR-2DLPP) to obtain a nonlinear representation. The theoretical analysis including the convergence and computational complexity of SILR-2DLPP are presented. To verify the performance of SILR-2DLPP and KSILR-2DLPP, six well-known image databases were used in the experiments. The experimental results show that the proposed methods are superior to the previous preserving projection methods for image classification.

2-D linear discriminant analysis (2DLDA) has been widely used in pattern recognition and image classification. 2DLDA selects discriminative features from the up and left corner of images. However, 2DLDA uses the Frobenius norm (F-norm), which is sensitive to noise or outliers in data, as a metric. In this paper, we propose a novel framework, called horizontal and vertical nuclear norm-based 2DLDA (HVNN-2DLDA) for image representation. In the proposed framework, HVNN-2DLDA methods (i.e., HNN-2DLDA and VNN-2DLDA) are proposed, and both use the nuclear norm as a criterion. The nuclear norm can provide more structure and global information for the reconstruction of noisy images. HNN-2DLDA and VNN-2DLDA represent images in the row and column directions, respectively. In addition, by combining the row and column directions, we propose a bilateral nuclear norm-based 2DLDA method called BNN-2DLDA. The advantage of BNN-2DLDA over HNN-2DLDA and VNN-2DLDA is that an image sample can be represented by both the row and the column directions instead of only the row or column direction. HVNN-2DLDA learns a set of local optimal projection vectors by maximizing the ratio of the nuclear norm of the between-class scatter matrix and the nuclear norm of the within-class scatter matrix. To verify the robustness and recognition performance in image classification of HVNN-2DLDA, six public image databases are used for experiments. The experimental results demonstrate the effectiveness and the feasibility of the proposed framework.

2-D neighborhood preserving projection (2DNPP) uses 2-D images as feature input instead of 1-D vectors used by neighborhood preserving projection (NPP). 2DNPP requires less computation time than NPP. However, both NPP and 2DNPP use the L2norm as a metric, which is sensitive to noise in data. In this paper, we proposed a novel NPP method called low-rank 2DNPP (LR-2DNPP). This method divided the input data into a component part that encoded low-rank features, and an error part that ensured the noise was sparse. Then, a nearest neighbor graph was learned from the clean data using the same procedure as 2DNPP. To ensure that the features learned by LR-2DNPP were optimal for classification, we combined the structurally incoherent learning and low-rank learning with NPP to form a unified model called discriminative LR-2DNPP (DLR2DNPP). By encoding the structural incoherence of the learned clean data, DLR-2DNPP could enhance the discriminative ability for feature extraction. Theoretical analyses on the convergence and computational complexity of LR-2DNPP and DLR-2DNPP were presented in details. We used seven public image databases to verify the performance of the proposed methods. The experimental results showed the effectiveness of our methods for robust image representation.

Compact hash code learning has been widely applied to fast similarity search owing to its significantly reduced storage and highly efficient query speed. However, it is still a challenging task to learn discriminative binary codes for perfectly preserving the full pairwise similarities embedded in the high-dimensional real-valued features, such that the promising performance can be guaranteed. To overcome this difficulty, in this paper, we propose a novel scalable supervised asymmetric hashing (SSAH) method, which can skillfully approximate the full-pairwise similarity matrix based on maximum asymmetric inner product of two different non-binary embeddings. In particular, to comprehensively explore the semantic information of data, the supervised label information and the refined latent feature embedding are simultaneously considered to construct the high-quality hashing function and boost the discriminant of the learned binary codes. Specifically, SSAH learns two distinctive hashing functions in conjunction of minimizing the regression loss on the semantic label alignment and the encoding loss on the refined latent features. More importantly, instead of using only part of similarity correlations of data, the full-pairwise similarity matrix is directly utilized to avoid information loss and performance degeneration, and its cumbersome computation complexity on n ×n matrix can be dexterously manipulated during the optimization phase. Furthermore, an efficient alternating optimization scheme with guaranteed convergence is designed to address the resulting discrete optimization problem. The encouraging experimental results on diverse benchmark datasets demonstrate the superiority of the proposed SSAH method in comparison with many recently proposed hashing algorithms.

Classical linear discriminant analysis (LDA) has the limitation that it requires the within-class scatter matrix to be nonsingular so that it can perform eigen-decomposition to obtain optimal solutions. To break through this limitation, many methods based on LDA have been proposed. However, these methods are either sensitive to outliers or lack joint sparsity for effective feature extraction. To release these problems, this paper proposes a locally joint sparse marginal embedding (LJSME) method. LJSME reconstructs the scatter matrices and utilizes the locality graph to weigh each pair of data, such that it is robust to outliers and able to preserve the neighborhood relationship of the data. Moreover, LJSME can easily avoid the small sample-size problem by a maximum margin criterion and obtain joint sparsity for effective feature extraction by using joint sparse regularization. The comprehensive analysis between the proposed LJSME and the related methods is presented, which indicates the advantages of the proposed method. A series of experiments was conducted to evaluate the performance of LJSME when compared with the state-of-the-art methods. The MATLAB code of LJSME can be downloaded fromhttps://github.com/TungmeeMo/LJSME.git.

Classical linear regression not only lacks of the flexibility in fitting the label, but also ignores to preserve the intrinsic local geometric structure of data, which leads to overfitting. In this paper, we propose a novel discriminative regression method, called adaptive locality preserving based discriminative regression (ALPDR), to address these problems. Firstly, a locality preserving constraint regularized by the adaptive weight is introduced to preserve the intrinsic geometric structures of data, in which the similar points of the same class are adaptively pulled together by the projection. Secondly, ALPDR directly learns the discriminative target matrix from data based on the given label information, which allows more freedom in label fitting and simultaneously enlarges the margins between different classes. Thirdly, ALPDR imposes a row-sparsity constraint on the projection, which enables the method to adaptively select the most discriminative features from data such that the negative influence of noises and redundant features can be eliminated. Finally, an efficient iterative algorithm is provided to optimize the model. Extensive experiments show that the proposed method outperforms the other state-of-art methods, which proves the effectiveness of the proposed method.

Nonnegative matrix factorization (NMF), known as a famous matrix factorization technique, has been widely used in pattern recognition and computer vision. NMF represents the input data matrix as a product of two nonnegative factors. As NMF is based on the Euclidean distance, which is sensitive to noise or errors in the data, some robust NMF methods are proposed. Mainly focusing on parts-based representation, these robust NMF methods often neglect global representation of data. In fact, the global geometry information of data is more robust than the local information about the noisy data in terms of image classification. In order to effectively improve the robustness of NMF and learn part-based and global representation of the data, a novel method low-rank nonnegative factorization (LRNF) is proposed in this paper. First, we assume that the data are grossly corrupted, and the$L_{1} $norm is used as a sparse constraint on the assumed noise matrix. Then, LRNF learns a low-rank matrix with the global representation ability. Finally, we make a nonnegative factorization of the learned low-rank matrix. We can obtain a base matrix, which preserves locality and globality properties of the data in the meantime. Extensive experiments have been conducted on nine real-world image databases to verify the performance of the proposed LRNF method by comparing with the state-of-the-art algorithms on robust dimensionality reduction.

Ridge regression (RR) and its extended versions are widely used as an effective feature extraction method in pattern recognition. However, the RR-based methods are sensitive to the variations of data and can learn only limited number of projections for feature extraction and recognition. To address these problems, we propose a new method called robust discriminant regression (RDR) for feature extraction. In order to enhance the robustness, the L2,1-norm is used as the basic metric in the proposed RDR. The designed robust objective function in regression form can be solved by an iterative algorithm containing an eigenfunction, through which the optimal orthogonal projections of RDR can be obtained by eigen decomposition. The convergence analysis and computational complexity are presented. In addition, we also explore the intrinsic connections and differences between the RDR and some previous methods. Experiments on some well-known databases show that RDR is superior to the classical and very recent proposed methods reported in the literature, no matter the L2-norm or the L2,1-norm-based regression methods. The code of this paper can be downloaded from http://www.scholat.com/laizhihui.

Recently, hash learning attracts great attentions since it can obtain fast image retrieval on large-scale datasets by using a series of discriminative binary codes. The popular methods include manifold-based hashing methods, which aim to learn the binary codes by embedding the original high-dimensional data into low-dimensional intrinsic subspace. However, most of these methods tend to relax the discrete constraint to compute the final binary codes in an easier way. Therefore, the information loss will increase. In this paper, we propose a novel jointly sparse regression model to minimize the locality information loss and obtain jointly sparse hashing method. The proposed model integrates locality, joint sparsity and rotation operation together with a seamless formulation. Thus, the drawback in previous methods using two separated and independent stages such as PCA-ITQ and the similar methods can be addressed. Moreover, since we introduce the joint sparsity, the feature extraction and jointly sparse feature selection can also be realized in a single projection operation, which has the potentials to select more discriminant features. The convergence of the proposed algorithm is proved, and the essences of the iterative procedures are also revealed. The experimental results on large-scale datasets demonstrate the performance of the proposed method.

Locality preserving projections (LPP) has been widely studied and extended in recent years, because of its promising performance in feature extraction. In this paper, we propose a modified version of the LPP by constructing a novel regression model. To improve the performance of the model, we impose a low-rank constraint on the regression matrix to discover the latent relations between different neighbors. By using the L2,1-norm as a metric for the loss function, we can further minimize the reconstruction error and derive a robust model. Furthermore, the L2,1-norm regularization term is added to obtain a jointly sparse regression matrix for feature selection. An iterative algorithm with guaranteed convergence is designed to solve the optimization problem. To validate the recognition efficiency, we apply the algorithm to a series of benchmark datasets containing face and character images for feature extraction. The experimental results show that the proposed method is better than some existing methods. The code of this paper can be downloaded from http://www.scholat.com/laizhihui.

This paper proposes a novel method, called robust latent subspace learning (RLSL), for image classification. We formulate an RLSL problem as a joint optimization problem over both the latent SL and classification model parameter predication, which simultaneously minimizes: 1) the regression loss between the learned data representation and objective outputs and 2) the reconstruction error between the learned data representation and original inputs. The latent subspace can be used as a bridge that is expected to seamlessly connect the origin visual features and their class labels and hence improve the overall prediction performance. RLSL combines feature learning with classification so that the learned data representation in the latent subspace is more discriminative for classification. To learn a robust latent subspace, we use a sparse item to compensate error, which helps suppress the interference of noise via weakening its response during regression. An efficient optimization algorithm is designed to solve the proposed optimization problem. To validate the effectiveness of the proposed RLSL method, we conduct experiments on diverse databases and encouraging recognition results are achieved compared with many state-of-the-arts methods.

Linear regression (LR) and some of its variants have been widely used for classification problems. Most of these methods assume that during the learning phase, the training samples can be exactly transformed into a strict binary label matrix, which has too little freedom to fit the labels adequately. To address this problem, in this paper, we propose a novel regularized label relaxation LR method, which has the following notable characteristics. First, the proposed method relaxes the strict binary label matrix into a slack variable matrix by introducing a nonnegative label relaxation matrix into LR, which provides more freedom to fit the labels and simultaneously enlarges the margins between different classes as much as possible. Second, the proposed method constructs the class compactness graph based on manifold learning and uses it as the regularization item to avoid the problem of overfitting. The class compactness graph is used to ensure that the samples sharing the same labels can be kept close after they are transformed. Two different algorithms, which are, respectively, based on -norm and -norm loss functions are devised. These two algorithms have compact closed-form solutions in each iteration so that they are easily implemented. Extensive experiments show that these two algorithms outperform the state-of-the-art algorithms in terms of the classification accuracy and running time.

Nonnegative matrix factorization (NMF), which aims at obtaining the nonnegative low-dimensional representation of data, has received wide attention. To obtain more effective nonnegative discriminant bases from the original NMF, in this paper, a novel method called nonnegative discriminant matrix factorization (NDMF) is proposed for image classification. NDMF integrates the nonnegative constraint, orthogonality, and discriminant information in the objective function. NDMF considers the incoherent information of both factors in standard NMF and is proposed to enhance the discriminant ability of the learned base matrix. NDMF projects the low-dimensional representation of the subspace of the base matrix to regularize the NMF for discriminant subspace learning. Based on the Euclidean distance metric and the generalized Kullback-Leibler (KL) divergence, two kinds of iterative algorithms are presented to solve the optimization problem. The between- and within-class scatter matrices are divided into positive and negative parts for the update rules and the proofs of the convergence are also presented. Extensive experimental results demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art discriminant NMF algorithms.

A common intrinsic limitation of the traditional subspace learning methods is the sensitivity to the outliers and the image variations of the object since they use the norm as the metric. In this paper, a series of methods based on the -norm are proposed for linear dimensionality reduction. Since the -norm based objective function is robust to the image variations, the proposed algorithms can perform robust image feature extraction for classification. We use different ideas to design different algorithms and obtain a unified rotational invariant (RI) dimensionality reduction framework, which extends the well-known graph embedding algorithm framework to a more generalized form. We provide the comprehensive analyses to show the essential properties of the proposed algorithm framework. This paper indicates that the optimization problems have global optimal solutions when all the orthogonal projections of the data space are computed and used. Experimental results on popular image datasets indicate that the proposed RI dimensionality reduction algorithms can obtain competitive performance compared with the previous norm based subspace learning algorithms.

This paper proposes a velocity learning method based on the directional Gaussian model to detect speeding events in surveillance scenarios. The proposed method is of an uncalibrated type, but yet has considered the influences of projective transformation on estimating the motion velocities in an image plane, which is convenient and feasible to use in real application. We have analyzed the theory that the velocity in the image plane varies due to the changes of the direction of the moving object with constant velocity in a real world plane. With the support of this theory, we propose to learn the velocities calculated on a certain position in different direction bins to tolerate the effect of projective transformation on velocity modeling. To facilitate the whole framework, two key issues have to be addressed. First, we have designed an improved Fisher model to optimize the direction bins, which reflect the major moving directions in a scenario. Second, we have adopted a weighted sampling strategy and surface fitting to solve the lack of sample problem during the learning process. Experiments conducted on real surveillance videos show that the proposed method can obtain competitive results compared with the state-of-the-art methods.

Robustness to noises, outliers, and corruptions is an important issue in linear dimensionality reduction. Since the sample-specific corruptions and outliers exist, the class-special structure or the local geometric structure is destroyed, and thus, many existing methods, including the popular manifold learning- based linear dimensionality methods, fail to achieve good performance in recognition tasks. In this paper, we focus on the unsupervised robust linear dimensionality reduction on corrupted data by introducing the robust low-rank representation (LRR). Thus, a robust linear dimensionality reduction technique termed low-rank embedding (LRE) is proposed in this paper, which provides a robust image representation to uncover the potential relationship among the images to reduce the negative influence from the occlusion and corruption so as to enhance the algorithm's robustness in image feature extraction. LRE searches the optimal LRR and optimal subspace simultaneously. The model of LRE can be solved by alternatively iterating the argument Lagrangian multiplier method and the eigendecomposition. The theoretical analysis, including convergence analysis and computational complexity, of the algorithms is presented. Experiments on some well-known databases with different corruptions show that LRE is superior to the previous methods of feature extraction, and therefore, it indicates the robustness of the proposed method. The code of this paper can be downloaded from http://www.scholat.com/laizhihui.

In this paper, we aim at learning compact and discriminative linear regression models. Linear regression has been widely used in different problems. However, most of the existing linear regression methods exploit the conventional zero-one matrix as the regression targets, which greatly narrows the flexibility of the regression model. Another major limitation of these methods is that the learned projection matrix fails to precisely project the image features to the target space due to their weak discriminative capability. To this end, we present an elastic-net regularized linear regression (ENLR) framework, and develop two robust linear regression models which possess the following special characteristics. First, our methods exploit two particular strategies to enlarge the margins of different classes by relaxing the strict binary targets into a more feasible variable matrix. Second, a robust elastic-net regularization of singular values is introduced to enhance the compactness and effectiveness of the learned projection matrix. Third, the resulting optimization problem of ENLR has a closed-form solution in each iteration, which can be solved efficiently. Finally, rather than directly exploiting the projection matrix for recognition, our methods employ the transformed features as the new discriminate representations to make final image classification. Compared with the traditional linear regression model and some of its variants, our method is much more accurate in image classification. Extensive experiments conducted on publicly available data sets well demonstrate that the proposed framework can outperform the state-of-the-art methods. The MATLAB codes of our methods can be available at http://www.yongxu.org/lunwen.html.

Two-dimensional locality preserving projections (2DLPP) that use 2D image representation in preserving projection learning can preserve the intrinsic manifold structure and local information of data. However, 2DLPP is based on the Euclidean distance, which is sensitive to noise and outliers in data. In this paper, we propose a novel locality preserving projection method called nuclear norm-based two-dimensional locality preserving projections (NN-2DLPP). First, NN-2DLPP recovers the noisy data matrix through low-rank learning. Second, noise in data is removed and the learned clean data points are projected on a new subspace. Without the disturbance of noise, data points belonging to the same class are kept as close to each other as possible in the new projective subspace. Experimental results on six public image databases with face recognition, object classification, and handwritten digit recognition tasks demonstrated the effectiveness of the proposed method.

Locality and label information of training samples play an important role in image classification. However, previous dictionary learning algorithms do not take the locality and label information of atoms into account together in the learning process, and thus their performance is limited. In this paper, a discriminative dictionary learning algorithm, called the locality-constrained and label embedding dictionary learning (LCLE-DL) algorithm, was proposed for image classification. First, the locality information was preserved using the graph Laplacian matrix of the learned dictionary instead of the conventional one derived from the training samples. Then, the label embedding term was constructed using the label information of atoms instead of the classification error term, which contained discriminating information of the learned dictionary. The optimal coding coefficients derived by the locality-based and label-based reconstruction were effective for image classification. Experimental results demonstrated that the LCLE-DL algorithm can achieve better performance than some state-of-the-art algorithms.

This paper presents a novel image feature representation method, called multi-channel micro-structure difference descriptor (MCMSDD) for image retrieval. With the local feature extraction from a micro-structure and MAX operator, MCMSDD integrates the advantages of multi-channel local binary encoding and color difference histogram , which are the fusion of color, texture and spatial distribution information. Although it extracts feature from full color image, the dimension of the feature vector is relatively low without learning and segmentation. To improve the performance of retrieval, a simple re-ranking algorithm is employed. Finally, the proposed MCMSDD is extensively tested on Corel-2K and Washington datasets, and the experimental results show that the proposed MCMSDD is more effective than the state-of-the-art.

Low-rank representation (LRR) has been successfully applied in exploring the subspace structures of data. However, in previous LRR-based semi-supervised subspace clustering methods, the label information is not used to guide the affinity matrix construction so that the affinity matrix cannot deliver strong discriminant information. Moreover, these methods cannot guarantee an overall optimum since the affinity matrix construction and subspace clustering are often independent steps. In this paper, we propose a robust semi-supervised subspace clustering method based on non-negative LRR (NNLRR) to address these problems. By combining the LRR framework and the Gaussian fields and harmonic functions method in a single optimization problem, the supervision information is explicitly incorporated to guide the affinity matrix construction and the affinity matrix construction and subspace clustering are accomplished in one step to guarantee the overall optimum. The affinity matrix is obtained by seeking a non-negative low-rank matrix that represents each sample as a linear combination of others. We also explicitly impose the sparse constraint on the affinity matrix such that the affinity matrix obtained by NNLRR is non-negative low-rank and sparse. We introduce an efficient linearized alternating direction method with adaptive penalty to solve the corresponding optimization problem. Extensive experimental results demonstrate that NNLRR is effective in semi-supervised subspace clustering and robust to different types of noise than other state-of-the-art methods.

As one of the most popular dimensionality reduction techniques, locality preserving projections (LPP) has been widely used in computer vision and pattern recognition. However, in practical applications, data is always corrupted by noises. For the corrupted data, samples from the same class may not be distributed in the nearest area, thus LPP may lose its effectiveness. In this paper, it is assumed that data is grossly corrupted and the noise matrix is sparse. Based on these assumptions, we propose a novel dimensionality reduction method, named low-rank preserving projections (LRPP) for image classification. LRPP learns a low-rank weight matrix by projecting the data on a low-dimensional subspace. We use the L21 norm as a sparse constraint on the noise matrix and the nuclear norm as a low-rank constraint on the weight matrix. LRPP keeps the global structure of the data during the dimensionality reduction procedure and the learned low rank weight matrix can reduce the disturbance of noises in the data. LRPP can learn a robust subspace from the corrupted data. To verify the performance of LRPP in image dimensionality reduction and classification, we compare LRPP with the state-of-the-art dimensionality reduction methods. The experimental results show the effectiveness and the feasibility of the proposed method with encouraging results.

Low-rank matrix completion aims to recover a matrix from a small subset of its entries and has received much attention in the field of computer vision. Most existing methods formulate the task as a low-rank matrix approximation problem. A truncated nuclear norm has recently been proposed as a better approximation to the rank of matrix than a nuclear norm. The corresponding optimization method, truncated nuclear norm regularization (TNNR), converges better than the nuclear norm minimization-based methods. However, it is not robust to the number of subtracted singular values and requires a large number of iterations to converge. In this paper, a TNNR method based on weighted residual error (TNNR-WRE) for matrix completion and its extension model (ETNNR-WRE) are proposed. TNNR-WRE assigns different weights to the rows of the residual error matrix in an augmented Lagrange function to accelerate the convergence of the TNNR method. The ETNNR-WRE is much more robust to the number of subtracted singular values than the TNNR-WRE, TNNR alternating direction method of multipliers, and TNNR accelerated proximal gradient with Line search methods. Experimental results using both synthetic and real visual data sets show that the proposed TNNR-WRE and ETNNR-WRE methods perform better than TNNR and Iteratively Reweighted Nuclear Norm (IRNN) methods.

Locally linear embedding (LLE) is one of the most well-known manifold learning methods. As the representative linear extension of LLE, orthogonal neighborhood preserving projection (ONPP) has attracted widespread attention in the field of dimensionality reduction. In this paper, a unified sparse learning framework is proposed by introducing the sparsity or L1-norm learning, which further extends the LLE-based methods to sparse cases. Theoretical connections between the ONPP and the proposed sparse linear embedding are discovered. The optimal sparse embeddings derived from the proposed framework can be computed by iterating the modified elastic net and singular value decomposition. We also show that the proposed model can be viewed as a general model for sparse linear and nonlinear (kernel) subspace learning. Based on this general model, sparse kernel embedding is also proposed for nonlinear sparse feature extraction. Extensive experiments on five databases demonstrate that the proposed sparse learning framework performs better than the existing subspace learning algorithm, particularly in the cases of small sample sizes.

In this paper, we propose a novel unsupervised dimensionality reduction (DR) method called orthogonal self-guided similarity preserving projections (OSSPP), which seamlessly integrates the procedures of an adjacency graph learning and DR into a one step. Specifically, OSSPP projects the data into a low-dimensional subspace and simultaneously performs similarity preserving learning by using the similarity preserving regularization term in which the reconstruction coefficients of the projected data are used to encode the similarity structure information. An interesting finding is that the problem to determine the reconstruction coefficients can be converted into a weighted non-negative sparse coding problem without any explicit sparsity constraint. Thus the projections obtained by OSSPP contain natural discriminating information. Experimental results demonstrate that OSSPP outperforms state-of-the-art methods in DR.

Microarray techniques have been used to delineate cancer groups or to identify candidate genes for cancer prognosis. As such problems can be viewed as classification ones, various classification methods have been applied to analyze or interpret gene expression data. In this paper, we propose a novel method based on robust principal component analysis (RPCA) to classify tumor samples of gene expression data. Firstly, RPCA is utilized to highlight the characteristic genes associated with a special biological process. Then, RPCA and RPCA+LDA (robust principal component analysis and linear discriminant analysis) are used to identify the features. Finally, support vector machine (SVM) is applied to classify the tumor samples of gene expression data based on the identified features. Experiments on seven data sets demonstrate that our methods are effective and feasible for tumor classification.

Tensor-based object recognition has been widely studied in the past several years. This paper focuses on the issue of joint feature selection from the tensor data and proposes a novel method called joint tensor feature analysis (JTFA) for tensor feature extraction and recognition. In order to obtain a set of jointly sparse projections for tensor feature extraction, we define the modified within-class tensor scatter value and the modified between-class tensor scatter value for regression. The k-mode optimization technique and the L(2,1)-norm jointly sparse regression are combined together to compute the optimal solutions. The convergent analysis, computational complexity analysis and the essence of the proposed method/model are also presented. It is interesting to show that the proposed method is very similar to singular value decomposition on the scatter matrix but with sparsity constraint on the right singular value matrix or eigen-decomposition on the scatter matrix with sparse manner. Experimental results on some tensor datasets indicate that JTFA outperforms some well-known tensor feature extraction and selection algorithms.

Previous graph-based semisupervised learning (G-SSL) methods have the following drawbacks: 1) they usually predefine the graph structure and then use it to perform label prediction, which cannot guarantee an overall optimum and 2) they only focus on the label prediction or the graph structure construction but are not competent in handling new samples. To this end, a novel nonnegative sparse graph (NNSG) learning method was first proposed. Then, both the label prediction and projection learning were integrated into linear regression. Finally, the linear regression and graph structure learning were unified within the same framework to overcome these two drawbacks. Therefore, a novel method, named learning a NNSG for linear regression was presented, in which the linear regression and graph learning were simultaneously performed to guarantee an overall optimum. In the learning process, the label information can be accurately propagated via the graph structure so that the linear regression can learn a discriminative projection to better fit sample labels and accurately classify new samples. An effective algorithm was designed to solve the corresponding optimization problem with fast convergence. Furthermore, NNSG provides a unified perceptiveness for a number of graph-based learning methods and linear regression methods. The experimental results showed that NNSG can obtain very high classification accuracy and greatly outperforms conventional G-SSL methods, especially some conventional graph construction methods.

Over the past few decades, a large number of algorithms have been developed for dimensionality reduction. Despite the different motivations of these algorithms, they can be interpreted by a common framework known as graph embedding. In order to explore the significant features of data, some sparse regression algorithms have been proposed based on graph embedding. However, the problem is that these algorithms include two separate steps: (1) embedding learning and (2) sparse regression. Thus their performance is largely determined by the effectiveness of the constructed graph. In this paper, we present a framework by combining the objective functions of graph embedding and sparse regression so that embedding learning and sparse regression can be jointly implemented and optimized, instead of simply using the graph spectral for sparse regression. By the proposed framework, supervised, semisupervised, and unsupervised learning algorithms could be unified. Furthermore, we analyze two situations of the optimization problem for the proposed framework. By adopting an ℓ2,1-norm regularization for the proposed framework, it can perform feature selection and subspace learning simultaneously. Experiments on seven standard databases demonstrate that joint graph embedding and sparse regression method can significantly improve the recognition performance and consistently outperform the sparse regression method.

This paper aims to explore the optimal feature selection with dimensionality reduction and jointly sparse representation scheme for classification. The proposed method is called Optimal Feature Selection Classification (OFSC). Our model simultaneously learns an orthogonal subspace for jointly sparse feature selection and representation via l2,1-norms regularization. To solve the proposed model, an alternately iterative algorithm is proposed to optimize both the jointly sparse projection matrix and representation matrix. Experimental results on three public face datasets and one action dataset validate the quick convergence of our algorithm and show that the proposed method is more competitive than the state-of-the-art methods.

As an important biometric feature, human gait has great potential in video-surveillance-based applications. In this paper, we focus on the matrix representation-based human gait recognition and propose a novel discriminant subspace learning method called sparse bilinear discriminant analysis (SBDA). SBDA extends the recently proposed matrix-representation-based discriminant analysis methods to sparse cases. By introducing the L1and L2norms into the objective function of SBDA, two interrelated sparse discriminant subspaces can be obtained for gait feature extraction. Since the optimization problem has no closed-form solutions, an iterative method is designed to compute the optimal sparse subspace using the L1and L2norms sparse regression. Theoretical analyses reveal the close relationship between SBDA and previous matrix-representation-based discriminant analysis methods. Since each nonzero element in each subspace is selected from the most important variables/factors, SBDA is potential to perform equivalent to or even better than the state-of-the-art subspace learning methods in gait recognition. Moreover, using the strategy of SBDA plus linear discriminant analysis (LDA), we can further improve the performance. A set of experiments on the standard USF HumanID and CASIA gait databases demonstrate that the proposed SBDA and SBDA + LDA can obtain competitive performance.

The recently proposed robust principle component analysis (RPCA) has been successfully applied in background subtraction. However, low-rank decomposition makes sense on the condition that the foreground pixels (sparsity patterns) are uniformly located at the scene, which is not realistic in real-world applications. To overcome this limitation, we reconstruct the input video frames and aim to make the foreground pixels not only sparse in space but also sparse in time. Therefore, we propose a joint video frame set division and RPCA-based method for background subtraction. In addition, we use the motion as a priori knowledge which has not been considered in the current subspace-based methods. The proposed method consists of two phases. In the first phase, we propose a lower bound-based within-class maximum division method to divide the video frame set into several subsets. In this way, the successive frames are assigned to different subsets in which the foregrounds are located at the scene randomly. In the second phase, we augment each subset using the frames with a small quantity of motion. To evaluate the proposed method, the experiments are conducted on real-world and public datasets. The comparisons with the state-of-the-art background subtraction methods validate the superiority of our method.

Representation-based classification methods are all constructed on the basis of the conventional representation, which first expresses the test sample as a linear combination of the training samples and then exploits the deviation between the test sample and the expression result of every class to perform classification. However, this deviation does not always well reflect the difference between the test sample and each class. With this paper, we propose a novel representation-based classification method for face recognition. This method integrates conventional and the inverse representation-based classification for better recognizing the face. It first produces conventional representation of the test sample, i.e., uses a linear combination of the training samples to represent the test sample. Then it obtains the inverse representation, i.e., provides an approximation representation of each training sample of a subject by exploiting the test sample and training samples of the other subjects. Finally, the proposed method exploits the conventional and inverse representation to generate two kinds of scores of the test sample with respect to each class and combines them to recognize the face. The paper shows the theoretical foundation and rationale of the proposed method. Moreover, this paper for the first time shows that a basic nature of the human face, i.e., the symmetry of the face can be exploited to generate new training and test samples. As these new samples really reflect some possible appearance of the face, the use of them will enable us to obtain higher accuracy. The experiments show that the proposed conventional and inverse representation-based linear regression classification (CIRLRC), an improvement to linear regression classification (LRC), can obtain very high accuracy and greatly outperforms the naive LRC and other state-of-the-art conventional representation based face recognition methods. The accuracy of CIRLRC can be 10% greater than that of LRC.

We modify the conventional principal component analysis (PCA) and propose a novel subspace learning framework, modified PCA (MPCA), using multiple similarity measurements. MPCA computes three similarity matrices exploiting the similarity measurements: 1) mutual information; 2) angle information; and 3) Gaussian kernel similarity. We employ the eigenvectors of similarity matrices to produce new subspaces, referred to as similarity subspaces. A new integrated similarity subspace is then generated using a novel feature selection approach. This approach needs to construct a kind of vector set, termed weak machine cell (WMC), which contains an appropriate number of the eigenvectors spanning the similarity subspaces. Combining the wrapper method and the forward selection scheme, MPCA selects a WMC at a time that has a powerful discriminative capability to classify samples. MPCA is very suitable for the application scenarios in which the number of the training samples is less than the data dimensionality. MPCA outperforms the other state-of-the-art PCA-based methods in terms of both classification accuracy and clustering result. In addition, MPCA can be applied to face image reconstruction. MPCA can use other types of similarity measurements. Extensive experiments on many popular real-world data sets, such as face databases, show that MPCA achieves desirable classification results, as well as has a powerful capability to represent data.

Multilinear/tensor extensions of manifold learning based algorithms have been widely used in computer vision and pattern recognition. This paper first provides a systematic analysis of the multilinear extensions for the most popular methods by using alignment techniques, thereby obtaining a general tensor alignment framework. From this framework, it is easy to show that the manifold learning based tensor learning methods are intrinsically different from the alignment techniques. Based on the alignment framework, a robust tensor learning method called sparse tensor alignment (STA) is then proposed for unsupervised tensor feature extraction. Different from the existing tensor learning methods, L1- and L2-norms are introduced to enhance the robustness in the alignment step of the STA. The advantage of the proposed technique is that the difficulty in selecting the size of the local neighborhood can be avoided in the manifold learning based tensor feature extraction algorithms. Although STA is an unsupervised learning method, the sparsity encodes the discriminative information in the alignment step and provides the robustness of STA. Extensive experiments on the well-known image databases as well as action and hand gesture databases by encoding object images as tensors demonstrate that the proposed STA algorithm gives the most competitive performance when compared with the tensor-based unsupervised learning methods.

In this brief, multilinear sparse principal component analysis (MSPCA) is proposed for feature extraction from the tensor data. MSPCA can be viewed as a further extension of the classical principal component analysis (PCA), sparse PCA (SPCA) and the recently proposed multilinear PCA (MPCA). The key operation of MSPCA is to rewrite the MPCA into multilinear regression forms and relax it for sparse regression. Differing from the recently proposed MPCA, MSPCA inherits the sparsity from the SPCA and iteratively learns a series of sparse projections that capture most of the variation of the tensor data. Each nonzero element in the sparse projections is selected from the most important variables/factors using the elastic net. Extensive experiments on Yale, Face Recognition Technology face databases, and COIL-20 object database encoded the object images as second-order tensors, and Weizmann action database as third-order tensors demonstrate that the proposed MSPCA algorithm has the potential to outperform the existing PCA-based subspace learning algorithms.

The classical linear discriminant analysis has undergone great development and has recently been extended to different cases. In this paper, a novel discriminant subspace learning method called sparse tensor discriminant analysis (STDA) is proposed, which further extends the recently presented multilinear discriminant analysis to a sparse case. Through introducing the L1 and L2 norms into the objective function of STDA, we can obtain multiple interrelated sparse discriminant subspaces for feature extraction. As there are no closed-form solutions, k-mode optimization technique and the L1 norm sparse regression are combined to iteratively learn the optimal sparse discriminant subspace along different modes of the tensors. Moreover, each non-zero element in each subspace is selected from the most important variables/factors, and thus STDA has the potential to perform better than other discriminant subspace methods. Extensive experiments on face databases (Yale, FERET, and CMU PIE face databases) and the Weizmann action database show that the proposed STDA algorithm demonstrates the most competitive performance against the compared tensor-based methods, particularly in small sample sizes.

Two-dimensional (2-D) image-matrix-based projection methods for feature extraction are widely used in many fields of computer vision and pattern recognition. In this paper, we propose a novel framework called sparse 2-D projections (S2DP) for image feature extraction. Different from the existing 2-D feature extraction methods, S2DP iteratively learns the sparse projection matrix by using elastic net regression and singular value decomposition. Theoretical analysis shows that the optimal sparse subspace approximates the eigensubspace obtained by solving the corresponding generalized eigenequation. With the S2DP framework, many 2-D projection methods can be easily extended to sparse cases. Moreover, when each row/column of the image matrix is regarded as an independent high-dimensional vector (1-D vector), it is proven that the vector-based eigensubspace is also approximated by the sparse subspace obtained by the same method used in this paper. Theoretical analysis shows that, when compared with the vector-based sparse projection learning methods, S2DP greatly saves both computation and memory costs. This property makes S2DP more tractable for real-world applications. Experiments on well-known face databases indicate the competitive performance of the proposed S2DP over some 2-D projection methods when facial expressions, lighting conditions, and time vary.

In this paper, a novel method called tangent space discriminant analysis is proposed for dimensionality reduction and feature extraction. Differing from the recently proposed manifold learning methods completely operating on raw feature space, TSDA completely uses the local tangent space to represent the local within-class geometry and local between-class geometry. Assume that the face images of different people reside on different intrinsically low-dimensional sub-manifolds, TSDA is developed to preserve the locality of each sub-manifold and simultaneously maximize the local separability of different sub-manifolds by using local tangent space alignment. Experimental results show that TSDA achieves higher recognition rates than a few the state-of-the-art techniques.

One of the major disadvantages of the linear dimensionality reduction algorithms, such as Principle Component Analysis (PCA) and Linear Discriminant Analysis (LDA), are that the projections are linear combination of all the original features or variables and all weights in the linear combination known as loadings are typically non-zero. Thus, they lack physical interpretation in many applications. In this paper, we propose a novel supervised learning method called Sparse Local Discriminant Projections (SLDP) for linear dimensionality reduction. SLDP introduces a sparse constraint into the objective function and obtains a set of sparse projective axes with directly physical interpretation. The sparse projections can be efficiently computed by the Elastic Net combining with spectral analysis. The experimental results show that SLDP give the explicit interpretation on its projections and achieves competitive performance compared with some dimensionality reduction techniques.

The general computational models of visual attention are to obtain multi-scale feature maps in terms of visual properties like intensity, color and orientation, and then combine them to get one saliency map. But due to the lack of object edge information and reasonable feature combination strategy, the visual saliency map of the image is a blur map. Being aware of these, we propose a new scheme for saliency extraction. In this paper, we firstly put forward a sparse embedding feature combination strategy, inspired by sparse representation. The strategy is used to combine the salient regions from the individual feature maps based on a novel feature sparse indicator that measures the contribution of each map to saliency. Then we combine traditional visual attention with edge information. Results on different scene images show that our method outperforms other traditional feature combination strategies.