Few-shot image classification (FSIC) aims to recognize novel categories from only a few labeled examples, making it inherently challenging under limited supervision. Existing approaches have attempted to alleviate this issue by incorporating explicit semantics like class names or knowledge graphs to guide learning. However, such methods often encounter semantic ambiguity due to their dependence on either overly simplistic semantic priors or resource-intensive external knowledge sources, which limits their potential. In this paper, we explore the frequency domain as an implicit and task-adaptive source of semantic information. We propose F2SST, a Frequency-to-Spatial Semantic Transfer framework that enhances feature learning by leveraging spectral signals as hidden semantics. Specifically, F2SST applies Fast Fourier Transform (FFT) to extract phase-invariant global frequency descriptors, followed by a lightweight Gated Spectral Attention (GSA) module that selectively emphasizes class-relevant frequency components. These enhanced spectral cues are then integrated into the spatial stream through a class-guided fusion mechanism, enabling more robust and semantically aligned representations. Extensive experiments on four standard benchmarks (miniImageNet, tieredImageNet, CIFAR-FS and FC100) demonstrate that F2SST consistently improves performance, validating the effectiveness of frequency-domain semantics in FSIC.

Accurately estimating the number of objects across different categories is crucial in remote sensing images for applications ranging from urban planning and environmental monitoring to disaster management. Although there are a few studies about multi-class object counting for remote sensing images, these methods may suffer from the issues of class imbalance, inter-class interference, or inability to capture fine-grained features. Thus, we propose a novel network called Multi-class Object Counting Network with Adaptive Class Alignment loss (ACA-MOCN) for multi-class object counting in remote sensing images. ACA-MOCN newly designs a Adaptive Class Alignment (ACA) loss function that can effectively balance the losses across multi-class objects and reduce interference between classes. In addition, ACA-MOCN also designs a Filtering Feature Pyramid Network (FFPN) structure to facilitate an effective feature fusion mechanism, enabling the integration of granular and detailed information. Experimental results on the two public datasets demonstrate that ACA-MOCN is superior when handling large-scale multi-class object counting tasks in remote sensing images.

At present, EEG-based emotion recognition has advanced significantly. However, models' discrimination ability across subjects remains limited because of ignoring the electrode structure and the cross-subject differences in EEG signals. To address this, we propose a discrimination-enhanced hierarchical network with structure optimization (SO-DHNet), a discrimination-enhanced hierarchical network with structure optimization for cross-subject EEG emotion recognition. It includes a hierarchical data generation (HDG) module that organizes EEG electrodes into five neuroanatomical regions, optimizes their structure, and generates 4D hierarchical data. Additionally, we introduce a random pairing training (RPT) strategy with a pairwise-similarity binary cross-entropy (PS-BCE) loss to maximize inter-class diversity and minimize intra-class variability. Experiments on SEED confirm SO-DHNet's effectiveness in cross-subject emotion recognition.

Entity Alignment (EA) aims to identify equivalent entities across multiple Knowledge Graphs (KGs). However, when applied to larger-scale KGs, most existing EA approaches suffer from the scalability issue due to excessive GPU memory and time consumption. To mitigate this, recent advances have introduced the Large-scale EA (LsEA) task, which divides large-scale KG pairs into smaller sub-graph pairs. Despite their promising results, several notable challenges remain, preventing these advances from achieving optimal performance: 1) How to effectively utilize critical structures when generating sub-tasks? 2) How to supplement high-quality context to enhance LsEA performance? 3) How to address scenarios without alignment seeds? To tackle these challenges, we propose a novel method called ELsEA. It comprises three main components: (1) Source and Target Graph Partition, using a Metis-based weighted partitioner and a counter-part candidate generator to partition source and target graphs respectively, aiming to utilize critical structures effectively; (2) Supplement High-quality Context, which utilizes a value-based informativeness-evaluation module and a neighbor enrichment module to assess each entity's informativeness effectively, then supplement high-quality context based on this informativeness; and (3) Seed-free Setup, introducing a mixed-info pseudo-seed generation strategy to mitigate name bias, generating accurate pseudo-seeds when alignment seeds are unavailable. Extensive experiments demonstrate that ELsEA outperforms state-of-the-art baselines. The code of ELsEA is available online11https://githuh.com/wx-qzhou/ELsEA.git.

Using low-rank adaptation to fine-tuning pretrained neural network models has attracted widespread attention due to its advantages of low resource requirements, high precision, and no additional inference delay. However, most existing methods are designed for large language models based on Transformer structure and lack adaptation to convolutional neural networks (CNN) widely used on Internet of Things (IoT) devices. In addition, IoT devices are usually deployed outdoors and collect a large amount of data affected by the environment. Providing a highly robust model is a prerequisite for providing high-quality services. To this end, this paper proposes a highly robust seed-filter adaptation method (SeFA) for pre-trained CNNs. SeFA introduces an adaptation branch with the same structure as the backbone network. In the adaptation branch, some filters are first designated seed filters and grouped. Then, other filters are generated based on the grouped seed filters and nonlinear transformation functions (NLFs) with different hyperparameters. The parameters of the seed filters are updated with model training, and the hyperparameters of the NLFs are randomly initialized and frozen. Both grouping seed filters and configuring NLFs with nonlearnable hyperparameters can improve the robustness of the model. This is because grouping seed filters can generate diverse filters on demand without increasing the model's complexity, and the NLFs' rules can regularize the model. The key idea of this paper is to propose SeFA with flexible controllable learnable parameters, high robustness and adaptability to pretrained CNN models, to facilitate fine-tuning of pre-trained CNN models on resource-constrained IoT devices to provide highly robust visual services. Experimental results on the CIFAR-10, CIFAR-10-C, CIFAR-100, CIFAR-100-C, and Icons50 datasets demonstrate that the proposed SeFA outperforms other state-of-the-art methods. Specifically, based on the ResNet152, on the CIFAR-10-C dataset, the accuracy of our SeFA is about$+7 {\%}$higher than that of the full fine-tuning method.

Multi-modal recommender systems (MMRSs) have demonstrated significant potential in mitigating data sparsity and cold start problems by leveraging diverse multi-modal data, such as text and images. To further improve the recommendation accuracy, some MMRSs have integrated knowledge graphs (KGs) to enrich the graph structure with meaningful relationships between entities, giving rise to the task of KG-based MMRSs. Despite the promising results achieved by existing studies on this task, (i) they overlook the substantial noise introduced within the auxiliary information (i.e., both KGs and multi-modal data), and (ii) most of them struggle to effectively align the knowledge from history user-item interactions and auxiliary information. To tackle these limitations, we propose a novel model entitled MAGIC (noise Mitigation and knowledge Aignment for knowledge Graph-based multI-modal reCommendation). Specifically, to tackle the limitation (i), we design a noise-aware heterogeneous aggregation layer in the KG-based modal enhancement module. To address the limitation (ii), MAGIC introduces adversarial learning in the CF-based adversarial learning module, and exploits contrastive learning in the fusion and prediction module. The experiments conducted on two extended real-world datasets from different domains demonstrate the superiority of MAGIC over state-of-the-art baselines.

Multi-Modal Knowledge Graphs (MMKGs), comprising relational triples and related multi-modal data (e.g., text and images), usually suffer from the problems of low coverage and incompleteness. To mitigate this, existing studies introduce a fundamental MMKG fusion task, i.e., Multi-Modal Entity Alignment (MMEA) that identifies equivalent entities across multiple MMKGs. Despite MMEA's significant advancements, effectively integrating MMKGs remains challenging, mainly stemming from two core limitations: 1) entity ambiguity, where real-world entities across different MMKGs may possess multiple corresponding counterparts or alternative identities; and 2) severe noise within multi-modal data. To tackle these limitations, a new task MMER (Multi-Modal Entity Resolution), which expands the scope of MMEA to encompass entity ambiguity, is introduced. To tackle this task effectively, we develop a novel model ADMH-ER (Adaptive Denoising Multi-modal Hybrid for Entity Resolution) that incorporates several crucial modules: 1) multi-modal knowledge encoders, which are crafted to obtain entity representations based on multi-modal data sources; 2) an adaptive denoising multi-modal hybrid module that is designed to tackle challenges including noise interference, multi-modal heterogeneity, and semantic irrelevance across modalities; and 3) a hierarchical multi-objective learning strategy, which is proposed to ensure diverse convergence capabilities among different learning objectives. Experimental results demonstrate that ADMH-ER outperforms state-of-the-art methods.

Deep learning models, including convolution-based and self-attention-based models, have been highly successful applied to remote sensing object detection owing to their powerful performance. However, convolution-based models are constrained by the locality of convolution, thereby limiting their detection performance; self-attention-based methods require a large number of training samples to achieve good performance and have low computational efficiency, which hinders their application in real-time detection tasks. To address these issues, we propose an enhanced partial feature aggregation network (EPFA-Net) for remote sensing object detection. In EPFA-Net, we design an enhanced partial feature aggregation (EPFA) module that consists of two parts. The first part is an efficient layer aggregation block that is used to extract local features. The second part adopts a partial feature aggregation (PFA) structure, enhancing gradient efficiency, and a shift-window attention block to extract global features, establishing long-range dependencies. Extensive experiments are conducted on two challenging remote sensing object detection datasets. Experimental results indicate that EPFA-Net is superior to four mainstream methods in detection performance. In addition, findings from ablation experiments suggest that EPFA-Net is efficient in inference speed because of the utilization of the PFA structure.

Medical report images have become the main information resource in intelligent medical treatment, online consultation, and medical research. While facilitating the use of relevant personnel, information security and privacy protection are particularly important. At present, there are related technologies for image desensitization or medical text desensitization, but there is a lack of research on Chinese medical report image desensitization. Thus, this paper proposes a desensitization framework for sensitive information (SIDF) in Chinese medical report images. SIDF contains five modules: sensitive information construction, text detection and text recognition, sensitive information extraction, sensitive information location, and sensitive information desensitization. Because all modules in SIDF can be flexibly designed according to specific requirements, it is convenient to use SIDF. In addition, we design an index of valid desensitization (VD) for measuring the desensitization performance of algorithms. We evaluate SIDF on the Chinese medical report images collected from Internet and private contributions. Experimental results show that our proposed desensitization framework is effective.

Limit order books (LOBs) can reflect the order trading situation and the interactive information of the financial market; thus, LOBs have been widely adopted to forecast the future trend of stocks. Existing methods either do not make full use of LOBs’ information or lack adaptability to the information when constructing a model for trend prediction. Therefore, this paper proposes an ensemble network based on position attention mechanism (PAM-ENet) for trend prediction of LOBs. In PAM-ENet, we fuse three sub-models: a convolutional neural network (CNN) and two gated recurrent unit (GRU) networks, which are used to deal with the basic, time-insensitive, and time-sensitive features information of LOBs, respectively. To integrate outputs of three sub-models, we design a fusion module based on the position attention mechanism (PAM) and use it twice. First, two GRU sub-models are fused. Then, the output vector of CNN sub-model is fused with that of the first PAM fusion module. PAM-ENet can not only capture locality from the CNN sub-model and global property from two GRU sub-models, but also obtain the dependency between elements at any position of output vectors by using PAM fusion modules. Experimental results on FI-2010 show that PAM-ENet can extract significant features and discriminate the price trend well.

Graph attention neural network (GAT) stands as a fundamental model within graph neural networks, extensively employed across various applications. It assigns different weights to different nodes for feature aggregation by comparing the similarity of features between nodes. However, as the amount and density of graph data increases, GAT’s computational demands rise steeply. In response, we present FastGAT, a simpler and more efficient graph attention neural network with global-aware adaptive computational node attention. FastGAT assigns a trainable attention weight to each node and updates it adaptively. Experiments show that FastGAT reduces training time on eight public datasets by 6.22% to 19.50% while maintaining the same performance. The code is available via https://github.com/szYang2000/FastGAT.

Crowd counting plays an important role in computer vision. However, existing methods mainly depend on largescale samples and then make a success in low and medium dense scenes. In addition, current methods suffer from noise introduced by generating density maps and lose the crowd location information. To remedy these issues, we propose a multihead multi-scale pixel localization network (M2PLNet), which can effectively localize the crowds and perform well in the counting task with highly dense and small-scale samples. To address challenges posed by occlusion, noise, and other factors, we design the centralized asymptotic feature pyramid (CAFP) module. This module effectively fuses information from multiple scales, enhancing the capability to identify and count individuals across various angles and distances. Our approach concludes with results obtained through four sets of classification and regression heads. As confirmed by a large number of experiments, M2PLNet achieves the state-of-the-art results on two datasets (ShangHaiTech PartA and UCF CC 50) that are highly dense and with small-scale samples, which fully demonstrates the effectiveness and superiority of M2PLNet. Code is available at: https://github.com/Jason-Mar1/M2PLNet/tree/master.

Graph neural networks (GNNs) have achieved success in social recommender systems due to its capacity of capturing high-order collaborative filtering effects. However, some issues, such as vector scale distortion and over-smoothing, are obstacles to improving the performance of GNN-based models. This paper investigates these two issues and proposes an enhanced self-supervised graph learning (ESGL) algorithm based on GNN for social recommendation. By introducing a scale regulation module, ESGL can solve the issue of vector scale distortion. The scale regulation module takes the output of GNN as its input and generates the scaling matrix that limits the scale of embeddings. In addition, ESGL adopts two kinds of self-supervised learning schemes to enhance the recommendation performance. One scheme is to learn the similarity between users, while the other scheme is the embedding-level contrastive learning. The latter regards the initial normalized embedding of a user node and its average normalized embedding generated in the rest layers as positive samples. Experimental results have shown that our model is effective in improving recommendations.

Abstract Software defect prediction (SDP) aims to distinguish between defective and nondefective instances, but the imbalance between these two classes often leads to reduced prediction performance. Conventional SDP approaches use oversampling techniques, such as synthetic oversampling, to tackle the problem of imbalanced data. However, these methods merely synthesize new instances based on traditional code features without considering actual defects at the code level. To address the issue of data imbalance while preserving semantic features of code samples, a mutation‐based data augmentation approach in SDP is proposed. The method utilizes the mutation operator to generate mutants that mutate nondefective instances and create new defective instances. Six projects from the PROMISE dataset are used to evaluate the approach, employing four traditional and two deep classifiers. The experimental results demonstrate the effectiveness of this method in improving defect prediction performance for both traditional and deep classifiers compared with other data augmentation methods.

Few-shot learning for image classification task aims to classify images from several novel classes with limited number of samples. Recent studies have shown that the deep local descriptors have better representation ability than image-level features, and achieve great success. However, most of these methods often use all local descriptors or over-screening local descriptors for classification. The former contains some task-irrelevant descriptors, which may misguide the final classification result. The latter is likely to lose some key descriptors. In this paper, we propose a novel Task-Aware Discriminant local descriptors Network (TADNet) to address these issues, which can adaptively select the discriminative query descriptors and eliminate the task-irrelevant query descriptors among the entire task. Specifically, TADNet assigns a value to each query descriptor by comparing its similarity to all support classes to represent its discriminant power for classification. Then the discriminative query descriptors can be preserved via a task-aware attention map. Extensive experiments on both fine-grained and generalized datasets demonstrate that the proposed TADNet outperforms the existing state-of-the-art methods.

Nantong blue calico is a kind of important intangible cultural heritages in China. To better safeguard and inherit it in a digital way, it is necessary to construct a large-scale dataset for Nantong blue calico. As so far, however, we could not find a public dataset for blue calico. The goal of this paper is to give a public image dataset which named$N$tBC consisting of Nantong blue calico patterns and provide a baseline result for the recognition of Nantong blue calico patterns. In this paper, we perform several baseline experiments on the NtBC dataset, including handcrafted and deep feature based classification methods. we compare some handcrafted methods and four kinds of popular convolutional neural networks (CNNs), including ResNet-50, AlexNet, GoogLeNet-V1 and VGGNet-16. Experimental results show that ResNet-50 yields an accuracy of 93.8% in the recognition performance, which shows that it is efficient to classify blue calico patterns through deep learning methods. As a consequence, this result provides the current best baseline result for Nantong blue calico image recognition. We believe our$N$tBC will facilitate future research on Chinese traditional patterns development, fine grained visual classification, and imbalanced learning fields. We make the dataset and pre-trained models publicly available at https://github.com/facebook/react.

The high dimensionality and sparsity of text data are major obstacles for sentiment analysis. These obstacles may be removed by using some technology such as feature selection. In this paper, we propose a filter feature selection method, called Discriminant Variance Criterion (DVC), for sentiment analysis. DVC utilizes feature and class variances to select discriminative features. Statistically, the feature variance of a feature is its total scatter. Apart from it, DVC also takes into account the class variance of a feature in each class, i.e., class scatter, by introducing the class information of data. As a result, DVC selects terms with a higher total scatter and a lower class scatter as the optimal feature subset. The effectiveness and efficiency of DVC is demonstrated by experimental results on five sentiment datasets using three widely used classifiers including Support Vector Machine (SVM), Multinomial Naive Bayes (MNB) and Logistic Regression (LR).

Forestry pest is like a non-smoke fire for causing a huge loss of forestry products; thus, timely detection and recognition for forestry pests can significantly reduce the loss. The performance of forestry pest recognition greatly depends on the quality of image dataset on which the state-of-the-art learning model can be trained. However, there is a lack of such a dataset that is specially collected for forestry pests. In this paper, we collect a forestry pest (FP60) dataset based on insect taxonomy, which contains 15 families with 7523 images that can be further divided into 60 species. This paper also designs a framework of family-species network (FSNet) for forestry pest recognition, which is a two-task model based on insect taxonomy. In FSNet, the first task is family classification that is to first detect forestry pests in a given image and then determine which families these pests belong to, and the second task is species classification that is to further identify the species of these pests based on their families. Extensive experimental results indicate that FSNet outperforms the state-of-the-art algorithms, especially in the case of increasing the number of pest categories. The FP60 dataset and algorithm codes used in this paper have been published on github.com/sk94747/FP60.

In imbalanced learning, oversampling is incredibly prevalent. However, it is disappointing that existing oversampling methods have their own limitations, such as new synthetic samples may be uninformative or noisy. To better address imbalanced learning tasks, this paper proposes a novel over-sampling method, named Density-induced Selection Probability-based Oversampling TEchnique (DSPOTE). To increase the number of samples in the minority class, DSPOTE designs a novel scheme for filtering noisy samples based on the Chebychev distance and a new way of calculating selection probability based on relative density. DSPOTE first filters noisy samples and then gets borderline ones from the minority class. Next, DSPOTE calculates the selection probabilities for all borderline samples and applies these probabilities to pick up borderline samples. Finally, DSPOTE generates synthetic samples for the minority class based on the selected borderline ones. Experimental results indicate that our method has good performance in terms of metrics, recall and AUC (Area Under the Curve), when compared with other eight methods.

High dimensionality of text data would degrade the performance of text classification for the existence of irrelevant terms. Thus, it is necessary to perform feature selection to remove irrelevant terms. As an effective feature selection method, Relative Discrimination Criterion (RDC) can identify the irrelevant terms according to discriminant information, but it is incapable of capturing redundancy between terms. Therefore, this paper proposes a novel text feature selection based on RDC, named De-redundancy Relative Discrimination Criterion (DRDC), which takes into account the redundancy between terms when assessing their importance. For one thing, DRDC takes advantage of RDC to measure the relevance of terms to categories. For another, DRDC utilizes mutual information to measure the redundancy between terms. During iterations, we separately normalize the scores of RDC and mutual information for balancing them and reducing the impact of marginal probabilities on mutual information. In the procedure of feature selection, DRDC iteratively picks up the term that has the maximum relevance to categories and the minimum redundancy with terms already in the current feature subset. In such a way, DRDC can find an optimal term subset. The effectiveness and efficiency of DRDC is clarified by experiments on R8 and 20Newsgroups data sets using three widely used classifiers.

The goal of data reduction is to remove noise features or samples from the original data and hence improve the performance of algorithms. Although Fisher-regularized support vector machine (FisherSVM) has good separability, FisherSVM lacks sparsity to reduce noise data. To alleviate this issue, we propose a novel sparse support vector machine with Fisher-regularizer (SF-SVM). The$L_{1}$-norm of model coefficients and the Fisher-regularizer term used in SF-SVM can induce a sparse model and improve the generalization performance, respectively. Besides, the proposed SF-SVM includes two versions: linear SF-SVM and kernel SF-SVM, which can perform feature and sample reduction with favorable sparsity, respectively. Experimental results on seven UCI datasets show that the proposed SF-SVM has a satisfactory data reduction ability and good classification performance.

In this paper, we technically propose an enriched prior guided framework, called Dual-constrained Deep Semi-Supervised Coupled Factorization Network (DS2CF-Net), for discovering hierarchical coupled data representation. To extract hidden deep features, DS2CF-Net is formulated as a partial-label and geometrical structure-constrained framework. Specifically, DS2CF-Net designs a deep factorization architecture using multilayers of linear transformations, which can coupled update both the basis vectors and new representations in each layer. To enable learned deep representations and coefficients to be discriminative, we also consider enriching the supervised prior by joint deep coefficients-based label prediction and then incorporate the enriched prior information as additional label and structure constraints. The label constraint can enable the intra-class samples to have same coordinate in feature space, and the structure constraint forces the coefficients in each layer to be block-diagonal so that the enriched prior using the self-expressive label propagation are more accurate. Our network also integrates the adaptive dual-graph learning to retain the local structures of both data and feature manifolds in each layer. Extensive experiments on image datasets demonstrate the effectiveness of DS2CF-Net for representation learning and clustering.

Object counting has become an important task in computer vision for its practical applications in surveillance system. Previous methods for object counting have achieved promising results in accuracy, but few researchers focus on the real-time performance of counting methods. In this paper, we propose an efficient and accurate light-weight network for object counting, called Partial Feature Aggregation Network (PFANet). In this novel method, a Partial Feature Aggregation (PFA) structure is designed to accelerate networks and improve the utilization of multi-scale features. Moreover, PFANet uses the dilated convolution to enlarge the receptive-filed of network. Experiments on two datasets indicate our network exceeds the existing real-time counting networks in both accuracy and efficiency.

Fisher-regularized support vector machine (Fish-erSVM) can approximatively fulfill the Fisher criterion and obtain good statistical separability, which is a combined method of the support vector machine and Fisher discriminant analysis. However, the hinge loss function is related to the shortest distance between two-class sets, and FisherSVM may be hence sensitive to noise. To remedy it, the pinball loss function, which is related to the quantile distance, is introduced into FisherSVM and then a Fisher-regularized support vector machine with pinball loss function (Pin-FisherSVM) is proposed, which combines the noise insensitivity of the pinball loss function with the statistical separability of FisherSVM well. Pin-FisherSVM can be cast as a quadratic programming, which results a globally optimal solution. Experimental results on artificial and real-world datasets demonstrate that our proposed method is insensitive to label noise or feature noise. Compared to FisherSVM, the proposed Pin-FisherSVM has the same computational complexity and exhibits superior noise insensitivity.

In this paper, we technically propose a novel framework called Deep Self-representative Concept Factorization Network (DSCF-Net), for clustering deep features. To improve the representation and clustering abilities, DSCF-Net explicitly considers discovering hidden deep semantic features, enhancing the robustness properties of the deep factorization to noise and preserving the local manifold structures of deep features. Specifically, DSCF-Net integrates the robust deep concept factorization, deep self-expressive representation and adaptive locality preserving feature learning into a unified framework. To discover hidden deep representations, DSCF-Net designs a hierarchical factorization architecture using multiple layers of linear transformations, where the hierarchical representation is performed by formulating the problem as optimizing the basis concepts in each layer to improve the representation indirectly. DSCF-Net also improves robustness by subspace recovery for sparse error correction firstly and then performs deep factorization in the recovered visual subspace. To obtain localitypreserving representations, we also present an adaptive deep self-representative weighting strategy by using the coefficient matrix as adaptive weights to keep the locality of representations. Extensive results show that DSCF-Net delivers state-of-the-art performance on several public databases.

In this article, we propose a structured robust adaptive dictionary pair learning (RA-DPL) framework for the discriminative sparse representation (SR) learning. To achieve powerful representation ability of the available samples, the setting of RA-DPL seamlessly integrates the robust projective DPL, locality-adaptive SRs, and discriminative coding coefficients learning into a unified learning framework. Specifically, RA-DPL improves existing projective DPL in four perspectives. First, it applies a sparse l2,1-norm-based metric to encode the reconstruction error to deliver the robust projective dictionary pairs, and the l2,1-norm has the potential to minimize the error. Second, it imposes the robust l2,1-norm clearly on the analysis dictionary to ensure the sparse property of the coding coefficients rather than using the costly l0/l1-norm. As such, the robustness of the data representation and the efficiency of the learning process are jointly considered to guarantee the efficacy of our RA-DPL. Third, RA-DPL conceives a structured reconstruction weight learning paradigm to preserve the local structures of the coding coefficients within each class clearly in an adaptive manner, which encourages to produce the locality preserving representations. Fourth, it also considers improving the discriminating ability of coding coefficients and dictionary by incorporating a discriminating function, which can ensure high intraclass compactness and interclass separation in the code space. Extensive experiments show that our RA-DPL can obtain superior performance over other state of the arts.

Due to the incorporation of dimensionality reduction, spectral clustering (SC) based methods have a unique advantage in dealing with high-dimensional data. However, data is mainly characterized by its distribution on multiple low-dimensional manifolds, which is ignored by some SC-based methods. We put forward a new spectral multi-manifold embedded clustering (SMEC) method in this paper, which incorporates the local geometric information of data into the traditional SC. Thus, the designed similarity matrix in SMEC is able to capture both the local and global discriminating information, which results in improved clustering. Experimental results on seven benchmark datasets demonstrate our proposed method's promising performance.

As a manifold learning algorithm, unsupervised large graph embedding (ULGE) has been proposed to deal with large-scale dataset for clustering. This paper improves ULGE and proposes a graph embedding-based ensemble learning (GEEL) algorithm. We take the dimensionality reduction algorithm in ULGE and the K-means clustering algorithm as an individual learner in our ensemble learning. For each individual learner, the K-means clustering method is first used to generate anchors. Then, the low-dimensional embedding of the sample data is obtained. Finally, the K-means clustering method is used again and performed on the low-dimensional data, which results in a clustering. The diversity of ensemble learning lies on the unstable of K-means. To combine multiple clusterings, we first match these clusterings with a reference clustering using the bestMap method, where the reference clustering is randomly chosen from multiple ones. A majority voting rule is adopted to these matched clusterings to generate the final clustering. A large number of experiments show the efficiency and effectiveness of the proposed method.

Multi-label image annotation is one of the most important open problems in machine learning and computer vision. In this paper, we propose a novel model for image annotation. Unlike existing works that usually use conventional visual features to annotate images, this paper adopts features based on convolutional neural network (CNN), which have shown potential to achieve outstanding performance. In particular, we use CNN to extract image features with higher semantic meaning and apply them to the image annotation method - Tag Propagation (TagProp). Experimental results on four challenging datasets indicate that our model makes a marked improvement as compared to the current state-of-the-art.

Single sample face recognition has always been a hot but difficult issue in face recognition. The existing methods solve this issue from selecting robust features or generating virtual samples. By considering selecting robust features and generating virtual samples simultaneously, this paper proposes a multi-scale support vector transformation (MSSVT) based method to generate multi-scale virtual samples for single image recognition. Experimental results on three face data sets verify that the proposed algorithm retains most information and has the best performance compared with other related algorithm.

The research and analysis on multi-source data is one of important tasks in information science. Compared with traditional single-source data learning algorithms, multi-source data learning ones can describe objects more real and complete. Meanwhile, the learning process of multi-source data is more in line with the cognitive mechanism of human brain. So far, the research on multi-source data learning algorithms includes three classes, multi-source data transfer learning, multi-source data collaborative learning and multi-source multi-view learning. The traditional multi-source multi-view learning algorithms lack the ability of handling with the data missing issue, which means that these algorithms require the multi-source data to be complete. This paper proposes a multi-source clustering algorithm. Based on the spectral properties of Laplace operator, we first obtain the complete representation of multi-source data. Then, we utilize the multi-view spectral embedding (MVSE) to construct the fusion model. Experimental results show that our proposed method can improve the ability of clustering efficiently in the case of data missing.

This paper proposes a novel Nonnegative and Adaptive Multi-view Clustering (NAMC) method. NAMC integrates the nonnegative matrix factorization (NMF), adaptive neighborhood learning and consensus adaptive similarity matrix fusion. More specifically, NAMC performs the nonnegative weight learning over the original data and the parts-based representations of NMF for more accurate measure and representation. For nonnegative adaptive feature extraction, our model first utilizes NMF to obtain the local parts-based representation of the original high-dimensional data. To keep the local structure of parts-based representations, we minimize the adaptive neighborhood reconstruction error. Then the optimal consensus similarity matrix can be iteratively obtained according to the nonnegative adaptive similarity matrix of each view. What's more, the proposed NAMC is totally self-weighted. Once the target graph is obtained in our model, it can be partitioned into specific clusters directly. Extensive simulations show that NAMC can achieve good performance on several public databases for multi-view clustering, compared with other related methods.

In this paper, we propose an analysis mechanism-based structured analysis discriminative dictionary learning analysis discriminative dictionary learning, framework. The ADDL seamlessly integrates ADDL, analysis representation, and analysis classifier training into a unified model. The applied analysis mechanism can make sure that the learned dictionaries, representations, and linear classifiers over different classes are independent and discriminating as much as possible. The dictionary is obtained by minimizing a reconstruction error and an analytical incoherence promoting term that encourages the subdictionaries associated with different classes to be independent. To obtain the representation coefficients, ADDL imposes a sparse -norm constraint on the coding coefficients instead of using or norm, since the - or -norm constraint applied in most existing DL criteria makes the training phase time consuming. The code-extraction projection that bridges data with the sparse codes by extracting special features from the given samples is calculated via minimizing a sparse code approximation term. Then we compute a linear classifier based on the approximated sparse codes by an analysis mechanism to simultaneously consider the classification and representation powers. Thus, the classification approach of our model is very efficient, because it can avoid the extra time-consuming sparse reconstruction process with trained dictionary for each new test data as most existing DL algorithms. Simulations on real image databases demonstrate that our ADDL model can obtain superior performance over other state of the arts.

We propose a robust inductive semi-supervised label prediction model over the embedded representation, termed adaptive embedded label propagation with weight learning (AELP-WL), for classification. AELP-WL offers several properties. First, our method seamlessly integrates the robust adaptive embedded label propagation with adaptive weight learning into a unified framework. By minimizing the reconstruction errors over embedded features and embedded soft labels jointly, our AELP-WL can explicitly ensure the learned weights to be joint optimal for representation and classification, which differs from most existing LP models that perform weight learning separately by an independent step before label prediction. Second, existing models usually precalculate the weights over the original samples that may contain unfavorable features and noise decreasing performance. To this end, our model adds a constraint that decomposes original data into a sparse component encoding embedded noise-removed sparse representations of samples and a sparse error part fitting noise, and then performs the adaptive weight learning over the embedded sparse representations. Third, our AELP-WL computes the projected soft labels by trading-off the manifold smoothness and label fitness errors over the adaptive weights and the embedded representations for enhancing the label estimation power. By including a regressive label approximation error for simultaneous minimization to correlate sample features with the embedded soft labels, the out-of-sample issue is naturally solved. By minimizing the reconstruction errors over features and embedded soft labels, classification error and label approximation error jointly, state-of-the-art results are delivered.

Both neighbourhood preserving embedding (NPE) and sparsity preserving projection (SPP) are unsupervised learning methods, where NPE can preserve the local neighbourhood information of a given dataset and SPP can preserve the sparsely reconstructive relationship of the dataset. However, it is not satisfactory when applying the two methods to classification tasks. First, a modified SPP is presented here. Then this study proposes a supervised sparse neighbourhood preserving embedded algorithm (SSNPE) based on NPE, the modified SPP and the label information of a given task. SSNPE inherits the merits of NPE and SPP, which can preserve not only the local neighbourhood information but also the sparsely reconstructive relationship. The connection between SSNPE and both NPE and SPP is discussed. Experimental results on the datasets of UCI, ORL and MNIST indicate that the proposed method is effective.

This study proposes a supervised orthogonal discriminant projection (SODP) based on double adjacency graphs (DAGs). SODP based on DAG (SODP‐DAG) aims to minimise the local within‐class scatter and simultaneously maximise both the local between‐class scatter and the non‐local scatter, where the local between‐class scatter and the local within‐class scatter are constructed by applying the DAG structure. By doing so, SODP‐DAG can keep the local within‐class structure for original data and find the optimal discriminant directions effectively. Moreover, four schemes are designed for constructing weight matrices in SODP‐DAG. To validate the performance of SODP‐DAG, the authors compared it with orthogonal discriminant projection, SODP and others on several publicly available datasets. Experimental results show the feasibility and effectiveness of SODP‐DAG.

This paper investigates the salient machine faults representation-based classification issue by dictionary learning. A novel structured latent label consistent dictionary learning (LLC-DL) model is proposed for joint discriminative salient representation and classification. Our LLC-DL deals with the tasks by solving one objective function that aims to minimize the structured reconstruction error, structured discriminative sparse-code error and classification error simultaneously. Also, LLC-DL decomposes given signals into a sparse reconstruction part over structured latent weighted discriminative dictionary, a salient feature extraction part and an error part fitting noise. Specifically, the dictionary is learnt atom by atom, where each dictionary atom is learnt with a latent vector that reduces the disturbance between interclass atoms. The structured coding coefficients are calculated via minimizing the reconstruction error and discriminative sparse code error simultaneously. The salient representations are learnt by embedding signals onto a projection and a robust linear classifier is then trained over the learned salient features directly so that features can be ensured to be optimal for classification, where robust l2,1-norm imposed on the classifier can make the prediction results more accurate. By including a salient feature extraction term, the classification approach of LLC-DL is very efficient, since there is no need to involve an extra time-consuming sparse reconstruction process with the well-trained dictionary for each test signal. Extensive simulations versify the effectiveness of our algorithm.

We propose two nuclear- and L2,1-norm regularized 2D neighborhood preserving projection (2DNPP) methods for extracting representative 2D image features. 2DNPP extracts neighborhood preserving features by minimizing a Frobenius norm-based reconstruction error that is very sensitive noise and outliers in given data. To make the distance metric more reliable and robust, and encode the neighborhood reconstruction error more accurately, we minimize the nuclear- and L2,1-norm-based reconstruction error, respectively and measure it over each image. Technically, we propose two enhanced variants of 2DNPP, nuclear-norm-based 2DNPP and sparse reconstruction-based 2DNPP. Besides, to optimize the projection for more promising feature extraction, we also add the nuclear- and sparse L2,1-norm constraints on it accordingly, where L2,1-norm ensures the projection to be sparse in rows so that discriminative features are learnt in the latent subspace and the nuclear-norm ensures the low-rank property of features by projecting data into their respective subspaces. By fully considering the neighborhood preserving power, using more reliable and robust distance metric, and imposing the low-rank or sparse constraints on projections at the same time, our methods can outperform related state-of-the-arts in a variety of simulation settings.

The typical classification rule for kernel sparse representation-based classifier (KSRC) is the reconstruction error minimization rule. Its computational complexity mainly depends on both the dimensionality of a subspace and the number of training samples. This paper presents an alternative classification rule, called reconstruction coefficient energy maximization, for KSRC and applies it to target recognition in synthetic aperture radar (SAR) images. The computational complexity of this rule is related to only the number of training samples, which is smaller than that of the reconstruction error minimization rule. Experimental results on the Moving and Stationary Target Acquisition and Recognition (MSTAR) public database indicate that KSRC is very promising in SAR image target recognition., and the reconstruction coefficient energy maximization rule outperforms the reconstruction error minimization rule in KSRC.

We propose a nuclear-norm regularized two-dimensional neighborhood preserving projection (2DNPP) for extracting representative 2D image features. Note that 2DNPP extracts neighborhood preserving features through minimizing the reconstruction error, but the Frobenius norm based metric is sensitive to noise and outliers. To make the distance metric more reliable and model the neighborhood reconstruction error more accurately, we impose the nuclear-norm on the neighborhood reconstruction error and measure it over each image. Technically, we propose a new variant of 2DNPP termed nuclear-norm based 2DNPP (N-2DNPP). Besides, to make delivered projection promising for feature extraction, we also include the nuclear-norm constraint on projection accordingly, where the low-rank projection can embed data into their respective subspaces. Our method can outperform related state-of-the-arts in a variety of simulation settings.

In this paper, we propose a semisupervised label consistent dictionary learning (SSDL) framework for machine fault classification. SSDL is a semisupervised extension of recent fully supervised label consistent dictionary learning approach, since the number of labeled machine data is usually limited in practice. To enable the supervised dictionary learning model to use both labeled and commonly readily available unlabeled data for enhancing performance, we propose to incorporate the merits of label prediction and present a joint label consistent dictionary learning and adaptive label prediction technique. In this setting, we first employ the existing label prediction model to estimate the labels of unlabeled training signals in a transductive fashion for enriching supervised prior. Then, we use predicted labeled data for label consistent dictionary learning. After that, we apply the discriminant sparse codes as the adaptive reconstruction weights for label prediction to update the estimated labels of unlabeled training data and the discriminative sparse codes matrix for label consistent dictionary learning so that classification performance can be enhanced. Thus, an informative dictionary, a sparse-code matrix, and an optimal multiclass classifier can be alternately obtained from one objective function. Besides, the tricky process of choosing optimal kernel width and neighborhood size can also be effectively voided in our scheme due to the adaptive weights. Extensive simulations on several machine fault datasets show that our SSDL method can deliver enhanced performance over other state-of-the-arts for machine fault classification.

Recovering low-rank and sparse subspaces jointly for enhanced robust representation and classification is discussed. Technically, we first propose a transductive low-rank and sparse principal feature coding (LSPFC) formulation that decomposes given data into a component part that encodes low-rank sparse principal features and a noise-fitting error part. To well handle the outside data, we then present an inductive LSPFC (I-LSPFC). I-LSPFC incorporates embedded low-rank and sparse principal features by a projection into one problem for direct minimization, so that the projection can effectively map both inside and outside data into the underlying subspaces to learn more powerful and informative features for representation. To ensure that the learned features by I-LSPFC are optimal for classification, we further combine the classification error with the feature coding error to form a unified model, discriminative LSPFC (D-LSPFC), to boost performance. The model of D-LSPFC seamlessly integrates feature coding and discriminative classification, so the representation and classification powers can be enhanced. The proposed approaches are more general, and several recent existing low-rank or sparse coding algorithms can be embedded into our problems as special cases. Visual and numerical results demonstrate the effectiveness of our methods for representation and classification.

This paper presents a new multi-class gene selection and classification method based on multiple support vector machine recursive feature elimination (SVM-RFE). For a multi-class DNA microarray problem, we solve it as multiple binary classification problems. First, the one-versus-all method is used to decompose the multi-class task into multiple binary problems. Second, an SVM-RFE is adopted to select genes for each binary problem. Then, an SVM classifier is used to train the selected gene data for a binary problem. Finally, we combine the outputs of multiple SVM classifiers. Experimental results on three DNA Microarray datasets show that the proposed method achieves higher classification accuracy.

Generally, in order to learn sparse representations for raw inputs via an auto-encoder, the Kullback-Leibler (KL) divergence as a sparsity regularizer is introduced to the loss function for penalizing active code units. In fact, there exist other sparsity regularizers except the KL divergence. This paper introduces some classical sparsity regularizers into auto-encoders, and empirically gives a survey on the auto-encoders with different sparsity regularizers. Specifically, we analyze another two sparsity regularizers which are usually used in sparse coding. In addition, we also consider the effect of different activation functions and different sparsity regularizers on learning performance of auto-encoders. Our experiments are conducted on the datasets of MNIST and COIL.

Similarity learning ranges over an extensive field in machine learning and pattern recognition. This paper deals with similarity learning based on multiple support vector data description (SVDD). It is well known that SVDD was proposed for one-class or two-class unbalanced learning problems. Thus, we propose a multiple SVDD (MSVDD) algorithm and apply it to multi-class learning problems. A SVDD model is trained by similar pairwise samples in the same class instead of all similar ones. In addition, the dissimilar pairwise samples are not considered in MSVDD. Experimental results validate that MSVDD is promising in similarity learning.

Detecting Edge in synthetic aperture radar (SAR) imagery is to extract contours across the investigated SAR image. Classical edge detection methods provide a limited efficiency in SAR imagery for the presence of speckle noise. This paper deals with a novel filtering method for edge detection in SAR imagery based on support value transform (SVT). By using SVT, we decompose a SAR image into a low-frequency component image and a series of support value images (SVIs). The low-frequency image contains the slowly variation information, or the contour information in the SAR image. Then, the classical edge detection methods can be used to extract the contours of the low-frequency image. Experimental results show that this novel edge detection scheme is promising.

1-norm support vector machine (SVM) has attracted substantial attentions for its good sparsity. However, the computational complexity of training 1-norm SVM is about the cube of the sample number, which is high. This paper replaces the hinge loss or the ε-insensitive loss by the squared loss in the 1-norm SVM, and applies orthogonal matching pursuit (OMP) to approximate the solution of the 1-norm SVM with the squared loss. Experimental results on toy and real-world datasets show that OMP can faster train 1-norm SVM and achieve similar learning performance compared with some methods available.

In this paper, we mainly present a Semi-Supervised Label Consistent KSVD (S2KSVD) algorithm for representing and classifying machine faults. The formulation of our S2KSVD is an improvement to the recent label consistent K-SVD (LC-KSVD), because LC-KSVD is a fully supervised approach, and needs to use supervised class information of all training data to compute a reconstructive & discriminative dictionary. But labeled signals are often expensive to obtain, while in contrast unlabeled signals can be easily captured with low expense from the real world. Thus, the application of LC-KSVD may be constrained in reality. To address this problem, we present S2KSVD through involving a computationally efficient label propagation (LP) process as a preprocessing step. The core idea is to employ the LP process to estimate the labels of unlabeled signals so that supervised prior knowledge that can significantly enhance classification can be increased. Simulation results on several machine fault datasets demonstrate that our algorithm delivers promising performance for machine fault classification.

This paper proposes an enhanced semi-supervised classification approach termed Nonnegative Sparse Neighborhood Propagation (SparseNP) that is an improvement to the existing neighborhood propagation due to the fact that the outputted soft labels of points cannot be ensured to be sufficiently sparse, discriminative, robust to noise and be probabilistic values. Note that the sparse property and strong discriminating ability of predicted labels is important, since ideally the soft label of each sample should have only one or few positive elements (that is, less unfavorable mixed signs are included) deciding its class assignment. To reduce the negative effects of unfavorable mixed signs on the learning performance, we regularize the l2,1-norm on the soft labels during optimization for enhancing the prediction results. The non-negativity and sum-to-one constraints are also included to ensure the outputted labels are probabilistic values. The proposed framework is solved in an alternative manner for delivering a more reliable solution so that the accuracy can be improved. Simulations show that satisfactory results can be obtained by the proposed SparseNP compared with other related approaches.

Supervised locally linear embedding (SLLE) has been proposed for classification tasks. SLLE can take full use of the label information and select neighbours only in the same class. However, SLLE uses the least squares (LSs) method for solving a set of linear equations to obtain linear representation coefficients, which relates to the inverse of a matrix. If the matrix is singular, the solution to the set of linear equations does not exist. Additionally, if the size of neighbourhood is not appropriate, some further neighbours along the manifold would be selected. To remedy those, this study deals with SLLE based on orthogonal matching pursuit (SLLE‐OMP) by introducing OMP into SLLE. In SLLE‐OMP, LS is replaced by OMP and OMP can reselect new neighbours from old ones. Experimental results on some real‐world datasets show that SLLE‐OMP can achieve better classification performance compared with SLLE.

Neighborhood preserving embedding (NPE) is an approximation to locally linear embedding (LLE), which has an ability to preserve local neighborhood structure on data manifold. As an unsupervised dimensionality reduction method, NPE can be applied to face recognition for preprocessing. However, NPE could not utilize the label information in the classification tasks. To make the data in a reduced subspace separable, this paper proposes a supervised neighborhood preserving embedding which could learn a projection matrix by using both the geometrical manifold structure and the label information of the given data. In addition, the projection matrix could be found by solving a linear set of equations. Experimental results on ORL and Yale face image datasets show that the proposed method has a high recognition rate.

The idea that with the help of proper dimensionality reduction, trying to make the samples with the same label be compact and the ones with the different labels be separate after projection, is introduced into classification problems with high-dimensional data. Based on the analysis of the drawbacks of Discriminant Neighborhood Embedding (DNE) and Locality-Based Discriminant Neighborhood Embedding (LDNE), being the two relatively successful Locally Discriminant Analysis methods proposed in recent years, this paper proposes a method called Similarity-balanced Discriminant Neighborhood Embedding (SBDNE). When constructing the adjacent graph, SBDNE fully takes into account the geometric construction of manifold and the problem of imbalance between the intra-class points and the inter-class points. By endowing these two kinds of samples with different similarities and selecting the near neighbors according to the similarity matrix, not only the structure in the original space can be preserved more efficiently, but also the choice of discriminative information increases. The method proposed here has a better recognition with comparisons to some classical methods, which fully shows that SBDNE method has the capacity to efficiently solve the classification problem.

Discriminant neighborhood embedding (DNE) algorithm is one of supervised linear dimensionality reduction methods. Its nonlinear version kernel discriminant neighborhood embedding (KDNE) is expected to behave well on classification tasks. However, since KDNE constructs an adjacent graph in the original space, the adjacency graph could not represent the adjacent information in the kernel mapping space. By introducing hidden space, this paper proposes a novel nonlinear method for DNE, called hidden space discriminant neighborhood embedding (HDNE). This algorithm first maps the data in the original space into a high dimensional hidden space by a set of nonlinear hidden functions, and then builds an adjacent graph incorporating neighborhood information of the dataset in the hidden space. Finally, DNE is used to find a transformation matrix which would map the data in the hidden space to a low-dimensional subspace. The proposed method is applied to ORL face and MNIST handwritten digit databases. Experimental results show that the proposed method is efficiency for classification tasks.

Locally Linear Embedding (LLE) is a sort of powerful nonlinear dimensionality reduction algorithms. The basic idea behind the LLE method is that each data point and its neighbors lie on or close to a locally linear patch of the manifold if there is sufficient data. Then the local geometry of these patches is described by using linear coefficients which can reconstruct each data point from its neighbors. However, LLE operates in a batch way and its dimension reduction cannot be generalized to unseen samples. If a test sample arrives, LLE must run repeatedly and the former computational results are discarded. Thus, some incremental methods have been proposed for LLE to solve this problem. In these incremental methods, the neighbor number is globally fixed, which may result in selecting points from another linear space as neighbors. This paper presents LLE based on orthogonal matching pursuit (OMP) and applies it to classification tasks. In the classification tasks, dimensionality reduction on test samples is implemented by applying dimension reduction on training samples. The new LLE method could select a more appropriate neighbors from the selected neighbors. OMP is applied to not only LLE for training samples, but also the incremental learning of LLE for test samples. Compared with other linear incremental methods, experimental results show that the proposed method is promising.

Deep networks are well known for their powerful function approximations. To train a deep network efficiently, greedy layer-wise pre-training and fine tuning are required. Typically, pre-training, aiming to initialize a deep network, is implemented via unsupervised feature learning, with multiple feature representations generated. However, in general only the last layer representation is to be employed because of its abstraction and compactness being the best with comparisons to the ones of lower layers. To make full use of the representations of all layers, this paper proposes a feature ensemble learning method based on sparse autoencoders for image classification. Specifically, we train three softmax classifiers by using the representations of different layers, instead of one classifier trained by applying the last layer representation. Of the three softmax classifiers, two are obtained by training stacked autoencoders with fine tuning, and the other one is obtained by directly using a concatenation of two representations. To improve accuracy and stability of a single softmax classifier, the ensemble of multiple classifiers is considered, and some Naive Bayes combination rules are introduced to integrate the three classifiers. Experimental results on the MNTST and COIL datasets are presented, with comparisons to other classification methods.

Similarity learning is one of the most fundamental notions in machine learning and pattern recognition. In real-world problems, the number of the paired-samples in similarity set is far less than the ones in dissimilarity set. In other word, there is an unbalanced problem in the paired-samples of similarity learning. This paper presents a scheme of SVM ensemble to solve it. In our scheme, we randomly select some of samples to construct paired-samples, not producing all the paired-samples, and introduces multiple classifiers to obtain higher stability and reliability. As a result, the SVM ensemble can effectively decrease the number of paired-samples in similarity learning and solve the unbalanced data learning to some degree. In the experiments, the SVM ensemble is compared with some classic unbalanced learning algorithms. The results on classification tasks show that the SVM ensemble gains better performance.

We investigate large-scale face identification in unconstrained videos with 1000 subjects. This problem is very challenging, and until now most studies have only considered the scenarios with a small number of subjects and videos captured in controlled laboratory environments. Our contributions in this paper are twofold. First, we set up a large-scale video database in an unconstrained environment, Celebrity-1000, with data collected from two popular video-sharing websites, YouTube and Youku, for face identification research. It contains 1000 celebrities from different countries, ~7000 videos, ~160 K tracking sequences, and ~2.4 M sampled frames. Second, we boost the efficiency of multitask joint sparse representation (MTJSR) algorithm for video-based face identification on Celebrity-1000. MTJSR is training free and can naturally integrate multiple frames of the same tracking sequence for collaborative inference, and thus is suitable for video-based face identification. We present a sparsity-induced scalable optimization method, which solves the large-scale MTJSR problem by sequentially solving a series of smaller-scale subproblems with theoretically guaranteed convergency. Extensive experiments show several orders-of-magnitude speedup with this new optimization method, and also demonstrate the superiorities of the accelerated MTJSR algorithm over several popular baseline algorithms.

Adaptive direction lifting-based wavelet transform (ADL) has better performance than conventional lifting both in image compression and de-noising. However, no architecture has been proposed to hardware implement it because of its high computational complexity and huge internal memory requirements. In this paper, we propose a four-stage pipelined architecture for 2 Dimensional (2D) ADL with fast computation and high data throughput. The proposed architecture comprises column direction estimation, column lifting, row direction estimation and row lifting which are performed in parallel in a pipeline mode. Since the column processed data is transposed, the row processor can reuse the column processor which can decrease the design complexity. In the lifting step, predict and update are also performed in parallel. For an 8×8 image sub-block, the proposed architecture can finish the ADL forward transform within 78 clock cycles. The architecture is implemented on Xilinx Virtex5 device on which the frequency can achieve 367 MHz. The processed time is 212.5 ns, which can meet the request of real-time system.

A progressive image compression scheme is investigated using reversible integer discrete cosine transform (RDCT) which is derived from the matrix factorization theory. Previous techniques based on DCT suffer from bad performance in lossy image compression compared with wavelet image codec. And lossless compression methods such as IntDCT, I2I-DCT and so on could not compare with JPEG-LS or integer discrete wavelet transform (DWT) based codec. In this paper, lossy to lossless image compression can be implemented by our proposed scheme which consists of RDCT, coefficients reorganization, bit plane encoding, and reversible integer pre- and post-filters. Simulation results show that our method is competitive against JPEG-LS and JPEG2000 in lossless compression. Moreover, our method outperforms JPEG2000 (reversible 5/3 filter) for lossy compression, and the performance is even comparable with JPEG2000 which adopted irreversible 9/7 floating-point filter (9/7F filter).

A generalized nonlinear discriminant analysis (GNDA) method is proposed, which implements Fisher discriminant analysis in a nonlinear mapping space. Linear discriminant analysis in the nonlinear mapping space corresponds to nonlinear discriminant analysis in an input space. GNDA suggests a unified framework of nonlinear discriminant analysis which includes the kernel Fisher discriminant analysis as a specific case. Experimental results on UCI data sets demonstrate the validity of our method.

Recently the study on the theory of wavelets shows that the wavelets have not only the multi-resolution property both in frequency and time domain, but also the good approximation ability. SVMs based on the statistical learning theory are a kind of general and effective learning machines, and have described for us the nice application blueprint in machine learning domain. There exists a bottleneck problem, or the pre-selection of kernel parameter for SVMs. In this paper, the orthogonal projection kernels of father wavelet (OPFW kernels) are introduced into SVMs. In doing so SVMs based on the OPFW kernels can have good performance in both approximation and generalisation. Simultaneously the parameter pre-selection of wavelet kernels can be implemented by discrete wavelet transform. Experiments on regression estimation illustrate the approximation and generalisation ability of our method.

Hidden space support vector machines (HSSVMs) are presented in this paper. The input patterns are mapped into a high-dimensional hidden space by a set of hidden nonlinear functions and then the structural risk is introduced into the hidden space to construct HSSVMs. Moreover, the conditions for the nonlinear kernel function in HSSVMs are more relaxed, and even differentiability is not required. Compared with support vector machines (SVMs), HSSVMs can adopt more kinds of kernel functions because the positive definite property of the kernel function is not a necessary condition. The performance of HSSVMs for pattern recognition and regression estimation is also analyzed. Experiments on artificial and real-world domains confirm the feasibility and the validity of our algorithms.

An admissible support vector (SV) kernel (the wavelet kernel), by which we can construct a wavelet support vector machine (SVM), is presented. The wavelet kernel is a kind of multidimensional wavelet function that can approximate arbitrary nonlinear functions. The existence of wavelet kernels is proven by results of theoretic analysis. Computer simulations show the feasibility and validity of wavelet support vector machines (WSVMs) in regression and pattern recognition.