Over the past few years, various word-level textual attack approaches have been proposed to reveal the vulnerability in existing deep neural networks and even large language models (LLMs) for Natural Language Processing (NLP). The textual attack aims to fool existing models into making erroneous predictions by altering the text without affecting the user’s understanding. However, current methods either struggle to construct semantically preserved adversarial texts and altered the semantics of the original text, or fail to consider the semantic perturbation constraints and are prone to invalid adversarial examples. In this paper, we propose an efficient and effective framework SubAttack to address these issues. SubAttack is a word-level adversarial textual attack method via antonym substitution, which replaces semantic indicator keywords to generate high-quality adversarial samples with considering both semantically preservation and semantic perturbation. Specifically, the process first involves tokenizing the text and performing part-of-speech tagging Identifying the semantic indicator keywords. Then, the antonym ranking is designed to decide the substitutions of candidate words to fit the context. Finally, while retaining the original text, the ranked antonyms are integrated into the text and the instructions are added for both semantically preservation and semantic perturbation. Extensive experiments reveal that state-of-the-art (SOTA) LLMs (e.g. Llama and QWen) are still vulnerable to our SubAttack. Further experiments show that the adversarial examples crafted by SubAttack usually have higher quality, exhibit better fluency and barely affect human performance and can bring more robustness improvement to victim models by adversarial training.

The personalized recommendation aims to address the information overload problem, which can find interesting items for users from massive amounts of information. The research paradigm of personalized recommendation evolved from deep neural networks to pre-trained language models (PLMs) like BERT and, more recently, into large language models (LLMs). However, it is always very difficult to find the target item among a massive number of data or information, which is not only time-consuming but also often has low accuracy. In this paper, we propose a Personalized Recommendation method with Clustering via Prompt-tuning (PRCP), a candidate item set is developed and a prompt-tuning model with a designed verbalizer is constructed for recommendation. Specifically, the target users are first selected by the similarity calculation, and items are then clustered by the preferences of similar users to form a candidate item set. Then the prompt-tuning model is introduced to predict the masked label for candidate items, and three different strategies are designed to expand the label word space for verbalizer optimization. Extensive experiments conducted on three datasets validated the effectiveness of the proposed method compared to other state-of-the-art baselines including LLMs.

Research on text simplification has been ongoing for many years. However, the task of document simplification (DS) remains a significant challenge due to the need to consider complex factors such as technical terminology, metaphors, and overall coherence. In this work, we introduce a novel multi-agent framework for document simplification (AgentSimp) based on large language models (LLMs). This framework emulates the collaborative process of a human expert team through the roles played by multiple agents, addressing the intricate demands of document simplification. We explore two communication strategies among agents (pipeline-style and synchronous) and two document reconstruction strategies (Direct and Iterative ). According to both automatic evaluation metrics and human evaluation results, the documents simplified by AgentSimp are deemed to be more thoroughly simplified and more coherent on a variety of articles across different types and styles.

Research on text simplification has been ongoing for many years, yet document simplification remains a significant challenge due to the need to address complex factors such as technical terminology, metaphors, and overall coherence. In this work, we introduce a novel multi-agent framework AgentSimp for document simplification, based on large language models. This framework simulates the collaborative efforts of a team of human experts through the roles played by multiple agents, effectively meeting the intricate demands of document simplification. We investigate two communication strategies among agents (pipeline-style and synchronous) and two document reconstruction strategies (Direct and Iterative). According to both automatic evaluation metrics and human evaluation results, AgentSimp produces simplified documents that are more thoroughly simplified and more coherent across various articles and styles.

The low resolution (LR) problem is rather challenging in face analysis. Most existing face hallucination methods assume that LR face images have only one resolution, but multiple resolutions may be available from different sources. To solve this issue, we propose a novel simultaneous facial canonical correlation representation learning method for face hallucination, which seeks latent correlation subspaces for multi-resolution views. Our method jointly solves multiple linear transformations by optimizing a correlation summation criterion of all pairs of resolutions. The neighborhood reconstruction is used to infer the HR facial canonical correlation representation of LR face inputs. Extensive experimental results show the superiority of our proposed method in terms of quantitative and qualitative evaluations.

Conventional image set methods typically learn from image sets stored in one location. However, in real-world applications, image sets are often distributed or collected across different positions. Learning from such distributed image sets presents a challenge that has not been studied thus far. Moreover, efficiency is seldom addressed in large-scale image set applications. To fulfill these gaps, this paper proposes Distributed Manifold Hashing (DMH), which models distributed image sets as a connected graph. DMH employs Riemannian manifold to effectively represent each image set and further suggests learning hash code for each image set to achieve efficient computation and storage. DMH is formally formulated as a distributed learning problem with local consistency constraint on global variables among neighbor nodes, and can be optimized in parallel. Extensive experiments on three benchmark datasets demonstrate that DMH achieves highly competitive accuracies in a distributed setting and provides faster classification and retrieval than state-of-the-arts.

Multiple kernel clustering (MKC) has received increasing attention in the community of machine learning, which takes advantage of multiple pre-specified kernels to perform clustering tasks. Traditional MKC algorithms cannot effectively deal with the incomplete views where some samples are missing. Thus, incomplete MKC (IMKC) has been developed to solve this problem and obtained promising results. Nevertheless, the samples may be noisy or limited in real-world applications, which will result in the performance deterioration of existing IMKC algorithms. To address this issue, in this paper we propose a simple yet effective clustering method for incomplete data, termed fractional-order embedding incomplete multi-kernel k-means clustering (FE-MKKM-IK). Specifically, FE-MKKM-IK introduces the idea of fractional-order embedding to reconstruct the kernel matrix computed by the samples. On this basis, a new incomplete multiple kernel k-means clustering is developed. Performance evaluation is conducted on four widely used datasets, which shows that FE-MKKM-IK is effective to cluster the incomplete data.

Face super-resolution (FSR) is a powerful technique for restoring high-resolution face images from the captured low-resolution ones with the assistance of prior information. Existing FSR methods based on explicit or implicit covariance matrices are difficult to reveal complex nonlinear relationships between features, as conventional covariance computation is essentially a linear operation process. Besides, the limited number of training samples and noise disturbance lead to the deviation of sample covariance matrices. To solve these issues, we propose a novel FSR method via using spectral canonical ℱ-correlation representation. The proposed method first defines intra-resolution and inter-resolution covariation matrices by considering the nonlinear relationship between different features, and then uses the fractional order idea to rebuild covariation matrices. The qualitative and quantitative results have validated the superiority of the proposed method.

As social networks grow exponentially, there is an increasing demand for video retrieval using natural language. Cross-modal hashing that encodes multi-modal data using compact hash code has been widely used in large-scale image-text retrieval, primarily due to its computation and storage efficiency. When applied to video-text retrieval, existing unsupervised cross-modal hashing extracts the frame- or word-level features individually, and thus ignores long-term dependencies. In addition, effectively exploiting the multi-modal structure is a remarkable challenge owing to the complex nature of video and text. To address the above issues, we propose Similarity Preserving Transformer Cross-Modal Hashing (SPTCH), a new unsupervised deep cross-modal hashing method for video-text retrieval. SPTCH encodes video and text by bidirectional transformer encoder that exploits their long-term dependencies. SPTCH constructs a multi-modal collaborative graph to model correlations among multi-modal data, and applies semantic aggregation by employing Graph Convolutional Network (GCN) on such graph. SPTCH designs unsupervised multi-modal contrastive loss and neighborhood reconstruction loss to effectively leverage inter- and intra-modal similarity structure among videos and texts. The empirical results on three video benchmark datasets illustrate that the proposed SPTCH generally outperforms state-of-the-arts in video-text retrieval.

The application of Auto-Encoder (AE) to multi-view representation learning has gained traction due to advancements in deep learning. While some current AE-based multi-view representation learning algorithms incorporate the geometric structure of the input data into their feature representation learning process, their use of a shallow structured graph regularization term can be restrictive when used in conjunction with deep models. Furthermore, current multi-view representation learning algorithms do not fully utilize the diversity and consistency presented in different views, leading to a reduction in the efficacy of feature learning. This paper introduces a novel approach, reconstructed graph constrained auto-encoders (RGCAE), for multi-view representation learning. Unlike existing methods, our approach incorporates deep adaptive graph regularization based on multi-layer perceptron to ensure the preservation of the geometric similarity graph, which is constructed based on the local invariance principle. By decoupling the feature representation learning from the preservation of the geometric structure among different views, our approach can better leverage the diversity presented in multi-view data. We obtain view-specific representations that preserve the geometric structure and then combine them by averaging to obtain a common representation. To ensure the consistency of the multi-view data, we minimize the loss between the view-specific and common representations. Consequently, our RGCAE approach can maintain the geometric structure of multi-view data and is better suited for integration with deep models. Extensive experiments on six datasets demonstrate that RGCAE obtained promising performance, compared with the state-of-the-art methods.

Lexical substitution (LS) aims at finding appropriate substitutes for a target word in a sentence.Recently, LS methods based on pretrained language models have made remarkable progress, generating potential substitutes for a target word through analysis of its contextual surroundings.However, these methods tend to overlook the preservation of the sentence's meaning when generating the substitutes.This study explores how to generate the substitute candidates from a paraphraser, as the generated paraphrases from a paraphraser contain variations in word choice and preserve the sentence's meaning.Since we cannot directly generate the substitutes via commonly used decoding strategies, we propose two simple decoding strategies that focus on the variations of the target word during decoding.Experimental results show that our methods outperform state-of-theart LS methods based on pre-trained language models on three benchmarks.

Lexical simplification (LS) methods based on pretrained language models have made remarkable progress, generating potential substitutes for a complex word through analysis of its contextual surroundings. However, these methods require separate pretrained models for different languages and disregard the preservation of sentence meaning. In this paper, we propose a novel multilingual LS method via paraphrase generation, as paraphrases provide diversity in word selection while preserving the sentence’s meaning. We regard paraphrasing as a zero-shot translation task within multilingual neural machine translation that supports hundreds of languages. After feeding the input sentence into the encoder of paraphrase modeling, we generate the substitutes based on a novel decoding strategy that concentrates solely on the lexical variations of the complex word. Experimental results demonstrate that our approach surpasses BERT-based methods and zero-shot GPT3-based method significantly on English, Spanish, and Portuguese.

Existing lexical substitution (LS) benchmarks were collected by asking human annotators to think of substitutes from memory, resulting in benchmarks with limited coverage and relatively small scales.To overcome this problem, we propose a novel annotation method to construct an LS dataset based on human and machine collaboration.Based on our annotation method, we construct the first Chinese LS dataset CHNLS which consists of 33,695 instances and 144,708 substitutes, covering three text genres (News, Novel, and Wikipedia).Specifically, we first combine four unsupervised LS methods as an ensemble method to generate the candidate substitutes, and then let human annotators judge these candidates or add new ones.This collaborative process combines the diversity of machine-generated substitutes with the expertise of human annotators.Experimental results that the ensemble method outperforms other LS methods.To our best knowledge, this is the first study for the Chinese LS task.

Canonical correlation analysis (CCA) is a classical yet powerful tool for learning two-view feature representation in various fields. But, most CCA approaches are based on the conventional covariance measure, which makes them difficult to uncover the complicatedly nonlinear relationship between distinct features. In this paper, we address the preceding problem and propose two novel CCA approaches in a supervised manner by using a general covariance metric. The proposed approaches not only consider the label information of training data, but also the nonlinear relationship between different features rather than samples, which leads to greater flexibility in many practical applications. A series of experimental results on five benchmark datasets demonstrate the effectiveness of our proposed methods in terms of classification accuracy.

Short text classification has provoked a vast amount of attention and research in recent decades. However, most existing methods only focus on the short texts that contain dozens of words like Twitter and Microblog, while pay far less attention to the extreme short texts like news headline and search snippets. Meanwhile, contemporary short text classification methods that extend the features via external knowledge sources always introduce lots of useless concepts, which may be detrimental to classification performance. Moreover, unlike traditional short text classification methods, the classification results of extreme short texts are often determined by a few even one or two keywords. To address these problems, we propose a novel hybrid classification method via Keywords Screening and Attention Mechanisms in extreme short text, called KSAM. More specifically, firstly, the attention-based BiLSTM is introduced in our method to enhance the role of keywords. Secondly, we screen the keywords in the extreme short text for obtaining the true class label, and the concepts concerning the keywords are retrieved from external open knowledge sources like DBpedia. Thirdly, the attention mechanisms are introduced to acquire the weight of these retrieved concepts. Finally, conceptual information is utilized to assist the classification of the extreme short text. Extensive experiments have demonstrated the effectiveness of our method compared to other state-of-the-art methods.

Abstract Idioms are a kind of idiomatic expression in Chinese, most of which consist of four Chinese characters. Due to the properties of non-compositionality and metaphorical meaning, Chinese idioms are hard to be understood by children and non-native speakers. This study proposes a novel task, denoted as Chinese Idiom Paraphrasing (CIP). CIP aims to rephrase idiom-containing sentences to non-idiomatic ones under the premise of preserving the original sentence’s meaning. Since the sentences without idioms are more easily handled by Chinese NLP systems, CIP can be used to pre-process Chinese datasets, thereby facilitating and improving the performance of Chinese NLP tasks, e.g., machine translation systems, Chinese idiom cloze, and Chinese idiom embeddings. In this study, we can treat the CIP task as a special paraphrase generation task. To circumvent difficulties in acquiring annotations, we first establish a large-scale CIP dataset based on human and machine collaboration, which consists of 115,529 sentence pairs. In addition to three sequence-to-sequence methods as the baselines, we further propose a novel infill-based approach based on text infilling. The results show that the proposed method has better performance than the baselines based on the established CIP dataset.

Video hashing learns compact representation by mapping video into low-dimensional Hamming space and has achieved promising performance in large-scale video retrieval. It is challenging to effectively exploit temporal and spatial structure in an unsupervised setting. To fulfill this gap, this paper proposes Contrastive Transformer Hashing (CTH) for effective video retrieval. Specifically, CTH develops a bidirectional transformer autoencoder, based on which visual reconstruction loss is proposed. CTH is more powerful to capture bidirectional correlations among frames than conventional unidirectional models. In addition, CTH devises multi-modality contrastive loss to reveal intrinsic structure among videos. CTH constructs inter-modality and intra-modality triplet sets and proposes multi-modality contrastive loss to exploit inter-modality and intra-modality similarities simultaneously. We perform video retrieval tasks on four benchmark datasets, i.e., UCF101, HMDB51, SVW30, FCVID using the learned compact hash representation, and extensive empirical results demonstrate the proposed CTH outperforms several state-of-the-art video hashing methods.

Canonical correlation analysis (CCA) matters in multi-view representation learning. But, CCA and its most variants are essentially based on explicit or implicit covariance matrices. It means that they have no ability to model the nonlinear relationship among features due to intrinsic linearity of covariance. In this paper, we address the preceding problem and propose a novel canonical F-correlation framework by exploring and exploiting the nonlinear relationship between different features. The framework projects each feature rather than observation into a certain new space by an arbitrary nonlinear mapping, thus resulting in more flexibility in real applications. With this frame-work as a tool, we propose a correlative covariation projection (CCP) method by using an explicit nonlinear mapping. Moreover, we further propose a multiset version of CCP dubbed MCCP for learning compact representation of more than two views. The proposed MCCP is solved by an iterative method, and we prove the convergence of this iteration. A series of experimental results on six benchmark datasets demonstrate the effectiveness of our proposed CCP and MCCP methods.

Multi-view hashing (MvH) learns compact hash code by efficiently integrating multi-view data, and has achieved promising performance in large-scale retrieval task. In real-world applications, multi-view data is often stored or collected in different locations, and learning hash code in such case is more challenging yet less studied. In addition, unsupervised MvHs hardly achieve impressive retrieval performance due to absence of supervision. To fulfill this gap, this paper introduces a novel unsupervised multi-view distributed hashing (UMvDisH) to learn hash code from multi-view data, which is distributed in different nodes of a network. UMvDisH jointly performs latent factor model and spectral clustering to generate latent hash code and pseudo label respectively in each node. The consistency between hash code and pseudo label improves discrimination of hash code. The proposed distributed learning problem is divided into a set of decentralized subproblems by imposing local consistency among neighbor nodes. As such, the subproblems can be solved in parallel, and training time can be reduced. The communication cost is low due to no exchange of training data. Experimental results on four benchmark image datasets including a very large-scale image dataset show that UMvDisH achieves comparable retrieval performance and trains faster than state-of-the-art unsupervised MvHs in the distributed setting.

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm.

Summary In this paper, we propose a composite nonlinear multiset canonical correlation projections (CNMCPs) framework where orthogonal constraints are imposed in each set. This makes CNMCP capable of learning uncorrelated low‐dimensional features with minimum redundancy in Hilbert space. With the CNMCP framework, we further present a particular algorithm called multikernel multiset canonical correlations or mKMCC, which introduces different weights into multiple nonlinear functions in all views. An alternating iterative optimization is designed for computational solution. Numerous experimental results on practical datasets have demonstrated the effectiveness and robustness of mKMCC, in contrast with existing kernel correlation learning approaches.

Deep hashing has greatly improved retrieval performance with the powerful learning capability of deep neural network. However, deep unsupervised hashing can hardly achieve impressive performance due to the lack of the semantic supervision. This letter proposes Unsupervised Discriminative Deep Hashing (UD2H) to fulfill this gap. UD2H is formulated to jointly perform hash code learning and clustering, and trained in an asymmetric manner to improve the efficiency. The cluster labels supervise the training of deep model to enable hash code discriminative. Based on the outputs of the deep model, UD2H adaptively constructs a similarity graph that considers the local and global structures. Experiments on three benchmark datasets show that the proposed UD$^2$H outperforms the state-of-the-art unsupervised deep hashing methods.

Component analysis (CA) is a powerful technique for learning discriminative representations in various computer vision tasks. Typical CA methods are essentially based on the covariance matrix of training data. But, the covariance matrix has obvious disadvantages such as failing to model complex relationship among features and singularity in small sample size cases. In this letter, we propose a general covariance measure to achieve better data representations. The proposed covariance is characterized by a nonlinear mapping determined by domain-specific applications, thus leading to more advantages, flexibility, and applicability in practice. With general covariance, we further present two novel CA methods for learning compact representations and discuss their differences from conventional methods. A series of experimental results on nine benchmark data sets demonstrate the effectiveness of the proposed methods in terms of accuracy.

Lexical simplification has attracted much attention in many languages, which is the process of replacing complex words in a given sentence with simpler alternatives of equivalent meaning. Although the richness of vocabulary in Chinese makes the text very difficult to read for children and non-native speakers, there is no research work for the Chinese lexical simplification (CLS) task. To circumvent difficulties in acquiring annotations, we manually create the first benchmark dataset for CLS, which can be used for evaluating the lexical simplification systems automatically. To acquire a more thorough comparison, we present five different types of methods as baselines to generate substitute candidates for the complex word that includes synonym-based approach, word embedding-based approach, BERT-based approach, sememe-based approach, and a hybrid approach. Finally, we design the experimental evaluation of these baselines and discuss their advantages and disadvantages. To our best knowledge, this is the first study for CLS task.

Lexical simplification (LS) aims at replacing complex words with simpler alternatives. LS commonly consists of three main steps: complex word identification, substitute generation, and substitute ranking. Existing LS methods focus on the contextual information of the complex word in the last step (substitute ranking). However, they miss out the following two facts: (1) The word complexity of a polysemous word is very closely related to its context; (2) The step of substitute generation regardless of the context will inevitably produce a large number of spurious candidates. Therefore, we propose a novel LS system LSBert based on pretrained language model BERT to address the aforementioned issues, which is capable of making use of the wider context when both identifying the words in need of simplification and generating substitute candidates for the complex words. Specifically, LSBert consists of a network for complex word identification by fine-tuning BERT and a network for substitute generation based on BERT. Experimental results show that LSBert performs well in both complex word identification and substitute generation, achieving state-of-the-art results in three benchmarks. To facilitate reproducibility, the code of the LSBert system is available at https://github.com/qiang2100/BERT-LS.

Lexical simplification (LS) aims to replace complex words in a given sentence with their simpler alternatives of equivalent meaning. Recently unsupervised lexical simplification approaches only rely on the complex word itself regardless of the given sentence to generate candidate substitutions, which will inevitably produce a large number of spurious candidates. We present a simple LS approach that makes use of the Bidirectional Encoder Representations from Transformers (BERT) which can consider both the given sentence and the complex word during generating candidate substitutions for the complex word. Specifically, we mask the complex word of the original sentence for feeding into the BERT to predict the masked token. The predicted results will be used as candidate substitutions. Despite being entirely unsupervised, experimental results show that our approach obtains obvious improvement compared with these baselines leveraging linguistic databases and parallel corpus, outperforming the state-of-the-art by more than 12 Accuracy points on three well-known benchmarks.

Face hallucination (FH) is an effective technique for super-resolving low-resolution (LR) face images. In real-world applications, a face image usually has multiple distinct low resolutions. Most existing FH methods can not effectively deal with multiple LR views simultaneously. To solve this issue, we present a multi-set partial least squares (MPLS) approach and its kernel extension for jointly learning the nonlinear consistency of multi-resolution facial features. With nonlinear MPLS, we present a novel simultaneous super-resolution coherent facial feature method for the face images with multiple LRs, which has capacity of jointly learning the nonlinear relationships between multiple facial resolutions. Experimental results demonstrate the effectiveness and robustness of our proposed FH method.

In pattern recognition, the representation-based classification (RBC) has attracted much attention recently. As a representative one of RBC, collaborative representation-based classification (CRC) and its variants have achieved promising classification performance in many visual classification tasks. However, most of the CRC methods cannot directly consider the class discrimination information of data that is very important for classification. To fully use the class discrimination information, we propose a novel discriminative group collaborative competitive representation-based classification method (DGCCR) in this paper. In the designed DGCCR model, the discriminative competitive relationships of classes, the discriminative decorrelations among classes and the weighted class-specific group constraints are simultaneously taken into account for strengthening the power of pattern discrimination. Experiments on three visual classification data sets demonstrate that the proposed DGCCR out-performs state-of-the-art RBC methods.

Face super-resolution (FSR) is an effective way to solve low-resolution (LR) problems in face analysis. But, most FSR methods only consider that LR face images have a single resolution, which is usually not consistent with practical situations due to the existence of multiple resolutions. To date, simultaneously learning the mappings from multiple LRs to high resolution (HR) has not been given proper attention. To solve this issue, we first propose a multi-set partial least squares (MPLS) approach to jointly deal with multi-set random variables via a recursive optimization. With MPLS, we then present a novel FSR method called MPLS-FH to simultaneously learn multiple resolution-specific mappings for various LR views from the same source. Concretely, MPLS-FH first divides multi-resolution face images into many patches. Then, it jointly learns the latent coherent features of principal-component embeddings of multi-resolution patches. Last, it super-resolves the input LR face by cross-resolution neighborhood search. Experimental results demonstrate the effectiveness of the proposed method in terms of quantitative and qualitative evaluations.

Dirichlet Multinomial Mixture (DMM) models have been successful in clustering short texts. However, the word co-occurrence information that can be captured by these models is limited to the short text corpus itself. If two words have strong relatedness but rarely co-occurring in short texts, these models can not fully capture the semantic relatedness between the two words. In this paper, we propose a novel model by incorporating word-word correlation into DMM, called WDMM. By constructing a sparse graph using word-word relationship, our model expands each short text using their neighboring words in each text that can help to solve the problem of sparseness in short texts. Therefore, the cluster label of each text is not only influenced by its words, but decided by their similar words in this corpus. Experimental results on real-world datasets demonstrated the substantial superiority of our WDMM model over the state-of-the-art methods.

Low-resolution (LR) face identification is always a challenge in computer vision. In this paper, we propose a new LR face recognition and reconstruction method using deep canonical correlation analysis (DCCA). Unlike linear CCA-based methods, our proposed method can learn flexible nonlinear representations by passing LR and high-resolution (HR) image principal component features through multiple stacked layers of nonlinear transformation. As the nonlinear transformation in deep neural networks is implicit, we apply radial basis function based neural network to learn an explicit mapping between principal components and correlational features. In addition, we also design two residual compensation methods for identification and vision enhancement, respectively. The proposed approach is compared with existing LR face recognition and reconstruction algorithms. A number of experimental results on benchmark datasets have demonstrated the effectiveness and robustness of our method.

Canonical correlation analysis (CCA) is an effective feature learning method, which has wide applications in pattern recognition and computer vision. However, CCA considers the correlation only between the one-to-one aligned samples in two views, ignoring the correlation between all the samples sharing the same label. In this paper, we propose a deep complete canonical correlation analysis (Deep Complete-CCA), which learns the relationships between all pairwise correspondences of sample points in the same classes. Unlike CCA, our method can learn discriminant representations that maximize the correlation between the two views while segregating the different classes on the learned space. We test Deep Complete-CCA on handwriting recognition and speech based emotion recognition using two popular MNIST and RAVDESS datasets. Experimental results show that our proposed method can obtain better performances than several related algorithms.

Low resolution is one of the main obstacles in the application of face recognition. Although many methods have been proposed to improve the problem, they assume that low-resolution (LR) face images have a uniform scale. In real scenarios, this prerequisite is very harsh. In this paper, we propose a scale-adaptive LR face recognition approach based on two-dimensional multi-set canonical correlation analysis (2DM-CCA), where face image matrix does not need to be previously transformed into a vector. In the proposed method, training sets with different resolutions are treated as different views, and then projected in parallel into a latent coherent space where the consistency of multi-view face data is maximally enhanced. When a new LR face image with an arbitrary scale is input, we first transform it by using the left and right projection matrices of an appropriate training view, and then reconstruct its high resolution facial feature by neighborhood reconstruction. Experimental results show that our proposed method is more effective and efficient than several existing methods.

Hashing techniques have recently gained increasing research interest in multimedia studies. Most existing hashing methods only employ single features for hash code learning. Multiview data with each view corresponding to a type of feature generally provides more comprehensive information. How to efficiently integrate multiple views for learning compact hash codes still remains challenging. In this article, we propose a novel unsupervised hashing method, dubbed multiview discrete hashing (MvDH), by effectively exploring multiview data. Specifically, MvDH performs matrix factorization to generate the hash codes as the latent representations shared by multiple views, during which spectral clustering is performed simultaneously. The joint learning of hash codes and cluster labels enables that MvDH can generate more discriminative hash codes, which are optimal for classification. An efficient alternating algorithm is developed to solve the proposed optimization problem with guaranteed convergence and low computational complexity. The binary codes are optimized via the discrete cyclic coordinate descent (DCC) method to reduce the quantization errors. Extensive experimental results on three large-scale benchmark datasets demonstrate the superiority of the proposed method over several state-of-the-art methods in terms of both accuracy and scalability.

Multiple view data with different feature representations have widely arisen in various practical applications. Due to the information diversity, fusing multiview features is very valuable for classification purpose. In this paper, we propose a new multifeature fusion method called fractional-order discriminative multiview correlation projection (FDMCP), which is based on fractional-order scatter matrices with class label information of the samples. FDMCP first defines supervised covariance matrices in each view. It then constructs fractional supervised scatter matrices. Experimental results on three benchmark face image datasets show that our proposed FDMCP approach outperforms generalized multiview linear discriminant analysis.

Before inferring the real number of clusters in short text clustering, Dirichlet Multinomial Mixture (DMM) model makes assumption that there are at most Kmax clusters. In some cases, it is difficult to choose a proper Kmax beforehand. In the paper, we propose a novel model based on Pitman-Yor Process to capture the power-law phenomenon of the cluster distribution. Specifically, each text chooses one of the active clusters or a new cluster with probabilities derived from the Pitman-Yor Process Mixture model (PYPM). Different from DMM model, our model does not require Kmax as input. Discriminative words and nondiscriminative words are identified automatically to help enhance text clustering. Parameters are estimated efficiently by collapsed Gibbs sampling. The experiments on real-world datasets validate the effectiveness of the proposed model in comparison with other state-of-theart models.

Summary Vertical data structure is very important for closed frequent itemset mining. All closed frequent itemsets can be found by simply using the operations of AND/OR. However, it consumes a large amount of storage space, especially in the case of large‐size dataset. This paper proposes an algorithm for mining closed frequent itemsets based on a new vertical data structure. The proposed data structure is helpful to save storage space by using a multi‐layer index. At the same time, numerous CPU and graphics processing unit can be employed in parallel to achieve high‐efficiency computing. Especially when dealing with large datasets, the proposed algorithm can obtain a high‐speed computing with the help of graphics processing unit. The improved vertical structure reduces the storage space of the data. The experimental results show that our proposed algorithm requires much less computation time than other related methods. Copyright © 2016 John Wiley & Sons, Ltd.

Due to the significant reduction in computational cost and storage, hashing techniques have gained increasing interests in facilitating large-scale cross-view retrieval tasks. Most cross-view hashing methods are developed by assuming that data from different views are well paired, e.g., text-image pairs. In real-world applications, however, this fully-paired multiview setting may not be practical. The more practical yet challenging semi-paired cross-view retrieval problem, where pairwise correspondences are only partially provided, has less been studied. In this paper, we propose an unsupervised hashing method for semi-paired cross-view retrieval, dubbed semi-paired discrete hashing (SPDH). In specific, SPDH explores the underlying structure of the constructed common latent subspace, where both paired and unpaired samples are well aligned. To effectively preserve the similarities of semi-paired data in the latent subspace, we construct the cross-view similarity graph with the help of anchor data pairs. SPDH jointly learns the latent features and hash codes with a factorization-based coding scheme. For the formulated objective function, we devise an efficient alternating optimization algorithm, where the key binary code learning problem is solved in a bit-by-bit manner with each bit generated with a closed-form solution. The proposed method is extensively evaluated on four benchmark datasets with both fully-paired and semi-paired settings and the results demonstrate the superiority of SPDH over several other state-of-the-art methods in term of both accuracy and scalability.

This paper presents a region-based composite kernel framework for spatial-spectral hyperspectral image classification, referred as RCK, by exploiting the local similarities of both the spectral and spatial features via superpixel segmentation. The proposed framework consists of three steps. In the first step, the original hyperspectral image together with its spatial feature image are segmented into several nonoverlapping regions by using an efficient superpixel segmentation algorithm. In the second step, a mean filtering is performed within each region of both the spectral feature image and the spatial feature image to generate the corresponding region-based spectral and spatial features, respectively. In the final step, both the obtained region-based features are combined and incorporated into a probabilistic kernel collaboration classifier by taking advantage of a composite kernel framework. Experimental results on two real hyperspectral images demonstrate the improvement of RCK over the traditional composite kernel framework, as well as its effectiveness as compared to some popular spatial-spectral techniques.

Locality-based feature learning has drawn more and more attentions recently. However, most of locality-based feature learning methods only consider a kind of local neighbor information, and such the locality-based methods are difficult to well reveal intrinsic geometrical structure of raw high-dimensional data. In this paper, we propose a novel multi-locality correlation feature learning algorithm for multi-view data, called multi-locality discrimination canonical correlation analysis (MLDCCA), which can learn nonlinear correlation features with strong discriminative power. Different from the locality-based methods, our algorithm not only employs multiple local patches of each raw data to well capture the intrinsic geometrical structure information, but also fully considers intraclass scatter information for further enhancing the class separability of the learned correlation features. Extensive experimental results on several real-word image datasets have demonstrated the effectiveness of our algorithm.

canonical correlations (M 2 CCs) framework for subspace learning. In the proposed framework, the input data of each original view are mapped into multiple higher dimensional feature spaces by multiple nonlinear mappings determined by different kernels. This makes M 2 CC can discover multiple kinds of useful information of each original view in the feature spaces. With the framework, we further provide a specific multi-view feature learning method based on direct summation kernel strategy and its regularized version. The experimental results in visual recognition tasks demonstrate the effectiveness and robustness of the proposed method.

Hashing techniques have been widely applied to large-scale cross-view retrieval tasks due to the significant advantage of binary codes in computation and storage efficiency. However, most existing cross-view hashing methods learn binary codes with continuous relaxations, which cause large quantization loss across views. To address this problem, in this letter, we propose a novel cross-view hashing method, where a common Hamming space is learned such that binary codes from different views are consistent and comparable. The quantization loss across views is explicitly reduced by two carefully designed regression terms from original spaces to the Hamming space. In our method, the l2,1-norm regularization is further exploited for discriminative feature selection. To obtain high-quality binary codes, we propose to jointly learn the codes and hash functions, for which an efficient iterative algorithm is presented. We evaluate the proposed method, dubbed Robust Cross-view Hashing (RCH), on two benchmark datasets and the results demonstrate the superiority of RCH over many other state-of-the-art methods in terms of retrieval performance and cross-view consistency.

This paper presents a new approach for accurate spatial-spectral classification of hyperspectral images, which consists of three main steps. First, a pixelwise classifier, i.e., the probabilistic-kernel collaborative representation classification (PKCRC), is proposed to obtain a set of classification probability maps using the spectral information contained in the original data. This is achieved by means of a kernel extension based on collaborative representation (CR) classification. Then, an adaptive weighted graph (AWG)-based postprocessing model is utilized to include the spatial information by refining the obtained pixelwise probability maps. Furthermore, to deal with scenarios dominated by limited training samples, we modify the postprocessing model by fixing the probabilistic outputs of training samples to integrate the spatial and label information. The proposed approach is able to cover different analysis scenarios by means of a fully adaptive processing chain (based on three steps) for hyperspectral image classification. All the techniques that integrate the proposed approach have a closed-form analytic solution and are easy to be implemented and calculated, exhibiting potential benefits for hyperspectral image classification under different conditions. Specifically, the proposed method is experimentally evaluated using two real hyperspectral imagery data sets, exhibiting good classification performance even when the number of training samples available a priori is very limited.

Hashing techniques have attracted broad research interests in recent multimedia studies. However, most of existing hashing methods focus on learning binary codes from data with only one single view, and thus cannot fully utilize the rich information from multiple views of data. In this paper, we propose a novel unsupervised hashing approach, dubbed multi-view latent hashing (MVLH), to effectively incorporate multi-view data into hash code learning. Specifically, the binary codes are learned by the latent factors shared by multiple views from an unified kernel feature space, where the weights of different views are adaptively learned according to the reconstruction error with each view. We then propose to solve the associate optimization problem with an efficient alternating algorithm. To obtain high-quality binary codes, we provide a novel scheme to directly learn the codes without resorting to continuous relaxations, where each bit is efficiently computed in a closed form. We evaluate the proposed method on several large-scale datasets and the results demonstrate the superiority of our method over several other state-of-the-art methods.

Multiset features extracted from the same patterns always represent different characteristics of data. Thus, it is very valuable to perform the extraction on multiple feature sets. This paper addresses the issue of multiset correlation feature extraction (MCFE) in multiple feature representations. A novel method is proposed to carry out the MCFE for classification, called multiset canonical correlations using globality-preserving projections (MCC-GPs), which can perform joint dimensionality reduction for high-dimensional data. MCC-GP integrates correlational characteristics of feature pairs and global geometric information of data in the transformed low-dimensional space. This makes MCC-GPs have better discriminant ability than a previous method proposed by the authors, called multiset integrated canonical correlation analysis (MICCA), which only considers correlations for recognition tasks. Furthermore, MCC-GP can subsume two popular feature extraction methods into its framework under some constraints. This also provides a new insight for these two methods. The proposed method is applied to pattern recognition and examined using the COIL-100 and ETH-80 object databases and AR, CMU PIE, and Yale face databases. Extensive experimental results show that MCC-GP outperforms MICCA and multiset canonical correlation analysis in terms of classification accuracy and efficiency.