Cross-scene hyperspectral image (HSI) recognition aims to assign a unique label to each pixel in the target scene by transferring knowledge from the source scene. Existing methods primarily rely on fully labeled source data and either partially labeled or unlabeled target data. No prior work has addressed the more challenging scenario of cross-scene recognition without label guidance in both scenes. To bridge this gap, we present the first study on cross-scene HSI clustering, proposing an anchor-guided discriminative subspace alignment and clustering (ADSAC) framework that follows a well-structured three-step learning paradigm to effectively mitigate distribution shifts. Specifically, we first develop an anchor-promoted graph learning (APGL) model to efficiently derive accurate clustering labels for the source scene by leveraging anchor-based structural information. Next, we propose a discriminative cross-scene subspace alignment (DCSA) model to improve feature discriminability and reduce distribution discrepancies. Finally, labels of the target scene are inferred after source clustering and cross-scene alignment. To solve the formulated models, we design tailored optimization algorithms to ensure high-quality learning. Extensive experiments demonstrate the superiority of the proposed framework over state-of-the-art methods.

Multi-view clustering of remote sensing data presents significant challenges, as it integrates diverse data representations to improve Earth observation. Although existing anchor graph-based methods have yielded promising results, they generally exhibit two key limitations: (1) the time-consuming process of directly exploring pixel clustering structures, and (2) insufficient modeling of high-order correlations among different views. To address these issues, we propose an Efficient Tensorized multi-view anchor graph clustering method with Affinity Propagation (ETAP) for remote sensing data. Based on superpixel preprocessing, anchor graphs are learned from view-specific pixels and anchors, while compressed anchor graphs are simultaneously learned from the view-specific anchors. An adaptive weighting scheme is introduced to facilitate the learning of these anchor graphs. To capture high-order correlations, tensor Schatten p-norm regularization is applied to the compressed anchor graphs. A connectivity constraint is introduced to uncover the clustering structures of anchors. Finally, pixel clustering structures are then efficiently revealed from the pseudo-labeled anchors through affinity propagation without requiring additional clustering steps. To solve the proposed formulation, we develop an alternating optimization algorithm. Extensive experiments on three public datasets demonstrate the efficacy and efficiency of the proposed method over state-of-the-art methods.

The hyperspectral anomaly detection (HAD) aims to identify potential anomalies from complex backgrounds. Most reconstruction-based autoencoders equally treat background pixels and anomalies or ignore potential spatial information. In this letter, we propose an HAD method based on multiscale memory autoencoder and spatial filtering, abbreviated as SFM2AE. Specifically, by introducing memory modules into different hidden layers of the autoencoder, multiscale reconstruction of background and anomaly pixels is achieved in the spectral domain. In addition, morphological filtering in the spatial domain is used to extract spatial structural information from anomalies. Joint spatial-spectral anomaly detection is achieved by combining multiscale memory autoencoder and spatial filtering. Experiments demonstrate superior detection performance of the proposed method over the state-of-the-art methods.

Hyperspectral image (HSI) classification is a key task in the field of remote sensing, aiming to assign category labels to each pixel by leveraging the spectral and spatial information in HSIs. Recently, many deep learning (DL) methods, such as convolutional neural networks (CNNs) and Transformers, have been applied to this task, achieving significant results. However, most existing patch-based DL methods often overlook the potential relationships between the central pixel and its surrounding pixels. Additionally, the unique spectral characteristics of HSIs, such as the high correlation between adjacent spectral bands and low dependence between distant bands, also require special attention. Based on this, we propose a novel dual-branch convolution-Transformer network with spectral-spatial attention (CTSSA), which can effectively aggregate both local and global spectral-spatial features. Specifically, CTSSA comprises two core modules: the Pyramid Spectral Attention Module (PSAM) and the Center Transformer Encoder (CenterTE). The former extracts highly discriminative spectral features through a hierarchical multi-scale attention mechanism, capturing subtle differences between adjacent spectral bands. The latter improves the original Transformer encoder (TE) by introducing a center-attention mechanism to model the global relationship between the central pixel and its surrounding pixels, thereby enhancing classification accuracy while reducing computational complexity. Experimental results on four public datasets (Salinas, Pavia University, Houston, and WHUHi-LongKou) demonstrate that, compared with nine other networks, CTSSA achieves satisfactory performance with fewer parameters and relatively high efficiency.

Band selection aims at selecting a subset of representative bands from original hyperspectral images (HSIs) to alleviate data redundancy. There are at least two issues existing in previous methods. First, most of them ignore global or local structural information without considering both two aspects. Second, the high-order correlations among spectral bands are not explored during learning. In this paper, we propose a tensorial global-local graph self-representation (TGSR) method for hyperspectral band selection. Specifically, we segment the HSI into diverse superpixels to show the inherent spectral-spatial structures. Based on the generated superpixels, we learn the global and local graphs to explore complex structural information from global pixels and local regions. To alleviate the computational burden, a transformation is designed for easy graph convolution of global graph and pixel spectral matrix. With global and local knowledge, we formulate a global-local graph self-representation model to conduct band correlation learning in a self-weighted manner. To explore the high-order correlations among bands, we reorganize the self-representation coefficient matrices into a tensor with low-rank constraint. We design an alternating optimization algorithm to solve the proposed model. The most representative band is selected from each band subset by performing spectral clustering on the constructed affinity matrix. Experiments on HSI datasets verify the effectiveness of our method over the state-of-the-art methods. The source code is released athttps://github.com/ZhangYongshan/TGSR.

Multimodal remote sensing image recognition aims to identify a category of land cover for every pixel with consistency and complementary information provided by different modalities. Most existing methods perform land cover recognition in a supervised manner with explicit label guidance. It is challenging to perform recognition without label guidance due to the complex spatial distribution and modality incompatibility, especially for large-scale data. In this article, we propose a dual graph learning affinity propagation (DGLAP) method for multimodal remote sensing image clustering. Based on the consistent spatial distribution from local regions, the proposed method learns an$N \times M$consensus anchor graph from N denoised pixels and M anchors by adaptive weighting different modalities along with projection learning. Meanwhile, an optimal$M \times M$compressed consensus anchor graph is learned from the updated anchors in different modalities with diverse adaptive contributions and connectivity constraint. Since$M \ll N$, clustering results can be efficiently obtained according to affinity propagation from the pseudolabeled anchors to the pixels without additional steps. An alternating optimization algorithm is devised to solve the proposed formulation. This is the first attempt to propose a ultraefficient graph-based clustering method with linear time complexity$\mathcal {O}(N)$and low time cost for large-scale multimodal remote sensing data. Extensive experiments on three datasets demonstrate the superiority of the proposed method over the state-of-the-art methods in both efficacy and efficiency. The code is released athttps://github.com/ZhangYongshan/DGLAP.

Hyperspectral image (HSI) clustering is challenging to divide all pixels into different clusters because of the absent labels, large spectral variability and complex spatial distribution. Anchor strategy provides an attractive solution to the computational bottleneck of graph-based clustering for large HSIs. However, most existing methods require separated learning procedures and ignore noisy as well as spatial information. In this paper, we propose a bipartite graph-based projected clustering (BGPC) method with local region guidance for HSI data. To take full advantage of spatial information, HSI denoising to alleviate noise interference and anchor initialization to construct bipartite graph are conducted within each generated superpixel. With the denoised pixels and initial anchors, projection learning and structured bipartite graph learning are simultaneously performed in a one-step learning model with connectivity constraint to directly provide clustering results. An alternating optimization algorithm is devised to solve the formulated model. The advantage of BGPC is the joint learning of projection and bipartite graph with local region guidance to exploit spatial information and linear time complexity to lessen computational burden. Extensive experiments demonstrate the superiority of the proposed BGPC over the state-of-the-art HSI clustering methods.

Hyperspectral image (HSI) clustering seeks to assign each pixel to a specific class without trained labels. This is a challenging task owing to the spatial and spectral complexity. Recently, anchor graph-based clustering has attracted considerable attention due to its flexibility in handling large-scale HSI data. However, these methods typically disregard noisy bands and require post-processing. To tackle these issues, we propose a structured-anchor projected clustering (SAPC) model for HSIs. In SAPC, the projection clustering is introduced into anchor graph learning to suppress noise, and the Laplacian rank constraints can quickly obtain the structure of anchors. Thus, we can directly obtain the clustering results through the anchor graph and the structured anchors. Moreover, we propose an iterative optimization method to efficiently solve the SAPC model. Extensive experiments show that our model achieves superior results.

Hyperspectral anomaly detection (HAD) aims to discern the objects deviated dramatically from their surrounding pixels. Some deep learning-based models integrating with the low-rank representation (LRR) have been proposed recently. The process of constructing dictionary in these methods is complex and the stability of the models is hard to maintain. To address these problems, in this paper, we propose a low-rank constrained memory autoencoder (LRMAE) for HAD. Specifically, we first train the memory autoencoder with a sparsity regularizer and a spectral consistency constraint in an unsupervised learning fashion and the embedded memory module is used to acquire the dictionary atoms for the construction of dictionary. The low-rank optimization process is conducted in the low-dimensional manifold space to obtain the final detection map. Substantial experiments are performed on three different datasets, and the final detection results demonstrate the superiority of the proposed model over other state-of-the-art methods.

Hyperspectral band selection has been proved to be effective in reducing redundant information for hyperspectral images (HSIs). Most existing band selection methods simply consider the relationship between bands by reshaping them into vectors and destroying the spatial structure. Moreover, the converted band vectors are usually high-dimensional, making the learning processing very time-consuming. To solve these problems, we propose a tensor decomposition based latent feature clustering (TDLFC) model for band selection. We maintain the tensor structure of the HSI and use CANDECOMP/PARAFAC (CP) decomposition to learn the latent low-dimensional representation of the bands to preserve spatial and spectral information. To avoid overfitting, we introduce a regularization term for the CP decomposition model. To solve the proposed model, we present an effective optimization algorithm as solution. Finally, the k-means algorithm is applied to the latent representation to get the band clustering results for band selection. Extensive experiments on three public HSI datasets show the superiority of our proposed model over the state-of-the-art methods.

Hyperspectral image (HSI) clustering has attracted great attention in the field of remote sensing. General anchor-based clustering methods often suffer from the problems of unstable anchor selection and insufficient utilization of spatial information, resulting in poor clustering performance. In this letter, a spectral-spatial superpixel anchor graph-based clustering (S3AGC) method is proposed for HSIs. Specifically, S3AGC further improves the clustering performance by simultaneously considering spatial and structural information as well as an advanced anchor selection strategy. Based on the spatial distribution, a useful HSI denoising solution is presented to reduce noise interference, and an effective anchor selection strategy is raised to alleviate the instability of random or clustering selection. Besides, we use a graph convolution method to embed structural information of spectral bands into the proposed framework. Experiments on HSI datasets verify the effectiveness of S3AGC.

Hyperspectral images (HSIs) are typically utilized in a wide variety of practical applications. HSI is replete with spatial and spectral information, which provides precise data for material detection. HSIs are characterized by a high degree of variations and undesirable pixel distributions, providing major processing challenges. This article introduces the fast hyperspectral image classification (FHIC) model, a rapid model for classifying HSIs and resolving their associated challenges. It uses the enhancing transformation reduction (ETR) method to address the HSI difficulties and enhance classes’ differentiation. It also uses exponential linear units (ELU) to smooth and speed the classification processing. The structure of the FHIC model is designed to be very flexible and suitable for a range of HSIs. The model reduced execution time and RAM consumption and provided superior performance compared to seven of the most advanced analysis models, for three well-known HSIs. In some cases, it was 60% faster than other models. In addition, this work presents a new and highly effective method for measuring the performance of the compared models in terms of their accuracy and processing speed to provide an easy evaluation method. The code of the FHIC model is available at this link: https://github.com/DalalAL-Alimi/FHIC.

Dimensionality reduction (DR) is important for feature extraction and classification of hyperspectral images (HSIs). Recently proposed superpixel-based DR models have shown promising performance, where superpixel segmentation techniques were applied to segment an HSI and then DR models like principal component analysis (PCA) or linear discriminant analysis (LDA) were employed to extract the local and/or global features. However, superpixelwise PCA based local features are unsatisfactory because PCA aims to extract features with high variance, which could be inefficient in superpixels with mixed objects or strong noise/outliers. In addition, superpixelwise unsupervised LDA based global features may neglect local (spatial-contextual) information. To address these issues, we propose a new spectral-spatial and superpixelwise unsupervised LDA (S3-ULDA) model for unsupervised feature extraction from HSIs. Specifically, the HSI is first segmented into various superpixels with pseudo labels. Then, superpixel based local reconstruction for HSI denoising is conducted. Next, superpixelwise unsupervised LDA (SuperULDA) is performed on both the original HSI and locally reconstructed data to extract global features. Then, superpixelwise unsupervised local Fisher discriminant analysis (SuperULFDA) is developed for local feature extraction, where each superpixel and its adjacent superpixels (along with their pseudo-labels) are fed into local Fisher discriminant analysis (LFDA) to extract local features. The superpixel-level local manifold structures can be effectively modeled by the proposed SuperULFDA. Finally, by fusing the extracted global and local features, novel global-local and spectral-spatial features can be obtained. Our experimental results on several benchmark HSIs demonstrate the superiority of the proposed method over state-of-the-art methods. The code of the proposed model is available at https://github.com/XinweiJiang/S3-ULDA.

This article presents FLGC, a simple yet effective fully linear graph convolutional network for semi-supervised and unsupervised learning. Instead of using gradient descent, we train FLGC based on computing a global optimal closed-form solution with a decoupled procedure, resulting in a generalized linear framework and making it easier to implement, train, and apply. We show that (1) FLGC is powerful to deal with both graph-structured data and regular data, (2) training graph convolutional models with closed-form solutions improve computational efficiency without degrading performance, and (3) FLGC acts as a natural generalization of classic linear models in the non-Euclidean domain (e.g., ridge regression and subspace clustering). Furthermore, we implement a semi-supervised FLGC and an unsupervised FLGC by introducing an initial residual strategy, enabling FLGC to aggregate long-range neighborhoods and alleviate over-smoothing. We compare our semi-supervised and unsupervised FLGCs against many state-of-the-art methods on a variety of classification and clustering benchmarks, demonstrating that the proposed FLGC models consistently outperform previous methods in terms of accuracy, robustness, and learning efficiency. The core code of our FLGC is released at https://github.com/AngryCai/FLGC .

Superpixel based label propagation models have been successfully used for Hyperspectral Images (HSIs) classification especially when the training data are limited. However, it is inevitable that there are segmentation errors leading to data in one superpixel containing samples from different classes which could decrease the classification accuracy. In order to address this issue, we propose Superpixel Correction based Label Propagation for HSIs classification. First, superpixel segmentation technique is adopted to segment a HSI into many superpixel blocks. Then, clustering model density peak is used to adaptively cluster the data in each superpixel block to correct the segmentation errors. Finally, based on the corrected superpixel segmentation we construct global-local and spatial-spectral similarity graphs which results into effective propagation matrix for label propagation. The proposed model is verified in two HSIs data sets, and the experimental results demonstrate that the proposed model is superior to several state-of-the-art methods in terms of classification accuracy, especially in the case of limited training samples.

Deep neural networks have gained increasing interest in hyperspectral image (HSI) processing. However, prior arts often neglect the high-order correlation among data points, failing to capture intraclass variations. In this letter, we present a unified neural network framework, termed as hypergraph-structured autoencoder (HyperAE), to leverage the high-order relationship among data and learn robust deep representation for downstream tasks. Technically, the proposed method adopts a deep autoencoder regularized by hypergraph structure as the backbone network, which is jointly trained with a task-specific branch, resulting in a multitask architecture. We separately combine the subspace clustering model and the softmax classifier into the HyperAE to deal with HSI unsupervised and semisupervised classification problems. Benefiting from the hypergraph, HyperAE endows traditional networks with the capacity of preserving the high-order structured information. We evaluate the proposed methods on three benchmarking HSI data sets, demonstrating that the proposed HyperAE dramatically outperforms many existing methods with significant margins in both unsupervised and semisupervised HSI classification problems.

The bilateral filter (BF) is a nonlinear filtering method, which can remove noise and retain better edge information. It has been widely used in the field of hyperspectral images (HSIs) filtering. In this letter, we propose a novel spectral-spatial information integration method based on the BF with multispatial domain (MBF). The proposed method includes three steps. First, principal component analysis (PCA) is used for the original HSI to obtain multiple components containing almost all information; second, multiple principal components are used as both spatial domain and range domain information for BF; finally, the extreme learning machine (ELM) is used for classification. To verify the effectiveness of the proposed approach, we evaluate performance on three benchmark data sets. Our method will improve the existing filtering methods by constructing multiple spatial domains for filtering, which will make more effective use of spatial features and solve the problem of lack of spatial information in HSIs. This method is compared with other filtering algorithms. Comparative experiments show that our proposed method can improve the classification accuracy. And the MBF information is more effective than the BF with single spatial domain information and other filtering methods.

Hyperspectral images (HSIs) consisting of abundant spectral bands could lead to the curse of dimensionality issue when performing HSIs classification. In this letter, an unsupervised dimensionality reduction (DR) method termed Laplacian regularized collaborative representation projection (LRCRP) is proposed, where Laplacian regularization and local enhancement are introduced into collaborative representation (CR) to construct adjacent graph and then to reduce the spectral dimension in graph embedding framework. As the constructed graph simultaneously preserves the local manifold and global information in HSIs, the proposed LRCRP could be used to extract effective low-dimensional features for accurate HSIs classification. The experimental results on two HSI datasets demonstrate the effectiveness of the proposed model. The source code the proposed model is available athttps://github.com/XinweiJiang/LRCRP.

Hyperspectral image (HSI) clustering has attracted a great deal of attention, owing to lower cost and higher application prospects. Deep subspace clustering has been proved to be an effective method to explore the sample relationship of HSI clustering. However, due to the complex distribution of HSI data, the problem of data cluster overlap occurs frequently. In the actual sample distribution, a sample may belong to multiple subspaces. The complex sample distribution brings challenges to subspace clustering. In this letter, we propose a deep mutual information subspace clustering network (DMISC) to find a more intuitive feature space for non-linear subspace clustering. Technically, we maximize the mutual information between the samples and their generated features to enlarge the inter-class dispersion and intra-class compactness. The deep subspace method can find a more suitable non-linear intrinsic relationship, benefitting from the generated feature distribution. We evaluate DMISC on four HSI data sets and compare the performances with 12 popular clustering methods. The experiment results demonstrate our method outperforms many prior unsupervised methods.

Deep subspace clustering has achieved remarkable performances in the unsupervised classification of hyperspectral images. However, previous models based on pixel-level self-expressiveness of data suffer from the exponential growth of computational complexity and access memory requirements with increasing number of samples, thus leading to poor applicability to large hyperspectral images. This paper presents a Neighborhood Contrastive Subspace Clustering network (NCSC), a scalable and robust deep subspace clustering approach, for unsupervised classification of large hyperspectral images. Instead of using a conventional autoencoder, we devise a novel superpixel pooling autoencoder to learn the superpixel-level latent representation and subspace, allowing a contracted self-expressive layer. To encourage a robust subspace representation, we propose a novel neighborhood contrastive regularization to maximize the agreement between positive samples in subspace. We jointly train the resulting model in an end-to-end fashion by optimizing an adaptively weighted multi-task loss. Extensive experiments on three hyperspectral benchmarks demonstrate the effectiveness of the proposed approach and its substantial advancement of state-of-the-art approaches.

Convolutional neural networks (CNNs) are widely used in hyperspectral image (HSI) classification. However, the network architecture of CNNs is often designed manually, which requires careful fine-tuning. Recently, many techniques for neural architecture search (NAS) have been proposed to design the network automatically but most of the methods are only concerned with the overall classification accuracy and ignore the balance between the floating point operations per second (FLOPs) and the number of parameters. In this paper, we propose a new multi-objective optimization (MO) method called MO-CNN to automatically design the structure of CNNs for HSI classification. First, a MO method based on continuous particle swarm optimization (CPSO) is constructed, where the overall accuracy, floating point operations (FLOPs) and the number of parameters are considered, to obtain an optimal architecture from the Pareto front. Then, an auxiliary skip connection strategy is added (together with a partial connection strategy) to avoid performance collapse and to reduce memory consumption. Furthermore, an end-to-end band selection network (BS-Net) is used to reduce redundant bands and to maintain spectral-spatial uniformity. To demonstrate the performance of our newly proposed MO-CNN in scenarios with limited training sets, a quantitative and comparative analysis (including ablation studies) is conducted. Our optimization strategy is shown to improve the classification accuracy, reduce memory and obtain an optimal structure for CNNs based on unbiased datasets.

As the most classical unsupervised dimension reduction algorithm, principal component analysis (PCA) has been widely used in hyperspectral images (HSIs) preprocessing and analysis tasks. Recently proposed superpixelwise PCA (SuperPCA) has shown promising accuracy where superpixels segmentation technique was first used to segment an HSI to various homogeneous regions and then PCA was adopted in each superpixel block to extract the local features. However, the local features could be ineffective due to the neglect of global information especially in some small homogeneous regions and/or in some large homogeneous regions with mixed ground truth objects. In this article, a novel spectral–spatial and SuperPCA (S3-PCA) is proposed to learn the effective and low-dimensional features of HSIs. Inspired by SuperPCA we further adopt superpixels-based local reconstruction to filter the HSIs and use the PCA-based global features as the supplement of local features. It turns out that the global–local and spectral–spatial features can be well exploited. Specifically, each pixel of an HSI is reconstructed by the nearest neighbors’ pixels in the same superpixel block, which could eliminate the noise and enhance the spatial information adaptively. After the local reconstruction-based data preprocessing, PCA is performed on each region and the entire HSI to obtain local and global features, respectively. Then we simply concatenate them to get the global–local and spectral–spatial features for HSIs classification. The experimental results on two HSIs data sets demonstrate the superiority of the proposed method over the state-of-the-art methods. The source code of the proposed model is available athttps://github.com/XinweiJiang/S3-PCA.

In image analysis, image samples from multiple sources may contain noisy features. Due to the difficulty of obtaining label information and complex intrinsic structures, performing unsupervised feature selection on multi-view data is a challenging problem. Most existing unsupervised multi-view feature selection methods may explore only the inter-view correlations at the view-level, and ignore the explicit correlations between features across multiple views. In this paper, we propose a tensor-based unsupervised multi-view feature selection (TUFS) method. Specifically, TUFS efficiently explores the full-order interactions among multi-view data without physically building a tensor. Besides, multiple local geometric structures for different views are constructed to facilitate unsupervised feature selection. To solve the proposed model, we design an alternating optimization algorithm. Experiments and comparisons on three image datasets demonstrate that the proposed TUFS yields better performance over the state-of-the-art methods.

Recently, deep learning-based methods have made great progress in hyperspectral image (HSI) classification (HSIC). Different from ordinary images, the intrinsic complexity of HSIs data still limits the performance of many common convolutional neural network (CNN) models. Thus, the network architecture becomes more and more complex to extract discriminative spectral-spatial features. For instance, 3-D CNN usually has a large number of trainable parameters, thus increasing the computational complexity of the HSIC. In this letter, we designed a cooperative spectral-spatial attention dense network (CS2ADN) that takes raw 3-D HSI data as input data. Specifically, the attention module consists of spectral and spatial axes, by which the salient spectral-spatial features will be emphasized. Furthermore, we combined these attention modules with the dense connection, which is termed as the lightweight dense block; it has a lower computation cost and achieves better classification performance. At the same time, we introduced the center loss, by jointly using the supervision of the center loss and the softmax loss, where the discriminative features could be clearly observed, particularly for small data sets. Experimental results on the biased and unbiased HSI data show that our method outperforms several state-of-the-art methods in HSIC with small training samples.

Hyperspectral image (HSI) clustering is a challenging task due to the high complexity of HSI data. Subspace clustering has been proven to be powerful for exploiting the intrinsic relationship between data points. Despite the impressive performance in the HSI clustering, traditional subspace clustering methods often ignore the inherent structural information among data. In this article, we revisit the subspace clustering with graph convolution and present a novel subspace clustering framework called graph convolutional subspace clustering (GCSC) for robust HSI clustering. Specifically, the framework recasts the self-expressiveness property of the data into the non-Euclidean domain, which results in a more robust graph embedding dictionary. We show that traditional subspace clustering models are the special forms of our framework with the Euclidean data. On the basis of the framework, we further propose two novel subspace clustering models by using the Frobenius norm, namely efficient GCSC (EGCSC) and efficient kernel GCSC (EKGCSC). Each model has a globally optimal closed-form solution, making it easier to implement, train, and apply in practice. Extensive experiments strongly evidence that EGCSC and EKGCSC dramatically outperform current models on three popular HSI data sets consistently.

Deep convolutional neural networks(CNNs) have been widely used in hyperspectral image(HSI) classification. However, these CNNs architectures are all handcrafted, which need professional knowledge and consume very significant time. In order to automatically search for cell-based CNNs architectures for HSI classification, we proposed an cell-based CNNs architecture search method by particle swarm optimization(PSO), which is capable of getting the global optimal architecture compared with the gradient descent method. First, the cell-based search space is structured. Then, a novel directly encoding strategy is devised to encode architectures into particles. Finally, PSO is used to search for the optimal deep architecture from the particle swarm. Furthermore, 1-D PSO-NET and 3-D PSO-NET based on PSO-NET are used as spectral and spectral-spatial HSI classifiers respectively. The experiments on two widely used datasets of hyperspectral image show that the method we proposed achieve good performance.

Extreme Learning Machine (ELM) has gained lots of research interest due to its universal approximation capability and fast learning speed. However, traditional ELMs are devised for regular Euclidean data, such as 2D grid and 1D sequence, and thus don't apply to non-Euclidean data, e.g., graph-structured data. To overcome this shortcoming, this paper presents a Graph Convolutional Extreme Learning Machine (termed as GCELM) for semi-supervised classification. Technically, a random graph convolutional layer is introduced to replace the random projection of original ELM, which endues ELM with the capability of dealing with graph-structured data directly. To generate a robust graph from the raw dataset, a self-representation model is adopted to construct a weighted graph. Extensive experiments on 27 UCI datasets demonstrate that GCELM outperforms many popular semi-supervised methods, and with faster learning speed. To the best of our knowledge, this is the first work that combines graph convolution with ELM.

Classification of hyperspectral images (HSIs) by making full use of the spectral and the spatial information has become a research hotspot in the field of remote sensing technology. Aiming at the problems of information redundancy and low utilization of spatial information, this letter proposes a visual saliency-based extended morphological profile (VS-EMP) scheme. First, the morphological features are extracted by the EMP from the HSIs on several principal components. Second, the local binary pattern (LBP) is performed to extract the texture features from morphological scenes. Third, saliency features are captured according to the texture features in an approach of Boolean mapping saliency (BMS). Finally, spectral-spatial features are constructed by feature fusion and are further used for the classification of the HSIs. A number of experiments are performed, including using different classifiers to verify the performance of the proposed scheme, comparing with related variant algorithms, comparing time with deep learning, and testing learning ability in the absence of labeled samples. Experimental results indicate that the proposed method is significantly superior to the previous methods.

Hyperspectral image (HSI) consists of hundreds of continuous narrowbands with high spectral correlation, which would lead to the so-called Hughes phenomenon and the high computational cost in processing. Band selection (BS) has been proven to be effective in avoiding such problems by removing redundant bands. However, many existing BS methods separately estimate the significance for every single band and cannot fully consider the nonlinear and global interaction between spectral bands. In this article, by assuming that a complete HSI band set can be reconstructed from its few informative bands, we propose a unified BS framework, BS Network (BS-Net). The framework consists of a band attention module (BAM), which aims to explicitly model the nonlinear interdependences between spectral bands, and a reconstruction network (RecNet), which is used to restore the original HSI from the learned informative bands, resulting in a flexible architecture. The resulting framework is end-to-end trainable, making it easier to train from scratch and to combine with many existing networks. We implement two versions of BS-Nets, respectively, using fully connected networks (BS-Net-FC) and convolutional neural networks (BS-Net-Conv), and extensively compare their results with popular existing BS approaches on three real hyperspectral data sets, showing that the proposed BS-Nets can accurately select informative band subset with less redundancy and outperform the competitors in terms of classification accuracy with competitive time cost.

In image analysis, image samples are always represented by multiple view features and associated with multiple class labels for better interpretation. However, multiple view data may include noisy, irrelevant and redundant features, while multiple class labels can be noisy and incomplete. Due to the special data characteristic, it is hard to perform feature selection on multi-view multi-label data. To address these challenges, in this paper, we propose a novel multi-view multi-label sparse feature selection (MSFS) method, which exploits both view relations and label correlations to select discriminative features for further learning. Specifically, the multi-labeled information is decomposed into a reduced latent label representation to capture higher level concepts and correlations among multiple labels. Multiple local geometric structures are constructed to exploit visual similarities and relations for different views. By taking full advantage of the latent label representation and multiple local geometric structures, the sparse regression model with an l2,1-norm and an Frobenius norm (F-norm) penalty terms is utilized to perform hierarchical feature selection, where the F-norm penalty performs high-level (i.e., view-wise) feature selection to preserve the informative views and the l2,1-norm penalty conducts low-level (i.e., row-wise) feature selection to remove noisy features. To solve the proposed formulation, we also devise a simple yet efficient iterative algorithm. Experiments and comparisons on real-world image datasets demonstrate the effectiveness and potential of MSFS.

Finding multiple roots of nonlinear equation systems (NESs) in a single run is one of the most important challenges in numerical computation. We tackle this challenging task by combining the strengths of the repulsion technique, diversity preservation mechanism, and adaptive parameter control. First, the repulsion technique motivates the population to find new roots by repulsing the regions surrounding the previously found roots. However, to find as many roots as possible, algorithm designers need to address a key issue: how to maintain the diversity of the population. To this end, the diversity preservation mechanism is integrated into our approach, which consists of the neighborhood mutation and the crowding selection. In addition, we further improve the performance by incorporating the adaptive parameter control. The purpose is to enhance the search ability and remedy the trial-and-error tuning of the parameters of differential evolution (DE) for different problems. By assembling the above three aspects together, we propose a repulsion-based adaptive DE, called RADE, for finding multiple roots of NESs in a single run. To evaluate the performance of RADE, 30 NESs with diverse features are chosen from the literature as the test suite. Experimental results reveal that RADE is able to find multiple roots simultaneously in a single run on all the test problems. Moreover, RADE is capable of providing better results than the compared methods in terms of both root rate and success rate.

Finding multiple roots of nonlinear equations systems (NESs) in a single run is an important yet difficult task. It requires to keep a balance between explorative and exploitative traits. In this paper, we present a random walk mutation-based differential evolution (DE) with estimation of distribution algorithm (EDA) to address this problem. The major characteristics are: i) the random walk mutation is capable of preserving the population diversity , which guides individuals to move toward different promising regions; ii) probability selection is employed to provide suitable parent individuals for evolution; iii) EDA is used to accelerate the convergence and obtains the roots. To evaluate the performance of our approach, 30 NESs with diverse features are selected as test suite. Experimental results indicate that the proposed approach is able to yield better performance compared with other state-of-the-art methods.

This paper presents a novel clustering method, named Laplacian regularized deep subspace clustering (LRDSC), for unsupervised hyperspectral image (HSI) classification. We introduce the Laplacian regularization into the subspace clustering to consider the manifold structure reflecting geometric information. To enable the subspace clustering, which works in linear space, to deal with the complicated HSI data with nonlinear characteristics, we combine the subspace clustering as a self-expressive layer with deep convolutional auto-encoder. Furthermore, the 3-D convolutions and deconvolutions with skip connections are utilized to make full extraction of the spectral-spatial information and full use of the historical feature maps produced by the network. We compare the results of the proposed method with six existing cluster methods on four real hyperspectral data sets, showing that the proposed method is able to achieve state-of-the-art performance.

Traditional bilateral filtering (BF) cannot extract hyperspectral image (HSI) features well when the center pixel of the neighborhood pixel set is a noise point in the process of filtering the HSI. In this letter, a trilateral smooth filtering (TRSF) is presented. The proposed algorithm avoids the above-mentioned limitation problem in the BF algorithm. TRSF is successfully applied to the feature extraction of three actual HSIs. To prove the effectiveness of the proposed algorithm, support vector machines are used to classify the extracted features. Experimental results show that the proposed feature extraction method is simple and effective.

Hyperspectral images (HSIs) consist of hundreds of continuous bands with high correlation, making it contain great abundant information. Band selection is an effective idea for removing redundant bands and preserving the physical significance at the same time. Popular sparse representation-based band selection commonly introduces an additional coefficient to combine error term and sparse constraint term, making it difficult to find out the optimal balance coefficient. In this letter, we propose a hybrid clustering-based band-selection approach based on using evolutionary multiobjective optimization to solve a sparse self-representation model constituted with two conflicting objectives. The proposed approach simultaneously minimizes two terms of the sparse representation model, avoiding the balance coefficient and producing a set of optimal solutions that are used to construct a similarity matrix for spectral clustering. Finally, a reduced band subset is determined by the cluster centers. We compare the results of the proposed approach with four existing band-selection methods for three real HSI data sets, showing that the proposed approach is able to effectively select representative bands with better classification accuracy.

Hyperspectral image (HSI) consists of hundreds of continuous narrow bands with high redundancy, resulting in the curse of dimensionality and an increased computation complexity in HSI classification. Many clustering-based band selection approaches have been proposed to deal with such a problem. However, a few of them consider the spectral and spatial relationship simultaneously. In this letter, we proposed a novel clustering-based band selection approach using deep subspace clustering (DSC). The proposed approach combines the subspace clustering task into a convolutional autoencoder by treating it as a self-expressive layer, enabling it to be trained end to end. The resulting network can fully extract the interaction of spectral bands based on using spatial information and nonlinear feature transformation. We compared the results of the proposed method with existing band selection methods for three widely used HSI data sets, showing that the proposed method is able to accurately select an informative band subset with remarkable classification accuracy.

Currently, deep neural networks (DNNs) are an important method for handling hyperspectral image (HSI) classification because of their good performance in image processing. However, DNNs' performance depends on a massive number of training data and hyperparameters that are carefully fine-tuned, which results in structural complexity and a time-consuming process. Deep forest is a novel deep learning method that does not need much training data and has a simple structure. In this paper, we first design a deep forest for spectral-based HSI classification and then propose an improved deep forest algorithm, named deep multigrained cascade forest (dgcForest), for spatial-based HSI classification. On the one hand, the cascade forest in dgcForest is used in multigrained scanning, which enhances the performance; on the other hand, a pooling layer is added after the multigrained scanning to reduce the output dimensions. To demonstrate that our proposed algorithm presents a good performance in HSI classification, we analyze the hyperparameters of deep forest and dgcForest and compare them with other methods on the biased and unbiased data sets, which illustrates that our method is superior to other state-of-the-art deep learning methods.

Unlabeled, multi-view data presents a considerable challenge in many real-world data analysis tasks. These data are worth exploring because they often contain complementary information that improves the quality of the analysis results. Clustering with multi-view data is a particularly challenging problem as revealing the complex data structures between many feature spaces demands discriminative features that are specific to the task and, when too few of these features are present, performance suffers. Extreme learning machines (ELMs) are an emerging form of learning model that have shown an outstanding representation ability and superior performance in a range of different learning tasks. Motivated by the promise of this advancement, we have developed a novel multi-view fusion clustering framework based on an ELM, called MVEC. MVEC learns the embeddings from each view of the data via the ELM network, then constructs a single unified embedding according to the correlations and dependencies between each embedding and automatically weighting the contribution of each. This process exposes the underlying clustering structures embedded within multi-view data with a high degree of accuracy. A simple yet efficient solution is also provided to solve the optimization problem within MVEC. Experiments and comparisons on eight different benchmarks from different domains confirm MVEC’s clustering accuracy.

Deep neural networks have been successfully applied to many data mining problems in recent works. The training of deep neural networks relies heavily upon gradient descent methods, however, which may lead to the failure of training due to the vanishing gradient (or exploding gradient) and local optima problems. In this paper, we present SEvoAE method based on using Evolutionary Multiobjective optimization (EMO) algorithm to train single layer auto-encoder, and sequentially learning deeper representation in a stacking way. SEvoAE is able to achieve accurate feature representation with good sparseness by globally simultaneously optimizing two conflicting objective functions and allows users to flexibly design objective functions and evolutionary optimizers. We compare results of the proposed method with existing architectures for seven classification problems, showing that the proposed method is able to outperform existing methods with a reduced risk of overfitting the training data.

Dimensionality Reduction (DR) which tries to discover low-dimensional feature representation embedded into the high-dimensional observations are significant for data visualization and data preprocessing. However, most DR models are designed for vector-valued data while only few of them are for functional data where samples are considered as continuous data such as curves or surfaces compared to discrete vector-valued data. Motivated by Functional Principal Component Analysis (FPCA), which generalizes the idea of Principal Component Analysis (PCA) to the Hilbert space of square-integrable functions, in this paper we propose Functional Locality Preserving Projection (FLPP), where classic Locality Preserving Projection (LPP) is extended for functional data analysis. Different from FPCA which only focuses on the global structure, FLPP could preserve local manifold structure embedded into the functional data, thus FLPP is capable of dealing with noise data. Experimental results on both synthetic data and real-world data verify that FLPP outperforms FPCA and typical LPP.

Face hallucination can improve the resolution of observed low-resolution (LR) face image to predict the high-resolution (HR) face image. In order to achieve good result performance, the training samples and local structure prior of face image are utilized by some approaches including Least Square Representation (LSR) and convex optimization to obtain the better representation coefficients. However, they do not pay more attention to the relationship between local-pixel structures of HR training samples and input LR face. Thus, the reconstruction coefficients they get are not optimal. Therefore, Optical Flow based face hallucination via weightedly-constrained representation(OFWCR) has been developed in this paper. Compared with LSR and Sparse Representation (SR), our method uses a warping HR training face image strategy to achieve better details from the input LR face. We also take into account the locality constraint in our effective representation scheme to reach locality and sparsity synchronously. Experiments show that our proposed scheme outperforms state-of-the-art approaches in common database.

As an unsupervised dimensionality reduction method, the principal component analysis (PCA) has been widely considered as an efficient and effective preprocessing step for hyperspectral image (HSI) processing and analysis tasks. It takes each band as a whole and globally extracts the most representative bands. However, different homogeneous regions correspond to different objects, whose spectral features are diverse. Therefore, it is inappropriate to carry out dimensionality reduction through a unified projection for an entire HSI. In this paper, a simple but very effective superpixelwise PCA (SuperPCA) approach is proposed to learn the intrinsic low-dimensional features of HSIs. In contrast to classical PCA models, the SuperPCA has four main properties: 1) unlike the traditional PCA method based on a whole image, the SuperPCA takes into account the diversity in different homogeneous regions, that is, different regions should have different projections; 2) most of the conventional feature extraction models cannot directly use the spatial information of HSIs, while the SuperPCA is able to incorporate the spatial context information into the unsupervised dimensionality reduction by superpixel segmentation; 3) since the regions obtained by superpixel segmentation have homogeneity, the SuperPCA can extract potential low-dimensional features even under noise; and 4) although the SuperPCA is an unsupervised method, it can achieve a competitive performance when compared with supervised approaches. The resulting features are discriminative, compact, and noise-resistant, leading to an improved HSI classification performance. Experiments on three public data sets demonstrate that the SuperPCA model significantly outperforms the conventional PCA-based dimensionality reduction baselines for HSI classification, and some state-of-the-art feature extraction approaches. The MATLAB source code is available at https://github.com/junjun-jiang/SuperPCA.

Transfer learning has attracted more and more attention, and many scholars proposed some useful strategies. Boosting is the main strategy for transfer learning. In boosting, resampling is preferred over reweighting, and it can be applied to any base learner. In this paper, we propose a weighted-resampling method for transfer learning, called TrResampling. Firstly, resampling is applied to the data with heaven weight in the source domain, and the resampled data is used with the target data as the training data to build a classifier. Then the TrAdaBoost algorithm is used to adjust the weights of source data and target data. We discuss Decision Tree, Naive Bayes, and SVM as the base learner in TrResampling, and choose the suitable for TrResampling. In order to illustrate the performance of the proposed algorithm, we compare TrResampling with the state-of-the-art algorithm TrAdaBoost and the base learner Decision Tree, experimental results on UCI data sets indicate that TrResampling is superior to TrAdaBoost and Decision Tree on many data sets.

Discriminative features are significant for hyper-spectral image (HSI) classification. In this letter, we apply the supervised dimensionality reduction (DR) model termed supervised latent linear Gaussian process latent variable model (SLLGPLVM) for feature extraction. As a semiparametric classification model, the new model has ability in simultaneous feature extraction and classification and demonstrates high classification accuracy with only a small training set. This is therefore suitable for HSI classification. Experimental results on six real HSI data sets show that the proposed SLLGPLVM outperforms several conventional supervised DR models and the support vector machine implemented in the original spectral space.

Due to the fact that a nonlinear equation system (NES) may contain multiple optimal solutions, solving NESs is one of the most important challenges in numerical computation. When applying evolutionary algorithms to solve NESs, two issues should be considered: 1) how to transform an NES into a kind of optimization problem and 2) how to develop an optimization algorithm to solve the transformed optimization problem. In this paper, we tackle the first issue by transforming an NES into a weighted biobjective optimization problem. By the above transformation, not only do all the optimal solutions of an original NES become the Pareto optimal solutions of the transformed biobjective optimization problem, but also their images are different points on a linear Pareto front in the objective space. In addition, we suggest an adaptive multiobjective differential evolution, the goal of which is to effectively locate the Pareto optimal solutions of the transformed biobjective optimization problem. Once these solutions are found, the optimal solutions of the original NES can also be obtained correspondingly. By combining the weighted biobjective transformation technique with the adaptive multiobjective differential evolution, we propose a generic framework for the simultaneous locating of multiple optimal solutions of NESs. Comprehensive experiments on 38 NESs with various features have demonstrated that our framework provides very competitive overall performance compared with several state-of-the-art methods.

Cooperative co-evolution (CC) is an explicit means of problem decomposition in multipopulation evolutionary algorithms for solving large-scale optimization problems. For CC, subpopulations representing subcomponents of a large-scale optimization problem co-evolve, and are likely to have different contributions to the improvement of the best overall solution to the problem. Hence, it makes sense that more computational resources should be allocated to the subpopulations with greater contributions. In this paper, we study how to allocate computational resources in this context and subsequently propose a new CC framework named CCFR to efficiently allocate computational resources among the subpopulations according to their dynamic contributions to the improvement of the objective value of the best overall solution. Our experimental results suggest that CCFR can make efficient use of computational resources and is a highly competitive CCFR for solving large-scale optimization problems.

The decomposition method in multiobjective evolutionary algorithms (MOEA/D) is an effective approach to evolve solutions along predefined weight vectors for solving multiobjective optimization problems (MOPs). However, obtaining evenly distributed weight vectors for different types of MOPs is a challenge problem especially when the true Pareto fronts (PFs) are unknown before a MOEA/D starts. In this paper, a new MOEA/D with a fast hypervolume archive (called FV-MOEA/D) is proposed to adaptively adjust the weight vectors for various shapes of PFs. The core idea of FV-MOEA/D is to periodically adjust weight vectors based on solutions in the proposed archive, in which convergence and diversity are maintained by maximizing hypervolume. Experimental studies on 58 benchmark MOPs in jMetal demonstrate that the proposed FV-MOEA/D not only reached higher hypervolumes when compare to five classical MOEAs i.e., NSGAII, SPEA2, IBEA, FV-MOEA and MOEA/D, but also obtained well distributed weight vectors on PFs with different geometrical characteristics.

Differential evolution (DE) is an efficient and powerful evolutionary algorithm for numerical optimization. In DE, different search strategies are presented. Generally, different strategies are suitable to different problems, whereas it is difficult to select the best one for a problem at hand. In this paper, we propose a multi-strategy based DE framework based on the classification technique. More specifically, a set of candidate trial vectors are generated by using the multiple strategies, and the best one is chosen as the trial vector according to the classification technique. As an illustration, the extreme learning machine (ELM) is selected as the classification method in our framework. Then, four state-of-the-art DE variants (i.e., SaDE, CoDE, EPSDE, and JADE) are integrated into the framework to implement multi-strategy adaptation in DE. The proposed variants are extensively evaluated on a suite of 13 benchmark optimization problems. Experimental results show that our approach is very competitive with state-of-the-art DE variants.

Hyperspectral image (HSI) classification has attracted much attention and extensive research efforts over the past decade. Due to few labeled samples versus high dimensional features, it is a challenging problem in practice. Recently, combining the pixel spectral information and the spatial (neighborhood) information has been verified to be effective for HSI classification. In this paper, we introduce a novel method for HSI classification using set-to-set distance (SSD). Based on the assumption that neighbor pixels tend to belong to the same class with high probability, we model a test pixel and its neighbor pixels as a testing set (or a neighbor set) inspired by bilateral filtering. Meanwhile, the training pixels belong to the same class are modeled as a training set. Therefore, the classification is based on comparisons of sets distances. Experiments on a real HSI dataset show that our proposed method outperforms a number of existing state-of-the-art approaches.

Unsupervised learning aims to discovery latent representation embedded in the observation, which is useful for data visualization, dimensionality reduction, and density modeling. Autoencoders have been successfully used to learn the latent variations in data, especially with the recent reintroduction by deep learning. For some specific tasks, there are supervised information or labels that can be used to further guide the unsupervised autoencoder model for finding latent representation. The Non-Parametrically Guided Autoencoder (NPGA) has been proved to be an effective model. It tries to utilize Gaussian Process Regression (GPR) to model the unknown mapping from unknown latent representation to extra supervised information. However for the discrete label information in classification tasks, using GPR could be unwise and inefficient. In this paper, we propose the Non-Parametrically Guided Autoencoder with Laplace Approximation (NPGA-LA) to effectively handle discrete labels. The idea of NPGA-LA is to make use of Gaussian Process Classification (GPC) rather than GPR to model the transformation between the latent space and the discrete label space. The experimental results verify the excellent performance of the newly developed method.

In classification problem, we assume that the samples around the class boundary are more likely to be incorrectly annotated than others, and propose boundary-conditional class noise (BCN). Based on the BCN assumption, we use unnormalized Gaussian and Laplace distributions to directly model how class noise is generated, in symmetric and asymmetric cases. In addition, we demonstrate that Logistic regression and Probit regression can also be reinterpreted from this class noise perspective, and compare them with the proposed models. The empirical study shows that, the proposed asymmetric models overall outperform the benchmark linear models, and the asymmetric Laplace-noise model achieves the best performance among all.

Extreme Learning Machine (ELM) is an elegant technique for training Single-hidden Layer Feedforward Networks (SLFNs) with extremely fast speed that attracts significant interest recently. One potential weakness of ELM is the random generation of the input weights and hidden biases, which may deteriorate the classification accuracy. In this paper, we propose a new Memetic Algorithm (MA) based Extreme Learning Machine (M-ELM) for classification problems. M-ELM uses Memetic Algorithm which is a combination of population-based global optimization technique and individual-based local heuristic search method to find optimal network parameters for ELM. The optimized network parameters will enhance the classification accuracy and generalization performance of ELM. Experiments and comparisons on 22 benchmark data sets demonstrate that M-ELM is able to provide highly competitive results compared with other state-of-the-art varieties of ELM algorithms.

Differential evolution (DE) is a powerful evolutionary algorithm (EA) for numerical optimization. Combining with the constraint-handling techniques, recently, DE has been successfully used for the constrained optimization problems (COPs). In this paper, we propose the adaptive ranking mutation operator (ARMOR) for DE when solving the COPs. The ARMOR is expected to make DE converge faster and achieve feasible solutions faster. In ARMOR, the solutions are adaptively ranked according to the situation of the current population. More specifically, the population is classified into three situations, i.e., infeasible situation, semi-feasible situation, and feasible situation. In the infeasible situation, the solutions are ranked only based on their constraint violations; in the semi-feasible situation, they are ranked according to the transformed fitness; while in the feasible situation, the objective function value is used to assign ranks to different solutions. In addition, the selection probability of each solution is calculated differently in different situations. The ARMOR is simple, and it can be easily combined with most of constrained DE (CDE) variants. As illustrations, we integrate our approach into three representative CDE variants to evaluate its performance. The 24 benchmark functions presented in CEC 2006 and 18 benchmark functions presented in CEC 2010 are chosen as the test suite. Experimental results verify our expectation that the ARMOR is able to accelerate the original CDE variants in the majority of test cases. Additionally, ARMOR-based CDE is able to provide highly competitive results compared with other state-of-the-art EAs.

In this paper, we formulate a novel graph-based learning problem, multi-graph classification (MGC), which aims to learn a classifier from a set of labeled bags each containing a number of graphs inside the bag. A bag is labeled positive, if at least one graph in the bag is positive, and negative otherwise. Such a multi-graph representation can be used for many real-world applications, such as webpage classification, where a webpage can be regarded as a bag with texts and images inside the webpage being represented as graphs. This problem is a generalization of multi-instance learning (MIL) but with vital differences, mainly because instances in MIL share a common feature space whereas no feature is available to represent graphs in a multi-graph bag. To solve the problem, we propose a boosting based multi-graph classification framework (bMGC). Given a set of labeled multi-graph bags, bMGC employs dynamic weight adjustment at both bag- and graph-levels to select one subgraph in each iteration as a weak classifier. In each iteration, bag and graph weights are adjusted such that an incorrectly classified bag will receive a higher weight because its predicted bag label conflicts to the genuine label, whereas an incorrectly classified graph will receive a lower weight value if the graph is in a positive bag (or a higher weight if the graph is in a negative bag). Accordingly, bMGC is able to differentiate graphs in positive and negative bags to derive effective classifiers to form a boosting model for MGC. Experiments and comparisons on real-world multi-graph learning tasks demonstrate the algorithm performance.

In differential evolution (DE) studies, there are many parameter adaptation methods, aiming at tuning the mutation factor F and the crossover probability CR . However, these methods still cannot resolve the issues of population premature convergence and population stagnation. To address these issues, in this paper, we investigate the population adaptation regarding population diversity at the dimensional level and propose a mechanism named auto-enhanced population diversity (AEPD) to automatically enhance population diversity. AEPD is able to identify the moments when a population becomes converging or stagnating by measuring the distribution of the population in each dimension. When convergence or stagnation is identified at a dimension, the population is diversified at that dimension to an appropriate level or to eliminate the stagnation issue. The AEPD mechanism was incorporated into a popular DE algorithm and it was tested on a set of 25 CEC2005 benchmark functions. The results showed that AEPD significantly improved the performance of the original algorithms. In addition, AEPD helped the algorithms become less sensitive to population size, a parameter widely considered problem dependent for many DE algorithms. The DE algorithm with AEPD also has a superior performance in comparison with several other peer algorithms.

It is well known that in evolutionary algorithms (EAs), different reproduction operators may be suitable for different problems or in different running stages. To improve the algorithm performance, the ensemble of multiple operators has become popular. Most ensemble techniques achieve this goal by choosing an operator according to a probability learned from the previous experience. In contrast to these ensemble techniques, in this paper we propose a cheap surrogate model-based multioperator search strategy for evolutionary optimization. In our approach, a set of candidate offspring solutions are generated by using the multiple offspring reproduction operators, and the best one according to the surrogate model is chosen as the offspring solution. Two major advantages of this approach are: 1) each operator can generate a solution for competition compared to the probability-based approaches and 2) the surrogate model building is relatively cheap compared to that in the surrogate-assisted EAs. The model is used to implement multioperator ensemble in two popular EAs, that is, differential evolution and particle swarm optimization. Thirty benchmark functions and the functions presented in the CEC 2013 are chosen as the test suite to evaluate our approach. Experimental results indicate that the new approach can improve the performance of single operator-based methods in the majority of the functions.

The performance of the Kalman filter (KF), which is recognized as an outstanding tool for dynamic system state estimation, heavily depends on its parameter R, called the measurement noise covariance matrix. However, it's difficult to get the exact value of R before the filter starts, and the value of R is likely to change with the measurement environment when the filter is working. To solve this problem, a new parameter adaptive Kalman filter is proposed in this paper. In this new Kalman filter, the initial value of R is offline decided by Evolutionary Algorithm (EA), and the value of R decided by EA is online updated by Fuzzy Inference System (FIS). A simulation experiment based on target tracking is carried out, and the results demonstrate that the new adaptive Kalman filter proposed in this paper (HydGeFuzKF) has a stronger adaptability to time-varying measurement noises than regular Kalman filter (RegularKF), Sage-Husa adaptive Kalman filter (SageHusaKF), the adaptive Kalman filter only based on genetic algorithm (GeneticKF) and the adaptive Kalman filter only based on fuzzy inference system (FuzzyKF).

In differential evolution (DE), the optimal value of the control parameters is problem-dependent. Many improved DE algorithms have been proposed with the aim of improving the effectiveness for solving general problems. As a very known adaptive DE algorithm, JADE adjusts the crossover probability CR of each individual by a norm distribution, in which the value of standard deviation is fixed, based on its historical record of success. The fixed and small standard deviation results in that the generated CR may not suitable for solving a problem. This paper proposed an improvement for the adaptation of CR, in which the standard deviation is adaptive. The diversity of values of CR was improved. This improvement was incorporated into the JADE algorithm and tested on a set of 25 scalable benchmark functions. The results showed that the adaptation of CR improved the performance of the JADE algorithm, particularly in comparisons with several other peer algorithms on high-dimensional functions.

In traditional multi-instance learning (MIL), instances are typically represented by using a single feature view. As MIL becoming popular in domain specific learning tasks, aggregating multiple feature views to represent multi-instance bags has recently shown promising results, mainly because multiple views provide extra information for MIL tasks. Nevertheless, multiple views also increase the risk of involving redundant views and irrelevant features for learning. In this paper, we formulate a new cross-view feature selection problem that aims to identify the most representative features across all feature views for MIL. To achieve the goal, we design a new optimization problem by integrating both multi-view representation and multi-instance bag constraints. The solution to the objective function will ensure that the identified top-m features are the most informative ones across all feature views. Experiments on two real-world applications demonstrate the performance of the cross-view feature selection for content-based image retrieval and social media content recommendation.

Generally, there are two objectives in the optimization of the measurement noise covariance matrix R of Kalman filter. However, most of the traditional optimization methods of Kalman filter only focus on one objective. In this paper, we proposed a new method to optimize the parameter R based on Multi-Objective Memetic Algorithm (MOMA). Compared with traditional methods, it can optimize multiple objectives simultaneously. In this method, the decision vector is the diagonal elements of matrix R, the first objective function f1 is the mean of the residual vectors, and the second objective function f2 is the degree of mismatching between the actual value of the residual covariance with its theoretical value. In the MOMA, the global search based on NSGA-II is utilized to minimize the two objective functions, and the local search based on Simulated Annealing (SA) is just used to minimize the f1. The experimental results demonstrate that the Kalman filter optimized by MOMA, namely MOMA-Kalman, can get much smaller filtering error than regular Kalman filter and other adaptive filter algorithms, such as SageHusa-Kalman and Fuzzy-Kalman.

Graph classification has traditionally focused on graphs generated from a single feature view. In many applications, it is common to have useful information from different channels/views to describe objects, which naturally results in a new representation with multiple graphs generated from different feature views being used to describe one object. In this paper, we formulate a new Multi-Graph-View learning task for graph classification, where each object to be classified contains graphs from multiple graph-views. This problem setting is essentially different from traditional single-graph-view graph classification, where graphs are from one single feature view. To solve the problem, we propose a Cross Graph-View Sub graph Feature based Learning (gCGVFL) algorithm that explores an optimal set of sub graphs, across multiple graph-views, as features to represent graphs. Specifically, we derive an evaluation criterion to estimate the discriminative power and the redundancy of sub graph features across all views, and assign proper weight values to each view to indicate its importance for graph classification. The iterative cross graph-view sub graph scoring and graph-view weight updating form a closed loop to find optimal sub graphs to represent graphs for multi-graph-view learning. Experiments and comparisons on real-world tasks demonstrate the algorithm's performance.

Memetic algorithms (MAs) represent one of the promising areas of evolutionary algorithms. However, there are many issues to be solved to design a robust MA. In this paper, we introduce an adaptive memetic algorithm, named GADE-DHC, which combines a genetic algorithm and a differential evolution algorithm as global search methods with a directional hill climbing (DHC) algorithm as local search method. In addition, a novel strategy is proposed to balance the intensity of global search methods and local search method, as well as the ratio between genetic algorithm and differential evolution algorithm. Experiments on several benchmark problems of diverse complexities have shown that the new approach is able to provide highly competitive results compared with other algorithms.

A Bayesian Network (BN) is a graphical model which can be used to represent conditional dependency between random variables, such as diseases and symptoms. A Bayesian Network Classifier (BNC) uses BN to characterize the relationships between attributes and the class labels, where a simplified approach is to employ a conditional independence assumption between attributes and the corresponding class labels, i.e., the Naive Bayes (NB) classification model. One major approach to mitigate NB's primary weakness (the conditional independence assumption) is the attribute weighting, and this type of approach has been proved to be effective for NB with simple structure. However, for weighted BNCs involving complex structures, in which attribute weighting is embedded into the model, there is no existing study on whether the weighting will work for complex BNCs and how effective it will impact on the learning of a given task. In this paper, we first survey several complex structure models for BNCs, and then carry out experimental studies to investigate the effectiveness of the attribute weighting strategies for complex BNCs, with a focus on Hidden Naive Bayes (HNB) and Averaged One-Dependence Estimation (AODE). Our studies use classification accuracy (ACC), area under the ROC curve ranking (AUC), and conditional log likelihood (CLL), as the performance metrics. Experiments and comparisons on 36 benchmark data sets demonstrate that attribute weighting technologies just slightly outperforms unweighted complex BNCs with respect to the ACC and AUC, but significant improvement can be observed using CLL.

Naive Bayes (NB) network is a popular classification technique for data mining and machine learning. Many methods exist to improve the performance of NB by overcoming its primary weakness - the assumption that attributes are conditionally independent given the class, using techniques such as backwards sequential elimination and lazy elimination. Some weighting technologies, including attribute weighting and instance weighting, have also been proposed to improve the accuracy of NB. In this paper, we propose a dual weighted model, namely DWNB, for NB classification. In DWNB, we firstly employ an instance similarity based method to weight each training instance. After that, we build an attribute weighted model based on the new training data, where the calculation of the probability value is based on the embedded instance weights. The dual instance and attribute weighting allows DWNB to tackle the conditional independence assumption for accurate classification. Experiments and comparisons on 36 benchmark data sets demonstrate that DWNB outperforms existing weighted NB algorithms.

In this paper, we formulate a new multi-graph learning task with only positive and unlabeled bags, where labels are only available for bags but not for individual graphs inside the bag. This problem setting raises significant challenges because bag-of-graph setting does not have features to directly represent graph data, and no negative bags exits for deriving discriminative classification models. To solve the challenge, we propose a puMGL learning framework which relies on two iteratively combined processes for multigraph learning: (1) deriving features to represent graphs for learning; and (2) deriving discriminative models with only positive and unlabeled graph bags. For the former, we derive a subgraph scoring criterion to select a set of informative subgraphs to convert each graph into a feature space. To handle unlabeled bags, we assign a weight value to each bag and use the adjusted weight values to select most promising unlabeled bags as negative bags. A margin graph pool (MGP), which contains some representative graphs from positive bags and identified negative bags, is used for selecting subgraphs and training graph classifiers. The iterative subgraph scoring, bag weight updating, and MGP based graph classification forms a closed loop to find optimal subgraphs and most suitable unlabeled bags for multi-graph learning. Experiments and comparisons on real-world multigraph data demonstrate the algorithm performance.

A suitable key agreement protocol plays an essential role in protecting the communications over open channels among users using voice over Internet protocol (VoIP). This study presents a robust and flexible password authenticated key agreement protocol with user anonymity for session initiation protocol (SIP) used by VoIP communications. Security analysis demonstrates that the proposed protocol enjoys many unique properties, such as user anonymity, no password table, session key agreement, mutual authentication, password updating freely, conveniently revoking lost smartcards and so on. Furthermore, the proposed protocol can resist the replay attack, the impersonation attack, the stolen‐verifier attack, the man‐in‐middle attack, the Denning‐Sacco attack and the offline dictionary attack with or without smartcards. Finally, the performance analysis shows that the protocol is more suitable for practical application in comparison with other related protocols.

A high quality distance function that measures the difference between instances is essential in many real-world applications and research fields. For example, in instance-based learning, the distance function plays the most important role. A large number of distance functions have been proposed. For nominal attributes, Value Difference Metric (VDM) is one of the state-of-the-art and widely used distance functions. However, it needs to estimate the conditional probabilities, which drops its efficiency in computing the distance between instances. Besides, a practical issue that arises in estimating the conditional probabilities is that the denominators can be zero or very small. This makes them either undefined or very large. Therefore, an efficient distance function that can measure the difference between two instances but without the practical issue confronting VDM is desirable. In this paper, we propose a novel distance function: Frequency Difference Metric (FDM). FDM is just based on the joint frequencies of class labels and attribute values, instead of the conditional probabilities. Extensive empirical studies show that FDM performs almost as well as VDM in terms of accuracy, but significantly outperforms VDM in terms of efficiency. This work provides a very simple, efficient, and effective distance function that can be widely used in many real-world applications and research fields.

ABSTRACT Session Initiation Protocol (SIP) is one of the most commonly used protocols for handling sessions for over Internet Protocol based communications, and the security of SIP is becoming increasingly important. Recently, Zhang et al. proposed a password‐authenticated key agreement protocol for SIP by using smart cards to protect the VoIP communications between users. Their protocol provided some unique features, such as mutual authentication, no password table needed, and password updating freely. In this study, we performed cryptanalysis of Zhang et al.'s protocol and found that their protocol was vulnerable to the impersonation attack although the protocol could withstand several other attacks. A malicious attacker could compute other users' privacy keys and then impersonated the users to cheat the SIP server. Furthermore, we proposed an improved password‐authentication key agreement protocol for SIP, which overcame the weakness of Zhang et al.'s protocol and was more suitable for Voice over Internet Protocol communications. Copyright © 2014 John Wiley & Sons, Ltd.

In differential evolution (DE), there are many adaptive algorithms proposed for parameters adaptation. However, they mainly aim at tuning the amplification factor F and crossover probability CR. When the population diversity is at a low level or the population becomes stagnant, the population is not able to improve any more. To enhance the performance of DE algorithms, in this paper, we propose a method of population adaptation. The proposed method can identify the moment when the population diversity is poor or the population stagnates by measuring the Euclidean distances between individuals of a population. When the moment is identified, the population will be regenerated to increase diversity or to eliminate the stagnation issue. The population adaptation is incorporated into the jDE algorithm and is tested on a set of 25 scalable CEC05 benchmark functions. The results show that the population adaptation can significantly improve the performance of the jDE algorithm. Even if the population size of jDE is small, the jDE algorithm with population adaptation also has a superior performance in comparisons with several other peer algorithms for high-dimension function optimization.

Multi-instance learning concerns about building learning models from a number of labeled instance bags, where each bag consists of instances with unknown labels. A bag is labeled positive if one or more multiple instances inside the bag is positive, and negative otherwise. For all existing multi-instance learning algorithms, they are only applicable to the setting where instances in each bag are represented by a set of well defined feature values. In this paper, we advance the problem to a multi-instance multi-graph setting, where a bag contains a number of instances and graphs in pairs, and the learning objective is to derive classification models from labeled bags, containing both instances and graphs, to predict previously unseen bags with maximum accuracy. To achieve the goal, the main challenge is to properly represent graphs inside each bag and further take advantage of complementary information between instance and graph pairs for learning. In the paper, we propose a Dual Embedding Multi-Instance Multi-Graph Learning (DE-MIMG) algorithm, which employs a dual embedding learning approach to (1) embed instance distributions into the informative sub graphs discovery process, and (2) embed discovered sub graphs into the instance feature selection process. The dual embedding process results in an optimal representation for each bag to provide combined instance and graph information for learning. Experiments and comparisons on real-world multi-instance multi-graph learning tasks demonstrate the algorithm performance.

In many real-world applications, it is often the case that the class distribution of instances is imbalanced and the costs of misclassification are different. Thus, class-imbalance and cost-sensitive learning have attracted much attention from researchers. Sampling is one of the widely used approaches in dealing with the class imbalance problem, which alters the class distribution of instances so that the minority class is well represented in the training data. In this paper, we study the effect of sampling the natural training data on state-of-the-art Bayesian network classifiers, such as Naive Bayes (NB), Tree Augmented Naïve Bayes (TAN), Averaged One-Dependence Estimators (AODE), Weighted Average of One-Dependence Estimators (WAODE), and Hidden naive Bayes (HNB) and propose sampled Bayesian network classifiers. Our experimental results on a large number of UCI datasets show that our sampled Bayesian network classifiers perform much better than the ones trained from the natural training data especially when the natural training data is highly imbalanced and the cost ratio is high enough.

Probability estimation from a given set of training examples is crucial for learning Naive Bayes (NB) Classifiers. For an insufficient number of training examples, the estimation will suffer from the zero-frequency problem which does not allow NB classifiers to classify instances whose conditional probabilities are zero. Laplace-estimate and M-estimate are two common methods which alleviate the zero-frequency problem by adding some fixed terms to the probability estimation to avoid zero conditional probability. A major issue with this type of design is that the fixed terms are pre-specified without considering the uniqueness of the underlying training data. In this paper, we propose an Artificial Immune System (AIS) based self-adaptive probability estimation method, namely AISENB, which uses AIS to automatically and self-adaptively select the optimal terms and values for probability estimation. The unique immune system based evolutionary computation process, including initialization, clone, mutation, and crossover, ensure that AISENB can adjust itself to the data without explicit specification of functional or distributional forms for the underlying model. Experimental results and comparisons on 36 benchmark datasets demonstrate that AISENB significantly outperforms traditional probability estimation based Naive Bayes classification approaches.

Naive Bayes (NB) is a popularly used classification method. One potential weakness of NB is the strong conditional independence assumption between attributes, which may deteriorate the classification accuracy. In this paper, we propose a new Artificial Immune System based Weighted Naive Bayes (AISWNB) classifier. AISWNB uses immunity theory in artificial immune systems to find optimal weight values for each attribute. The adjusted weight values will alleviate the conditional independence assumption and help calculate the conditional probability in an accurate way. Because AISWNB uses artificial immune system search mechanism to find optimal weights, it does not need to know the importance of individual attributes nor the relevance among attributes. As a result, it can obtain optimal weight value for each attribute during the learning process. Experiments and comparisons on 36 benchmark data sets demonstrate that AISWNB outperforms other state-of-the-art attribute weighted NB algorithms.

Due to being fast, easy to implement and relatively effective, some state-of-the-art naive Bayes text classifiers with the strong assumption of conditional independence among attributes, such as multinomial naive Bayes, complement naive Bayes and the one-versus-all-but-one model, have received a great deal of attention from researchers in the domain of text classification. In this article, we revisit these naive Bayes text classifiers and empirically compare their classification performance on a large number of widely used text classification benchmark datasets. Then, we propose a locally weighted learning approach to these naive Bayes text classifiers. We call our new approach locally weighted naive Bayes text classifiers (LWNBTC). LWNBTC weakens the attribute conditional independence assumption made by these naive Bayes text classifiers by applying the locally weighted learning approach. The experimental results show that our locally weighted versions significantly outperform these state-of-the-art naive Bayes text classifiers in terms of classification accuracy.

Differential evolution (DE) has been proven to be one of the most powerful global numerical optimization algorithms in the evolutionary algorithm family. The core operator of DE is the differential mutation operator. Generally, the parents in the mutation operator are randomly chosen from the current population. In nature, good species always contain good information, and hence, they have more chance to be utilized to guide other species. Inspired by this phenomenon, in this paper, we propose the ranking-based mutation operators for the DE algorithm, where some of the parents in the mutation operators are proportionally selected according to their rankings in the current population. The higher ranking a parent obtains, the more opportunity it will be selected. In order to evaluate the influence of our proposed ranking-based mutation operators on DE, our approach is compared with the jDE algorithm, which is a highly competitive DE variant with self-adaptive parameters, with different mutation operators. In addition, the proposed ranking-based mutation operators are also integrated into other advanced DE variants to verify the effect on them. Experimental results indicate that our proposed ranking-based mutation operators are able to enhance the performance of the original DE algorithm and the advanced DE algorithms.

The lack of labeled training data is a common issue in many machine learning applications. Semi-supervised learning addresses this issue by self-labeling unlabelled examples. Transfer learning tackles it from a different way: borrow labeled examples from a different but related domain (source domain) by assigning weights to those examples based on their suitability on the new domain (target domain). However, it is quite challenging to figure out the suitability. In this paper, we propose a different way for utilizing the labeled examples from source domain. That is, we use them only for labelling the unlabelled examples in the target domain. In this self-labelling, we use the idea of Tri-training. We call our new algorithm: TriTransfer. In TriTransfer, three initial classifiers are generated from the source data and the originally labeled data in the target domain, and an unlabeled example is labeled and added to the labeled data for a classifier if other two classifiers agree on its label. After an expanded labeled data set is obtained, we re-train the classifier. We repeat this process until no more change can be made. At the end, the final classifier, which is a weighted combination of the three classifiers, is output. We conduct an extensive empirical study on 34 UCI datasets, which shows that TriTransfer performs better than the state-of-art algorithms Transfer Boost, Tritraining, and NaiveBayes.

Naive Bayes (NB) is a probability-based classification model which is based on the attribute independence assumption. However, in many real-world data mining applications, its attribute independence assumption is often violated. Responding to this fact, researchers have made a substantial amount of effort to improve the classification accuracy of NB by weakening its attribute independence assumption. For a recent example, averaged one-dependence estimators (AODE) is proposed, which weakens its attribute independence assumption by averaging all models from a restricted class of one-dependence classifiers. However, all one-dependence classifiers in AODE have same weights and are treated equally. According to our observation, different one-dependence classifiers should have different weights. Therefore, in this article, we proposed an improved model called weighted average of one-dependence estimators (WAODE) by assigning different weights to these one-dependence classifiers. In our WAODE, four different weighting approaches are designed and thus four different versions are created. For simplicity, we respectively denote them by WAODE-MI, WAODE-ACC, WAODE-CLL and WAODE-AUC. The experimental results on a large number of UCI datasets published on the main website of Weka platform show that our WAODE significantly outperform AODE.

Differential evolution (DE) is a simple yet powerful evolutionary algorithm for global numerical optimization over continuous domain, which has been widely used in many areas. Although DE is good at exploring the search space, it is slow at the exploitation of the solutions. To alleviate this drawback, in this paper, we propose a generalized hybrid generation scheme, which attempts to enhance the exploitation and accelerate the convergence velocity of the original DE algorithm. In the hybrid generation scheme the operator with powerful exploitation is hybridized with the original DE operator. In addition, a self-adaptive exploitation factor is introduced to control the frequency of the exploitation operation. In order to evaluate the performance of our proposed generation scheme, two operators, the migration operator of biogeography-based optimization and the "DE/best/1" mutation operator, are employed as the exploitation operator. Moreover, 23 benchmark functions (including 10 test functions provided by CEC2005 special session) are chosen from the literature as the test suite. Experimental results confirm that the new hybrid generation scheme is able to enhance the exploitation of the original DE algorithm and speed up its convergence rate.

Differential evolution (DE) is a simple, yet efficient, evolutionary algorithm for global numerical optimization, which has been widely used in many areas. However, the choice of the best mutation strategy is difficult for a specific problem. To alleviate this drawback and enhance the performance of DE, in this paper, we present a family of improved DE that attempts to adaptively choose a more suitable strategy for a problem at hand. In addition, in our proposed strategy adaptation mechanism (SaM), different parameter adaptation methods of DE can be used for different strategies. In order to test the efficiency of our approach, we combine our proposed SaM with JADE, which is a recently proposed DE variant, for numerical optimization. Twenty widely used scalable benchmark problems are chosen from the literature as the test suit. Experimental results verify our expectation that the SaM is able to adaptively determine a more suitable strategy for a specific problem. Compared with other state-of-the-art DE variants, our approach performs better, or at least comparably, in terms of the quality of the final solutions and the convergence rate. Finally, we validate the powerful capability of our approach by solving two real-world optimization problems.

Differential evolution (DE) is a simple yet powerful evolutionary algorithm for global numerical optimization. Different strategies have been proposed for the offspring generation; but the selection of which of them should be applied is critical for the DE performance, besides being problem-dependent. In this paper, the probability matching technique is employed in DE to autonomously select the most suitable strategy while solving the problem. Four credit assignment methods, that update the known performance of each strategy based on the relative fitness improvement achieved by its recent applications, are analyzed. To evaluate the performance of our approach, thirteen widely used benchmark functions are used. Experimental results confirm that our approach is able to adaptively choose the suitable strategy for different problems. Compared to classical DE algorithms and to a recently proposed adaptive scheme (SaDE), it obtains better results in most of the functions, in terms of the quality of the final results and convergence speed.

In this paper, we propose a hybrid Differential Evolution (DE) algorithm based on the fuzzy C-means clustering algorithm, referred to as FCDE. The fuzzy C-means clustering algorithm is incorporated with DE to utilize the information of the population efficiently, and hence it can generate good solutions and enhance the performance of the original DE. In addition, the population-based algorithmgenerator is adopted to efficiently update the population with the clustering offspring. In order to test the performance of our approach, 13 high-dimensional benchmark functions of diverse complexities are employed. The results show that our approach is effective and efficient. Compared with other state-of-the-art DE approaches, our approach performs better, or at least comparably, in terms of the quality of the final solutions and the reduction of the number of fitness function evaluations (NFFEs).

Traditionally, the performance of a classifier is measured by its classification accuracy or error rate. In fact, probability-based classifiers also produce the class probability estimation (the probability that a test instance belongs to the predicted class). This information is often ignored in classification, as long as the class with the highest class probability estimation is identical to the actual class. In many data mining applications, however, classification accuracy and error rate are not enough. For example, in direct marketing, we often need to deploy different promotion strategies to customers with different likelihood (class probability) of buying some products. Thus, accurate class probability estimations are often required to make optimal decisions. In this paper, we firstly review some state-of-the-art probability-based classifiers and empirically investigate their class probability estimation performance. From our experimental results, we can draw a conclusion: C4.4 is an attractive algorithm for class probability estimation. Then, we present a locally weighted version of C4.4 to scale up its class probability estimation performance by combining locally weighted learning with C4.4. We call our improved algorithm locally weighted C4.4, simply LWC4.4. We experimentally test LWC4.4 using the whole 36 UCI data sets selected by Weka. The experimental results show that LWC4.4 significantly outperforms C4.4 in terms of class probability estimation.

Because learning an optimal Bayesian network classifier is an NP-hard problem, learning-improved naive Bayes has attracted much attention from researchers. In this paper, we summarize the existing improved algorithms and propose a novel Bayes model: hidden naive Bayes (HNB). In HNB, a hidden parent is created for each attribute which combines the influences from all other attributes. We experimentally test HNB in terms of classification accuracy, using the 36 UCI data sets selected by Weka, and compare it to naive Bayes (NB), selective Bayesian classifiers (SBC), naive Bayes tree (NBTree), tree-augmented naive Bayes (TAN), and averaged one-dependence estimators (AODE). The experimental results show that HNB significantly outperforms NB, SBC, NBTree, TAN, and AODE. In many data mining applications, an accurate class probability estimation and ranking are also desirable. We study the class probability estimation and ranking performance, measured by conditional log likelihood (CLL) and the area under the ROC curve (AUC), respectively, of naive Bayes and its improved models, such as SBC, NBTree, TAN, and AODE, and then compare HNB to them in terms of CLL and AUC. Our experiments show that HNB also significantly outperforms all of them.

Recently, using multiobjective optimization concepts to solve the constrained optimization problems (COPs) has attracted much attention. In this paper, a novel multiobjective differential evolution algorithm, which combines several features of previous evolutionary algorithms (EAs) in a unique manner, is proposed to COPs. Our approach uses the orthogonal design method to generate the initial population; also the crossover operator based on the orthogonal design method is employed to enhance the local search ability. In order to handle the constraints, a novel constraint-handling method based on Pareto dominance concept is proposed. An archive is adopted to store the nondominated solutions and a relaxed form of Pareto dominance, called ε-dominance, is used to update the archive. Moreover, to utilize the archive solution to guide the search, a hybrid selection mechanism is proposed. Experiments have been conducted on 13 benchmark COPs. And the results prove the efficiency of our approach. Compared with five state-of-the-art EAs, our approach provides very good results, which are highly competitive with those generated by the compared EAs in constrained evolutionary optimization. Furthermore, the computational cost of our approach is relatively low.

Improving naive Bayes (simply NB)15,28 for classification has received significant attention. Related work can be broadly divided into two approaches: eager learning and lazy learning.1 Different from eager learning, the key idea for extending naive Bayes using lazy learning is to learn an improved naive Bayes for each test instance. In recent years, several lazy extensions of naive Bayes have been proposed. For example, LBR,30 SNNB,27 and LWNB.8 All these algorithms aim to improve naive Bayes' classification performance. Indeed, they achieve significant improvement in terms of classification, measured by accuracy. In many real-world data mining applications, however, an accurate ranking is more desirable than an accurate classification. Thus a natural question is whether they also achieve significant improvement in terms of ranking, measured by AUC (the area under the ROC curve).2,11,17 Responding to this question, we conduct experiments on the 36 UCI data sets18 selected by Weka12 to investigate their ranking performance and find that they do not significantly improve the ranking performance of naive Bayes. Aiming at scaling up naive Bayes' ranking performance, we present a novel lazy method ICNB (instance cloned naive Bayes) and develop three ICNB algorithms using different instance cloning strategies. We empirically compare them with naive Bayes. The experimental results show that our algorithms achieve significant improvement in terms of AUC. Our research provides a simple but effective method for the applications where an accurate ranking is desirable.