Density-Based Spatial Clustering of Applications with Noise (DBSCAN), a well-known density-based clustering algorithm, has gained widespread popularity and usage due to its effectiveness in identifying clusters of arbitrary shapes and handling noisy data. However, it encounters challenges in producing satisfactory cluster results when confronted with datasets of varying density scales, a common scenario in real-world applications. In this paper, we propose a novel Adaptive and Robust DBSCAN with Multi-agent Reinforcement Learning cluster framework, namely AR-DBSCAN. First, we model the initial dataset as a two-level encoding tree and categorize the data vertices into distinct density partitions according to the information uncertainty determined in the encoding tree. Each partition is then assigned to an agent to find the best clustering parameters without manual assistance. The allocation is density-adaptive, enabling AR-DBSCAN to effectively handle diverse density distributions within the dataset by utilizing distinct agents for different partitions. Second, a multi-agent deep reinforcement learning guided automatic parameter searching process is designed. The process of adjusting the parameter search direction by perceiving the clustering environment is modeled as a Markov decision process. Using a weakly-supervised reward training policy network, each agent adaptively learns the optimal clustering parameters by interacting with the clusters. Third, a recursive search mechanism adaptable to the data's scale is presented, enabling efficient and controlled exploration of large parameter spaces. Extensive experiments are conducted on nine artificial datasets and a real-world dataset. The results of offline and online tasks show that AR-DBSCAN not only improves clustering accuracy by up to 144.1% and 175.3% in the Normalized Mutual Information (NMI) and the Adjusted Rand Index (ARI) metrics, respectively, but also is capable of robustly finding dominant parameters.

Active learning (AL) is a semi-supervised learning paradigm with human-machine interaction and a limited annotation budget. However, few AL studies have explored distribution inconsistency between the data and the population. In this paper, we consider a basic form of the aforementioned issue,i.e., the training data is non-independently and identically distributed (non-IID) sampled from a class uniformly distributed population. Accordingly, we propose a naïve sample selection plugin, namely generalized active stratified sampling (GASS), to rebalance the sample size of each class during AL iterative process, resulting in a progressive approximation to the population. We generalize statistical stratified sampling to support the uncertainty strata criterion, forming the statistical foundation of GASS. This method, as a plugin, can seamlessly collaborate with popular information-based strategies. GASS shows superior rebalancing capabilities by analyzing the statistical moment and the class imbalanced index under the Probably Approximately Correct (PAC) theory. Furthermore, models derived with GASS have low Rademacher complexity (RC), indicating low generalization error bounds, and GASS also exhibits strong robustness to prediction perturbations. Experiments were conducted on 5 benchmark image datasets, and the results show that GASS significantly boosts the test accuracy by about$2.38\%$/$3.19\%$(paired$t$-test$p=0.01$/0.04) and reduces the empirical RC by about$1.42\%$/$1.94\%$(paired$t$-test$p=0.01$/0.05) on average in class imbalanced/balanced scenarios, respectively. This study establishes a potential benchmark for information-based AL.

Predicting protein-ligand binding affinity from a structural level is a crucial but challenging problem in computational drug discovery (CDD). Although such works have benefited largely from the advancements of machine learning or deep learning techniques, developing an approach that performs well in predictions while producing decent interpretability still needs further exploration. In this study, we proposed a binding affinity prediction model (UAAM), which leverages the concept of thermodynamics cycle and can be interpreted from the perspective of energy. Moreover, it adopts an attention mechanism to investigate important atom pairs in each protein-ligand binding structure and quantifies the uncertainty levels of those attention scores using entropy. A loss function with a regularization term on the uncertainty levels is employed in UAAM for better extraction of important atom pairs. Compared with baseline approaches on a benchmark database, the proposed UAAM model performed well and was able to generate valuable knowledge at the atom-pair level. This work will contribute concretely to the sustainable development of the CDD community.

Multi-granularity formal concept analysis is a research field that integrates granular computing and formal concept analysis. It provides valuable opportunities to analyze problems at multiple levels of granularity, which has been becoming a key tool for data mining and knowledge discovery. In multi-granularity formal concept analysis, it is an important challenge to select the optimal granularity. Existing methods for optimal granularity selection typically focus on a single granularity of attributes, without considering meso-granularity layers or the combination of different attribute granularities. Furthermore, these methods fail to reduce redundant attributes effectively. This paper aims to solve this issue and further improve the performance of classification. Specifically, we first introduce the meso-granularity class-attribute blocks to address these limitations. Then, we introduce information gain and design an optimal meso-granularity selection algorithm by combining meso-granularity class-attribute blocks with decision information. Furthermore, we propose the concept of core attributes and define their internal and external significance, which is beneficial for attribute reduction. Finally, the optimal meso-granularity combination is selected by using Bayesian optimization , which improves the efficiency of the meso-granularity combination for classification. The effectiveness of the proposed model is validated through experiments on real-world datasets. The experimental results demonstrate that the proposed model outperforms existing methods, achieving the best classification performance among the compared methods. All codes have been released at https://github.com/yqlinux/OMSFC .

Outliers are intrinsic to recommender systems (RSs) due to user uncertainty and bring large deviations to the total loss. Classical loss functions such as$L_{1}$and$L_{2}$do not consider this issue, thus biasing the model toward abnormal ratings. Advanced ones such as Logcosh, Huber, and Wing losses try to control model biasing; however, they do not consider the rating ranges of recommender systems. In this article, we propose a Seagull loss combining quadratic and softsign functions to handle this issue. The quadratic function aims to preserve or appropriately increase losses caused by small deviations, while softsign function aims to suppress losses caused by large deviations. It is distinguished from three types of popular losses. Compared with$L_{1}$and$L_{2}$, it controls the upper bound, therefore suppressing the influence of outliers. Compared with Logcosh and Huber, it not only adjusts the loss of small deviations but also suppresses the loss of large deviations. Compared with Wing loss, it considers the characteristics of data in RSs and therefore is more suitable to the matrix factorization model. Experiments are undertaken on six real-world recommendation datasets in comparison with six popular loss functions. Results show that Seagull loss function has superiority to all losses above on mean absolute error (MAE) and root mean square error (RMSE) and slight inferiority to$L_{2}$and Wing on hit ratio (HR), mean average precision (MAP), and normalized discounted cumulative gain (NDCG).

Neighborhood rough sets are an effective model for handling numerical and categorical data entangled with vagueness, imprecision, or uncertainty. However, existing neighborhood rough set models and their feature selection methods treat each sample equally, whereas different types of samples inherently play different roles in constructing neighborhood granules and evaluating the goodness of features. In this study, the sample weight information is first introduced into neighborhood rough sets, and a novel weighted neighborhood rough set model is consequently constructed. Then, considering the lack of sample weight information in practical data, a margin-based weight optimization function is designed, based on which a gradient descent algorithm is provided to adaptively learn sample weights through maximizing sample margins. Finally, an average granule margin measure is put forward for feature selection, and a forward-adding heuristic algorithm is developed to generate an optimal feature subset. The proposed method constructs the weighted neighborhood rough sets using sample weights for the first time and is able to yield compact feature subsets with a large margin. Extensive experiments and statistical analysis on UCI datasets show that the proposed method achieves highly competitive performance in terms of feature reduction rate and classification accuracy when compared with other state-of-the-art methods.

Concept-cognitive learning is an emerging area of cognitive computing, which refers to continuously learning new knowledge by imitating the human cognition process. However, the existing research on concept-cognitive learning is still at the level of complete cognition as well as cognitive operators, which is far from the real cognition process. Meanwhile, the current classification algorithms based on concept-cognitive learning models (CCLMs) are not mature enough yet since their cognitive results highly depend on the cognition order of attributes. To address the above problems, this article presents a novel concept-cognitive learning method, namely, stochastic incremental incomplete concept-cognitive learning method (SI2CCLM), whose cognition process adopts a stochastic strategy that is independent of the order of attributes. Moreover, a new classification algorithm based on SI2CCLM is developed, and the analysis of the parameters and convergence of the algorithm is made. Finally, we show the cognitive effectiveness of SI2CCLM by comparing it with other concept-cognitive learning methods. In addition, the average accuracy of our model on 24 datasets is 82.02%, which is higher than the compared 20 classification algorithms, and the elapsed time of our model also has advantages.

Heterogeneous domain adaptation (HDA) is expected to achieve effective knowledge transfer from a label-rich source domain to a heterogeneous target domain with scarce labeled data. Most prior HDA methods strive to align the cross-domain feature distributions by learning domain invariant representations without considering the intrinsic semantic correlations among categories, which inevitably results in the suboptimal adaptation performance across domains. Therefore, to address this issue, we propose a novel semantic correlation transfer (SCT) method for HDA, which not only matches the marginal and conditional distributions between domains to mitigate the large domain discrepancy, but also transfers the category correlation knowledge underlying the source domain to target by maximizing the pairwise class similarity across source and target. Technically, the domainwise and classwise centroids (prototypes) are first computed and aligned according to the feature embeddings. Then, based on the derived classwise prototypes, we leverage the cosine similarity of each two classes in both domains to transfer the supervised source semantic correlation knowledge among different categories to target effectively. As a result, the feature transferability and category discriminability can be simultaneously improved during the adaptation process. Comprehensive experiments and ablation studies on standard HDA tasks, such as text-to-image, image-to-image, and text-to-text, have demonstrated the superiority of our proposed SCT against several state-of-the-art HDA methods.

Deep neural networks (DNNs) have demonstrated remarkable success on various learning problems, but they face a formidable challenge in the form of adversarial attacks. Especially, when dealing with complex classification tasks for numerous classes with a hierarchical structure, the adversarial robustness of a DNN model may drop seriously. In this paper, we investigate the adversarial robustness of DNN models on such complex classification tasks. In response, we propose a two-stage hierarchical classification framework, which is composed of a coarse-grained classifier and a series of fine-grained classifiers. A data correction sampling module is designed between the two stages, in order to mitigate the influence of misclassification caused by the coarse-grained classifier; and a discriminative filter learning module is employed in the fine-grained classification, in order to gain better distinguish abilities among fine-grained categories. Experiments on the well-known dataset CIFAR-100 and a newly-constructed hierarchical dataset mini-ImageNet76 demonstrate that employing a hierarchical framework can effectively improve the model robustness on such complex classification tasks.

Representing data in different spaces becomes more powerful and suitable for solving downstream learning tasks. The membership degrees obtained through fuzzy C-means (FCM) clustering cannot capture data structures sufficiently, as they represent samples from a single Euclidean geometrical perspective. To address this issue, we propose a novel fuzzy clustering model guided by spectral rotation and scaling (FCSR). In FCSR, both spectral embeddings and membership degrees are considered as new representations of data. They can complement each other from different perspectives which enables the model to engage more structural properties of the data. The process of solving the problem of membership degrees not only inherits the merits of traditional FCM but also preserves data neighborhood structures revealed by the spectral decomposition based on an affinity matrix. Furthermore, to improve the adaptability and extensibility of FCSR, the projected and kernel versions of FCSR (FCSR-P and FCSR-K) are formed. We demonstrate that FCSR-P is suitable for high-dimensional scenarios and FCSR-K can improve the linear separability among data. Extensive experiments conducted on various well-known data sets illustrate the validity of the proposed ideas.

Segmentation on the left atrium from magnetic resonance imaging scans is a crucial task for the clinical diagnosis of cardiac diseases. Automatic segmentation methods based on deep learning have demonstrated their advantages in recent years, but they often suffer from the difficulty to process high uncertainty of some regions. Focusing on the cognitive uncertainty in deep neural networks, we build an uncertainty-aware model based on anchor fuzzy sets in this article. A fuzzy entropy measure defined on membership functions is employed to quantify the uncertainty induced by an anchor's membership degrees to the individual categories. A key insight in this study is that the anchor regions with higher uncertainty are considered to include hard-generalized cases. Based on this prior of high uncertainty about image segmentation, the uncertainty-aware model exhibits the self-learning attention that sheds light on the regions the deep learning should focus on. In addition, our segmentation model with an uncertainty-aware plug-in unit highlights its nonlocal perception due to Fourier convolution and global self-attention mechanism. Experiments show that our model can achieve competitive results on left atrium segmentation tasks. This article provides some useful guidelines for using uncertainty modeling to improve the performance of deep learning systems for image segmentation tasks.

Graph-based clustering approaches, especially the family of spectral clustering, have been widely used in machine learning areas. The alternatives usually engage a similarity matrix that is constructed in advance or learned from a probabilistic perspective. However, unreasonable similarity matrix construction inevitably leads to performance degradation, and the sum-to-one probability constraints may make the approaches sensitive to noisy scenarios. To address these issues, the notion of typicality-aware adaptive similarity matrix learning is presented in this study. The typicality (possibility) rather than the probability of each sample being a neighbor of other samples is measured and adaptively learned. By introducing a robust balance term, the similarity between any pairs of samples is only related to the distance between them, yet it is not affected by other samples. Therefore, the impact caused by the noisy data or outliers can be alleviated, and meanwhile, the neighborhood structures can be well captured according to the joint distance between samples and their spectral embeddings. Moreover, the generated similarity matrix has block diagonal properties that are beneficial to correct clustering. Interestingly, the results optimized by the typicality-aware adaptive similarity matrix learning share the common essence with the Gaussian kernel function, and the latter can be directly derived from the former. Extensive experiments on synthetic and well-known benchmark datasets demonstrate the superiority of the proposed idea when comparing with some state-of-the-art methods.

Generalized zero-shot learning (GZSL) aims to train a model for classifying data samples under the condition that some output classes are unknown during supervised learning. To address this challenging task, GZSL leverages semantic information of the seen (source) and unseen (target) classes to bridge the gap between both seen and unseen classes. Since its introduction, many GZSL models have been formulated. In this review paper, we present a comprehensive review on GZSL. First, we provide an overview of GZSL including the problems and challenges. Then, we introduce a hierarchical categorization for the GZSL methods and discuss the representative methods in each category. In addition, we discuss the available benchmark data sets and applications of GZSL, along with a discussion on the research gaps and directions for future investigations.

Fine-grained visual categorization is a challenging issue owing to high intra-class and low inter-class variances. Classical approaches rely on pre-trained models or many fine annotations. In this paper, we observe that spatial and frequency information provides distinct image views, and propose a new spatial–frequency feature fusion (SFFF) perspective to handle this challenging issue. Specifically, we design a heterogeneous feature extraction loss function, construct a global and local fusion SFFF network, and propose an importance-sparsity selection strategy. For feature extraction, we focus on the frequency domain feature learning network, extract fine-grained features, and achieve feature complementarity. For feature selection, we propose importance ranking and sparse regularity to constrain spatial–frequency features. For feature fusion, we design a spatial–frequency loss and an inter-layer switching strategy to achieve local-global collaboration. Comparative experiments were performed on popular fine-grained datasets and classic datasets such as CUB200-2011, Stanford Cars, Stanford Dogs, FGVC-Aircraft, and CIFAR100. The effectiveness and outstanding performance of SFFF are confirmed by comparisons with more than 40 state-of-the-art fine-grained categorization methods. Ablation studies and visualizations are provided to facilitate an understanding of our approach.

It is necessary to improve the performance of some special classes or to particularly protect them from attacks in adversarial learning. This article proposes a framework combining cost-sensitive classification and adversarial learning together to train a model that can distinguish between the protected and unprotected classes, such that the protected classes are less vulnerable to adversarial examples. We find in this framework an interesting phenomenon during the training of deep neural networks, called the Min–Max property, that is, the absolute values of most parameters in the convolutional layer approach 0 while the absolute values of a few parameters are significantly larger, becoming bigger. Based on this Min–Max property which is formulated and analyzed in a view of random distribution, we further build a new defense model against adversarial examples for adversarial robustness improvement. An advantage of the built model is that it performs better than the standard one and can combine with adversarial training to achieve improved performance. It is experimentally confirmed that, regarding the average accuracy of all classes, our model is almost as same as the existing models when an attack does not occur and is better than the existing models when an attack occurs. Specifically, regarding the accuracy of protected classes, the proposed model is much better than the existing models when an attack occurs.

Fuzzy$\beta$-covering rough sets, as an effective extension of covering-based rough sets, have been concerned by many researchers. All fuzzy$\beta$-covering rough set models are constructed under a corresponding fuzzy$\beta$-covering approximation space. However, fuzzy$\beta$-covering is difficult to find directly from the real data. Fortunately, fuzzy information granulation provides a reasonable and effective way to obtain fuzzy$\beta$-coverings from the real data. Since fuzzy information granulation is capable of generating multiple fuzzy$\beta$-coverings, we introduce the notion of multifuzzy$\beta$-covering approximation spaces. Fuzzy$\beta$-covering approximation spaces are a special case of multifuzzy$\beta$-covering approximation spaces. Besides, we employ fuzzy$\beta$-neighborhood operators with reflexivity and symmetry to characterize the similarity between samples. In this article, we first present the definition of multifuzzy$\beta$-covering approximation spaces and investigate some useful properties about fuzzy$\beta$-covering. Second, several information measures are explored in the context of multifuzzy$\beta$-covering approximation spaces. On this basis, a novel heuristic fuzzy$\beta$-covering reduction method with the measure of monotone conditional entropy is proposed. Moreover, a general framework of attribute reduction based on fuzzy$\beta$-covering reduction is also designed. Finally, through the comparative and experimental analyses with other four state-of-the-art attribute reduction methods, the effectiveness and superiority of the proposed method are verified.

All six papers in this special section engage in different streams but extremely relevant domain vectors of Cognitive Cyber-Physical Systems (CCPS) with artificial intelligence (AI) based solutions in medical informatics. Highlights recent trends in the scientific community and presents emergent technologies, implementations, applications concerning the CPSS. CPSS is witnessing rapid transformation as an interdisciplinary technology that blends physical components and computing devices to enable AI-based solutions. CCPS will be playing a significant role that integrates machine learning/AI techniques and resulting in dramatic improvements for medical informatics and the future of human-augmentation. CPHMS coordinates supervisory medical systems and medical resources everywhere; there is a great scope towards health consciousness and healthy society. Medical Cyber-Physical Systems (MCPS) in healthcare towards critical integration in network of medical devices. MCPS is the next generation computing that is comprised of tightly coupled computational and communication components of medical automation systems such as clinical decision, early detection of health infectious, disease prevention, rapid analysis of health hazards and so on. CCPS and MCPS research would be created new models, new design, and integration models for large scale systems in comprehensive, holistic medical automation systems. With recent enlargements in the big data processing, cognitive data science and AI, it is now possible to create even more realistic digital twins that properly model different operating situations and characteristics to process the medical intelligence systems.

Dominance-based rough approximation discovers inconsistencies from ordered criteria and satisfies the requirement of the dominance principle between single-valued domains of condition attributes and decision classes. When the ordered decision system (ODS) is no longer single-valued, how to utilize the dominance principle to deal with multivalued ordered data is a promising research direction, and it is the most challenging step to design a feature selection algorithm in interval-valued ODS (IV-ODS). In this article, we first present novel thresholds of interval dominance degree (IDD) and interval overlap degree (IOD) between interval values to make the dominance principle applicable to an IV-ODS, and then, the interval-valued dominance relation in the IV-ODS is constructed by utilizing the above two developed parameters. Based on the proposed interval-valued dominance relation, the interval-valued dominance-based rough set approach (IV-DRSA) and their corresponding properties are investigated. Moreover, the interval dominance-based feature selection rules based on IV-DRSA are provided, and the relevant algorithms for deriving the interval-valued dominance relation and the feature selection methods are established in IV-ODS. To illustrate the effectiveness of the parameters variation on feature selection rules, experimental evaluation is performed using 12 datasets coming from the University of California-Irvine (UCI) repository.

Practical data often suffer from missing attribute values and lack of class labels. A reasonable machine learning scenario involves obtaining certain values and labels at cost on request. In this article, we propose the cost-sensitive active learning through unified evaluation and dynamic selection (CALS) algorithm to handle the learning task in this new scenario. For data representation, we consider misclassification cost, label query cost, and attribute query cost. For the cost/benefit estimation, we design a unified assessment of attribute values and labels with softmax regression. For the selection of attribute value and label, we propose an optimal acquisition scheme with permutation and greedy strategies. We perform experiments with synthetic, benchmark, and domain datasets. The results of the significance test verify the effectiveness of CALS and its superiority over cost-sensitive active learning and missing data imputation algorithms.

In real-life applications, monotonic classification is a widespread task, where the improvement of a particular input value cannot result in an inferior output. A common drawback of the existing algorithms for monotonic classification is their sensitivity to noise data which particularly refer to monotonicity violations in the monotonic circumstance. Motivated by weakening the impact of noises, the fuzzy monotonicK-nearest neighbor (FMKNN) is proposed in this article, which constructs monotonic classifiers by taking advantage of the fuzzy dominance relation between a pair of instances, especially that between incomparable instances for the first time. Through tuning the thresholds of fuzzy dominance relation degrees, FMKNN intends to decrease the disturbance caused by noises which considerably affect the selection range of theK-nearest neighbors in different extent. The experimental results show that the best average improvement degrees of FMKNN in terms of theKNN-based and non-KNN-based classifiers on all the involved datasets arrive at 28$\%$, 11$\%$, and 29$\%$with respect to ACCU, MAE, and NMI, respectively, which demonstrates the superiority of our proposed FMKNN over other state-of-the-art monotonic classifiers including the monotonic fuzzyK-nearest neighbor (MFKNN) which disperses the impact of noise data by converting crisp class labels into class membership vectors.

Multi-instance learning (MIL) is more general and challenging than traditional supervised learning in that labels are given at the bag level. The popular feature mapping approaches convert each bag into an instance in the new feature space. However, most of them hardly maintain the distinguishability of bags, and the MIL model does not support self-reinforcement. In this article, we propose the multi-instance ensemble learning with discriminative bags (ELDB) algorithm with two new techniques. The bag selection technique obtains a discriminative bag set (dBagSet) according to two parts. First, considering the space and label distribution of the data, the bag selection process is optimized through discriminative analysis to obtain the basic dBagSet. Second, with the state and action transfer strategy, a dBagSet with better distinguishability is obtained through self-reinforcement. The ensemble technique trains a series of classifiers with these dBagSets and obtains the final weighted model. The experimental results show that ELDB is superior to the state-of-the-art MIL mapping solutions.

With the development of social network platforms, discussion forums, and question answering websites, a huge number of short messages that typically contain a few words for an individual document are posted by online users. In these short messages, emotions are frequently embedded for communicating opinions, expressing friendship, and promoting influence. It is quite valuable to detect emotions from short messages, but the corresponding task suffers from the sparsity of feature space. In this article, we first generate term groups co-occurring in the same context to enrich the number of features. Then, two basic supervised topic models are proposed to associate emotions with topics accurately. To reduce the time cost of parameter estimation, we further propose an accelerated algorithm for our basic models. Extensive evaluations using three short corpora validate the efficiency and effectiveness of the accelerated models for predicting the emotions of unlabeled documents, in addition to generate the topic-level emotion lexicons.

Kinship verification in the wild is an interesting and challenging problem. The goal of kinship verification is to determine whether a pair of faces are blood relatives or not. Most previous methods for kinship verification can be divided as handcrafted features-based shallow learning methods and convolutional neural network (CNN)-based deep-learning methods. Nevertheless, these methods are still facing the challenging task of recognizing kinship cues from facial images. The reason is that the family ID information and the distribution difference of pairwise kin-faces are rarely considered in kinship verification tasks. To this end, a family ID-based adversarial convolutional network (AdvKin) method focused on discriminative Kin features is proposed for both small-scale and large-scale kinship verification in this article. The merits of this article are four-fold: 1) for kin-relation discovery, a simple yet effective self-adversarial mechanism based on a negative maximum mean discrepancy (NMMD) loss is formulated as attacks in the first fully connected layer; 2) a pairwise contrastive loss and family ID-based softmax loss are jointly formulated in the second and third fully connected layer, respectively, for supervised training; 3) a two-stream network architecture with residual connections is proposed in AdvKin; and 4) for more fine-grained deep kin-feature augmentation, an ensemble of patch-wise AdvKin networks is proposed (E-AdvKin). Extensive experiments on 4 small-scale benchmark KinFace datasets and 1 large-scale families in the wild (FIW) dataset from the first Large-Scale Kinship Recognition Data Challenge, show the superiority of our proposed AdvKin model over other state-of-the-art approaches.

Rule-based classifier, that extract a subset of induced rules to efficiently learn/mine while preserving the discernibility information, plays a crucial role in human-explainable artificial intelligence. However, in this era of big data, rule induction on the whole datasets is computationally intensive. So far, to the best of our knowledge, no known method focusing on accelerating rule induction has been reported. This is the first study to consider the acceleration technique to reduce the scale of computation in rule induction. We propose an accelerator for rule induction based on fuzzy rough theory; the accelerator can avoid redundant computation and accelerate the building of a rule classifier. First, a rule induction method based on consistence degree, called consistence-based value reduction (CVR), is proposed and used as basis to accelerate. Second, we introduce a compacted search space termed key set, which only contains the key instances required to update the induced rule, to conduct value reduction. The monotonicity of key set ensures the feasibility of our accelerator. Third, a rule-induction accelerator is designed based on key set, and it is theoretically guaranteed to display the same results as the unaccelerated version. Specifically, the rank preservation property of key set ensures consistency between the rule induction achieved by the accelerator and the unaccelerated method. Finally, extensive experiments demonstrate that the proposed accelerator can perform remarkably faster than the unaccelerated rule-based classifier methods, especially on datasets with numerous instances.

This paper proposes a novel method called Jointly Sparse Locality Regression (JSLR) for feature extraction and selection. JSLR utilizes joint L2,1-norm minimization on regularization term, and also introduces the locality to characterize the local geometric structure of the data. There are three main contributions in JSLR for face recognition. Firstly, it eliminates the drawback in ridge regression and Linear Discriminant Analysis (LDA) that when the number of the classes is too small, not enough projections can be obtained for feature extraction. Secondly, by using the local geometric structure as the regularization term, JSLR is able to preserve local information and find an embedding subspace which can detect the most essential data manifold structure. Moreover, since the L2,1-norm based loss function is robust to outliers in data points, JSLR provides the joint sparsity for robust feature selection. The theoretical connections of the proposed method and the previous regression methods are explored and the convergence of the proposed algorithm is also proved. Experimental evaluation on several well-known data sets shows the merits of the proposed method on feature selection and classification.

The initialization of neural networks with random weights (NNRW) has a significant impact on model performance. However, there is no suitable way to solve this problem so far. In this paper, the relationship between meta features of a dataset and the initialization of NNRW is studied. Specifically, we construct seven regression datasets with known attributes' distributions, then initialize NNRW with different distributions and trained them based on the datasets to get the corresponding models respectively. The relationship between the attributes' distributions of the datasets and the initialization of NNRW is analyzed by the performance of the models. Several interesting phenomena are observed: firstly, initializing NNRW with the Gaussian distribution can help the model to have a faster convergence rate than ones with the Gamma and Uniform distribution. Secondly, if one or more attributes in a dataset that follow the Gamma distribution, using Gamma distribution to initialize NNRW may result in a slower convergence rate and easy overfitting. Thirdly, initializing NNRW with a specific distribution with smaller variances can always achieve faster convergence rate and better generalization performance than the one with larger variances. The above experimental results are not sensitive to the activation function and the type of NNRW. Some theoretical analyses about the above observations are also given in the study.

With wide adoption of Internet of Things (IoT) across the world, the IoT devices are facing more and more intensive computation task nowadays. However, the IoT devices are usually limited by their computing capability and battery lifetime. Mobile edge computing provides new opportunities for developments of IoT, since edge computing servers which are close to devices can provide more powerful computing resources. The IoT devices can offload the intensive computing tasks to edge computing servers, while saving their own computing resources and reducing energy consumption. However, the benefits come at the cost of higher latency, mainly due to additional transmission time, and it may be unacceptable for many IoT applications. In this paper, we try to find a tradeoff between the energy consumption and latency, in order to satisfy user demands of various IoT applications. We formalize the problem into a constrained multiobjective optimization problem and find the optimal solutions by a modified fast and elitist nondominated sorting genetic algorithm (NSGA-II). To improve the performance of the algorithm, we propose a novel problem-specific encoding scheme and genetic operators in the proposed modified NSGA-II. We also conduct extensive simulation experiments to evaluate the proposed algorithm and its sensitivity under certain major parameters. The experimental results show that the proposed algorithm can find a large number of optimal solutions to adjust the corresponding offloading decision according to the real-world situation.

Finding customer groups from transaction data is very important for retail and e-commerce companies. Recently, a "Purchase Tree" data structure is proposed to compress the customer transaction data and a local PurTree spectral clustering method is proposed to cluster the customer transaction data. However, in the PurTree distance, the node weights for the children nodes of a parent node are set as equal and the differences between different nodes are not distinguished. In this paper, we propose a two-level subspace weighting spectral clustering (TSW) algorithm for customer transaction data. In the new method, a PurTree subspace metric is proposed to measure the dissimilarity between two customers represented by two purchase trees, in which a set of level weights are introduced to distinguish the importance of different tree levels and a set of sparse node weights are introduced to distinguish the importance of different tree nodes in a purchase tree. TSW learns an adaptive similarity matrix from the local distances in order to better uncover the cluster structure buried in the customer transaction data. Simultaneously, it learns a set of level weights and a set of sparse node weights in the PurTree subspace distance. An iterative optimization algorithm is proposed to optimize the proposed model. We also present an efficient method to compute a regularization parameter in TSW. TSW was compared with six clustering algorithms on ten benchmark data sets and the experimental results show the superiority of the new method.

Images are uploaded to the Internet over time which makes concept drifting and distribution change in semantic classes unavoidable. Current hashing methods being trained using a given static database may not be suitable for nonstationary semantic image retrieval problems. Moreover, directly retraining a whole hash table to update knowledge coming from new arriving image data may not be efficient. Therefore, this paper proposes a new incremental hash-bit learning method. At the arrival of new data, hash bits are selected from both existing and newly trained hash bits by an iterative maximization of a 3-component objective function. This objective function is also used to weight selected hash bits to re-rank retrieved images for better semantic image retrieval results. The three components evaluate a hash bit in three different angles: 1) information preservation; 2) partition balancing; and 3) bit angular difference. The proposed method combines knowledge retained from previously trained hash bits and new semantic knowledge learned from the new data by training new hash bits. In comparison to table-based incremental hashing, the proposed method automatically adjusts the number of bits from old data and new data according to the concept drifting in the given data via the maximization of the objective function. Experimental results show that the proposed method outperforms existing stationary hashing methods, table-based incremental hashing, and online hashing methods in 15 different simulated nonstationary data environments.

With the help of automated classification, severe bugs can be rapidly identified so that the latent damage to software projects can be minimized. However, bug report datasets commonly suffer from disproportionate number of category samples. When presented with the situation of class imbalance, most standard classification learning approaches fail to properly learn the distributive characteristics of the samples and tend to result in unfavorable performance to predict class label. In this case, imbalanced learning becomes critical to advance classification algorithms. In this paper, we propose an improved synthetic minority oversampling technique to avoid the degraded performance caused by class imbalance in bug report datasets. Moreover, to lessen the chance of occasionalities in random sampling process, we propose a repeated sampling technique to train different, but related classifiers. Finally, an ensemble algorithm based on Choquet fuzzy integral is employed to combine the wisdom of crowds and make better decisions. We conduct comprehensive experiments on several bug report datasets from real-world bug repositories. The results demonstrate that the proposed method boosts the classification performance across the classes of the data. Specifically, compared with various ensemble learning techniques, the Choquet fuzzy integral achieves outstanding results on integrating multiple random oversampling techniques.

Fuzzy twin support vector machine (FTSVM) is an effective machine learning technique that is able to overcome the negative impact of noise and outliers in tackling data classification problems. In the FTSVM, the degree of membership function in the sample space describes the space between input data and class center, while ignoring the position of input data in the feature space and simply miscalculated the ledge support vectors as noises. This paper presents an intuitionistic FTSVM (IFTSVM) that combines the idea of intuitionistic fuzzy number with twin support vector machine (TSVM). An adequate fuzzy membership is employed to reduce the noise created by the pollutant inputs. Two functions, i.e., linear and nonlinear, are used to formulate two nonparallel hyperplanes. An IFTSVM not only reduces the influence of noises, it also distinguishes the noises from the support vectors. Further, this modification can minimize a newly formulated structural risk and improve the classification accuracy. Two artificial and eleven benchmark problems are employed to evaluate the effectiveness of the proposed IFTSVM model. To quantify the results statistically, the bootstrap technique with the 95% confidence intervals is used. The outcome shows that an IFTSVM is able to produce promising results as compared with those from the original support vector machine, fuzzy support vector machine, FTSVM, and other models reported in the literature.

In data streaming environments such as a smart grid, it is impossible to restrict each data chunk to have the same number of samples in each class. Hence, in addition to the concept drift, classification problems in streaming data environments are inherently imbalanced. However, streaming imbalanced and concept drifting problems in the power system and smart grid have rarely been studied. Incremental learning aims to learn the correct classification for the future unseen samples from the given streaming data. In this paper, we propose a new incremental ensemble learning method to handle both concept drift and class imbalance issues. The class imbalance issue is tackled by an imbalance-reversed bagging method that improves the true positive rate while maintains a low false positive rate. The adaptation to concept drift is achieved by a dynamic cost-sensitive weighting scheme for component classifiers according to their classification performances and stochastic sensitivities. The proposed method is applied to a case study for the electricity pricing in Australia to predict whether the price of New South Wales will be higher or lower than that of Victorias in a 24-h period. Experimental results show the effectiveness of the proposed algorithm with statistical significance in comparison to the state-of-the-art incremental learning methods.

A general deep learning (DL) mechanism for a multiple hidden layer feed-forward neural network contains two parts, i.e., 1) an unsupervised greedy layer-wise training and 2) a supervised fine-tuning which is usually an iterative process. Although this mechanism has been demonstrated in many fields to be able to significantly improve the generalization of neural network, there is no clear evidence to show which one of the two parts plays the essential role for the generalization improvement, resulting in an argument within the DL community. Focusing on this argument, this paper proposes a new DL approach to train multilayer feed-forward neural networks. This approach uses restricted Boltzmann machine (RBM) as the layer-wise training and uses the generalized inverse of a matrix as the supervised fine-tuning. Different from the general deep training mechanism like back-propagation (BP), the proposed approach does not need to iteratively tune the weights, and therefore, has many advantages such as quick training, better generalization, and high understandability, etc. Experimentally, the proposed approach demonstrates an excellent performance in comparison with BP-based DL and the traditional training method for multilayer random weight neural networks. To a great extent, this paper demonstrates that the supervised part plays a more important role than the unsupervised part in DL, which provides some new viewpoints for exploring the essence of DL.

This paper introduces a new type-2 intuitionistic exponential triangular fuzzy number. Basic generalized exponential triangular intuitionistic fuzzy numbers are formulated by (α,β)-cuts. Some of properties and theorems of this type of fuzzy number with graphical representations have been studied and some examples are given to show the effectiveness of the proposed method. Also, the ranking function of the generalized exponential triangular intuitionistic fuzzy number is computed. This ranking method is based on the centroid concept and Euclidean distance. Based on the ranking method, we develop an approach to solving an intuitionistic fuzzy assignment problem where cost is not deterministic numbers but imprecise ones. Then, we solve an intuitionistic fuzzy transportation problem where transportation cost, source, and demand were generalized type-2 intuitionistic fuzzy numbers by the ranking method for Euclidean distance. Intuitionistic fuzzy set theory has been used for analyzing the fuzzy system reliability. We have taken the intuitionistic fuzzy failure to start of an automobile as known basic fault events such as Ignition failure, Battery internal shortage, Spark plug failure and fuel pump failure using Type-2 Intuitionistic Exponential Triangular Fuzzy Number. Our computational procedure is very simple to implement for calculations in intuitionistic fuzzy failure. The major advantage of using Intuitionistic fuzzy sets over fuzzy sets is that intuitionistic fuzzy sets separate the positive and the negative evidence for the membership of an element in a set. Furthermore, the proposed technique can be suitably utilized to solve the start of an automobile problem, because the result of system failure in this method is significant. Finally, the proposed method has been compared with other existing method through numerical examples.

One of the central problems in object recognition is to develop appropriate representations for the objects in images. The authors present a novel approach for image representation that is based on graphs. In the proposed image graph, each node represents a patch and edges are added between neighbouring nodes. First, class‐specific match‐set graphs are generated by matching the image graphs that are in the same categories, and the multi‐image matching problem is solved by applying a seed‐expansion strategy. Then, the matches between the match‐set graphs and an image graph are considered to be the object patches in the image. Finally, the features extracted from these patches are used for the image representation. Extensive experiments are conducted to demonstrate that their approach can obtain state‐of‐the‐art results on several challenging datasets.

Recently, Integrated computing (IC) provides a promising solution to the industry for building the Internet of Things (IoT) systems and make innovation at a rapid pace. The new era of IC with reference to IoT for Industrial applications has three main components: 1) intelligent devices; 2) intelligent system of systems; and 3) end-to-end analytics. This Special Issue is integrating computational intelligence (CI) paradigms, advanced data analytics optimization opportunities to bring more compute to the IoT. CI paradigms are more appropriate for handling uncertainty and complexity of solving real work problems compared to traditional statistical approaches and tools presently being utilized. In fact, recent literatures have addressed the inherent power of fusion of CI approaches. Moreover, it can provide the effective solutions for machine understanding of data (structured/semi structured), optimization problems, specifically, dealing with incomplete or inconsistent information, with limited computational capability related to IoT.

The generalization ability of a classifier learned from a training set is usually dependent on the classifier's uncertainty, which is often described by the fuzziness of the classifier's outputs on the training set. Since the exact dependency relation between generalization and uncertainty of a classifier is quite complicated, it is difficult to clearly or explicitly express this relation in general. This paper shows a specific study on this relation from the viewpoint of complexity of classification by choosing extreme learning machines as the classification algorithms. It concludes that the generalization ability of a classifier is statistically becoming better with the increase of uncertainty when the complexity of the classification problem is relatively high, and the generalization ability is statistically becoming worse with the increase of uncertainty when the complexity is relatively low. This paper tries to provide some useful guidelines for improving the generalization ability of classifiers by adjusting uncertainty based on the problem complexity.

Feature selection based on fuzzy rough sets is an effective approach to select a compact feature subset that optimally predicts a given decision label. Despite being studied extensively, most existing methods of fuzzy rough set based feature selection are restricted to computing the whole dataset in batch, which is often costly or even intractable for large datasets. To improve the time efficiency, we investigate the incremental perspective for fuzzy rough set based feature selection assuming data can be presented in sample subsets one after another. The key challenge for the incremental perspective is how to add and delete features with the subsequent arrival of sample subsets. We tackle this challenge with strategies of adding and deleting features based on the relative discernibility relations that are updated as subsets arrive sequentially. Two incremental algorithms for fuzzy rough set based feature selection are designed based on the strategies. One updates the selected features as each sample subset arrives, and outputs the final feature subset where no sample subset is left. The other updates the relative discernibility relations but only performs feature selection where there is no further subset arriving. Experimental comparisons suggest our incremental algorithms expedite fuzzy rough set based feature selection without compromising performance.

Face recognition technique has been widely applied to personal identification systems due to its satisfying performance. However, its security may be a crucial issue, since many studies have shown that face recognition systems may be vulnerable in an adversarial environment, in which an adversary can camouflage as a legitimate user in order to mislead the system. Although face liveness detection methods have been proposed to distinguish real and fake faces, they are either time-consuming, costly, or sensitive to noise and illumination. This paper proposes a face liveness detection method with flash against 2D spoofing attack. Flash not only can enhance the differentiation between legitimate and illegitimate users, but it also reduces the influence of environmental factors. Two images are taken from a subject, one with flash and another without flash. Four texture and 2D structure descriptors with low computational complexity are used to capture information of the two images in our model. Advantages of our method include low installation cost of flash and no user cooperation required. A data set of 50 subjects collected under different scenarios is used in the experiments to evaluate the proposed method. The experimental results indicate that the proposed model performs better than existing liveness detection methods in different environmental scenarios. This paper confirms that the use of flash successfully improves face liveness detection in terms of accuracy, robustness, and running time.

Feature selection is viewed as an important preprocessing step for pattern recognition, machine learning, and data mining. Neighborhood is one of the most important concepts in classification learning and can be used to distinguish samples with different decisions. In this paper, a neighborhood discrimination index is proposed to characterize the distinguishing information of a neighborhood relation. It reflects the distinguishing ability of a feature subset. The proposed discrimination index is computed by considering the cardinality of a neighborhood relation rather than neighborhood similarity classes. Variants of the discrimination index, including joint discrimination index, conditional discrimination index, and mutual discrimination index, are introduced to compute the change of distinguishing information caused by the combination of multiple feature subsets. They have the similar properties as Shannon entropy and its variants. A parameter, named neighborhood radius, is introduced in these discrimination measures to address the analysis of real-valued data. Based on the proposed discrimination measures, the significance measure of a candidate feature is defined and a greedy forward algorithm for feature selection is designed. Data sets selected from public data sources are used to compare the proposed algorithm with existing algorithms. The experimental results confirm that the discrimination index-based algorithm yields superior performance compared to other classical algorithms.

Multiple-instance active learning (MIAL) is a paradigm to collect sufficient training bags for a multiple-instance learning (MIL) problem, by selecting and querying the most valuable unlabeled bags iteratively. Existing works on MIAL evaluate an unlabeled bag by its informativeness with regard to the current classifier, but neglect the internal distribution of its instances, which can reflect the diversity of the bag. In this paper, two diversity criteria, i.e., clustering-based diversity and fuzzy rough set based diversity, are proposed for MIAL by utilizing a support vector machine (SVM) based MIL classifier. In the first criterion, a kernel k-means clustering algorithm is used to explore the hidden structure of the instances in the feature space of the SVM, and the diversity degree of an unlabeled bag is measured by the number of unique clusters covered by the bag. In the second criterion, the lower approximations in fuzzy rough sets are used to define a new concept named dissimilarity degree, which depicts the uniqueness of an instance so as to measure the diversity degree of a bag. By incorporating the proposed diversity criteria with existing informativeness measurements, new MIAL algorithms are developed, which can select bags with both high informativeness and diversity. Experimental comparisons demonstrate the feasibility and effectiveness of the proposed methods.

When traditional sample selection methods are used to compress large data sets, the computational complexity turns out to be very high and it is really time consuming. To avoid these shortcomings, we propose a new method to select samples based on non-stable cut points. With the basic characteristic of convex function that its extreme values occur at the endpoints of intervals, the method measures the extent of a sample being endpoints by labeling non-stable cut points. Then we can select the samples with higher endpoint extent, which can avoid calculating the distances between samples. This method aims to compress the data sets and improve the computational efficiency without affecting the classification accuracy. Experiments show that the proposed algorithm performs very well on the compression of data sets with higher imbalance degree. Meanwhile, the method is experimentally confirmed to have strong noise-resistance.

Extreme learning machine (ELM) is an efficient training algorithm for single-hidden layer feed-forward neural networks (SLFNs). Two pruned-ELM named P-ELM1 and P-ELM2 are proposed by Rong et al. P-ELM1 and P-ELM2 employ [Formula: see text] and information gain to measure the association between the class labels and individual hidden node respectively. But for the continuous value data sets, it is inevitable for P-ELM1 and P-ELM2 to evaluate the probability distributions of the data sets with discretization methods for calculating [Formula: see text] and information gain, while the discretization will lead to information loss. Furthermore, the discretization will result in high computational complexity. In order to deal with the problems, based on tolerance rough sets, this paper proposed an improved pruned-ELM algorithm, which can overcome the drawbacks mentioned above. Experimental results along with statistical analysis on 8 UCI data sets show that the improved algorithm outperforms the pruned-ELM in computational complexity and testing accuracy.

Extreme learning machine (ELM) is a learning algorithm for single-hidden layer feedforward neural networks (SLFNs) which randomly chooses hidden nodes and analytically determines the output weights of SLFNs. After the input weights and the hidden layer biases are chosen randomly, ELM can be simply considered a linear system. However, the learning time of ELM is mainly spent on calculating the Moore-Penrose inverse matrices of the hidden layer output matrix. This paper focuses on effective computation of the Moore-Penrose inverse matrices for ELM, several methods are proposed. They are the reduced QR factorization with column Pivoting and Geninv ELM (QRGeninv-ELM), tensor product matrix ELM (TPM-ELM). And we compare QRGeninv-ELM, TPM-ELM with the relational algorithm of Moore-Penrose inverse matrices for ELM, the relational algorithms are: Cholesky factorization of singular matrix ELM (Geninv-ELM), QR factorization and Ginv ELM (QRGinv-ELM), the conjugate Gram-Schmidt process ELM (CGS-ELM). The experimental results and the statistical analysis of the experimental results both demonstrate that QRGeninv-ELM, TPM-ELM and Geninv-ELM are faster than other kinds of ELM and can reach comparable generalization performance.

A key issue in decision tree (DT) induction with continuous valued attributes is to design an effective strategy for splitting nodes. The traditional approach to solving this problem is adopting the candidate cut point (CCP) with the highest discriminative ability, which is evaluated by some frequency based heuristic measures. However, such methods ignore the class permutation of examples in the node, and they cannot distinguish the CCPs with the same or similar frequency information, thus may fail to induce a better and smaller tree. In this paper, a new concept, i.e., segment of examples, is proposed to differentiate the CCPs with same frequency information. Then, a new hybrid scheme that combines the two heuristic measures, i.e., frequency and segment, is developed for splitting DT nodes. The relationship between frequency and the expected number of segments, which is regarded as a random variable, is also given. Experimental comparisons demonstrate that the proposed scheme is not only effective to improve the generalization capability, but also valid to reduce the size of the tree.

We investigate essential relationships between generalization capabilities and fuzziness of fuzzy classifiers (viz., the classifiers whose outputs are vectors of membership grades of a pattern to the individual classes). The study makes a claim and offers sound evidence behind the observation that higher fuzziness of a fuzzy classifier may imply better generalization aspects of the classifier, especially for classification data exhibiting complex boundaries. This observation is not intuitive with a commonly accepted position in “traditional” pattern recognition. The relationship that obeys the conditional maximum entropy principle is experimentally confirmed. Furthermore, the relationship can be explained by the fact that samples located close to classification boundaries are more difficult to be correctly classified than the samples positioned far from the boundaries. This relationship is expected to provide some guidelines as to the improvement of generalization aspects of fuzzy classifiers.

Similar to the ensemble learning for classification, regression ensemble also tries to improve the prediction accuracy through combining several “weak” estimators which are usually high-variance and thus unstable. In this paper, we propose a new scheme of fusing the weak Priestley-Chao Kernel Estimators (PCKEs) based on Choquet fuzzy integral, which differs from all the existing models of regressor fusion. The new scheme uses Choquet fuzzy integral to fuse several target outputs from different PCKEs, in which the optimal bandwidths are obtained with cross-validation criteria. The key of applying fuzzy integral to PCKE fusion is the determination of fuzzy measure. Considering the advantage of particle swarm optimization (PSO) algorithm on convergence rate, we use three different PSO algorithms, i.e., standard PSO (SPSO), Gaussian PSO (GPSO) and GPSO with Gaussian jump (GPSOGJ), to determine the general and λ fuzzy measures. The finally experimental results on 6 standard testing functions show that the new paradigm for regression ensemble based on fuzzy integral is more accurate and stable in comparison with any individual PCKE. This demonstrates the feasibility and effectiveness of our proposed regression ensemble model.

The objective of link prediction for social network is to estimate the likelihood that a link exists between two nodes x and y. There are some well-known local information-based link prediction algorithms (LILPAs) which have been proposed to handle this essential and crucial problem in the social network analysis. However, they can not adequately consider the so-called local information: the degrees of x and y, the number of common neighbors of nodes x and y, and the degrees of common neighbors of x and y. In other words, not any LILPA takes into account these local information simultaneously. This limits the performances of LILPAs to a certain degree and leads to the high variability of LILPAs. Thus, in order to make full use of all the local information and obtain a LILPA with highly-predicted capability, an ordered weighted averaging (OWA) operator based link prediction ensemble algorithm (LPEOWA) is proposed by integrating nine different LILPAs with aggregation weights which are determined with maximum entropy method. The final experimental results on benchmark social network datasets show that LPEOWA can obtain higher prediction accuracies which is measured by the area under the receiver operating characteristic curve (AUC) in comparison with nine individual LILPAs.

Border Gateway Protocol (BGP) is the core component of the Internet's routing infrastructure. Abnormal routing behavior impairs global Internet connectivity and stability. Hence, designing and implementing anomaly detection algorithms is important for improving performance of routing protocols. While various machine learning techniques may be employed to detect BGP anomalies, their performance strongly depends on the employed learning algorithms. These techniques have multiple variants that often work well for detecting a particular anomaly. In this paper, we use the decision tree and fuzzy rough set methods for feature selection. Decision tree and extreme learning machine classification techniques are then used to maximize the accuracy of detecting BGP anomalies. The proposed techniques are tested using Internet traffic traces.

Online sequential extreme learning machine (OS-ELM) proposed by Liang et al. employ sequential learning strategy to learn the target concept from the data. Compared with the original ELM, OS-ELM can learn data one-by-one or chunk-by-chunk with fixed or varying chunk size with almost same performance as ELM. While compared with other state-of-the-art sequential algorithms such as SGBP, RAN and GAP-RBF, OS-ELM has faster learning speed and better generalization ability. However, similar to ELM, OS-ELM also has instability in different trials of simulations. In addition, for large data sets, OS-ELM will not halt when there are training samples not be learned, this phenomenon results in long learning time. In order to deal with the problems, this paper proposes an algorithm named E-OS-ELM for integrating OS-ELM to classify large data sets. The experimental results show that the proposed method is effective and efficient; it can effectively overcome the drawbacks mentioned above.

An important way to improve the performance of naive Bayesian classifiers (NBCs) is to remove or relax the fundamental assumption of independence among the attributes, which usually results in an estimation of joint probability density function (p.d.f.) instead of the estimation of marginal p.d.f. in the NBC design. This paper proposes a non-naive Bayesian classifier (NNBC) in which the independence assumption is removed and the marginal p.d.f. estimation is replaced by the joint p.d.f. estimation. A new technique of estimating the class-conditional p.d.f. based on the optimal bandwidth selection, which is the crucial part of the joint p.d.f. estimation, is applied in our NNBC. Three well-known indexes for measuring the performance of Bayesian classifiers, which are classification accuracy, area under receiver operating characteristic curve, and probability mean square error, are adopted to conduct a comparison among the four Bayesian models, i.e., normal naive Bayesian, flexible naive Bayesian (FNB), the homologous model of FNB (FNBROT), and our proposed NNBC. The comparative results show that NNBC is statistically superior to the other three models regarding the three indexes. And, in the comparison with support vector machine and four boosting-based classification methods, NNBC achieves a relatively favorable classification accuracy while significantly reducing the training time.

Fusing a number of classifiers can generally improve the performance of individual classifiers, and the fuzzy integral, which can clearly express the interaction among the individual classifiers, has been acknowledged as an effective tool of fusion. In order to make the best use of the individual classifiers and their combinations, we propose in this paper a new scheme of classifier fusion based on upper integrals, which differs from all the existing models. Instead of being a fusion operator, the upper integral is used to reasonably arrange the finite resources, and thus to maximize the classification efficiency. By solving an optimization problem of upper integrals, we obtain a scheme for assigning proportions of examples to different individual classifiers and their combinations. According to these proportions, new examples could be classified by different individual classifiers and their combinations, and the combination of classifiers that specific examples should be submitted to depends on their performance. The definition of upper integral guarantees such a conclusion that the classification efficiency of the fused classifier is not less than that of any individual classifier theoretically. Furthermore, numerical simulations demonstrate that most existing fusion methodologies, such as bagging and boosting, can be improved by our upper integral model.

A new ontology learning model called domain ontology graph (DOG) is proposed in this paper. There are two key components in the DOG, i.e., the definition of the ontology graph and the ontology learning process. The former defines the ontology and knowledge conceptualization model from the domain-specific text documents; the latter offers the necessary method of semiautomatic domain ontology learning and generates the corresponding ontology graphs. Two kinds of ontological operations are also defined based on the proposed DOG, i.e., document ontology graph generation and ontology-graph-based text classification. The simulation studies focused upon Chinese text data are used to demonstrate the potential effectiveness of our proposed strategy. This is accomplished by generating DOGs to represent the domain knowledge and conducting the text classifications based on the generated ontology graph. The experimental results show that the new method can produce significantly better classification accuracy (e.g., with 92.3% in f-measure) compared with other methods (such as 86.8% in f-measure for the term-frequency-inverse-document-frequency approach). The high performance demonstrates that our presented ontological operations based on the ontology graph knowledge model are effectively developed.

In flexible Naive Bayesian (FNB), the excellent qualities of Gaussian kernel have been demonstrated by the theoretical analyses and experimental comparisons with normal Naive Bayesian(NNB). There are also several types of kernel functions commonly used for probability density estimation, i.e., uniform, triangular, epanechnikov, biweight, triweight and cosine. We call them discontinuous kernels. In this paper, we verify the feasibility and efficiency of applying these alternative kernels in FNB. Our works mainly focus on three aspects: firstly, we give the application conditions of these kernels for the given domain data by analyzing the structural difference between the discontinuous kernel and Gaussian kernel; secondly, the equivalent probability is proposed to improve the capabilities of discontinuous kernels when such problem of kernel incapability occurs; finally, we carry out the experimental demonstration of our proposed method based on 15 UCI datasets. The results show that the discontinuous kernels can obtain better classification accuracies with the help of equivalent probabilities.

Instance selection also named sample selection is an important preprocessing step for pattern classification. Almost all of the existing instance selection methods are developed for specific classifiers, such as nearest neighbor (NN) classifier, support vector machine (SVM) classifier. Few of them are designed for single hidden layer feed-forward neural networks (SLFNs) classifier. Based on sample entropy, this paper presents an instance selection method for efficient data classification with extreme learning machine (ELM), which is used to train a SLFN. The proposed method is compared with four state-of-the-art approaches by a series of experiments. The experimental results show that the proposed method can provide similar generalization performance but lower computation time complexity.

The total error rate (TER) has been presented as a minimum classification error model for the single-layer feed-forward network (SLFN) learning. The TER, which uses one-against-all (OAA) for multi-class classification, may cause unbalanced data set especially for large number of training data in multi-class classification and then often has a bad influence on the accuracy. This paper proposes a new method, called multi-class total error rate (MTER) to deal with this problem. The MTER, which uses a unified learning mode of regression and multi-class classification and minimizes the error rate for each class, can approximate any target functions. It implies that a balanced data set can be obtained and the training process can be simplified. Experiments show that MTER has a higher accuracy and lower computational complexity in comparison with some learning algorithms such as ELM and TER. The experiments also show that the MTER has a similar performance with LIBSVM.

Sample selection is to select a number of representative samples from a large database such that a learning algorithm can have a reduced computational cost and an improved learning accuracy. This paper gives a new sample selection mechanism, i.e., the maximum ambiguity-based sample selection in fuzzy decision tree induction. Compared with the existing sample selection methods, this mechanism selects the samples based on the principle of maximal classification ambiguity. The major advantage of this mechanism is that the adjustment of the fuzzy decision tree is minimized when adding selected samples to the training set. This advantage is confirmed via the theoretical analysis of the leaf-nodes' frequency in the decision trees. The decision tree generated from the selected samples usually has a better performance than that from the original database. Furthermore, experimental results show that generalization ability of the tree based on our selection mechanism is far more superior to that based on random selection mechanism.

One-Class Support Vector Machine (OCSVM) is a general approach for novelty detection in the fields of machine learning and pattern classification. At the same time, AdaBoost is a famous ensemble method which can improve the performance of its base classifiers. However, the base classifiers in the AdaBoost method prefer to be weak classifiers. Since OCSVM is regarded as a strong classifier, the traditional AdaBoost method may not improve the classification performance of OCSVM. Therefore, to construct the AdaBoost method for OCSVM, we modify the traditional AdaBoost method to make it fit for OCSVM. Experimental results on three synthetic data sets and eight UCI benchmark data sets demonstrate that the proposed method is superior to its related methods.

Feature-weight learning (FWL) methods can be used to determine the importance degrees of each feature for constructing clusters or classifiers. In this paper, four FWL methods for unsupervised learning and two for supervised learning are surveyed. The FWL based models, i.e. feature-weighted fuzzy c-means (FWFCM) and feature-weighted support vector machine (FWSVM) are also reviewed. Through carefully selected experiments we find that FWFCM and FWSVM may improve the performances of their corresponding traditional fuzzy c-mean (FCM) and support vector machine (SVM), respectively. Moreover, the computational cost of FWL_Hung is least for unsupervised learning even though it may produce unsuitable feature weights in some extreme cases, while FWL_MI is most effective for supervised learning.

As a relatively new computing tool to deal with vagueness and uncertainty, rough set has drawn much attention of many researchers. By now many kinds of extended models of rough set have been proposed and investigated. However, there is no comprehensive survey on extended models of rough set in the literature, this paper attempts to present such a survey according to three expansion ways, i.e. direct extension of the definition of rough set, extension in the certainty environment, and extension in the uncertainty environment. Some relationships between the extended models and the classical model are also given. This article shows the improvement and the development process of rough set theory.

There is a phenomenon that binary decision trees generated for continuous attributes have lower prediction accuracy on near boundary examples than total testing dataset. In this paper, we propose a new approach by fuzzifying crisp rules into fuzzy IF-THEN rules and using fuzzy matching operator (∧, +) to overcome this problem. Experimental results show that this method can obtain good performance.

The fuzzy-rough set (FRS) methodology, as a useful tool to handle discernibility and fuzziness, has been widely studied. Some researchers studied on the rough approximation of fuzzy sets, while some others focused on studying one application of FRS: attribute reduction (i.e., feature selection). However, constructing classifier by using FRS, as another application of FRS, has been less studied. In this paper, we build a rule-based classifier by using one generalized FRS model after proposing a new concept named as ¿consistence degree¿ which is used as the critical value to keep the discernibility information invariant in the processing of rule induction. First, we generalized the existing FRS to a robust model with respect to misclassification and perturbation by incorporating one controlled threshold into knowledge representation of FRS. Second, we propose a concept named as ¿consistence degree¿ and by the strict mathematical reasoning, we show that this concept is reasonable as a critical value to reduce redundant attribute values in database. By employing this concept, we then design a discernibility vector to develop the algorithms of rule induction. The induced rule set can function as a classifier. Finally, the experimental results show that the proposed rule-based classifier is feasible and effective on noisy data.

This paper improves a method of sample selection based on maximum entropy. Compared with the original method, the improved one takes the probability distribution of unlabeled instances into consideration. It selects the instances which can reduce the uncertainty of the whole unlabeled set to a great extent. The uncertainty reduction of the whole unlabeled set caused by an instance is measured by the instance's uncertainty and its influence index on the whole unlabeled set. To calculate the influence index conveniently, we introduces the similar matrix, the elements of which are the similarities measured by the distances between instances. The new method avoids the drawbacks that some abnormal or isolated samples may be selected by original method. Thus it can select the instances with more representation and more capability to resist noises. Our experimental results show that the performance of the classifier built from samples selected by the new algorithm is better than those selected by original method in the same time complexity.

Motivated by overcoming the drawbacks of traditional decision tree and improving the efficiency of large margin learning based multi-stage decision tree when dealing with multi-class classification problems, this paper proposes a novel Multi-stage Decision Tree algorithm based on inter-class and inner class margin of SVM. This new algorithm is well designed for multi-class classification problem based on the maximum margin of SVM and the cohesion and coupling theory of clustering. Considering the multi-class classification problem as a clustering problem, this new algorithm attempts to convert the multi-class classification problem into a two-class classification problem such that the highest cohesion degree within classes while lowest coupling degree between classes, where the margin of SVM is considered as the measurement of the degree. Then for each two-class problem, this paper uses traditional C4.5 algorithm to generate each stage decision tree which splits a dataset into two subsets for the further induction. Recursively, the Multi-stage decision tree is obtained. Numerical simulations and theoretical analysis show this new multi-stage decision tree improves the performance of traditional decision tree and decreases the computational complexity a lot compare with large margin learning based multi-stage decision tree.

When fuzzy IF-THEN rules initially extracted from data have not a satisfying performance, we consider that the rules require refinement. Distinct from most existing rule-refinement approaches that are based on the further reduction of training error, this paper proposes a new rule-refinement scheme that is based on the maximization of fuzzy entropy on the training set. The new scheme, which is realized by solving a quadratic programming problem, is expected to have the advantages of improving the generalization capability of initial fuzzy IF-THEN rules and simultaneously overcoming the overfitting of refinement. Experimental results on a number of selected databases demonstrate the expected improvement of generalization capability and the prevention of overfitting by a comparison of both training and testing accuracy before and after the refinement.

Fuzzy Production Rules (FPRs) are widely used in expert systems to represent uncertainty concepts. In order to enhance the representation capability and to improve the reasoning-accuracy of FPRs, some useful knowledge representation parameters such as certainty factor, local weight and global weight have been included in FPRs. However, the acquisition of the values of these parameters is difficult and time-consuming. Usually the principle to determine these parameters is to further reduce the training error. This paper proposes a new principle, i.e., the maximum entropy principle, for solving these parameters. Firstly we present a parametric tuning method based on the maximization of fuzzy entropy on the training set, then a genetic algorithm-based optimization technique is applied to determine the values of the weights in FPRs. Experimental results demonstrate a number of advantages of our method such as automatic acquisition of the weights, avoiding the over-fitting to a great extent and non-changing the number of the initial FPRs.

Kernel function plays a very important role in the performance of SVM. In order to improve generalization capability of SVM classifier, this paper proposes a new mechanism to optimize the parameters of combined kernel function by using large margin learning theory and a genetic algorithm, which aims to search the optimal parameters for the combined kernel function. This approach leads SVM to attain the maximum margin in the training dataset. The combined kernel function and the parameters obtained by the proposed approach leads to a better performance and results in a better SVM classifier. Both numerical simulation results and theoretical analysis show the effectiveness and feasibility of the proposed approach.

In this paper, we focus on kernel learning vector quantization (KLVQ) for handling novelty detection. The two key issues are addressed: the existing KLVQ methods are reviewed and revisited, while the reformulated KLVQ is applied to tackle novelty detection problems. Although the calculation of kernelising the learning vector quantization (LVQ) may add an extra computational cost, the proposed method exhibits better performance over the LVQ. The numerical study on one synthetic data set confirms the benefit in using the proposed KLVQ.

This paper proposes a fast fuzzy classifier of multicategory support vector machines (FMSVM) based on support vector domain description (SVDD). The main idea is that the proposed FMSVM is obtained by directly considering all data in one optimization formulation, using a fuzzy membership to each input point. The fuzzy membership is determined by support vector domain description (SVDD). For making support vector machine (SVM) more practical, we use an implement of the modified sequential minimal optimization (SMO) that can quickly solve SVM quadratic programming (QP) problems without any extra matrix storage or the use of numerical QP optimization steps at all. Compared with the existing SVMs, the newly proposed FMSVM that uses the L2-norm in the objective function shows improvement with regards to accuracy of classification and reduction of the effects of noises and outliers. The experiment also shows the efficiency of the modified SMO for expediting the training of SVM.

Fuzzy rough sets are the generalization of traditional rough sets to deal with both fuzziness and vagueness in data. The existing researches on fuzzy rough sets are mainly concentrated on the construction of approximation operators. Less effort has been put on the attributes reduction of databases with fuzzy rough sets. This paper mainly focuses on the attributes reduction with fuzzy rough sets. After analyzing the previous works on attributes reduction with fuzzy rough sets, we introduce formal concepts of attributes reduction with fuzzy rough sets and completely study the structure of attributes reduction. An algorithm using discernibility matrix to compute all the attributes reductions is developed. Based on these lines of thought, we set up a solid mathematical foundation for attributes reduction with fuzzy rough sets. The experimental results show that the idea in this paper is feasible and valid.

Owing to the growth of Internet and computer technology, pattern recognition for large-scale datasets has become one of the hot research topics. The major challenges are to reduce the human efforts involved and to improve the efficiency. Traditional passive learning methods require labeling of all training samples may not be feasible in large-scale recognition problems because of the requirement of large-scale class labeling for the huge number of training samples. In the literatures, there are many studies on active learning methods, which does not require all training samples to be labeled and it selects training samples for labeling based on the knowledge of the current classifier. In this paper, we present an active learning method using localized generalization error of candidate sample as selection criterion. Our method uses the generalization error of candidate sample, so theoretically it should have a better performance than other methods. From the experiment results, our method outperforms other methods in both yielding higher prediction accuracy on testing dataset and selecting fewer training samples. Furthermore, we propose a heuristics improvement based on the Q-neighborhood idea of the localized generalization error model to reduce the number of samples being selected and the computational time.

Support vector machine (SVM) theory was originally developed on the basis of a linearly separable binary classification problem. The inverse problem of SVM is how to split a given dataset into two clusters such that the margin between the two clusters attains maximum. Due to the computational complexity, it is difficult to give an exact and feasible solution to the inverse problem. This paper makes an attempt to reduce the complexity of the inverse problem by clustering. It is demonstrated that the maximum margin between the two clusters is equivalent to the distance between the two closest points in convex hulls in the linearly separable case. For the inseparable case, the maximum margin between the two sets is equivalent to the distance between the two closest points in the reduced convex hulls.

The generalization capability of a classifier will probably be degenerated when the classifier is generated from a dataset containing redundancy. To remove the redundancy, sample selection methods which choose the most valuable and representative instances from the original date set, can be used to obtain a subset of the original dataset. It is expected that the classifier trained from the subset can achieve no lower generalization capability than the classifier trained from the original data set. This paper proposes a sample selection method based on maximum entropy of testing instances in the fuzzy decision tree induction, and also gives the related theoretical analyses.

Intrusion detection is an important part of assuring the reliability of computer systems. Different intrusion detection approaches vary with different patterns used and different intrusions addressed. However, what patterns are effective in constructing a detection system is still a challenge. This paper attempts to apply the traditional covariance matrix concept to the detection of multiple known and unknown network anomalies. With respect to the initiation of typical flood-based network intrusions, the proposed approach takes the measure of covariance matrix to reflect the changes of sequential correlativity of the network traffic when flood-based attacks happen. The differences among covariance matrices of network samples collected in temporal sequences of fixed and equal length are directly evaluated to detect multiple network anomalies. Extensive experiments on the subset of KDDCUP 1999 dataset show that the covariance matrix, as a new pattern, can be directly utilized to construct an effective detection system for flood-based attacks. It also points out that utilizing the covariance matrix in the detection of flood-based attacks can achieve higher performance over traditional approaches.

The generalization error bounds found by current error models using the number of effective parameters of a classifier and the number of training samples are usually very loose. These bounds are intended for the entire input space. However, support vector machine (SVM), radial basis function neural network (RBFNN), and multilayer perceptron neural network (MLPNN) are local learning machines for solving problems and treat unseen samples near the training samples to be more important. In this paper, we propose a localized generalization error model which bounds from above the generalization error within a neighborhood of the training samples using stochastic sensitivity measure. It is then used to develop an architecture selection technique for a classifier with maximal coverage of unseen samples by specifying a generalization error threshold. Experiments using 17 University of California at Irvine (UCI) data sets show that, in comparison with cross validation (CV), sequential learning, and two other ad hoc methods, our technique consistently yields the best testing classification accuracy with fewer hidden neurons and less training time.

This paper presents a covariance-matrix modeling and detection approach to detecting various flooding attacks. Based on the investigation of correlativity changes of monitored network features during flooding attacks, this paper employs statistical covariance matrices to build a norm profile of normal activities in information systems and directly utilizes the changes of covariance matrices to detect various flooding attacks. The classification boundary is constrained by a threshold matrix, where each element evaluates the degree to which an observed covariance matrix is different from the norm profile in terms of the changes of correlation between the monitored network features represented by this element. Based on Chebyshev inequality theory, we give a practical (heuristic) approach to determining the threshold matrix. Furthermore, the result matrix obtained in the detection serves as the second-order features to characterize the detected flooding attack. The performance of the approach is examined by detecting Neptune and Smurf attacks-two common distributed Denial-of-Service flooding attacks. The evaluation results show that the detection approach can accurately differentiate the flooding attacks from the normal traffic. Moreover, we demonstrate that the system extracts a stable set of the second-order features for these two flooding attacks

Motivated by improving the existing decision tree performance of dealing with multi-class problems, this paper proposes a new algorithm named multi-stage decision tree (MDT). The MDT algorithm is based on the relationship between the margin of SVM hyper-planes and their generalization capability and tries to find the large margin among the clusters. First the MDT algorithm converts the multi-class problem into two-class problem by large margin learning of SVM hyper-planes, and then for each two-class problem, it uses traditional decision tree induction algorithm to generate a decision tree which splits a dataset into two subsets for the further induction. Recursively, the multi-stage decision tree is obtained finally. Numerical simulations show the effectiveness and high accuracy of MDT algorithm. Initial experiments show that the MDT algorithm has the potential application to text classification with many classes.

This paper investigates the impact of merging branches on decision tree induction. The main concerns are whether the comprehensibility, the size and the generalization accuracy of a decision tree can be improved if an appropriate merging strategy is selected and applied. Based on information gain principle, this paper theoretically analyzes the complexity of a decision tree before and after merging branches, and designs an algorithm of merging branches MID, which is based on the support vector machine margin enlargement. Experimental results show that the MID has the comprehensibility and the generalization accuracy significantly better than the traditional decision tree algorithm without branch merging.

Intrusion detection is an important part of assuring the reliability of computer systems. From the viewpoint of feature space partition of detectors, this paper investigates one of the limitations of two traditional anomaly detection technologies - NN-based anomaly detection and statistical detection approaches in detecting novel attacks. A high dimensional covariance matrix feature space and an on-line detection algorithm are proposed to detect various known and unknown attacks. An illustrative example of detecting various known and unknown probing attacks is provided.

This paper presents an interactive weighted fuzzy reasoning algorithm for rule based systems based on interactive weighted fuzzy Petri nets (IWFPN). The fuzzy production rules in the knowledge base of a rule based system are modelled by IWFPN, where the weights appearing in the rules are represented by fuzzy numbers. In order to model and handle the interaction existing among attributes in a more flexible way, this paper also proposes a fuzzy number valued fuzzy integral (called extended Choquet integral) to draw the reasoning conclusion, where the fuzzy measure refers to an extended g-lambda measure, or a non-additive nonnegative fuzzy number valued set function specified by a domain expert. The proposed algorithm can allow the rule based systems to perform fuzzy reasoning in a more intelligent manner.

Case-based maintenance is an important issue in Case-Based Reasoning (CBR) System. Generally speaking, the larger the case-base, the more accurate the solution. However, if the case base is too large, it may include many redundant cases and the case retrieve will not be effective. Moreover redundant cases will affect the solution accuracy. Therefore, removing redundant cases is a fundamental issue in maintaining CBR systems. In this paper, a new approach based on the Generalization Capability of cases to select the representative cases for Case-Based Maintenance is proposed. Using this method, most redundant cases can be deleted and the most representative cases can be identified and retained. The experiments show that the proposed method can greatly remove the redundant cases as well as preserve a satisfying degree of accuracy of solutions when it is used for classification tasks.

Rough sets and fuzzy sets have been proved to be powerful mathematical tools to deal with uncertainty, it soon raises a natural question of whether it is possible to connect rough sets and fuzzy sets. The existing generalizations of fuzzy rough sets are all based on special fuzzy relations (fuzzy similarity relations, T-similarity relations), it is advantageous to generalize the fuzzy rough sets by means of arbitrary fuzzy relations and present a general framework for the study of fuzzy rough sets by using both constructive and axiomatic approaches. In this paper, from the viewpoint of constructive approach, we first propose some definitions of upper and lower approximation operators of fuzzy sets by means of arbitrary fuzzy relations and study the relations among them, the connections between special fuzzy relations and upper and lower approximation operators of fuzzy sets are also examined. In axiomatic approach, we characterize different classes of generalized upper and lower approximation operators of fuzzy sets by different sets of axioms. The lattice and topological structures of fuzzy rough sets are also proposed. In order to demonstrate that our proposed generalization of fuzzy rough sets have wider range of applications than the existing fuzzy rough sets, a special lower approximation operator is applied to a fuzzy reasoning system, which coincides with the Mamdani algorithm.

When fuzzy production rules are used to approximate reasoning, interaction exists among rules that have the same consequent. Due to this interaction, the weighted average model frequently used in approximate reasoning does not work well in many real-world problems. In order to model and handle this interaction, this paper proposes to use a nonadditive nonnegative set function to replace the weights assigned to rules having the same consequent, and to draw the reasoning conclusion based on an integral with respect to the nonadditive nonnegative set function, rather than on the weighted average model. Handling interaction in fuzzy production rule reasoning in this way can lead to a good understanding of the rules base and an improvement of reasoning accuracy. This paper also investigates how to determine from data the nonadditive set function that cannot be specified by a domain expert.

While ordinal classification problems are common in many situations, induction of ordinal decision trees has not been very extensiveness studied. They are commonly treated as nominal classification problem or regression problem in tree induction. On the other hand a monotonic decision tree is often desirable to aid decision making in such situations as credit rating and student admission. This paper proposes a novel approach called MDT to monotonic decision tree induction. Experiments show that generally this new approach produces decision trees that are more succinct and more effective predictors of the original implicit ordering, apart from being monotonic.

Case-base maintenance in Case-Based Reasoning (CBR) Systems is an important issue. If the case base is too large, it may include many redundant cases which will affect the solution accuracy. Therefore, removing redundant cases is a n important issue in CBR systems. In this paper, a new approach based on the Generalization Capability of cases to select the representative cases for Case-Base Maintenance is proposed. With this method, most redundant cases can be deleted and the most representative cases can be retained. The experiments show that the proposed method can greatly remove the redundant cases while preserving a satisfying degree of accuracy of solutions when applied to solve classification problems.

This paper presents a new rough set model called mutual-relation-based rough set model which is a generalization of Pawlak's model. Based on the new model, fuzzy core and fuzzy reduction are also proposed. They are the fuzzy generalization of core and reduction of the new model. Two new heuristic algorithms to get reduction of the new model are constructed and illustrated with a simple example.

Feature subset selection is a well-known pattern recognition problem, which aims to reduce the number of features used in classification or recognition. This reduction is expected to improve the performance of classification algorithms in terms of speed, accuracy and simplicity. Most existing feature selection investigations focus on the case that the feature values are real or nominal, very little research is found to address the fuzzy-valued feature subset selection and its computational complexity. This paper focuses on a problem called optimal fuzzy-valued feature subset selection (OFFSS), in which the quality-measure of a subset of features is defined by both the overall overlapping degree between two classes of examples and the size of feature subset. The main contributions of this paper are that: 1) the concept of fuzzy extension matrix is introduced; 2) the computational complexity of OFFSS is proved to be NP-hard; 3) a simple but powerful heuristic algorithm for OFFSS is given; and 4) the feasibility and simplicity of the proposed algorithm are demonstrated by applications of OFFSS to fuzzy decision tree induction and by comparisons with three different feature selection techniques developed recently.

Similarity-based clustering is a simple but powerful technique which usually results in a clustering graph for a partitioning of threshold values in the unit interval. The guiding principle of similarity-based clustering is "similar objects are grouped in the same cluster." To judge whether two objects are similar, a similarity measure must be given in advance. The similarity measure presented in the paper is determined in terms of the weighted distance between the features of the objects. Thus, the clustering graph and its performance (which is described by several evaluation indices defined in the paper) will depend on the feature weights. The paper shows that, by using gradient descent technique to learn the feature weights, the clustering performance can be significantly improved. It is also shown that our method helps to reduce the uncertainty (fuzziness and nonspecificity) of the similarity matrix. This enhances the quality of the similarity-based decision making.

A novel post-processing system for a handwritten Chinese character recognition system based on a neural network classifier is presented. The recognition results for input character images, namely candidate characters and their confidence scores, as the observed features of the recognizer are classified into the most probable characters. The confusing character set is established by analyzing large-scale recognition experimental results, and the statistical characteristics for a recognizer are expressed as confusing character sets. 3755 character categories in the GB2312-80 character-set are clustered into several hundreds of groups through searching the transitive closure of the similarity matrix associated with the confusing characters of each character category. A group of neural networks for these category groups is established and trained to be a classifier in the post-processing to recover the unrecognized characters and adjust confidence scores of the candidate characters when a candidate sequence for each individual character image is given. The experimental results show that an average accuracy rate improvement of 5.6% and 3.8% for an online and an offline handwritten Chinese character recognition system are achieved respectively.

In this article we propose a case‐base maintenance methodology based on the idea of transferring knowledge between knowledge containers in a case‐based reasoning (CBR) system. A machine‐learning technique, fuzzy decision‐tree induction, is used to transform the case knowledge to adaptation knowledge. By learning the more sophisticated fuzzy adaptation knowledge, many of the redundant cases can be removed. This approach is particularly useful when the case base consists of a large number of redundant cases and the retrieval efficiency becomes a real concern of the user. The method of maintaining a case base from scratch, as proposed in this article, consists of four steps. First, an approach to learning feature weights automatically is used to evaluate the importance of different features in a given case base. Second, clustering of cases is carried out to identify different concepts in the case base using the acquired feature‐weights knowledge. Third, adaptation rules are mined for each concept using fuzzy decision trees. Fourth, a selection strategy based on the concepts of case coverage and reachability is used to select representative cases. In order to demonstrate the effectiveness of this approach as well as to examine the relationship between compactness and performance of a CBR system, experimental testing is carried out using the Traveling and the Rice Taste data sets. The results show that the testing case bases can be reduced by 36 and 39 percent, respectively, if we complement the remaining cases by the adaptation rules discovered using our approach. The overall accuracies of the two smaller case bases are 94 and 90 percent, respectively, of the originals.

In this paper, the Hopfield neural network with delay (HNND) is studied from the standpoint of regarding it as an optimizing computational model. Two general updating rules for networks with delay (GURD) are given based on Hopfield-type neural networks with delay for optimization problems and characterized by dynamic thresholds. It is proved that in any sequence of updating rule modes, the GURD monotonously converges to a stable state of the network. The diagonal elements of the connection matrix are shown to have an important influence on the convergence process, and they represent the relationship of the local maximum value of the energy function to the stable states of the networks. All the ordinary discrete Hopfield neural network (DHNN) algorithms are instances of the GURD. It can be shown that the convergence conditions of the GURD may be relaxed in the context of applications, for instance, the condition of nonnegative diagonal elements of the connection matrix can be removed from the original convergence theorem. A new updating rule mode and restrictive conditions can guarantee the network to achieve a local maximum of the energy function with a step-by-step algorithm. The convergence rate improves evidently when compared with other methods. For a delay item considered as a noise disturbance item, the step-by-step algorithm demonstrates its efficiency and a high convergence rate. Experimental results support our proposed algorithm.

Fuzzy decision tree induction is an important way of learning from examples with fuzzy representation. Since the construction of optimal fuzzy decision tree is NP-hard, the research on heuristic algorithms is necessary. In this paper, three heuristic algorithms for generating fuzzy decision trees are analyzed and compared. One of them is proposed by the authors. The comparisons are twofold. One is the analytic comparison based on expanded attribute selection and reasoning mechanism; the other is the experimental comparison based on the size of generated trees and learning accuracy. The purpose of this study is to explore comparative strengths and weaknesses of the three heuristics and to show some useful guidelines on how to choose an appropriate heuristic for a particular problem.

The guiding principle of case-based reasoning (CBR) is the CBR-hypothesis which assumes that "similar problems have similar solutions". This principle requires a model to compute the problem-similarity in terms of individual features. One frequently used model is to consider the weighted average of feature-similarities as an overall similarity measure. Due to some inherent interaction among diverse features, the weighted average model does not work well in many real-world problems. This paper proposes using a non-linear integral tool to address such a problem. Five fuzzy integrals with respect to a fuzzy measure or a nonadditive set function are discussed in this paper. The interaction among the features is considered to be reflected in the non-additive set function, and the overall similarity is computed by using the integral model instead of using the weighted average model. Because the weighted average can be regarded as a special case of nonlinear integral, this paper to some extent generalizes the application scope of traditional CBR techniques based on similarity.

Although there have been many approaches to fuzzy clustering, the clustering based on a similarity matrix is still a popular technique which performs by means of transforming the similarity matrix into its transitive closure. The clustering performance depends strongly on the similarity matrix in which elements are determined according to a distance metric in many situations. For a given case library in which diverse similarity measures can be defined, different similarity matrixes result in different clustering results. This paper introduces the concept of feature weight and then incorporates this concept into the process of computing similarity between two cases, such that the similarity matrix relies on these feature weights. The purpose of this paper is to improve the clustering performance by adjusting these weights in terms of a neural-fuzzy technique. To learn the feature weights, a neural network is designed for minimizing an objective function. For achieving a local minimum of the objective function, the gradient-descent technique is used to train this network. Several indexes for measuring the quality of a clustering result are defined in this paper to compare the performance.

Although the induction of fuzzy decision tree (FDT) has been a very popular learning methodology due to its advantage of comprehensibility, it is often criticized to result in poor learning accuracy. Thus, one fundamental problem is how to improve the learning accuracy while the comprehensibility is kept. This paper focuses on this problem and proposes using a hybrid neural network (HNN) to refine the FDT. This HNN, designed according to the generated FDT and trained by an algorithm derived in this paper, results in a FDT with parameters, called weighted FDT. The weighted FDT is equivalent to a set of fuzzy production rules with local weights (LW) and global weights (GW) introduced in our previous work (1998). Moreover, the weighted FDT, in which the reasoning mechanism incorporates the trained LW and GW, significantly improves the FDTs' learning accuracy while keeping the FDT comprehensibility. The improvements are verified on several selected databases. Furthermore, a brief comparison of our method with two benchmark learning algorithms, namely, fuzzy ID3 and traditional backpropagation, is made. The synergy between FDT induction and HNN training offers new insight into the construction of hybrid intelligent systems with higher learning accuracy.