Reliable data transmission in noisy environments poses a significant challenge, where retransmissions are costly in terms of bandwidth, time, and energy. Traditional error detection methods, such as Cyclic Redundancy Check (CRC) and Low-Density Parity Check (LDPC) codes, focus on bit-level accuracy and retransmit all corrupted packets regardless of task relevance. In this paper, we focus on semantic communication, where not all bits contribute equally to the end task. To make this concrete, we study image transmission, where certain regions carry higher semantic importance than others. We leverage DI-NOv2 attention maps and Vision Transformers to identify high-importance regions and employ reinforcement learning to optimize retransmission policies accordingly. This enables selective retransmissions guided by semantic value rather than uniform error detection. Experiments on CIFAR-100 with simulated span corruption show that our framework reduces retransmissions from over 90% with CRC to below 25%, while preserving high classification accuracy above 89%. These findings highlight the potential of Bit Error Tolerance communication systems for efficient and task-aware transmission in noisy environments.

In this paper, we apply the method of control theory to machine learning, proposing a new multiplication update algorithm combined with adaptive control theory, we name it MRAC-MU algorithm. A new parameter updating law is obtained according to Lyapunov stability theorem. Using the same object function as the exponential gradient (EG) algorithm, which is the key online learning method to multiplicative updates algorithm, Experiments are used to validate the proposed algorithm has a better result than EG algorithm in prediction accuracy.

This paper proposes a new online learning algorithm which is based on adaptive control (AC) theory, thus, we call this proposed algorithm as AC algorithm. Comparing to the gradient descent (GD) and exponential gradient (EG) algorithm which have been applied to online prediction problems, we find a new form of AC theory for online prediction problems and investigate two key questions: how to get a new update law which has a tighter upper bound on the error than the square loss? How to compare the upper bound for accumulated losses for the three algorithms? We obtain a new update law which fully utilizes model reference AC theory. Moreover, we present upper bound on the worst-case expected loss for AC algorithm and compare it with previously known bounds for the GD and EG algorithm. The loss bound we get in this paper is a time-varying function, which provides increasingly accurate estimates for upper bound. The AC algorithm has a much smaller loss only if the number of the samples meets certain conditions which can be seen in this paper. We also performed experiments which show that our update law is reasonably feasible and our upper bound is quite tight on both simple artificial and real data sets. The main contributions of this paper are twofold. First of all, we develop a new online algorithm called AC algorithm, and second, we obtain improved bounds, see Theorems 2-4 in this paper.

In this paper, an automatic generation of domain-specific stopwords from a large labeled corpus is proposed. In the majority of text mining tasks, stopwords are removed according to a standard stopword list and/or using high and low document frequencies. In this paper, a new approach for stopword extraction, based on the notion of backward filter-level performance and data sparsity index, is proposed. First, based on the proposed model to evaluate the extracted stopwords, we examine high document frequency filtering for stopword reduction. Secondly, a new algorithm for building general and domain-specific stopword lists is proposed. For the method, it is assumed that a set of candidate stopwords must have a minimum information content and prediction capacity that is measured by the performance of a classifier. We show that to avoid obtaining the classifier performance, it can be estimated by the sparsity of the training dataset. Moreover, it is confirmed that even if a given term ranking measure can perform well for the feature selection, the measure is not necessarily efficient for selecting poor features (stopwords). According to the comparative study, the newly devised approach offers more promising results that guarantee a minimum information loss by filtering out most stopwords.

Situation prediction is a crucial part of active Advanced Driver Assistance Systems (ADAS) to prevent rear, lateral and other collisions. Majority of road crashes can be prevented if the ADAS issue a warning about a potential threat at least one-half second prior to the prominent accident. To take the suitable maneuver, active safety systems should assess succinctly the danger caused by other errant drivers and analyse surrounding drivers intent. This study presents a game theory impact assessment and decision making model that allows drivers to assess threat level caused during different road scenarios. Our model is based on Fuzzy Multi-Entity Bayesian Network (Fuzzy-MEBN) enriched by a game theory component. Illustrative scenarios are provided to show the merit of our model.

Clustering has raised as an important problem in many different domains like biology, computer vision, text analysis and robotics. Thus, many different clustering techniques were developed to address this essential problem and propose astonishing solutions to conquer it. However, traditional clustering techniques suffer either from their limitations to detect specific shapes like K-means and PAM or from their limitations to detect clusters with specific densities as in DBSCAN and SNN. Moreover, exploiting the data relations and similarities has been proven to provide better insights to enhance the clustering quality as shown in spectral clustering and affinity propagation. Our observations have shown that using variance of similarities between each data point and its neighbors can well distinguish between within-cluster points, points connecting two clusters and outlier points. Therefore, we have utilized this variance measure to calculate each data point density and developed a Local Variance-based Clustering (LVC) technique that employs this measure to cluster the data. Experimental results show that LVC outperforms spectral clustering and affinity propagation in clustering quality using control charts, ecoli and images datasets, while maintaining a good running time. In addition, results show that LVC can detect topics from Twitter with higher topic recall by 15% and higher term precision by 3% over DBSCAN.

Driver inattentiveness constitutes the main cause of road accidents, which makes it a major factor in road safety. In this paper, we propose a comprehensive framework to address the road safety problem by tackling it from a high-level information fusion standpoint, considering vehicular ad hoc networks (VANETs) as the deployment platform. The proposed framework relies on the multientity Bayesian networks (MEBNs), which exploit the expressiveness of first-order logic for semantic relations, and the strength of the Bayesian networks in handling uncertainty. First, the entities that influence the inattention phenomenon, as well as both their causal and semantic relationships, are identified. Next, an MEBN-based high-level information fusion framework is proposed through which entities, situations, and their relationships in specific contexts are modeled using MEBN fragments. Furthermore, MEBN inference is used to assess the situations of interest by estimating their states. To demonstrate the capabilities of the proposed framework, a collision warning system simulator has been developed, which evaluates the likelihood of a vehicle being in a near-collision situation using a wide variety of local and global information sources available in various VANET environments. If the threat of being in a near-collision situation is determined to be high, then the driver is warned accordingly. Our experimental results for two distinct single-vehicle and multivehicle categories of driving scenarios, as well as a novel hybrid MEBN inference, demonstrate the capability of the proposed framework to efficiently achieve situation and threat assessment on the road.

In this paper, a soft+hard data fusion model is proposed that is capable of combining the data generated from human-based sources with those generated by physical sensors. The basis of this model is our previously introduced Fuzzy extension to the Mutli-Entity Bayesian Network (MEBN) language, which is a High-Level Information Fusion (HLIF) framework capable of expressing the semantic and causal relationships between the entities constituting a world model, as well as managing their ambiguity and uncertainty. In our proposed model, the unstructured soft data is presented by undergoing a novel soft-data-association process, through which the data is semantically analyzed, and accurately structured in a fuzzy random variable. Moreover, the clique tree inference algorithm for Bayesian Networks is modified to handle fuzzy evidence in Fuzzy-MEBN. The simulation results, in transportation domain, show that our improved HLIF model is capable of handling both soft and hard data, and consequently, provide the user with more precise situation assessment.

The Kernel Mean Matching (KMM) is an elegant algorithm that produces density ratios between training and test data by minimizing their maximum mean discrepancy in a kernel space. The applicability of KMM to large-scale problems is however hindered by the quadratic complexity of calculating and storing the kernel matrices over training and test data. To address this problem, this paper proposes a novel ensemble algorithm for KMM, which divides test samples into smaller partitions, estimates a density ratio for each partition and then fuses these local estimates with a weighted sum. Our theoretical analysis shows that the ensemble KMM has a lower error bound than the centralized KMM, which uses all the test data at once to estimate the density ratio. Considering its suitability for distributed implementation, the proposed algorithm is also favorable in terms of time and space complexities. Experiments on benchmark datasets confirm the superiority of the proposed algorithm in terms of estimation accuracy and running time.

This paper presents a novel comprehensive Fuzzy extension to Multi-Entity Bayesian Networks (MEBN) that is deemed a well-studied and theoretically rich language that expressively handles semantics analysis, and effectively model uncertainty management. However, MEBN lack the capability of modeling the inherent conceptual and structural ambiguity that is delivered with the knowledge gained through human language. In this paper, Fuzzy MEBN that is a new version of MEBN which is based on First-order Fuzzy Logic, and Fuzzy Bayesian Networks is introduced. Furthermore, its applicability is evaluated by implementing an application related to Vehicular Ad-hoc Networks area. The results demonstrate that Fuzzy MEBN is capable of dealing with ambiguous semantical and uncertain causal relationships between the knowledge entities very efficiently.

In this paper, we adapt two existing methods to perform semi-supervised temporal clustering: Aligned Cluster Analysis (ACA), a temporal clustering algorithm, and Constrained Spectral Clustering, a semi-supervised clustering algorithm. In the first method, we add side information in the form of pair wise constraints to its objective function, and in the second, we add a temporal search to its framework. We also extend both methods by propagating the constraints throughout the whole similarity matrix. In order to validate the advantage of the proposed semi-supervised methods to temporal clustering, we evaluate them in comparison to their original versions as well as another semi-supervised temporal cluster on three temporal datasets. The results show that the proposed methods are competitive and provide good improvement over the unsupervised approaches.

The kernel k-means is an effective method for data clustering which extends the commonly-used k-means algorithm to work on a similarity matrix over complex data structures. It is, however, computationally very complex as it requires the complete kernel matrix to be calculated and stored. Further, its kernelized nature hinders the parallelization of its computations on modern scalable infrastructures for distributed computing. In this paper, we are defining a family of kernelbased low-dimensional embeddings that allows for scaling kernel k-means on MapReduce via an efficient and unified parallelization strategy. Afterwards, we propose two practical methods for low-dimensional embedding that adhere to our definition of the embeddings family. Exploiting the proposed parallelization strategy, we present two scalable MapReduce algorithms for kernel k-means. We demonstrate the effectiveness and efficiency of the proposed algorithms through an empirical evaluation on benchmark datasets.

Covariate shift is a challenging problem in supervised learning that results from the discrepancy between the training and test distributions. An effective approach which recently drew a considerable attention in the research community is to reweight the training samples to minimize that discrepancy. In specific, many methods are based on developing Density-ratio (DR) estimation techniques that apply to both regression and classification problems. Although these methods work well for regression problems, their performance on classification problems is not satisfactory. This is due to a key observation that these methods focus on matching the sample marginal distributions without paying attention to preserving the separation between classes in the reweighted space. In this paper, we propose a novel method for Discriminative Density-ratio (DDR) estimation that addresses the aforementioned problem and aims at estimating the density ratio of joint distributions in a class-wise manner. The proposed algorithm is an iterative procedure that alternates between estimating the class information for the test data and estimating a new density ratio for each class. To incorporate the estimated class information of the test data, a soft matching technique is proposed. In addition, we employ an effective criterion which adopts mutual information as an indicator to stop the iterative procedure while resulting in a decision boundary that lies in a sparse region. Experiments on synthetic and benchmark datasets demonstrate the superiority of the proposed method.

Recently, a supervised dictionary learning (SDL) approach based on the Hilbert-Schmidt independence criterion (HSIC) has been proposed that learns the dictionary and the corresponding sparse coefficients in a space where the dependency between the data and the corresponding labels is maximized. In this paper, two multiview dictionary learning techniques are proposed based on this HSIC-based SDL. While one of these two techniques learns one dictionary and the corresponding coefficients in the space of fused features in all views, the other learns one dictionary in each view and subsequently fuses the sparse coefficients in the spaces of learned dictionaries. The effectiveness of the proposed multiview learning techniques in using the complementary information of single views is demonstrated in the application of speech emotion recognition (SER). The fully-continuous sub-challenge (FCSC) of the AVEC 2012 dataset is used in two different views: baseline and spectral energy distribution (SED) feature sets. Four dimensional affects, i.e., arousal, expectation, power, and valence are predicted using the proposed multiview methods as the continuous response variables. The results are compared with the single views, AVEC 2012 baseline system, and also other supervised and unsupervised multiview learning approaches in the literature. Using correlation coefficient as the performance measure in predicting the continuous dimensional affects, it is shown that the proposed approach achieves the highest performance among the rivals. The relative performance of the two proposed multiview techniques and their relationship are also discussed. Particularly, it is shown that by providing an additional constraint on the dictionary of one of these approaches, it becomes the same as the other.

Quantitative ultrasound (QUS) spectroscopic techniques in conjunction with maximum mean discrepancy (MMD) have been proposed to detect, and to classify noninvasively the levels of cell death in response to cancer therapy administration in tumor models. Evaluation of xenograft tumor responses to cancer treatments were carried out using conventional-frequency ultrasound at different times after chemotherapy exposure. Ultrasound data were analyzed using spectroscopic techniques and multi-parametric QUS spectral maps were generated. MMD was applied as a distance criterion, measuring alterations in each tumor in response to chemotherapy, and the extent of cell death was classified into less/more than 20% and 40% categories. Statistically significant differences were observed between "pre-" and "post-treatment" groups at different times after chemotherapy exposure, suggesting a high capability of proposed framework for detecting tumor response noninvasively. Promising results were also obtained for categorizing the extent of cell death response in each tumor using the proposed framework, with gold standard histological quantification of cell death as ground truth. The best classification results were obtained using MMD when applied on histograms of QUS parametric maps. In this case, classification accuracies of 84.7% and 88.2% were achieved for categorizing extent of tumor cell death into less/more than 20% and 40%, respectively.

Given a very large data set distributed over a cluster of several nodes, this paper addresses the problem of selecting a few data instances that best represent the entire data set. The solution to this problem is of a crucial importance in the big data era as it enables data analysts to understand the insights of the data and explore its hidden structure. The selected instances can also be used for data preprocessing tasks such as learning a low-dimensional embedding of the data points or computing a low-rank approximation of the corresponding matrix. The paper first formulates the problem as the selection of a few representative columns from a matrix whose columns are massively distributed, and it then proposes a MapReduce algorithm for selecting those representatives. The algorithm first learns a concise representation of all columns using random projection, and it then solves a generalized column subset selection problem at each machine in which a subset of columns are selected from the sub-matrix on that machine such that the reconstruction error of the concise representation is minimized. The paper then demonstrates the effectiveness and efficiency of the proposed algorithm through an empirical evaluation on benchmark data sets.

In this paper, we consider an invariant Generalized Hough Transform (GHT) as a shape based extractor to improve the quality of the urban land-cover classification. Dense urban environment sensed by Very High-Resolution (VHR) optical sensors is one of the most challenging problems in pattern analysis and machine intelligence systems in remote sensing. We propose a three stage framework for extracting urban land-cover: a spectral cluster-based segmentation to segment and extract basic urban classes followed by two serialized classifications to extract structures of interest from the segmented data. The first classification uses Particle Swarm Optimization and shows a significant classification performance of 80-90% of roads of VHR remote sensing data over urban areas. Next, the classified data are piped into the third stage in which GHT is used to classify building areas. The suggested framework is successful in enhancing the building areas detection with an accuracy improvement of 30- 40%.

Particle Swarm Optimization (PSO) is a stochastic optimization approach that originated from early attempts to simulate the behavior of birds looking for food. Estimation of distributions algorithms (EDAs) are a class of evolutionary algorithms that build and maintain a probabilistic model capturing the search space characteristics and continuously use this model to generate new individuals. In this work, we propose a new PSO and EDA hybrid algorithm that uses the particles' distribution in the search space in order to adjust the search space bounds, hence, restricting the particles movement as well as their allowable maximum velocity. The algorithms is augmented with a mechanism to overcome premature convergence and escape local minima. The algorithm is compared to the standard PSO algorithm using a suite of well-known benchmark optimization functions. Experimental results show that the proposed algorithm has a promising performance.

Even though facial expressions have universal meaning in communications, their appearances show a large amount of variation due to many factors, such as different image acquisition setups, different ages, genders, and cultural backgrounds etc. Collecting enough amounts of annotated samples for each target domain is impractical, this paper investigates the problem of facial expression recognition in the more challenging situation, where the training and testing samples are taken from different domains. To address this problem, after observing the fact of unsatisfactory performance of the Kernel Mean Matching (KMM) algorithm, we propose a supervised extension that matches the distributions in a class-to-class manner, called Supervised Kernel Mean Matching (SKMM). The new approach stands out by taking into consideration both matching the distributions and preserving the discriminative information between classes at the same time. The extensive experimental studies on four cross-dataset facial expression recognition tasks show promising improvements of the proposed method, in which a small number of labeled samples guide the matching process.

Feature ranking is widely used in text classification. One problem with feature ranking methods is their non-robust behavior when applied to different data sets. In other words, the feature ranking methods behave differently from one data set to the

In data mining applications, data instances are typically described by a huge number of features. Most of these features are irrelevant or redundant, which negatively affects the efficiency and effectiveness of different learning algorithms. The selection of relevant features is a crucial task which can be used to allow a better understanding of data or improve the performance of other learning tasks. Although the selection of relevant features has been extensively studied in supervised learning, feature selection with the absence of class labels is still a challenging task. This paper proposes a novel method for unsupervised feature selection, which efficiently selects features in a greedy manner. The paper first defines an effective criterion for unsupervised feature selection which measures the reconstruction error of the data matrix based on the selected subset of features. The paper then presents a novel algorithm for greedily minimizing the reconstruction error based on the features selected so far. The greedy algorithm is based on an efficient recursive formula for calculating the reconstruction error. Experiments on real data sets demonstrate the effectiveness of the proposed algorithm in comparison to the state-of-the-art methods for unsupervised feature selection.

Feature ranking is widely employed to deal with high dimensionality in text classification. The main advantage of feature ranking methods is their low cost and simple algorithms. However, they suffer from some drawbacks which cause low performance compared to wrapper approach feature selection methods. In this paper, three major drawbacks of feature ranking methods are discussed. First, we show that feature ranking methods are highly problem dependent. For designing an effective feature ranking method and appropriate ranking threshold, we need background knowledge including the data set characteristics as well as the classifier to be used. Second, the feature ranking methods are univariate functions, while the nature of text classification is multivariate. It means that in these methods, correlation between terms is ignored. Finally, they fail in multiple class problems with unbalanced class distribution because they pay more attention to the simpler and larger classes. In this paper, these drawbacks, especially the last two issues, are experimentally investigated using a set of extensive numerical experiments with several data sets and feature scoring measures.

Global optimization is the problem of finding the global optimum of any given function in a certain search space. Simulated Annealing (SA) and Genetic Algorithms (GA) are among the well-known techniques used for global optimization. Adjusting the parameters of SA such as the temperature schedule and the neighborhood range plays an important role in the performance of the algorithm. Furthermore, many studies in literature showed that the best values for SA parameters depend on the optimization problem. We introduce a novel hybrid approach that uses SA to solve an optimization problem and uses GA simultaneously to adapt the parameters of SA. This new approach is referred to as Geno-Simulated Annealing (GSA). It does not require any predefined values for the parameters of SA. To evaluate the performance of the proposed approach, we used seven well-known benchmark optimization functions. The obtained results indicate the superiority of the proposed approach as compared to a similar approach and to conventional SA.

In text categorization (TC), which is a supervised technique, a feature vector of terms or phrases is usually used to represent the documents. Due to the huge number of terms in even a moderate-size text corpus, high dimensional feature space is an intrinsic problem in TC. Random subspace method (RSM), a technique that divides the feature space to smaller ones each submitted to a (base) classifier (BC) in an ensemble, can be an effective approach to reduce the dimensionality of the feature space. Inspired by a similar research on functional magnetic resonance imaging (fMRI) of brain, here we address the estimation of ensemble parameters, i.e., the ensemble size (L) and the dimensionality of feature subsets (M) by defining three criteria: usability, coverage, and diversity of the ensemble. We will show that relatively medium M and small L yield an ensemble that improves the performance of a single support vector machine, which is considered as the state-of-the-art in TC.

Relevance feedback (RFB) involves requesting some user judgments for an initial set of search results and then using these judgments to improve search results. Typical queries may have multiple possible interpretations or facets, only one of which is relevant to a user's need, but top search results may be dominated by one interpretation or facet. Thus, if the user is only given the top results to inspect, none of them may be relevant. One way to solve this is to intentionally diversify the top few results to cover multiple interpretations. This paper proposes the use of density-based clustering for the purpose of results diversification in the context of RFB. Other traditional clustering algorithms are also used for a comparative study. Clustering is compared to a baseline that nominates the top results from an initial ranked list and compared to using the top results after re-ranking using Maximal Marginal Relevance. The results show that density-based clustering achieves the best results with a statistically significant improvement of 12% over the baseline.

Most of text categorization techniques are based on word and/or phrase analysis of the text. Statistical analysis of a term frequency captures the importance of the term within a document only. However, two terms can have the same frequency in there documents, but one term contributes more to the meaning of its sentences than the other term. Thus, the underlying model should identify terms that capture the semantics of text. In this case, the model can capture terms that present the concepts of the sentence, which leads to discovering the topic of the document. A new concept‐based model that analyzes terms on the sentence, document, and corpus levels rather than the traditional analysis of document only is introduced. The concept‐based model can effectively discriminate between nonimportant terms with respect to sentence semantics and terms which hold the concepts that represent the sentence meaning. A set of experiments using the proposed concept‐based model on different datasets in text categorization is conducted in comparison with the traditional models. The results demonstrate the substantial enhancement of the categorization quality using the sentence‐based, document‐based and corpus‐based concept analysis.

IrisCode is an iris recognition algorithm developed in 1993 and continuously improved by Daugman. It has been extensively applied in commercial iris recognition systems. IrisCode representing an iris based on coarse phase has a number of properties including rapid matching, binomial impostor distribution and a predictable false acceptance rate. Because of its successful applications and these properties, many similar coding methods have been developed for iris and palmprint identification. However, we lack a detailed analysis of IrisCode. The aim of this paper is to provide such an analysis as a way of better understanding IrisCode, extending the coarse phase representation to a precise phase representation, and uncovering the relationship between IrisCode and other coding methods. Our analysis demonstrates that IrisCode is a clustering algorithm with four prototypes; the locus of a Gabor function is a 2-D ellipse with respect to a phase parameter and can be approximated by a circle in many cases; Gabor function can be considered as a phase-steerable filter and the bitwise hamming distance can be regarded as a bitwise phase distance. We also discuss the theoretical foundation of the impostor binomial distribution. We use this analysis to develop a precise phase representation which can enhance accuracy. Finally, we relate IrisCode and other coding methods.

In this paper, we present a design methodology for integrating heterogeneous classifier ensembles by employing a diversity-based hybrid classifier fusion approach, whose aggregator module consists of two classifier combiners, to achieve an improved classification performance for motor unit potential classification during electromyographic (EMG) signal decomposition. Following the so-called overproduce and choose strategy to classifier ensemble combination, the developed system allows the construction of a large set of base classifiers, and then automatically chooses subsets of classifiers to form candidate classifier ensembles for each combiner. The system exploits kappa statistic diversity measure to design classifier teams through estimating the level of agreement between base classifier outputs. The pool of base classifiers consists of different kinds of classifiers: the adaptive certainty-based, the adaptive fuzzy k -NN, and the adaptive matched template filter classifiers; and utilizes different types of features. Performance of the developed system was evaluated using real and simulated EMG signals, and was compared with the performance of the constituent base classifiers. Across the EMG signal datasets used, the developed system had better average classification performance overall, especially in terms of reducing classification errors. For simulated signals of varying intensity, the developed system had an average correct classification rate CCr of 93.8% and an error rate Er of 2.2% compared to 93.6% and 3.2%, respectively, for the best base classifier in the ensemble. For simulated signals with varying amounts of shape and/or firing pattern variability, the developed system had a CCr of 89.1% with an Er of 4.7% compared to 86.3% and 5.6%, respectively, for the best classifier. For real signals, the developed system had a CCr of 89.4% with an Er of 3.9% compared to 84.6% and 7.1%, respectively, for the best classifier.

Data partitioning methods such as bagging and boosting have been extensively used in multiple classifier systems. These methods have shown a great potential for improving classification accuracy. This study is concerned with the analysis of training data distribution and its impact on the performance of multiple classifier systems. In this study, several feature-based and class-based measures are proposed. These measures can be used to estimate statistical characteristics of the training partitions. To assess the effectiveness of different types of training partitions, we generated a large number of disjoint training partitions with distinctive distributions. Then, we empirically assessed these training partitions and their impact on the performance of the system by utilizing the proposed feature-based and class-based measures. We applied the findings of this analysis and developed a new partitioning method called "Clustering, Declustering, and Selection" (CDS). This study presents a comparative analysis of several existing data partitioning methods including our proposed CDS approach.

Most of the common techniques in text mining are based on the statistical analysis of a term, either word or phrase. Statistical analysis of a term frequency captures the importance of the term within a document only. However, two terms can have the same frequency in their documents, but one term contributes more to the meaning of its sentences than the other term. Thus, the underlying text mining model should indicate terms that capture the semantics of text. In this case, the mining model can capture terms that present the concepts of the sentence, which leads to discovery of the topic of the document. A new concept-based mining model that analyzes terms on the sentence, document, and corpus levels is introduced. The concept-based mining model can effectively discriminate between nonimportant terms with respect to sentence semantics and terms which hold the concepts that represent the sentence meaning. The proposed mining model consists of sentence-based concept analysis, document-based concept analysis, corpus-based concept-analysis, and concept-based similarity measure. The term which contributes to the sentence semantics is analyzed on the sentence, document, and corpus levels rather than the traditional analysis of the document only. The proposed model can efficiently find significant matching concepts between documents, according to the semantics of their sentences. The similarity between documents is calculated based on a new concept-based similarity measure. The proposed similarity measure takes full advantage of using the concept analysis measures on the sentence, document, and corpus levels in calculating the similarity between documents. Large sets of experiments using the proposed concept-based mining model on different data sets in text clustering are conducted. The experiments demonstrate extensive comparison between the concept-based analysis and the traditional analysis. Experimental results demonstrate the substantial enhancement of the clustering quality using the sentence-based, document-based, corpus-based, and combined approach concept analysis.

This paper proposes the use of a particle swarm optimization algorithm to the Field Programmable Gate Arrays (FPGA) placement problem. Two different versions of the particle swarm optimization algorithm are proposed. The first is a discrete version that solves the FPGA placement problem entirely in the discrete domain, while the second version is continuous in nature. Both versions are applied to several well- known FPGA benchmarks and the results are compared to those obtained by an academic placement tool that is based on adaptive simulated annealing. Results show that the proposed methods are competitive for small and medium-sized problems. For large-sized problems, the proposed methods provide very close results.

This paper focuses on evaluating and comparing a number of clustering methods used in color image segmentation of high resolution remote sensing images. Despite the enormous progress in the analysis of remote sensing imagery over the past three decades, there is a lack of guidance on how to select an image segmentation method suitable for the image type and size. Clustering has been widely used as a segmentation approach therefore, choosing an appropriate clustering method is very critical to achieve better results. In this paper we compare five clustering methods that have been suggested for segmentation of images. We focus on segmentation of urban areas in high resolution remote sensing images. Effective clustering extracts regions which correspond to land uses in urban areas. Ground truth images are used to evaluate the performance of clustering methods. The comparison shows that the average accuracy of road extraction is above 75%. The results show the potential of clustering high resolution aerial images starting from the three RGB bands only. The comparison gives some guidance and tradeoffs involved in using each.

Classification of data with imbalanced class distribution has encountered a significant drawback of the performance attainable by most standard classifier learning algorithms which assume a relatively balanced class distribution and equal misclassification costs. This paper provides a review of the classification of imbalanced data regarding: the application domains; the nature of the problem; the learning difficulties with standard classifier learning algorithms; the learning objectives and evaluation measures; the reported research solutions; and the class imbalance problem in the presence of multiple classes.

In distributed data mining, adopting a flat node distribution model can affect scalability. To address the problem of modularity, flexibility and scalability, we propose a Hierarchically-distributed Peer-to-Peer (HP2PC) architecture and clustering algorithm. The architecture is based on a multi-layer overlay network of peer neighborhoods. Supernodes, which act as representatives of neighborhoods, are recursively grouped to form higher level neighborhoods. Within a certain level of the hierarchy, peers cooperate within their respective neighborhoods to perform P2P clustering. Using this model, we can partition the clustering problem in a modular way across neighborhoods, solve each part individually using a distributed K-means variant, then successively combine clusterings up the hierarchy where increasingly more global solutions are computed. In addition, for document clustering applications, we summarize the distributed document clusters using a distributed keyphrase extraction algorithm, thus providing interpretation of the clusters. Results show decent speedup, reaching 165 times faster than centralized clustering for a 250-node simulated network, with comparable clustering quality to the centralized approach. We also provide comparison to the P2P K-means algorithm and show that HP2PC accuracy is better for typical hierarchy heights. Results for distributed cluster summarization match those of their centralized counterparts with up to 88% accuracy.

Most term weighting schemes for text document clustering depend on the term frequency based analysis of the text contents. A shortcoming of these indexing schemes, which consider only the occurrences of the terms in a document, is that they have some limitations in filtering out noise in most cases. In this paper, we propose a novel weighting approach using fusion technique that can be combined with wavelet-based estimation to achieve consistent improvements in the clustering. Our approach involves three steps: (1) term frequency (TF) weighting scheme, (2) multiple wavelets estimating, and (3) data fusion. Specifically, we apply the wavelet with different scales to produce different estimation values of the original TF, and use the fusion of these different values as new features for clustering the documents. The conducted experiments of clustering the documents from RETURES corpus verify that our weighting schemes using wavelet and fusion techniques reduces effectively the noise and improves clustering performance evaluated using the entropy and F_measure.

Extracting different clusters of the given data is an appealing topic in swarm intelligence applications. This paper introduces multiple cooperative swarms and single swarm clustering approaches and provides mathematical descriptions explaining why the former approach outperform the other one. Moreover, the stability analysis is proposed to obtain the model order of the data using multiple cooperative swarms clustering approach. The proposed clustering approach is evaluated using three data sets and its performance is compared with that of other clustering techniques.

Clustering techniques are helpful in understanding gene regulation, cellular processes, and subtypes of cells. A major thrust of gene expression analysis over the last twenty years has been the acquisition of enormous amount of various distributed sources of gene expression datasets. Thus, it is becoming increasingly important to perform clustering of distributed data in-place, without the need to pool it first into a central node. The general goal of distributed clustering is achieving a level of speedup than the centralized approaches. A recent study shows that centralized cooperative clustering outperforms the non-cooperative centralized clustering approaches. In this paper a novel distributed cooperative partitional-divisive clustering in a peer-to-peer network is presented. The distributed CPDC approach is based on intermediate cooperation between the Partitional k-means and the divisive bisecting k-means in a distributed Peer-to-Peer network to produce better global solutions. Computational experiments were conducted to test the performance of the distributed CPDC approach using different gene expression datasets. Undertaken experimental results show that the performance of the distributed CPDC method is better than that of the non-cooperative distributed k-means and distributed bisecting k-means. Thus a new cooperative technique for distributed gene expression repositories is efficiently presented to discover regularities and genes that may span multiple nodes.

Extracting natural groups of the unlabeled data is known as clustering. To improve the stability and robustness of the clustering outputs, clustering ensembles have emerged recently. In this paper, an ensemble of particle swarm clustering algorithms is proposed. That is, the members of the ensemble are based on the cooperative swarms clustering approaches. The performance of the proposed particle swarm clustering ensemble is evaluated using di®erent data sets and is compared to that of other clustering techniques.

Several validity indices have been designed to evaluate solutions obtained by clustering algorithms. Traditional indices are generally designed to evaluate center-based clustering, where clusters are assumed to be of globular shapes with defined centers or representatives. Therefore they are not suitable to evaluate clusters of arbitrary shapes and densities, where clusters have no defined centers or representatives, but formed based on the connectivity of patterns to their neighbours. In this work, a novel validity measure based on a density-based criterion is proposed. It is based on the concept that densities of clusters can be distinguished by the neighbourhood distances between patterns. It is suitable for clusters of any shapes and of different densities. The main concepts of the proposed measure are explained and experimental results that support the proposed measure are given.

This paper presents some investigations on tradeoff between exploration and exploitation of opposition-based Q(lambda) with non-Markovian update (NOQ(lambda)) in a dynamic environment. In the previous work the authors applied NOQ(lambda) to the deterministic GridWorld problem. In this paper, we have implemented the NOQ(lambda) algorithm for a simple elevator control problem to test the behavior of the algorithm for non-deterministic and dynamic environment. We also extend the NOQ(lambda) algorithm by introducing the opposition weight to find a better tradeoff between exploration and exploitation for the NOQ(lambda) technique. The value of the opposition weight increases as the number of steps increases. Hence, it has more positive effects on the Q-value updates for opposite actions as the learning progresses. The performance of NOQ(lambda) method is compared with Q(lambda) technique. The experiments indicate that NOQ(lambda) performs better than Q(lambda).

The constrained L(1) estimation is an attractive alternative to both the unconstrained L(1) estimation and the least square estimation. In this letter, we propose a cooperative recurrent neural network (CRNN) for solving L(1) estimation problems with general linear constraints. The proposed CRNN model combines four individual neural network models automatically and is suitable for parallel implementation. As a special case, the proposed CRNN includes two existing neural networks for solving unconstrained and constrained L(1) estimation problems, respectively. Unlike existing neural networks, with penalty parameters, for solving the constrained L(1) estimation problem, the proposed CRNN is guaranteed to converge globally to the exact optimal solution without any additional condition. Compared with conventional numerical algorithms, the proposed CRNN has a low computational complexity and can deal with the L(1) estimation problem with degeneracy. Several applied examples show that the proposed CRNN can obtain more accurate estimates than several existing algorithms.

In this paper, a geometry-based image retrieval system is developed for multi-object images. We model both shape and topology of image objects using a structured representation called curvature tree (CT). The hierarchy of the CT reflects the inclusion relationships between the image objects. To facilitate shape-based matching, triangle-area representation (TAR) of each object is stored at the corresponding node in the CT. The similarity between two multi-object images is measured based on the maximum similarity subtree isomorphism (MSSI) between their CTs. For this purpose, we adopt a recursive algorithm to solve the MSSI problem and a very effective dynamic programming algorithm to measure the similarity between the attributed nodes. Our matching scheme agrees with many recent findings in psychology about the human perception of multi-object images. Experiments on a database of 13500 real and synthesized medical images and the MPEG-7 CE-1 database of 1400 shape images have shown the effectiveness of the proposed method.

Over the past few years, there has been a renewed interest in the consensus clustering problem. Several new methods have been proposed for finding a consensus partition for a set of n data objects that optimally summarizes an ensemble. In this paper, we propose new consensus clustering algorithms with linear computational complexity in n. We consider clusterings generated with random number of clusters, which we describe by categorical random variables. We introduce the idea of cumulative voting as a solution for the problem of cluster label alignment, where, unlike the common one-to-one voting scheme, a probabilistic mapping is computed. We seek a first summary of the ensemble that minimizes the average squared distance between the mapped partitions and the optimal representation of the ensemble, where the selection criterion of the reference clustering is defined based on maximizing the information content as measured by the entropy. We describe cumulative vote weighting schemes and corresponding algorithms to compute an empirical probability distribution summarizing the ensemble. Given the arbitrary number of clusters of the input partitions, we formulate the problem of extracting the optimal consensus as that of finding a compressed summary of the estimated distribution that preserves maximum relevant information. An efficient solution is obtained using an agglomerative algorithm that minimizes the average generalized Jensen-Shannon divergence within the cluster. The empirical study demonstrates significant gains in accuracy and superior performance compared to several recent consensus clustering algorithms.

This study explores a new domain representation method for natural language processing based on an application of possibility theory. In our method, domain-specific information is extracted from natural language documents using a mathematical process based on Rieger's notion of semantic distances, and represented in the form of possibility distributions. We implement the distributions in the context of a possibilistic domain classifier, which is trained using the SchoolNet corpus.

This paper proposes a generalized least absolute deviation (GLAD) method for parameter estimation of autoregressive (AR) signals under non-Gaussian noise environments. The proposed GLAD method can improve the accuracy of the estimation of the conventional least absolute deviation (LAD) method by minimizing a new cost function with parameter variables and noise error variables. Compared with second- and high-order statistical methods, the proposed GLAD method can obtain robustly an optimal AR parameter estimation without requiring the measurement noise to be Gaussian. Moreover, the proposed GLAD method can be implemented by a cooperative neural network (NN) which is shown to converge globally to the optimal AR parameter estimation within a finite time. Simulation results show that the proposed GLAD method can obtain more accurate estimates than several well-known estimation methods in the presence of different noise distributions.

Clustering techniques organize a collection of objects into cohesive groups called clusters such that objects in the same cluster are more similar to each other than objects in different clusters. There are many clustering approaches proposed in the literature with different quality/complexity tradeoffs. Combining multiple clustering is an approach to overcome the deficiency of single algorithms and further enhance their performances. Current approaches to combining multiple clusterings use end-result cooperation (e.g. ensemble clustering and hybrid clustering) between the clustering algorithms. Inherent drawbacks of the end-result cooperation are: the computational complexity of ensemble clustering and the idle wasted time in the hybrid approaches. In this paper, the k-means and the bisecting k-means clustering algorithms are both combined using intermediate-cooperation strategy for the aim of obtaining better clustering solutions than non-cooperative algorithms. Undertaken experimental results show that the quality of the clustering solutions obtained from the cooperative partitional-divisive clustering (CPDC) model is better than those obtained from the non-cooperative algorithms over a number of gene expression datasets.

The huge number of gene expressions resulting from a single microarray experiment, together with the large number of tumor samples, needs efficient methods that can extract hidden information and structure in such data sets. Clustering is a common analysis tool used to find groups of gene expression patterns. However, analysis of large clusters can be an infeasible task in large sets. In this work, a method is proposed to capture the main structure of the data by identifying core gene expressions. This reduces the data to only a subset of representatives used to grasp the main behavior of gene expression. When integrated with clustering, it becomes feasible to analyze clusters of large sizes, and to identify main expression patterns and relations between them. The importance of using a connected-based clustering is emphasized in order to reveal the gradual change between core gene expressions, something which cannot be achieved using traditional clustering algorithms. Analysis is done on breast cancer data to illustrate the significance of the proposed methodology.

Particle Swarm Optimization (PSO) is a stochastic approach that was originally developed to simulate the behavior of birds and was successfully applied to many applications. In the field of evolutionary algorithms, researchers attempted many techniques in order to build probabilistic models that capture the search space properties and use these models to generate new individuals. Two approaches have been recently introduced to incorporate building a probabilistic model of the promising regions in the search space into PSO. This work proposes a new method for building this model into PSO, which borrows concepts from population-based incremental learning (PBIL) . The proposed method is implemented and compared to existing approaches using a suite of well-known benchmark optimization functions.

Clustering methods have been extensively used for gene expression data analysis to detect groups of related genes. The clusters provide useful information to analyze gene function, gene regulation and cellular patterns. Most existing clustering algorithms, though, discover only coherent gene expression patterns, and do not handle connected patterns. Coherent and connected patterns correspond to globular and arbitrary shaped clusters, respectively, in low dimensional spaces. For high dimensional gene expression data, two connected patterns can be two similar patterns with time lags in a time series data, or in general, two different patterns that are connected by an intermediate pattern that is related to both of them. Discovering such connected patterns has important biological implications not revealed by groups of coherent patterns. In this paper, a novel algorithm that finds connected patterns, in gene expression data, is proposed. Using a novel merge criterion, it can distinguish clusters based on distances between patterns, thus avoiding the effect of noise and outliers. Moreover, the algorithm uses a metric based on Pearson correlation to find neighbours, which renders it a lower complexity than related algorithms. Both time series and non temporal gene expression data sets are used to illustrate the efficiency of the proposed algorithm. Results on the serum and the leukaemia data sets reveal interesting biologically significant information

It is believed that modeling temporal structure of the speech data may be useful for the problem of speech emotion recognition (T. Nwe et al., 2003). In this paper, Gaussian mixture vector autoregressive model is proposed as a statistical classifier for this task. The main motivation behind using such a model is its ability to model the dependency among extracted speech feature vectors as well as the multi-modality in their distribution. When applied to the Berlin emotional speech database, the proposed technique provides a classification accuracy of 76% versus 71% for the hidden Markov model, 67% for the k-nearest neighbors, 55% for feed-forward neural networks. The model gives also better discrimination between high-arousal, low arousal, and neutral emotions than the HMM.

In this paper, a text classifier framework with a feature ranking scheme is proposed to extract social structures from text data. It is assumed that only a small subset of relations between the individuals in a community is known. With this assumption, the social network extraction is translated into a classification problem. The relations between two individuals are represented by merging their document vectors and the given relations are used as labels of training data. By this transformation, a text classifier such as Rocchio is used for learning the unknown relations. We show that there is a link between the intrinsic sparsity of social networks and class imbalance. Furthermore, we show that feature ranking methods usually fail in problem with unbalanced data. In order to deal with this deficiency and re-balance the unbalanced social data, a local feature ranking method, which is called reverse discrimination, is proposed.

Pitch control is a basic function of an Automatic Flight Control System (AFCS). Due to the complexity of problems, stochastic behavior, and the disturbing of the environment, traditional techniques, such as, linear feedback control, quantitative feedback theory, and adaptive control, which are all based on the explicit aerodynamic model of an aircraft, are not efficient in designing pitch controllers. This paper adopts multiple Reinforcement Learning (RL) algorithms and Cerebellar Model Articulation Controller (CMAC) techniques to design a pitch controller. In order to improve learning and control performances, a learn system named "Aggregated Multiple Reinforcement Learning System (AMRLS)" is proposed, which combines the outcomes of individual RL algorithms by using several aggregation methods. The goal of this paper is to demonstrate that the improved RL based control technology can be applied effectively to pitch control problem.

Most of text categorization techniques are based on word and/or phrase analysis of the text. Statistical analysis of a term frequency captures the importance of the term within a document only. However, two terms can have the same frequency in their documents, but one term contributes moreto the meaning of its sentences than the other term. Thus, the underlying model should indicate terms that capture these mantics of text. In this case, the model can capture terms that present the concepts of the sentence, which leads todiscover the topic of the document. A new concept-based model that analyzes terms on the sentence and document levels rather than the traditional analysis of document only is introduced. The concept-based model can effectively discriminate between non-important terms with respect to sentence semantics and terms which hold the concepts that represent the sentence meaning. The proposed model consists of concept-based statistical analyzer, conceptual ontological graph representation,and concept extractor. The term which contributes to the sentence semantics is assigned two different weights by the concept-based statistical analyzer and the conceptual ontological graph representation. These two weights are combined into a new weight. The concepts that have maximum combined weights are selected by the concept extractor. A set of experiments using the proposed concept-basedmodel on different datasets in text categorization is conducted. The experiments demonstrate the comparison between traditional weighting and the concept-based weighting obtained by the combined approach of the concept-based statistical analyzer and the conceptual ontological graph. The evaluation of results is relied on two quality measures, the Macro-averaged F1 and the Error rate. These quality measures are improved when the newly developedconcept-based model is used to enhance the quality of thetext categorization.

In distributed data mining models, adopting a flat node distribution model can affect scalability. To address the problem of modularity, flexibility and scalability, we propose a hierarchically-distributed peer-to-peer architecture and algorithm for data clustering (HP2PC). The architecture is based on a multi-layer overlay network of peer neighborhoods. Supernodes, which act as representatives of neighborhoods, are recursively grouped to form higher level neighborhoods. Peers at a certain level of the hierarchy cooperate within their respective neighborhoods to perform clustering. Using this model, we can partition the clustering problem in a modular way, solve each part individually, then successively combine clusterings up the hierarchy where increasingly global solutions are computed. The algorithm was applied to a distributed document clustering problem and achieved decent speedup with comparable clustering quality to the centralized approach.

In this paper, we propose the scheme of multiple input representation-adaptive ensemble generation and aggregation(MIR-AEGA) for the classification of time series data. MIR-AEGA employs a set of heterogeneous classifiers, each of which takes a different representation of time series data as the input. MIR-AEGA adopts an "overfitting and selection" strategy. In the training phase, different ensembles of classifiers are adaptively generated by fitting the validation data ' globally in different degrees. The test data are then classified by each of the generated ensembles. The final decision is made by taking consideration into both the ability of each ensemble to fit the validation data locally and the possible overfitting effects. We claim that MIR-AEGA has two advantages (1) By using multiple representations, it exploits the temporal information of time series data as much as possible, thus could improve the overall performance (2) By tweaking the trade-off between the ability to fit the validation data and the overfitting effects, we expect the performance of this method is reliable in different situations. In this paper, the performance of MIR- AEGA is also assessed experimentally in comparison with other benchmark techniques. The experimental results demonstrate the good performance and the reliability of MIR-AEGA for the classification of time series data.

Reinforcement learning (RL) is a learning technique that learns an optimal policy in case of knowing almost nothing about the dynamics of the environment under consideration. When RL is combined with function approximation schemes, the learning performance is greatly influenced by RL algorithms and learning parameters. This paper proposes a new on-line multiple learning and aggregating architecture, "aggregated multiple reinforcement learning system (AMRLS)". Instead of searching for the optimal learning parameters or featurization schemes, AMRLS attempts to aggregate the outcomes of different learners to produce a better policy. This architecture is tested on the mountain car problem with the aggregation of several related tiling and learning parameters. Experimental results show that AMRLS can improve the learning performance over the use of a single RL algorithm.

Data clustering plays an important role in many disciplines, where there is a need to learn the inherent grouping structure of the data in an unsupervised manner. It is well known that no clustering method can adequately handle all sorts of cluster structures and properties (e.g. shape, size, overlapping, and density). Combining multiple clustering methods is an approach to overcome the deficiency of single algorithms and further enhance their performances. Current approaches to multiple clusterings use ensemble clustering to generate aggregated solution from multiple clusterings or using a hybrid cascaded refinement to enhance the end-result clusters produced by a former clustering algorithm(s). A disadvantage of the cluster ensemble is the highly computational load of combing the clustering results especially for large and high dimensional datasets. A drawback of the hybrid approaches is that, one (or more) of the clustering algorithms stays idle until the previous algorithm(s) finishes its clustering. In this paper we propose a Cooperative Hard-Fuzzy Clustering (CHFC) model based on intermediate cooperation between the hard c-means (KM) andfuzzyc-means (FCM) to produce better clustering solutions. Our experimental results over artificial, real, and text documents datasets show that the quality of the clustering solutions obtained from the CHFC model is better than those obtained from both the KM and the FCM and also better than those obtained from hybrid cascaded models.

Automatic building of social networks requires extracting pair-wise relations of the individuals. In this paper, supervised learning of social networks from a set of documents is proposed. Given a small subset of relations between the individuals, the problem of learning social network is translated into a text classification problem. Relation between each pair of individuals is represented by a vector of words produced from merging all documents associated with these two individuals. The known relation is used as a label for the relation vector. The merged documents and their given labels, are used as training data. By this transformation, a text classifier such as SVM can be used for learning the unknown relations. We show that there is a link between the intrinsic sparsity of social networks and class distribution imbalance of the training data. In order to re-balance the unbalanced training data, a multiple resampling method, including undersampling of the majority and oversampling of the minority class, is employed. The proposed framework is applied to a Friend Of A Friend (FOAF) data set and evaluated by the macro-averaged F-measure.

Feature ranking is one of the dimensionality reduction methods. Because of its simplicity and low cost, it is widely used in text classification. One problem with feature ranking methods is their non-robust behavior when applied to different data sets. In other words, the feature ranking methods behave differently from one data set to the other. The problem is more complex when we consider that the performance of feature ranking methods is different when being used by different classifiers. In this paper, a new method based on combining feature rankings is proposed to find the best features among a set of feature rankings. Four preferential voting method are employed to combine feature rankings obtained by eight well-known ranking measures. According to the results, combining methods can offer reliable results that are very close to the best solution without the need to use a classifier. The proposed method is applied to the text classification problem and evaluated on three well-known data sets using SVM classifier.

Many outlier detection methods identify outliers ignoring any structure in data. However, it is sometimes beneficial to integrate outlierness and a method that groups data, such as clustering. This enhances both outlier and cluster analysis. In this paper, a fuzzy approach is proposed for integrating results from an outlier detection method and a clustering algorithm. A universal set of clusters is proposed which combines clusters obtained from clustering, and a virtual cluster for the outliers. The approach has two phases; the first computes patterns' initial memberships for the outlier cluster, and the second calculates memberships for the universal clusters, using an iterative membership propagation technique. The proposed approach is general and can combine any outlier detection method with any clustering algorithm. Both low and high dimensional data sets are used to illustrate the impact of the proposed approach.

Metric trees are designed for improving efficiency of similarity search in high dimensional data. Searching a metric tree always terminates at leaf nodes, restricting the hierarchical search to a certain leaf level, and the linear search in leafs to a particular leaf size. This may result in performing additional unneeded distance comparisons, which increases the search time as the number of dimensions increases. It is proposed that the search adapts itself to the query in question, so as to avoid unneeded distance comparisons. Thus, the hierarchical search should terminate when no more search- space pruning can be useful, and search then continues linearly through the un-pruned data space. Hierarchical search can thus stop any where above or below the original leaf level of a metric tree. Search rules are proposed to adapt the search to the query parameters. A modification to the metric tree is also suggested to adopt the proposed rules. High dimensional gene expression data sets are used to evaluate the new algorithm, showing speed ups of 55% compared to traditional search.

A new clustering technique by the use of multiple swarms is proposed. The proposed technique mimics the behavior of biological swarms which explore food situated in several places. We model the clustering problem using particle swarm optimization (PSO) approach. The proposed method considers multiple cooperating swarms to find centers of clusters. By assigning a portion of the solution space to each swarm, the exploration ability to find the solution is enhanced. Moreover, the cooperation among swarms increases the between-class distance. The proposed method outperforms k-means clustering as well as conventional PSO-based clustering techniques

An automatic taxonomy extraction algorithm is proposed. Given a set of terms or terminology related to a subject domain, the proposed approach uses Google page count to estimate the dependency links between the terms. A taxonomic link is an asymmetric relation between two concepts. In order to extract these directed links, neither mutual information nor normalized Google distance can be employed. Using the new measure of information theoretic inclusion index, term dependency matrix, which represents the pair-wise dependencies, is obtained. Next, using a proposed algorithm, the dependency matrix is converted into an adjacency matrix, representing the taxonomy tree. In order to evaluate the performance of the proposed approach, it is applied to several domains for taxonomy extraction.

Most of the common techniques in text retrieval are based on the statistical analysis of a term either as a word or a phrase. Statistical analysis of a term frequency captures the importance of the term within a document only. Thus, to achieve a more accurate analysis, the underlying representation should indicate terms that capture the semantics of text. In this case, the representation can capture terms that present the concepts of the sentence, which leads to discover the topic of the document. A new concept-based representation, called Conceptual Ontological Graph (COG), where a concept can be either a word or a phrase and totally dependent on the sentence semantics, is introduced. The aim of the proposed representation is to extract the most important terms in a sentence and a document with respect to the meaning of the text. The COG representation analyzes each term at both the sentence and the document levels. This is different from the classical approach of analyzing terms at the document level. First, the proposed representation denotes the terms which contribute to the sentence semantics. Then, each term is chosen based on its position within the COG representation. Lastly, the selected terms are associated to their documents as features for the purpose of indexing before text retrieval. The COG representation can effectively discriminate between non-important terms with respect to sentence semantics and terms which hold the key concepts that represent the sentence meaning. Large sets of experiments using the proposed COG representation on different datasets in text retrieval are conducted. Experimental results demonstrate the substantial enhancement of the text retrieval quality using the COG representation over the traditional techniques. The evaluation of results relies on two quality measures, the bpref and P(10). Both the quality measures improved when the newly developed COG representation is used to enhance the quality of the text retrieval results.

Identification of a general nonlinear noisy system viewed as an estimation of a predictor function is studied in this article. A measurement fusion method for the predictor function estimate is proposed. In the proposed scheme, observed data are first fused by using an optimal fusion technique, and then the optimal fused data are incorporated in a nonlinear function estimator based on a robust least squares support vector machine (LS-SVM). A cooperative learning algorithm is proposed to implement the proposed measurement fusion method. Compared with related identification methods, the proposed method can minimize both the approximation error and the noise error. The performance analysis shows that the proposed optimal measurement fusion function estimate has a smaller mean square error than the LS-SVM function estimate. Moreover, the proposed cooperative learning algorithm can converge globally to the optimal measurement fusion function estimate. Finally, the proposed measurement fusion method is applied to ARMA signal and spatial temporal signal modeling. Experimental results show that the proposed measurement fusion method can provide a more accurate model.

Modern statistical techniques used in the field of natural language processing are limited in their applications by the fact they suffer from the loss of most of the semantic information contained in text documents. Fuzzy techniques have been proposed as a way to correct this problem through the modelling of the relationships between words while accommodating the ambiguities of natural languages. However, these techniques are currently either restricted to modelling the effects of simple words or are specialized in a single domain. In this paper, we propose a novel statistical-fuzzy methodology to represent the actions described in a variety of text documents by modelling the relationships between subject-verb-object triplets. The research will focus in the first place on the technique used to accurately extract the triplets from the text, on the necessary equations to compute the statistics of the subject-verb and verb-object pairs, and on the formulas needed to interpolate the fuzzy membership functions from these statistics and on those needed to de fuzzify the membership value of unseen triplets. Taken together, these sets of equations constitute a comprehensive system that allows the quantification and evaluation of the meaning of text documents, while being general enough to be applied to any domain. In the second phase, this paper will proceed to experimentally demonstrate the validity of our new methodology by applying it to the implementation of a fuzzy classifier conceived especially for this research. This classifier is trained using a section of the Brown Corpus, and its efficiency is tested with a corpus of 20 unseen documents drawn from three different domains. The positive results obtained from these experimental tests confirm the soundness of our new approach and show that it is a promising avenue of research.

To deal with the problem of restoring degraded images with non-Gaussian noise, this paper proposes a novel cooperative neural fusion regularization (CNFR) algorithm for image restoration. Compared with conventional regularization algorithms for image restoration, the proposed CNFR algorithm can relax need of the optimal regularization parameter to be estimated. Furthermore, to enhance the quality of restored images, this paper presents a cooperative neural fusion (CNF) algorithm for image fusion. Compared with existing signal-level image fusion algorithms, the proposed CNF algorithm can greatly reduce the loss of contrast information under blind Gaussian noise environments. The performance analysis shows that the proposed two neural fusion algorithms can converge globally to the robust and optimal image estimate. Simulation results confirm that in different noise environments, the proposed two neural fusion algorithms can obtain a better image estimate than several well known image restoration and image fusion methods.

Different cooperative models have been proposed in the past few years using particle swarm optimization (PSO). These models relied on having more than one swarm running in a parallel or serial fashion while exchanging information among them. The information shared among the cooperating swarms was mostly their global bests. This work adopts the approach of selecting a particle from one swarm replacing a particle in another swarm. The work compares four different approaches for information exchange based on three different performance measures. The work also tests how this behavior changes while increasing the number of exchanged particles. Experiments are run using four benchmark optimization functions.

We propose here to develop a methodology to extract semantic knowledge from plain written English documents and represent it using a formal mathematical expression, in order to facilitate its use in practical applications. Our fundamental conjecture is that most of the semantic information of a sentence lies in the action described by that sentence. Consequently, we focus on extracting from the text the words whose relationships represent actions, and on modelling those relationships. We then demonstrate the practicality of our methodology by applying it to a domain classifier.

Most of text mining techniques are based on word and/or phrase analysis of the text. The statistical analysis of a term (word or phrase) frequency captures the importance of the term within a document. However, to achieve a more accurate analysis, the underlying mining technique should indicate terms that capture the semantics of the text from which the importance of a term in a sentence and in the document can be derived. A new concept-based mining model that relies on the analysis of both the sentence and the document, rather than, the traditional analysis of the document dataset only is introduced. The proposed mining model consists of a concept-based analysis of terms and a concept-based similarity measure. The term which contributes to the sentence semantics is analyzed with respect to its importance at the sentence and document levels. The model can efficiently find significant matching terms, either words or phrases, of the documents according to the semantics of the text. The similarity between documents relies on a new concept-based similarity measure which is applied to the matching terms between documents. Experiments using the proposed concept-based term analysis and similarity measure in text clustering are conducted. Experimental results demonstrate that the newly developed concept-based mining model enhances the clustering quality of sets of documents substantially.

Classification of data with imbalanced class distribution has posed a significant drawback of the performance attainable by most standard classifier learning algorithms, which assume a relatively balanced class distribution and equal misclassification costs. This learning difficulty attracts a lot of research interests. Most efforts concentrate on bi-class problems. However, bi-class is not the only scenario where the class imbalance problem prevails. Reported solutions for bi-class applications are not applicable to multi-class problems. In this paper, we develop a cost-sensitive boosting algorithm to improve the classification performance of imbalanced data involving multiple classes. One barrier of applying the cost-sensitive boosting algorithm to the imbalanced data is that the cost matrix is often unavailable for a problem domain. To solve this problem, we apply Genetic Algorithm to search the optimum cost setup of each class. Empirical tests show that the proposed cost-sensitive boosting algorithm improves the classification performances of imbalanced data sets significantly.

Reinforcement learning (RL) is a machine learning method that can learn an optimal strategy for a system without knowing the mathematical model of the system. Many RL algorithms are successfully applied in various fields. However, each algorithm has its advantages and disadvantages. With the increasing complexity of environments and tasks, it is difficult for a single learning algorithm to cope with complicated learning problems with high performance. This motivated us to combine some learning algorithms to improve the learning quality. This paper proposes a new multiple learning architecture, "Aggregated Multiple Reinforcement Learning System (AMRLS)". AMRLS adopts three different learning algorithms to learn individually and then combines their results with aggregation methods. To evaluate its performance, AMRLS is tested on two different environments: a Cart-pole System and a Maze environment. The presented simulation results reveal that aggregation not only provides robustness and fault tolerance ability, but also produces more smooth learning curves and needs fewer learning steps than individual learning algorithms.

The problem of delayed reward in reinforcement learning is usually tackled by implementing the mechanism of eligibility traces. In this paper we introduce an extension of eligibility traces to solve one of the challenging problems in reinforcement learning. The concept of opposition traces is proposed in this work to deal with large state space problems in reinforcement learning applications. We combine the idea of opposition and eligibility traces to construct the opposition-based Q(lambda). The results are compared with the conventional Watkins' Q(lambda) and reflect a remarkable performance increase.

A cooperative recurrent neural network (CRNN) algorithm for parameter estimation of autoregressive (AR) signals is proposed in this paper. The proposed CRNN algorithm is based on a generalized least absolute deviation (GLAD) method, which generalizes significantly the conventional least absolute deviation method. Compared with second-order and high-order statistic algorithms, the proposed CRNN algorithm can obtain robustly an optimal AR parameter estimation without requiring measurement Gaussian noise. Unlike existing cooperative neural network algorithms, the proposed CRNN algorithm has a global convergence and a novel weighting cooperation scheme to integrate single neural network output automatically. Simulation results shows that the more accurate estimates can be attained by the proposed CRNN algorithm in the presence of non-Gaussian colored noise.

A novel cooperative neural learning (CNL) algorithm based on a new linearly constrained least absolute deviation (LCLAD) method for data fusion is proposed in this paper. The state model of the proposed CNL algorithm combines adaptively three recurrent modular neural networks and is sample for implementation using both software and hardware. Unlike the conventional LAD approach, the propose LCLAD method can obtain the optimal fusion solution. Compared with the minimum variance method and linearly constrained least square method, the proposed LCLAD method can minimize an augmented least absolute deviation energy of the linearly fused information and has the robustness performance in non-Gaussian noise environments. Illustrative examples of signal and image fusion show that the quality of the solution can be more enhanced by the proposed CNL algorithm.

Document clustering has been traditionally studied as a centralized process. There are scenarios when centralized clustering does not serve the required purpose; e.g. documents spanning multiple digital libraries need not be clustered in one location, but rather clustered at each location, then enriched by receiving more information from other locations. A distributed collaborative approach for document clustering is proposed in this paper. The main objective here is to allow peers in a network to form independent opinions of local document grouping, followed by exchange of cluster summaries in the form of keyphrase vectors. The nodes then expand and enrich their local solution by receiving recommended documents from their peers based on the peer judgement of the similarity of local documents to the exchanged cluster summaries. Results show improvement in final clustering after merging peer recommendations. The approach allows independent nodes to achieve better local clustering by having access to distributed data without the cost of centralized clustering, while maintaining the initial local clustering structure and coherency.

Early diagnosis of plant faults / deviations is a critical factor for optimized and safe plant operation and maintenance. Although smart controllers and diagnosis systems are available and widely used in chemical plants, however, some faults couldn't be detected. Major reason is the lack of learning techniques that can learn from operational running data and previous abnormal cases. In addition, operator and maintenance engineer opinions and observations are not well used, while useful diagnosis knowledge is ignored. This research paper presents the framework of the proposed learning mechanisms in different stages of integrated fault diagnostic system, which is called FDS. The proposed idea will support plant operation and maintenance planning as well as overall plant safety.

Life on this planet is full of astonishing examples of cooperation. Individual species depend upon one another for sustenance, often forming surprising alliances to achieve a common goal: continuance of the species. The majority of living things also display amazing altruism in order to protect and provide the best care for their offspring, incomparable to any form of sacrifice shown by human beings. Studying the cooperation patterns between living things and their intelligent behaviors has been source of inspiration for many new algorithms, theories and systems. This paper addresses the concept of cooperation and why it is important. It highlights the available biologically-inspired models and algorithms, and their potential applications. The paper also presents a general typology of cooperation patterns, which can help to understand how systems could work cooperatively in an intelligent manner.

Automatic generation of a social network requires extracting pair-wise relations of the individuals. In this research, learning social network from incomplete relationship data is proposed. It is assumed that only a small subset of relations between the individuals is known. With this assumption, the social network extraction is translated into a text classification problem. The relations between two individuals are modeled by merging their document vectors and the given relations are used as labels of training data. By this transformation, a text classifier such as SVM is used for learning the unknown relations. We show that there is a link between the intrinsic sparsity of social networks and class distribution imbalance of the training data. In order to re-balance the unbalanced training data, a minority class down-sampling strategy is employed. The proposed framework is applied to a true FOAF (friend of a friend) database and evaluated by the macro-averaged F-measure

In this paper, new wavelet-based affine invariant functions for shape representation are presented. Unlike the previous representation functions, only the approximation coefficients are used to obtain the proposed functions. One of the derived functions is computed by applying a single wavelet transform; the other function is calculated by applying two different wavelet transforms with two different wavelet families. One drawback of the previously derived detail-based invariant representation functions is that they are sensitive to noise at the finer scale levels, which limits the number of scale levels that can be used. The experimental results in this paper demonstrate that the proposed functions are more stable and less sensitive to noise than the detail-based functions.

Biometric authentication systems are widely applied because they offer inherent advantages over classical knowledge-based and token-based personal-identification approaches. This has led to the development of products using palmprints as biometric traits and their use in several real applications. However, as biometric systems are vulnerable to replay, database, and brute-force attacks, such potential attacks must be analyzed before biometric systems are massively deployed in security systems. This correspondence proposes a projected multinomial distribution for studying the probability of successfully using brute-force attacks to break into a palmprint system. To validate the proposed model, we have conducted a simulation. Its results demonstrate that the proposed model can accurately estimate the probability. The proposed model indicates that it is computationally infeasible to break into the palmprint system using brute-force attacks.

One of the many issues that confront traditional statistical approaches of natural language understanding (NLU) is on how to overcome the insufficient co-occurrence information caused by the limited boundary of statistical approaches. Researches have long used the imparting of human knowledge into statistical approaches, including definition of rules and collections of hierarchy of concepts. However, these are difficult to define even for a domain expert. They are also very much people and domain dependent. This study proposes a fuzzy approach to tackle these issues in a way as to provide a methodology for logical reorganizing context in order to tackle the issue of boundary limitation, to create the more reasonable and understandable word association which will be referenced as membership degree in latter stage, and to make the processes of imparting of human knowledge easier and less domain dependent. The accomplishment of these tasks could be achieved through the concept of precisiated natural language (PNL)

This paper investigates the idea of having multiple swarms working separately and cooperating with each other to solve an optimization problem. Many factors that influence the behavior of this approach haven't been properly studied. This paper investigates two factors that affect this approach behavior. These factors are: (i) the communication strategy adopted if the number of swarms is raised above two, and (ii) the number of cooperating swarms. Experiments run on different benchmark optimization functions show that adopting a circular communication strategy gives better results than just sharing the global best of all the swarms. Increasing the number of cooperating swarms provides better results provided that the appropriate synchronization period is selected.

In this paper, a new 2D shape multiscale triangle-area representation (MTAR) is proposed. This representation utilizes a simple geometric principle, the area of a triangle, in obtaining a robust and efficient shape representation. The use of the wavelet transform for decomposing the boundary of the shapes improves the efficiency and robustness of the representation. The MTAR is more robust to the affine transformation, less affected by noise, and more selective than similar methods, e.g., the curvature scale-space CSS. Two tests, using MPEG-7 CE-shape-1 database, show that MTAR achieves better performance than the CSS under affine transformation and in the general shape retrieval.

Automatic cheque image processing is an important task in document image processing. Although many methods have been developed, it is still a challenge to find effective methods for some of the tasks, one of these is how to extract the filled-in strokes from a given cheque image. The problem becomes more difficult when a cheque has a complicated background. Most current methods are only applicable to simple binary images. This paper presents a method for processing complicated grey images in two steps. Both the extraction of reference lines and filled-in strokes are included. Our method is based on the construction of a pseudo 2D wavelet with adjustable rectangular supports. The experimental result shows good performance in the present method, which is also effective for slightly skewed cheque image.

A new framework for the context and speaker independent recognition of emotions from voice, based on a richer and more natural representation of the speech signal, is proposed. The utterance is viewed as consisting of a series of voiced segments and not as a single object. The voiced segments are first identified and then described using statistical measures of spectral shape, intensity, and pitch contours, calculated at both the segment and the utterance level. Utterance classification is performed by combining the segment classification decisions using a fixed combination scheme. The performance of two learning algorithms, support vector machines and K nearest neighbors, is compared. The proposed approach yields an overall classification accuracy of 87% for 5 emotions, outperforming previous results on a similar database.

This paper discusses document clustering using ant colony algorithm and validity index. Clusterings are formed on the plane by ants walking, picking up or dropping down projected document vectors with different probability. The proposed model uses a clustering validity index not only to evaluate the performance of the algorithm, but also to find the optimal number of clusters and reduce outliers. Experiments on data from the Reuters-21578 collection show that the proposed model has better performance than that of LF algorithm and ART neural networks.

This paper investigates the idea of having two cooperating swarms exchanging information in order to solve an optimization problem. The information being exchanged between the two swarms is an important factor that affects the quality of the obtained solution. The information to be shared between the two swarms has to be carefully selected depending on the model being used. This paper compares two types of information to be exchanged, namely the global best and the best particle. It is shown that exchanging the best particles leads to better results than only sharing the global best. This paper also addresses the idea of exchanging the best p particles between the two swarms.

We live in a society that depends on high-tech devices for assistance with everyday tasks, including everything from transportation to health care, communication, and entertainment. Tedious tactile input interfaces to these devices result in inefficient use of our time. Appropriate use of natural hand gestures will result in more efficient communication if the underlying meaning is understood. Overcoming natural hand gesture understanding challenges is vital to meet the needs of these increasingly pervasive devices in our every day lives. This work presents a graph-based approach to understand the meaning of hand gestures by associating dynamic hand gestures with known concepts and relevant knowledge. Conceptual-level processing is emphasized to robustly handle noise and ambiguity introduced during generation, data acquisition, and low-level recognition. A simple recognition stage is used to help relax scalability limitations of conventional stochastic language models. Experimental results show that this graph-based approach to hand gesture understanding is able to successfully understand the meaning of ambiguous sets of phrases consisting of three to five hand gestures. The presented approximate graph-matching technique to understand human hand gestures supports practical and efficient communication of complex intent to the increasingly pervasive high-tech devices in our society.

In this paper, an algorithm for matching and classifying 2-D shapes that undergo affine transformation is developed. The algorithm uses the 1-D dyadic wavelet transform (DWT) to decompose a shape's boundary into multiscale levels. The curve moment invariants of the approximation coefficients are used as the shape features. Two different dissimilarities are calculated from the Euclidean distances between the decomposed scale levels of the shapes. These dissimilarities are used in shape matching and clustering by using hierarchical clustering algorithm with Ward's linkage rules. The presented algorithm is invariant to the affine transformation and to the boundary starting point variation. The algorithm is also capable of finding and clustering similar shapes even if there are small deformations between their boundaries.

The need for the design of complex and incremental training algorithms in multiple neural network systems has motivated us to study combining methods from the cooperation perspective. One way of achieving effective cooperation is through sharing resources such as information and components. The degree and method by which multiple classifier systems share training resources can be a measure of cooperation. Despite the growing number of interests in data modification techniques, such as bagging and k-fold cross-validation, there is no guidance for whether sharing or not sharing training patterns results in higher accuracy and under what conditions. We implemented several partitioning techniques and examined the effect of sharing training patterns by varying the size of overlap between 0-100% of the size of training subsets. Under most conditions studied, multinet systems showed improvement over the presence of larger overlap subsets.

Document clustering is one of the popular techniques that assist users in organizing collections of documents. Two successful models of unsupervised neural networks, self-organizing map (SOM) and adaptive resonance theory (ART), have shown promising results in this task. Most of the existing neural network based document clustering techniques rely on a "bag of words" document representation. Each word in the document is considered as a separate feature, ignoring the word order. We investigate the use of phrases rather than words as document features applied to our proposed document clustering technique, called hierarchical SOMART (HSOMART), which is a hierarchical network built up from independent SOM and ART neural networks. We describe a phrase grammar extraction technique, and the proposed HSOMART. The experimental results of clustering documents from the REUTERS corpus using the extracted phrases as features show an improvement in the clustering performance evaluated using the entropy and F-measure.

Existing binarization methods are categorized as either global or local. In this paper, we present a new category, where the image is considered a collection of subimages. Each subimage provides a statistical model for the handwritten characters that can be used to optimize the binarization of other subimages based on gray-level and stroke-run features. The proposed method uses these multimodels to iteratively arrive at the optimal threshold for each subimage. It can be applied to different types of documents where prior knowledge about the noisiness of the subimages is not available. Experimental results showed significant improvement in the binarization quality in comparison with other well-established algorithms.

Document clustering techniques mostly rely on single term analysis of the document data set, such as the vector space model. To achieve more accurate document clustering, more informative features including phrases and their weights are particularly important in such scenarios. Document clustering is particularly useful in many applications such as automatic categorization of documents, grouping search engine results, building a taxonomy of documents, and others. This article presents two key parts of successful document clustering. The first part is a novel phrase-based document index model, the document index graph, which allows for incremental construction of a phrase-based index of the document set with an emphasis on efficiency, rather than relying on single-term indexes only. It provides efficient phrase matching that is used to judge the similarity between documents. The model is flexible in that it could revert to a compact representation of the vector space model if we choose not to index phrases. The second part is an incremental document clustering algorithm based on maximizing the tightness of clusters by carefully watching the pair-wise document similarity distribution inside clusters. The combination of these two components creates an underlying model for robust and accurate document similarity calculation that leads to much improved results in Web document clustering over traditional methods.

Existing skeletonization methods operate directly onthe binary image ignoring the gray-level information.In this paper we propose a new method for the skeletonization of handwritten characters that uses gray-level information and capitalizes on their elongatedpattern properties. The method controls the development of the skeleton while iteratively binarizing thegray-level image. Two types of iterations are performed: the iterative skeletonization and deletion ofboundary pixels, which is nested within the iterativebinarization of the gray-level image. Detailed analysis of the skeletonization process is presented to showits superior performance related to the prevention offlooding water and end point shrinkage and to noiseimmunity.

In this paper, we present an improved image registration algorithm based on the virtual circles features. Virtual circles are extracted efficiently by finding the local maxima in the distance transform of the edge maps. Virtual circles are grouped into collinear set to which a line is fitted to determine the direction. The radii of the virtual circles in the group as well as the direction of the fitted line are used in the registration algorithm. The Hausdorff fraction is also used as a similarity measure to determine the optimal transformation. This algorithm can find large translation, rotation, and scale differences between two images. Further more, it has linear complexity in terms of the number of virtual circles extracted.

Existing binarization methods are categorized as either global or local. In this paper we present a new category, where the image is considered as a collection of sub-images. Each sub-image provides a statistical model for the handwritten characters that will be used to optimize the binarization of other sub-images. This method can be applied to different types of documents and doesn't require any prior knowledge about the noisiness of the sub-images.

Availability of large full-text document collections in electronic form has created a need for tools and techniques that assist users in organizing these collections. Document clustering is one of the popular methods used for this purpose. In this paper, we propose the neural network based document clustering method by using a hierarchically organized network built up from independent Self-Organizing Map (SOM) and Adaptive Resonance Theory (ART) neural networks. We present clustering results using the REUTERS corpus and show an improvement in clustering performance using both entropy and F-measure as evaluation measures.

An intrusion detection system (IDS) is an art to detect network intrusions by monitoring the network traffic patterns. Generally, an IDS uses only a single-layer detection structure; therefore it cannot adjust its structure adaptively and automatically. In this paper, two hierarchical IDSs, the serial hierarchical and parallel hierarchical IDSs, are proposed. Both of them are based on radial basis function (RBF) neural networks. Because of the short training time and high accuracy of the RBF neural networks, two hierarchical IDSs can monitor network traffic in real-time, train new classifiers for novel intrusions automatically, and modify their structures adaptively after new classifiers are trained.

This paper presents a clustering ensemble using three colonies of ants, each colony having different ant speed model: constant, random, and randomly decreasing. The algorithm is a two-phase process. Initially clusterings are visually formed on the plane by ants walking, picking up or dropping down projected data objects with different probability, and then a hypergraph model is used to combine clusterings. Results on synthetic and real data sets are given to show that the number of clusters can be adaptively determined and clustering ensembles can improve the clustering performance.

Clustering of large collections of text documents is a key process in providing a higher level of knowledge about the underlying inherent classification of the documents. Web documents, in particular, are of great interest since managing, accessing, searching, and browsing large repositories of Web content requires efficient organization. Incremental clustering algorithms are always preferred to traditional clustering techniques, since they can be applied in a dynamic environment such as the Web. An incremental document clustering algorithm is introduced, which relies only on pair-wise document similarity information. Clusters are represented using a cluster similarity histogram, a concise statistical representation of the distribution of similarities within each cluster, which provides a measure of cohesiveness. The measure guides the incremental clustering process. Complexity analysis and experimental results are discussed and show that the algorithm requires less computational time than standard methods while achieving a comparable or better clustering quality.

This paper presents an automated system for multiple sensor placement based on the coordinated decisions of independent, intelligent agents. The problem domain is such that a single sensor system would not provide adequate information for a given sensor task. Hence, it is necessary to incorporate multiple sensors in order to obtain complete information. The overall goal of the system is to provide the surface coverage necessary to perform feature inspection on one or more target objects in a cluttered scene. This is accomplished by a group of cooperating intelligent sensors. In this system, the sensors are mobile, the target objects are stationary and each agent controls the position of a sensor and has the ability to communicate with other agents in the environment. By communicating desires and intentions, each agent develops a mental model of the other agents' preferences, which is used to avoid or resolve conflict situations. In this paper we utilize cameras as the sensors. The experimental results illustrate the feasibility of the autonomous deployment of the sensors and that this deployment can occur with sufficient accuracy as to allow the inspection task to be performed.

We report on a fuzzy logic-based language understanding system applied to speech recognition. This system acquires conceptual knowledge from corpus data and organizes such knowledge into fuzzy logic inference rules. The system parses speech recognition results into conceptual structures in a robust manner, and thus is able to tolerate noise caused by speech recognition errors. We discuss the fuzzy inference rule learning method and explain its organization. Experimental results that demonstrate the ability of the system to deal with complex speech input instances are reported.

Document clustering techniques mostly rely on single term analysis of the document data set, such as the vector space model. To better capture the structure of documents, the underlying data model should be able to represent the phrases in the document as well as single terms. We present a novel data model, the document index graph, which indexes web documents based on phrases, rather than single terms only. The semi-structured web documents help in identifying potential phrases that when matched with other documents indicate strong similarity between the documents. The document index graph captures this information, and finding significant matching phrases between documents becomes easy and efficient with such model. The similarity between documents is based on both single term weights and matching phrases weights. The combined similarities are used with standard document clustering techniques to test their effect on the clustering quality. Experimental results show that our phrase-based similarity, combined with single-term similarity measures, enhances web document clustering quality significantly.

Binarization of document images with poor contrast, strong noise complex patterns and variable modalities in the gray-scale histograms is a challenging problem. We present a binarization algorithm based on an image dependent model to address this problem for a cheque processing application. The proposed algorithm seeks an optimal threshold that would eliminate the background noise, while preserving as much character stroke data as possible. The strategy is based on the use of information extracted from one clean part of the image, referred to as the "model" sub-image, to optimize the binarization in another problematic part of the image, referred to as the "target" sub-image. Experiments with 4200 cheque images, provided by our industrial partner, showed significant improvement in the binarization quality in comparison with other well-established algorithms.

Image registration is the process of determining the transformation which best matches, according to some similarity measure, two images of the same scene taken at different times or from different view points. We propose a new image registration method based on the similarity measure called Hausdorff fraction and a novel set of features called virtual circles. This method is guaranteed to find the best homothetic transformation, if only two virtual circles are preserved between the model and the scene. Another advantage of this method is that it is a general method that works well for most types of images. The time complexity of this method is O(n/sup 2/ +nmE/sub m/), where n and m are the number of virtual circles in scene and model respectively, and E/sub m/ is the number of edge points in the model. However, using an heuristic called circularity criterion, the number of virtual circles can be reduced allowing for faster execution times, without much loss in robustness.

We present a framework for clustering and classifying cheque images according to their payee-line content. The features used in the clustering and classification processes are extracted from the wavelet domain by means of thresholding and counting of wavelet coefficients. The feasibility of this framework is tested on a database of 2620 cheque images. This database consists of cheques from 10 different accounts. Each account is written by a different person. Clustering and classification are performed separately on each account using distance-based techniques. We achieved correct-classification rates of 86% and 81% for the supervised and unsupervised learning cases, respectively. These rates are the average of correct-classification rates obtained from the 10 different accounts.

The process of CAD model acquisition from existing objects is desirable in many industrial applications. An automated range image segmentation module is an essential building block and is key to speeding up this process. This paper presents a segmentation method based on local approximation of scan lines and uses adequate edge models to detect noise pixels as well as position and orientation discontinuities. This is followed by an adaptive grouping process to find a geometric representation of the different surface regions of the object. The output of the segmentation module is then used to automatically generate a surface CAD model of the scene. Experimental results on a large number of real range images demonstrate the efficiency and robustness of the method.

In a classification problem, improved accuracy can be obtained in many situations by using the combination of several classifiers instead of a single one. Turner and Gosh (1999) derived the error reduction that can be obtained by combining unbiased classifiers with independent errors using a simple average. We present an extension of this result by finding the improvement obtained when combining classifiers using weighted average. We also prove that for unbiased classifiers with independent errors the best combination of N classifiers corresponds to a weighted average, where the combination coefficient of each classifier is equal to 1/N. This means that in these cases the simple average should be used. We present experiments illustrating our results.

An approach to reducing the computation time taken by neural nets for the searching process is introduced. We combine both fast and cooperative modular neural nets to enhance the performance of the detection process. Such an approach is applied to identify human faces automatically in cluttered scenes. In the detection phase, neural nets are used to test whether a window of 20/spl times/20 pixels contains a face or not. The major difficulty in the learning process comes from the large database required for face/nonface images. A simple design for cooperative modular neural nets is presented to solve this problem by dividing these data into three groups. Such division results in reduction of computational complexity and thus decreasing the time and memory needed during the test of an image. Simulation results for the proposed algorithm show a good performance.

In this paper, a new approach to reduce the computation time taken by neural nets for the searching process is introduced. We combine both fast and cooperative modular neural nets to enhance the performance of the detection process. Such an approach is applied to identify human faces automatically in cluttered scenes. In the detection phase, neural nets are used to test whether a window of 20/spl times/20 pixels contains a face or not. The major difficulty in the learning process comes from the large database required for face/nonface images. A simple design for cooperative modular neural nets is presented to solve this problem by dividing these data into three groups. Such division results in reduction of computational complexity and thus decreasing the time and memory needed during the test of an image. Simulation results for the proposed algorithm show a good performance.

Automatic recognition of individuals is a significant problem in the development of pattern recognition. In this paper, we introduce a simple technique for personal identification through human faces in cluttered scenes based on neural nets. In the detection phase, neural nets are used to test whether a window of 20/spl times/20 pixels contains a face or not. A major difficulty in learning process comes from the large database required for face/nonface images. We solve this problem by dividing these data into two groups. Such division results in a reduction of computational complexity and thus decreasing the time and memory needed during the test of an image. For the recognition phase, feature measurements are made through Fourier descriptors. Such a feature is modified to reduce the number of neurons in the hidden layer. Simulation results for the proposed algorithm show a good performance compared with previous results.

In this paper, we introduce a powerful solution for complex problems which are required to be solved using neural nets. This is done by using modular neural nets (MNNs) that divide the input space into several homogenous regions. Such an approach is applied to implement XOR functions, 16 logic functions on one bit level, and 2-bit digital multiplier. Compared to previous non-modular designs, a salient reduction in the order of computations and hardware requirements is obtained.

The modular neural network offers several advantages over classical non-modular neural network approaches to complex pattern classification problems. However, the accuracy of the modular approach depends greatly on the accurate fusion of the individual classification decisions. The paper presents a method for improving the overall accuracy of modular neural networks by incorporating an adaptive decision fusion mechanism. The proposed algorithm offers significant improvement over typical modular networks by evolving a more informed decision fusion mechanism that can greatly improve the final classification decision for complex classification tasks.

Combining decisions from several classifiers can be used to improve on the results of handwritten characters recognition. There are different methods to combine these decisions, most of which are static. We present an architecture that integrates learning into the voting scheme used to aggregate individual decisions. The focus of the work is to make the decision fusion a more adaptive process. This approach makes use of feature detectors responsible for gathering information about the input to perform adaptive decision aggregation. The approach is tested on handwritten Arabic character recognition. The results showed an improvement over any individual classifier, as well as different static classifier combining schemes.

Modular Neural Networks (MNNs) is a rapidly growing field in artificial Neural Networks (NNs) research. This paper surveys the different motivations for creating MNNs: biological, psychological, hardware, and computational. Then, the general stages of MNN design are outlined and surveyed as well, viz., task decomposition techniques, learning schemes and multi-module decision-making strategies. Advantages and disadvantages of the surveyed methods are pointed out, and an assessment with respect to practical potential is provided. Finally, some general recommendations for future designs are presented.

A genetic algorithm is developed to find the ridges in paper fingerprints. It is based on the fact that the ridges of the fingerprints are parallel. When scanning the fingerprint, line by line, the ideal noise-free gray level distribution should yield lines of black and white. The widths of these lines are not constant. The proposed genetic algorithm generates black and white lines of different widths. The widths change until we get the best match with the original fingerprint.

This paper presents a new digital image compression scheme which exploits one of the human visual system properties-namely that of, recognizing images by their regions-to achieve high compression ratios. It also assigns a variable bit-count to each image region that is proportional to the amount of information the region conveys to the viewer. The new scheme copes with image non-stationarity by adaptively segmenting the image into variable-block sized regions, and classifying them into statistically and perceptually different classes. Then, blocks in each class are separately encoded. Based on extensive testing, the performance of the new scheme surpasses the performance of the JPEG standard and goes beyond its compression limits. In most test cases, the new compression scheme results in a maximum compression ratio that is at least twice of JPEG, while exhibiting lower objective and subjective image degradations. Moreover, the performance of the new block-based compression is comparable to the performance of the state-of-the-art wavelet-based compression technique and provides a good alternative when adaptability to image content is of interest.

The central issue in the design of multirobot systems is the coordination of the robots' behaviors. Traditionally, this has been done by hand-coding complex strategies. Recent work has focussed on how strategies can be learned, but many of these systems suffer from convergence, credit assignment and scalability problems. This paper proposes a new approach for learning multirobot coordination strategies that addresses these concerns. The effectiveness of the technique is demonstrated using the noncooperative prisoners' dilemma and the cooperative predator and prey domains.

This paper presents a method to reorganize rules in knowledge bases with the objective of improving their performance. Knowledge reorganization is achieved through the combination of rule compression and abstraction techniques. The effectiveness of this methodology is evaluated in terms of pattern matching activity and execution times using knowledge bases from several application areas.

An experimental approach to robotic grasping is presented. This approach is based on developing a generic representation of grasping rules, which allows learning them from experiments between the object and the robot. A modular connectionist design arranged in subsumption layers is used to provide a mapping between sensory inputs and robot actions. Reinforcement feedback is used to select between different grasping rules and to reduce the number of failed experiments. This is particularly critical for applications in the personal service robot environment. Simulated experiments on a 15-object database show that the system is capable of learning grasping rules for each object in a finite number of experiments as well as generalizing from experiments on one object to grasping from another.

In this correspondence, a modified version of Hunt's (1980) image model is used to interpret the distribution of image data vectors. The model suggests that the diagonal line of the coordinates system is a good approximation of the principal axis of the image data vector set. The validity of the model is supported by experiments. Following this suggestion, an effective nearest neighbor search method for vector quantization of image data is developed. The method is based on partitioning the vector space using hyperplanes which are perpendicular to the diagonal direction of the coordinate system. The validity of the method is assessed by analyzing its complexity and comparing its performance to those of existing algorithms on a number of images.

Reasoning tasks such as simulation and planning involve deriving behavior of a system from a model of the system. The information needed to solve such problems can be represented as model behavior pairs (MBPs). The problem can be stated as one or more incomplete MBPs. The problem-solving method can be expressed as a sequence of MBP completions and comparisons. A language for representing and manipulating models, behaviors, and MBPs is presented. It is independent of any specific modeling domain. An important class of model transformation operators is the behavior-preserving model transformation operators. Because they preserve behavior, they can be used to simplify a model without compromising its value for problem solving. This sort of operator can speed up computations significantly. It can be used either to select an appropriate sub-model for a specific problem or to decompose a problem into a sequence of subproblems. A behavior-preserving pruning operator is presented and shown to work in three modeling domains: discrete event simulation (DES), planning, and qualitative physics (QP). The significance of this work lies in the domain independence of the language and operators. It provides a representation midway between the computer-oriented concepts of programming languages (and knowledge representation schemes) and the problem oriented concepts of the real world. The benefits that can result from such a representation are easy mapping of problem-to-solution method, easy communication between solution methods (when more than one reasoning technique is required to solve a problem) and efficient solution of problems.

In this paper we present an experimental approach to robotic grasping that is based on mapping grasping rules to a generic representation that can then be learned by experiments. Furthermore, grasping rules acquired in this format can then be used on different objects using different grippers. During experimentation, reinforcement learning is used to minimize the number of failed experiments. Results show that the system is able to learn how to grasp various objects while maintaining a small number of experiments.

This paper is concerned with segmenting light intensity images for the sake of compressing them using lossy compression techniques. Among the most commonly used techniques for image segmentation is quad-tree partitioning. In this technique, block variance based criteria are usually used to measure the smoothness of the segmented blocks and to consequently classify them. Block variance, however, does not consider the pixel value distribution within the block. Instead of using the block variance as a segmentation and classification measure, we propose using the mean squared deviation from the neighboring pixels mean. The proposed measure is capable of differentiating between blocks not only according to block pixel values but also according to their distribution within the block. This leads to a much better image segmentation and consequently to higher image compression ratios with lower image degradation. The results show the superiority of the proposed measure over the block variance measure.

Repertory grid technique plays a central role in the elicitation methodology of many well-reported knowledge acquisition tools or workbenches. However, the dependability of these systems is low where the technique breaks down or proves inadequate due to limited expressive power and other problems. The paper introduces an alternate approach based on Personal Construct Theory that elicits an expert's knowledge as a network of terms that constitutes a propositional formalism. An extended example is used to both highlight the difficulties encountered using repertory grids and illustrate how these are overcome using the proposed approach. The results of an empirical study are presented where an experienced clinician compared the knowledge structures that she constructed for a diagnostic task using each elicitation technique. Furthermore, although the network representation is amenable to inductive learning methods for generating production rules, an inference method is demonstrated which reveals the formalism's categorical reasoning potential. The authors conclude that it is more appropriate to classify such methods as either mediating or immediate rather than the knowledge structures they employ. The paper contributes to a better understanding of constructivist formalisms developed for knowledge acquisition.>

This paper describes a three layer neural network for the reconstruction of images from their contour data. To avoid a large number of nodes within the defined neural network, we are applying, recursively, a quadrature segmentation of the input contour map image to obtain smaller adjacent regions having a number of points less than or equal to a specified maximum. The contour points in the end regions are then used to train a three layer back-propagation neural network to be used for reconstructing an approximation of the original image. It is shown that the neural network has a better performance than other available classical algorithms.>

ODEXPERT is a prototype knowledge-based system which selects the appropriate numerical solvers for initial value ordinary differential equations. It is capable of deriving some knowledge about the input problem by performing automated tests to detect properties and structures in the problem which guide the selection process.

Presents two new methods for the best codeword searching based on different space partitioning strategies: the concentric hypersphere partitioning nearest neighbor search and the equal-average hyperplane partitioning nearest neighbor search.>

Deals with the extraction of binary character/graphics images from grayscale document images with background pictures, shadows, highlight, smear and smudge. The authors review four published extraction techniques, and present a new technique. They then propose speed, memory requirement, stroke width restriction, parameter number, parameter setting and human subjective evaluation of result images as six aspects for evaluating and analysing extraction techniques. The result of systematically evaluating and analyzing both new and published techniques with experiments on images of typical check images and poor-quality text documents with respect to these six aspects is reported.>

A representation for uncertain states within the context of robot task planning is described. This includes the probabilistic description of objects, relationships between objects, and operators. It is one component of a task planner designed for use in uncertain task domains. The planner is based on a game theoretic-approach which views the environment as an opponent (often fallible). Multiple outcomes arising from operator application in an uncertain world are attributed to moves of the opponent. These are called reaction operators. Their representation and effect on the planning process are discussed.>

article Free Access Share on Automatic Partitioning of Stiff Systems and Exploiting the Resulting Structure Authors: W. H. Enright Department of Computer Science, University of Toronto, Toronto, Ont., Canada M5S 1A7 Department of Computer Science, University of Toronto, Toronto, Ont., Canada M5S 1A7View Profile , M. S. Kamel Department of Computer Science, University of Toronto, Toronto, Ont., Canada M5S 1A7 Department of Computer Science, University of Toronto, Toronto, Ont., Canada M5S 1A7View Profile Authors Info & Claims ACM Transactions on Mathematical SoftwareVolume 5Issue 4Dec. 1979 pp 374–385https://doi.org/10.1145/355853.355855Published:01 December 1979Publication History 36citation429DownloadsMetricsTotal Citations36Total Downloads429Last 12 Months23Last 6 weeks7 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF