This paper proposes a new model of "novelty detection" for image sequence analysis using neural networks. This model uses the concept of artificially generated negative data to form closed decision boundaries using a multilayer perceptron. The neural network output is novelty filtered by thresholding the output of multiple networks (one per known class) to which the sample is input and clustered for determining which clusters represent novel classes. After labeling these novel clusters, new networks are trained on this data. We perform experiments with video-based image sequence data containing a number of novel classes. The performance of the novelty filter is evaluated using two performance metrics and we compare our proposed model on the basis of these with five baseline novelty detectors. We also discuss the results of retraining each model after novelty detection. On the basis of Chi-square performance metric, we prove at 5 percent significance level that our optimized novelty detector performs at the same level as an ideal novelty detector that does not make any mistakes.

The tracking of objects using distributed multiple sensors is an important field of work in the application areas of autonomous robotics, military applications, and mobile systems. In this survey, we review a number of computationally intelligent methods that are used for developing robust tracking schemes through sensor data fusion. The survey discusses the application of the various algorithms at different layers of the JDL model and highlights the weaknesses and strengths of the approaches in the context of different applications.

The main aim of this paper is to propose a novel set of metrics that measure the quality of the image enhancement of mammographic images in a computer-aided detection framework aimed at automatically finding masses using machine learning techniques. Our methodology includes a novel mechanism for the combination of the metrics proposed into a single quantitative measure. We have evaluated our methodology on 200 images from the publicly available digital database for screening mammograms. We show that the quantitative measures help us select the best suited image enhancement on a per mammogram basis, which improves the quality of subsequent image segmentation much better than using the same enhancement method for all mammograms.

For the purposes of forecasting (or classification) tasks neural networks (NNs) are typically trained with respect to Euclidean distance minimization. This is commonly the case irrespective of any other end user preferences. In a number of situations, most notably time series forecasting, users may have other objectives in addition to Euclidean distance minimization. Recent studies in the NN domain have confronted this problem by propagating a linear sum of errors. However this approach implicitly assumes a priori knowledge of the error surface defined by the problem, which, typically, is not the case. This study constructs a novel methodology for implementing multiobjective optimization within the evolutionary neural network (ENN) domain. This methodology enables the parallel evolution of a population of ENN models which exhibit estimated Pareto optimality with respect to multiple error measures. A new method is derived from this framework, the Pareto evolutionary neural network (Pareto-ENN). The Pareto-ENN evolves a population of models that may be heterogeneous in their topologies inputs and degree of connectivity, and maintains a set of the Pareto optimal ENNs that it discovers. New generalization methods to deal with the unique properties of multiobjective error minimization that are not apparent in the uni-objective case are presented and compared on synthetic data, with a novel method based on bootstrapping of the training data shown to significantly improve generalization ability. Finally experimental evidence is presented in this study demonstrating the general application potential of the framework by generating populations of ENNs for forecasting 37 different international stock indexes.

We present a new framework for novelty detection. The framework evaluates neural networks as adaptive classifiers that are capable of novelty detection and retraining on the basis of newly discovered information. We apply our newly developed model to the application area of object recognition in video. We detail the tools and methods needed for novelty detection such that data from unknown classes can be reliably rejected without any a priori knowledge of its characteristics. The rejected data is postprocessed to determine which samples can be manually labeled of a new type and used for retraining. We compare the proposed framework with other novelty detection methods and discuss the results of adaptive retraining of neural network to recognize further unseen data containing the newly added objects.

In this paper, we study two measures of classification complexity based on feature space partitioning: purity and neighborhood separability. The new measures of complexity are compared with probabilistic distance measures and a number of other nonparametric estimates of classification complexity on a total of 10 databases from the University of California, Irvine, (UCI) repository.

Multiobjective evolutionary algorithms (MOEAs) have been the subject of numerous studies over the past 20 years. Recent work has highlighted the use of an active archive of elite, nondominated solutions to improve the optimization speed of these algorithms. However, preserving all elite individuals is costly in time (due to the linear comparison with all archived solutions needed before a new solution can be inserted into the archive). Maintaining an elite population of a fixed maximum size (by clustering or other means) alleviates this problem, but can cause retreating (or oscillitory) and shrinking estimated Pareto fronts - which can affect the efficiency of the search process. New data structures are introduced to facilitate the use of an unconstrained elite archive, without the need for a linear comparison to the elite set for every new individual inserted. The general applicability of these data structures is shown by their use in an evolution-strategy-based MOEA and a genetic-algorithm-based MOEA. It is demonstrated that MOEAs using the new data structures run significantly faster than standard, unconstrained archive MOEAs, and result in estimated Pareto fronts significantly ahead of MOEAs using a constrained archive. It is also shown that the use of an unconstrained elite archive permits robust criteria for algorithm termination to be used, and that the use of the data structure can also be used to increase the speed of algorithms using /spl epsi/-dominance methods.

The class of geometric deformable models, also known as level sets, has brought tremendous impact to medical imagery due to its capability of topology preservation and fast shape recovery. In an effort to facilitate a clear and full understanding of these powerful state-of-the-art applied mathematical tools, this paper is an attempt to explore these geometric methods, their implementations and integration of regularizers to improve the robustness of these topologically independent propagating curves/surfaces. This paper first presents the origination of level sets, followed by the taxonomy of level sets. We then derive the fundamental equation of curve/surface evolution and zero-level curves/surfaces. The paper then focuses on the first core class of level sets, known as "level sets without regularizers." This class presents five prototypes: gradient, edge, area-minimization, curvature-dependent and application driven. The next section is devoted to second core class of level sets, known as "level sets with regularizers." In this class, we present four kinds: clustering-based, Bayesian bidirectional classifier-based, shape-based and coupled constrained-based. An entire section is dedicated to optimization and quantification techniques for shape recovery when used in the level set framework. Finally, the paper concludes with 22 general merits and four demerits on level sets and the future of level sets in medical image segmentation. We present applications of level sets to complex shapes like the human cortex acquired via MRI for neurological image analysis.

In this paper, the concept of long memory systems for forecasting is developed. The pattern modelling and recognition system and fuzzy single nearest neighbour methods are introduced as local approximation tools for forecasting. Such systems are used for matching the current state of the time-series with past states to make a forecast. In the past, the PMRS system has been successfully used for forecasting the Santa Fe competition data. In this paper, we forecast the FTSE 100 and 250 financial returns indices, as well as the stock returns of five FTSE 100 companies and compare the results of the two different systems with those of exponential smoothing and random walk on seven different error measures. The results show that pattern recognition based approaches in time-series forecasting are highly accurate. Simple theoretical trading strategies are also mentioned, highlighting real applications of the system.

Suspicious regions are identified following the bilateral image subtraction of left and right breast image pairs. The study uses the nipple as a common rotational point thereby facilitating an alignment with the highest correlation prior to subtraction. Within this study, 144 breast images from the MIAS database are considered. Five co-occurrence matrices are constructed at four different distances for each suspicious region. Twelve texture features defined by Haralick et. al. (1973) are considered. Two further features defined by Chan et. al (1997), inertia and difference average, are also computed giving a total of fourteen texture measures. Following classification of six principal components calculated for the extracted features using an artificial neural network and 10-fold cross-validation, an average recognition rate of 77% was achieved. Using the receiver operating characteristic analysis, the overall sensitivity of the technique measured by the value of Az, was found to be 0.74.

This paper develops a polygon feature selection method for the classification of temporal data from two or more sources on the basis of quantifying structural changes with time. The study focuses on the analysis of EEG data. The paper shows results on the feature classification using a modified fuzzy nearest neighbour method. The transformed inputs are ideally suited for the effective classification of EEG data. The results show that the developed polygon feature selection method can be used robustly in signal applications for source separation. Recognition rates vary for each EEG channel data between 90-99% correct recognition. It is expected that several applications including time-series analysis, signal processing and speech recognition will benefit from the findings in this paper.

This paper uses a modified Hough transform method to extract features from the cursive handwritten digit and characters in CEDAR database. The technique does not require the detection of complex structural primitives such as loops. The handwriting images are divided into uniform regions that are analysed for the presence of horizontal, vertical and diagonal segments. The total number of such segments found in these regions are used as input to a linear classifier (discriminant analysis) and a nonlinear classifier (nearest neighbour). The results are produced on the complete test sets specified in CEDAR database as well as a leave-one-out cross-validation is performed. On the digit data, the results show a recognition rate of around 94% correct recognition on the test set and 87.5% using the leave-one-out method. Character recognition results range between 67% correct on a test set and 64% connect using leave-one-out cross-validation.

In this paper we compare four classification techniques for classifying texture data of various natural objects found in forward-looking infrared (FLIR) images. The techniques compared include linear discriminant analysis, mean classifier and two different models of k-nearest neighbour methods. Hermite functions are used for texture feature extraction from segmented regions of interest in natural scenes taken as a video sequence. A total of 2680 samples for a total of twelve different classes are used for object recognition. The results on correctly identifying twelve natural objects in scenes are compared across the four classifiers on both unnormalised and normalised data. On unnormalised data, the average best recognition rate obtained using a ten fold cross-validation is 96.5%, and on unnormalised data it is 86.1% with a single nearest neighbour technique.

We study the identification of masses in digital mammograms using texture analysis. A number of texture measures are calculated for bilateral difference images showing regions of interest. The measurements are made on co-occurrence matrices in four different direction giving a total of seventy features. These features include the ones proposed by Haralick et al. (1973) and Chan et al. (1997). We study a total of 144 breast images from the MIAS database. The dimensionality of the dataset is reduced using principal components analysis (PCA), PCA components are classified using both multilayer perceptron networks using backpropagation (MLP) and radial basis functions based on Gaussian kernels (RBF). The two methods are compared on the same data across a ten fold cross-validation. The results are generated on the average recognition rate over these folds on correctly recognising masses and normal regions. Further analysis is based on the receiver operating characteristic (ROC) plots. The best results show recognition rates of 77% correct recognition and an area under the ROC curve value Az of 0.74.

The classification of helical structures is of fundamental importance in several domains including natural sciences, engineering and medicine. Helixes are complicated structures to separate due to their highly nonlinear temporal nature. Several studies have examined the famous two-spiral problem using several types of neural architectures. It has been observed that neural networks can be reliably tested on such benchmarks to get estimates of their true ability for application for the real world problems. Past experiences on the use of neural networks has shown that the raw helix data needs significant transformation before good results are possible using a neural network. In this paper, we consider the classification of two spirals in three dimension using standard neural networks. The input features are extracted from the helix data on its direction and distance between successive points. The experiments test the neural network solution on a range of helixes that differ from the helix learnt to various extents.

In this paper we apply artificial neural networks for classifying texture data of various natural objects found in FLIR images. Hermite functions are used for texture feature extraction from segmented regions of interest in natural scenes taken as a video sequence. A total of 2680 samples for a total of twelve different classes are used for object recognition. The results on correctly identifying twelve natural objects in scenes are compared across ten folds of the cross-validation study. Neural networks are found to be extremely effective in robust classification of our data giving an average recognition rate of 91.8%.

Time‐series prediction is important in physical and financial domains. Pattern recognition techniques for time‐series prediction are based on structural matching of the current state of the time‐series with previously occurring states in historical data for making predictions. This paper describes a Pattern Modelling and Recognition System (PMRS) which is used for forecasting benchmark series and the US S&P financial index. The main aim of this paper is to evaluate the performance of such a system on noise free and Gaussian additive noise injected time‐series. The results show that the addition of Gaussian noise leads to better forecasts. The results also show that the Gaussian noise standard deviation has an important effect on the PMRS performance. PMRS results are compared with the popular Exponential Smoothing method.

The study of time-dependent univariate systems plays an important role in several physical and applied sciences. Time-series behavior of such systems is mostly complex in nature and sophisticated mathematical modelling tools are needed for making accurate forecasts. These forecasts can be used for specific purposes in different domains, for example, to plan resources, develop market strategies, or control–understand complex systems. In the majority of successful applications of mathematical techniques used for forecasting, single point predictions are made. In this paper, a pattern recognition technique called the pattern modelling and recognition system (PMRS) is explored for making multiple forecasts into the future with four different time series. These multiple forecasts define a predicted behavioral profile of these univariate systems. These predicted profiles are compared against the actual behavior of the studied systems on a number of error measures. The results show that the structural primitives used for multiple forecasts are a very promising method of profiling the true behavior of univariate systems. © 2000 John Wiley & Sons, Inc.

The goal of character recognition research is to simplify and automate the development of character recognition algorithms. We describe an approach based on applying preprocessing to data sets of Latin characters and then applying a machine learning approach to the data sets to build a knowledge base able to classify unseen pre-processed characters. The machine learning method, Induct/RDR, has a number of features that make it particularly suitable for character recognition. It has the potential to integrate automatic analysis with a manual knowledge acquisition methodology if further refinement is required. Initial results on hand-printed Latin characters show the recognition accuracy of up to 90.2% on unseen cases for the machine learning system.

The main motivation for this research is three fold: a) temporal data classification is a challenging problem for pattern recognition tools and sophisticated tools are consistently sought to improve classifier performance on such type of data; b) traditional methods of treating temporal data often fail to adequately use temporal relationships between data points in their separation; and c) the development of neural tools based on new ideas capable of handling highly non-linear and noisy data will be of significant use in time-series, signal processing and speech applications. In this study we experiment with two types of data benchmarks: speech classification benchmark from NIST and 3D extension of the classical spiral benchmark from the Carnegie repository. It has been demonstrated in the past that ordinary neural network techniques for classification working on raw inputs of these benchmarks are inadequate. We propose a polygon method of temporal feature selection from time-dependent data and investigate neural network performance using a standard MLP architecture on such input.

The main aim of this paper is to introduce the single nearest neighbor approach for pattern recognition and the concept of incremental learning of a fuzzy classifier where decision making is based on data available up to time t rather than what may be available at the start of the trial, i.e. at t = 0. The single nearest neighbor method is explained in the context of solving the classic two-spiral benchmark. The proposed approach is further tested on the electronic nose coffee data to judge its performance on a real problem. This paper illustrates: (1) a novel fuzzy classifier system based on the single nearest neighbor method, (2) its application to the spiral benchmark taking the incremental pattern recognition approach, and (3) results obtained when solving the two-spiral problem with both nonincremental and incremental methods and coffee classification with the nonincremental method. The results show that incremental learning leads to improved recognition performance for spiral data and it is possible to study the behavioral characteristics of the classifier with possibility related parameters.

We describe a pattern recognition based tool for forecasting. We compare the results of forecasting with this tool against the exponential smoothing method on Santa Fe series data and US financial index. The results show that the pattern recognition based tool is highly accurate on standard error measures.

Pattern recognition techniques for time-series forecasting are beginning to be realised as an important tool for predicting chaotic behaviour of dynamic systems. In this paper we develop the concept of a pattern modelling and recognition system which is used for predicting future behaviour of time-series using local approximation. In this paper we compare this forecasting tool with neural networks. We also study the effect of noise filtering on the performance of the proposed system. Fourier analysis is used for noise-filtering the time-series. The results show that Fourier analysis is an important tool for improving the performance of the proposed forecasting system. The results are discussed on three benchmark series and the real US S&P financial index.

The recognition of Chinese characters has been an area of great interest for many years, and a large number of research papers and reports have already been published in this area. There are several major problems with Chinese character recognition: Chinese characters are distinct and ideographic, the character size is very large and many structurally similar characters exist in the character set. Thus, classification criteria are difficult to generate. This article presents a new technique for the recognition of hand-printed Chinese characters using statistical pattern classification. Conventional methods have relied on hand-constructed dictionaries which are tedious to construct, and difficult to make tolerant to variation in writing styles. The article also discusses Chinese character recognition using dominant point feature extraction, and statistical pattern classification. The system was tested with 500 characters (each character has 40 samples), and the rate of recognition obtained was 84.45%. This strongly supports the usefulness of the proposed measures for Chinese character classification.

In this paper the author describes a new method, called string distance measurement (SDM), for recognizing handwritten characters. The advantage of this technique is that it can be applied in a generic manner to different applications which involve shape recognition and may be successfully modified for individual applications. The technique is based on the measurement of gradient change. The technique is expected to perform better in uncertain and noisy environments compared to the existing methods. The paper describes the technique, and estimates the performance rates through a cross-validation study with neural networks using SDM pattern recognition.