This paper introduces a learning algorithm that can be used for training reformulated radial basis function neural networks (RBFNNs) capable of identifying uncertainty in data classification. This learning algorithm trains a special class of reformulated RBFNNs, known as cosine RBFNNs, by updating selected adjustable parameters to minimize the class-conditional variances at the outputs of their radial basis functions (RBFs). The experiments verify that quantum neural networks (QNNs) and cosine RBFNNs trained by the proposed learning algorithm are capable of identifying uncertainty in data classification, a property that is not shared by cosine RBFNNs trained by the original learning algorithm and conventional feed-forward neural networks (FFNNs). Finally, this study leads to a simple classification strategy that can be used to improve the classification accuracy of QNNs and cosine RBFNNs by rejecting ambiguous feature vectors based on their responses.

This article presents the results of a study aimed at the development of a system for short-term electric power load forecasting. This was attempted by training feedforward neural networks (FFNNs) and cosine radial basis function (RBF) neural networks to predict future power demand based on past power load data and weather conditions. This study indicates that both neural network models exhibit comparable performance when tested on the training data but cosine RBF neural networks generalize better since they outperform considerably FFNNs when tested on the testing data. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 591–605, 2005.

This paper proposes a framework for training feedforward neural network models capable of handling class overlap and imbalance by minimizing an error function that compensates for such imperfections of the training set. A special case of the proposed error function can be used for training variance-controlled neural networks (VCNNs), which are developed to handle class overlap by minimizing an error function involving the class-specific variance (CSV) computed at their outputs. Another special case of the proposed error function can be used for training class-balancing neural networks (CBNNs), which are developed to handle class imbalance by relying on class-specific correction (CSC). VCNNs and CBNNs are compared with conventional feedforward neural networks (FFNNs), quantum neural networks (QNNs), and resampling techniques. The properties of VCNNs and CBNNs are illustrated by experiments on artificial data. Various experiments involving real-world data reveal the advantages offered by VCNNs and CBNNs in the presence of class overlap and class imbalance.

This paper presents adaptive multiuser detectors for code-division multiple-access (CDMA) signals in wireless communication systems. Directly estimated adaptive (DEA) detectors are developed by formulating CDMA detection as an inverse problem in the presence of channel-induced interference and noise. The detector parameters are computed by a fully sequential adaptive algorithm that requires no matrix inversion, and can be implemented online as the data arrive at the receiver. The proposed DEA detector is experimentally evaluated in terms of its robustness to noise, resistance to the near-far problem, and ability to handle multipath fading signals. This experimental study indicates that the proposed DEA detector requires shorter training sequences of bits to achieve the performance levels attained by existing adaptive implementations of the minimum mean-square error detector.

This paper presents the development of regularized optical flow computation methods and an evaluation of their performance in the extraction of quantitative motion information from video recordings of neonatal seizures. A general formulation of optical flow computation is presented and a mathematical framework for the development of practical tools for computing optical flow is outlined. In addition, this paper proposes an alternative formulation of the optical flow problem that relies on a discrete approximation of a family of quadratic functionals. These regularized optical flow computation methods are used to extract motion strength signals from video recordings of neonatal seizures.

This paper presents the development of soft clustering and learning vector quantization (LVQ) algorithms that rely on a weighted norm to measure the distance between the feature vectors and their prototypes. The development of LVQ and clustering algorithms is based on the minimization of a reformulation function under the constraint that the generalized mean of the norm weights be constant. According to the proposed formulation, the norm weights can be computed from the data in an iterative fashion together with the prototypes. An error analysis provides some guidelines for selecting the parameter involved in the definition of the generalized mean in terms of the feature variances. The algorithms produced from this formulation are easy to implement and they are almost as fast as clustering algorithms relying on the Euclidean norm. An experimental evaluation on four data sets indicates that the proposed algorithms outperform consistently clustering algorithms relying on the Euclidean norm and they are strong competitors to non-Euclidean algorithms which are computationally more demanding.

Abstract: This paper presents the results of a study on short‐term electric power load forecasting based on feedforward neural networks. The study investigates the design components that are critical in power load forecasting, which include the selection of the inputs and outputs from the data, the formation of the training and the testing sets, and the performance of the neural network models trained to forecast power load for the next hour and the next day. The experiments are used to identify the combination of the most significant parameters that can be used to form the inputs of the neural networks in order to reduce the prediction error. The prediction error is also reduced by predicting the difference between the power load of the next hour (day) and that of the present hour (day). This is a promising alternative to the commonly used approach of predicting the actual power load. The potential of the proposed method is revealed by its comparison with two existing approaches that utilize neural networks for electric power load forecasting.

The paper presents a procedure developed to extract quantitative information from video recordings of neonatal seizures in the form of temporal motor activity signals. The motor activity signals are extracted by tracking selected anatomical sites during the seizure using adaptive block matching. The motion of a block of pixels is quantified by searching for the most similar block of pixels in subsequent frames; this search is facilitated by employing various update strategies to account for the changing appearance of the block. Experiments indicate that the temporal motor activity signals extracted by the proposed procedure constitute an effective representation of videotaped clinical events and can be used for seizure recognition and characterization.

The paper presents a procedure developed to extract quantitative information from video recordings of neonatal seizures in the form of temporal motion strength signals. These signals are obtained by applying nonlinear filtering, segmentation, and morphological filtering on the differences between adjacent frames. The experiments indicate that temporal motion strength signals constitute an effective representation of videotaped clinical events and can be used for seizure recognition and characterization.

This paper presents the results of a study aimed at the development of a system for short-term electric power load forecasting. This was attempted by training feedforward neural networks (FFNNs) and cosine radial basis function (RBF) neural networks to predict future power demand based on past power load data and weather conditions. This comparison indicates that both neural network models exhibit comparable performance when tested on the training data but cosine RBF neural networks generalize better since they outperform considerably FFNNs on the testing data.

This paper presents the results of a study that compared the performance of an entropy-constrained routing algorithm with that of routing methods based on neural optimization. The entropy-constrained algorithm was developed to allow multiple nodes of a communication network to compete for each position of a route and it is implemented as a deterministic annealing process. Routing is also performed by the routron and by a routing method relying on the Hopfield-Tank approach to optimization. This experimental study reveals the superiority of the entropy-constrained routing algorithm, which produces consistently the best routes in a small fraction of the time required for route discovery by the neural optimization methods.

This paper introduces variance-controlled neural networks (VCCNs), which were developed for handling class overlap. VCNNs have the same architecture as conventional feedforward neural networks; however, their training relies on a different error function that involves variances of the network outputs. The proposed approach is benchmarked against two statistical methods, conventional feedforward neural networks, and quantum neural networks for the removal of bird-contaminated data recorded by a 1290 MHz wind profiler. The experiments indicate that VCNNs are more reliable for handling the ambiguous data involved in this application compared with the statistical methods, conventional feedforward neural networks, or quantum neural networks.

This paper introduces a new family of reformulated radial basis function (RBF) neural networks, which are referred to as cosine RBFs. These RBF models are developed by relaying upon an axiomatic approach proposed for constructing reformulated RBF neural networks suitable for gradient descent learning. This approach reduces the search for RBF models to the selection of admissible generator functions. Cosine RBF models are constructed by linear generator functions of a special form. A set of experiments on two data sets indicate that cosine RBFs outperform considerably conventional RBF neural networks with Gaussian radial basis functions. Cosine RBFs are also strong competitors to existing reformulated RBF models trained by gradient descent and conventional feedforward neural networks with sigmoid hidden units.

This article presents a new family of reformulated radial basis function (RBF) neural networks that employ adjustable weighted norms to measure the distance between the training vectors and the centers of the radial basis functions. The reformulated RBF model introduced in this article incorporates norm weights that can be updated during learning to facilitate the implementation of the desired input-output mapping. Experiments involving classification and function approximation tasks verify that the proposed RBF neural networks outperform conventional RBF neural networks and reformulated RBF neural networks employing fixed Euclidean norms. Reformulated RBF neural networks with adjustable weighted norms are also strong competitors to conventional feedforward neural networks in terms of performance, implementation simplicity, and training speed. © 2003 Wiley Periodicals, Inc.

This paper presents the development of soft clustering and learning vector quantization (LVQ) algorithms that rely on multiple weighted norms to measure the distance between the feature vectors and their prototypes. Clustering and LVQ are formulated in this paper as the minimization of a reformulation function that employs distinct weighted norms to measure the distance between each of the prototypes and the feature vectors under a set of equality constraints imposed on the weight matrices. Fuzzy LVQ and clustering algorithms are obtained as special cases of the proposed formulation. The resulting clustering algorithm is evaluated and benchmarked on three data sets that differ in terms of the data structure and the dimensionality of the feature vectors. This experimental evaluation indicates that the proposed multinorm algorithm outperforms algorithms employing the Euclidean norm as well as existing clustering algorithms employing weighted norms.

Presents a systematic approach for constructing reformulated radial basis function (RBF) neural networks, which was developed to facilitate their training by supervised learning algorithms based on gradient descent. This approach reduces the construction of radial basis function models to the selection of admissible generator functions. The selection of generator functions relies on the concept of the blind spot, which is introduced in the paper. The paper also introduces a new family of reformulated radial basis function neural networks, which are referred to as cosine radial basis functions. Cosine radial basis functions are constructed by linear generator functions of a special form and their use as similarity measures in radial basis function models is justified by their geometric interpretation. A set of experiments on a variety of datasets indicate that cosine radial basis functions outperform considerably conventional radial basis function neural networks with Gaussian radial basis functions. Cosine radial basis functions are also strong competitors to existing reformulated radial basis function models trained by gradient descent and feedforward neural networks with sigmoid hidden units.

This paper presents a variable bit-rate encoding scheme developed for digital video compression based on spatiotemporal decomposition and vector quantization. This encoding scheme exploits the interframe redundancy present in image sequences by employing replenishment of code-vector labels. The experiments indicated that code-vector label replenishment results in a significant reduction of the number of bits required for encoding the frames of image sequences. The proposed video-compression system outperformed the H.261 video-compression standard at high compression ratios, and produced reconstructed video of comparable visual quality at medium and low compression ratios.

This paper presents two methods developed to extract quantitative information from video recordings of neonatal seizures in the form of temporal motion strength and motor activity signals. Motion strength signals are extracted by measuring the area of the body parts that move during the seizure and the relative speed of motion using a combination of spatiotemporal subband decomposition of video, nonlinear filtering, and segmentation. Motor activity signals are extracted by tracking selected anatomical sites during the seizure using a modified version of a feature-tracking procedure developed for video, known as the Kanade-Lucas-Tomasi (KLT) algorithm. The experiments indicate that the temporal signals produced by the proposed methods provide the basis for differentiating myoclonic from focal clonic seizures and distinguishing these types of neonatal seizures from normal infant behaviors.

Builds upon an axiomatic approach proposed for constructing reformulated radial basis function (RBF) neural networks suitable for gradient descent learning. This approach reduces the construction of RBF models to the selection of admissible generator functions. The selection of generator functions relies on criteria resulting from the analysis of the sensitivity of reformulated RBF models to gradient descent learning. The results of the study outlined in the paper are verified by a series of experiments on speech data.

Introduces reformulated radial basis function (RBF) neural networks with adjustable weighted norms. This work extends and improves an axiomatic approach that reduced the construction of RBF models to the selection of admissible generator functions. The RBF models proposed in this paper are constructed by linear generator functions and employ weighted norms that can be updated during learning to facilitate the implementation of the desired input-output mapping. Experiments on speech data verify that the proposed RBF models outperform conventional RBF neural networks with Gaussian radial basis functions and reformulated RBF neural networks constructed by linear generator functions and employing fixed Euclidean norms.

Presents the development of a neural-network-based bird removal approach for wind profiler data. Bird removal was attempted by training traditional feedforward neural networks (FFNNs) and quantum neural networks (QNNs) to identify and remove bird-contaminated data recorded by a 1290 MHz wind profiler. A series of experiments evaluated several sets of features extracted from wind profiler data, various FFNNs and QNNs of different sizes, and criteria employed for identifying birds in wind profiler data based on the outputs of the trained neural networks.

This paper presents the NEURO-BRA, a neural-network-based bird removal approach for wind profiler data. The NEURO-BRA was developed by training quantum neural networks to identify and remove bird-contaminated data recorded by a 1290-MHz wind profiler using a set of input features computed from the wind profiler measurements. This experimental investigation indicated that the NEURO-BRA was capable of removing over 90% of the bird-contaminated data recorded by the 1290-MHz wind profiler.

This paper presents the development and investigates the properties of ordered weighted learning vector quantization (LVQ) and clustering algorithms. These algorithms are developed by using gradient descent to minimize reformulation functions based on aggregation operators. An axiomatic approach provides conditions for selecting aggregation operators that lead to admissible reformulation functions. Minimization of admissible reformulation functions based on ordered weighted aggregation operators produces a family of soft LVQ and clustering algorithms, which includes fuzzy LVQ and clustering algorithms as special cases. The proposed LVQ and clustering algorithms are used to perform segmentation of magnetic resonance (MR) images of the brain. The diagnostic value of the segmented MR images provides the basis for evaluating a variety of ordered weighted LVQ and clustering algorithms.

This paper proposes a framework for developing a broad variety of soft clustering and learning vector quantization (LVQ) algorithms based on gradient descent minimization of a reformulation function. According to the proposed axiomatic approach to le

This paper evaluates a segmentation technique for magnetic resonance (MR) images of the brain based on fuzzy algorithms for learning vector quantization (FALVQ). These algorithms perform vector quantization by updating all prototypes of a competitive network through an unsupervised learning process. Segmentation of MR images is formulated as an unsupervised vector quantization process, where the local values of different relaxation parameters form the feature vectors which are represented by a relatively small set of prototypes. The experiments evaluate a variety of FALVQ algorithms in terms of their ability to identify different tissues and discriminate between normal tissues and abnormalities.

This paper presents an axiomatic approach for constructing radial basis function (RBF) neural networks. This approach results in a broad variety of admissible RBF models, including those employing Gaussian RBF's. The form of the RBF's is determined by a generator function. New RBF models can be developed according to the proposed approach by selecting generator functions other than exponential ones, which lead to Gaussian RBF's. This paper also proposes a supervised learning algorithm based on gradient descent for training reformulated RBF neural networks constructed using the proposed approach. A sensitivity analysis of the proposed algorithm relates the properties of RBF's with the convergence of gradient descent learning. Experiments involving a variety of reformulated RBF networks generated by linear and exponential generator functions indicate that gradient descent learning is simple, easily implementable, and produces RBF networks that perform considerably better than conventional RBF models trained by existing algorithms.

This paper presents an axiomatic approach to soft learning vector quantization (LVQ) and clustering based on reformulation. The reformulation of the fuzzy c-means (FCM) algorithm provides the basis for reformulating entropy-constrained fuzzy clustering (ECFC) algorithms. This analysis indicates that minimization of admissible reformulation functions using gradient descent leads to a broad variety of soft learning vector quantization and clustering algorithms. According to the proposed approach, the development of specific algorithms reduces to the selection of a generator function. Linear generator functions lead to the FCM and fuzzy learning vector quantization (FLVQ) algorithms while exponential generator functions lead to ECFC and entropy-constrained learning vector quantization (ECLVQ) algorithms. The reformulation of LVQ and clustering algorithms also provides the basis for developing uncertainty measures that can identify feature vectors equidistant from all prototypes. These measures are employed by a procedure developed to make soft LVQ and clustering algorithms capable of identifying outliers in the data set. This procedure is evaluated by testing the algorithms generated by linear and exponential generator functions on speech data.

This work evaluates the performance of an image compression system based on wavelet-based subband decomposition and vector quantization. The images are decomposed using wavelet filters into a set of subbands with different resolutions corresponding to different frequency bands. The resulting subbands are vector quantized using the Linde-Buzo-Gray (LBG) algorithm and various fuzzy algorithms for learning vector quantization (FALVQ). These algorithms perform vector quantization by updating all prototypes of a competitive neural network through an unsupervised learning process. The quality of the multiresolution codebooks designed by these algorithms is measured on the reconstructed images belonging to the training set used for multiresolution codebook design and the reconstructed images from a testing set.

This paper presents an approach for detecting microcalcifications in digital mammograms employing wavelet-based subband image decomposition. The microcalcifications appear in small clusters of few pixels with relatively high intensity compared with their neighboring pixels. These image features can be preserved by a detection system that employs a suitable image transform which can localize the signal characteristics in the original and the transform domain. Given that the microcalcifications correspond to high-frequency components of the image spectrum, detection of microcalcifications is achieved by decomposing the mammograms into different frequency subbands, suppressing the low-frequency subband, and, finally, reconstructing the mammogram from the subbands containing only high frequencies. Preliminary experiments indicate that further studies are needed to investigate the potential of wavelet-based subband image decomposition as a tool for detecting microcalcifications in digital mammograms.

This paper presents competitive neural trees (CNeT's) for pattern classification. The CNeT contains m-ary nodes and grows during learning by using inheritance to initialize new nodes. At the node level, the CNeT employs unsupervised competitive learning. The CNeT performs hierarchical clustering of the feature vectors presented to it as examples, while its growth is controlled by forward pruning. Because of the tree structure, the prototype in the CNeT close to any example can be determined by searching only a fraction of the tree. This paper introduces different search methods for the CNeT, which are utilized for training as well as for recall. The CNeT is evaluated and compared with existing classifiers on a variety of pattern classification problems.

Derives an interpretation for a family of competitive learning algorithms and investigates their relationship to fuzzy c-means and fuzzy learning vector quantization. These algorithms map a set of feature vectors into a set of prototypes associated with a competitive network that performs unsupervised learning. Derivation of the new algorithms is accomplished by minimizing an average generalized distance between the feature vectors and prototypes using gradient descent. A close relationship between the resulting algorithms and fuzzy c-means is revealed by investigating the functionals involved. It is also shown that the fuzzy c-means and fuzzy learning vector quantization algorithms are related to the proposed algorithms if the learning rate at each iteration is selected to satisfy a certain condition.

This paper presents a general methodology for the development of fuzzy algorithms for learning vector quantization (FALVQ). The design of specific FALVQ algorithms according to existing approaches reduces to the selection of the membership function assigned to the weight vectors of an LVQ competitive neural network, which represent the prototypes. The development of a broad variety of FALVQ algorithms can be accomplished by selecting the form of the interference function that determines the effect of the nonwinning prototypes on the attraction between the winning prototype and the input of the network. The proposed methodology provides the basis for extending the existing FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms. This paper also introduces two quantitative measures which establish a relationship between the formulation that led to FALVQ algorithms and the competition between the prototypes during the learning process. The proposed algorithms and competition measures are tested and evaluated using the IRIS data set. The significance of the proposed competition measure is illustrated using FALVQ algorithms to perform segmentation of magnetic resonance images of the brain.

This paper proposes a framework for constructing and training radial basis function (RBF) neural networks. The proposed growing radial basis function (GRBF) network begins with a small number of prototypes, which determine the locations of radial basis functions. In the process of training, the GRBF network gross by splitting one of the prototypes at each growing cycle. Two splitting criteria are proposed to determine which prototype to split in each growing cycle. The proposed hybrid learning scheme provides a framework for incorporating existing algorithms in the training of GRBF networks. These include unsupervised algorithms for clustering and learning vector quantization, as well as learning algorithms for training single-layer linear neural networks. A supervised learning scheme based on the minimization of the localized class-conditional variance is also proposed and tested. GRBF neural networks are evaluated and tested on a variety of data sets with very satisfactory results.

This paper introduces quantum neural networks (QNNs), a class of feedforward neural networks (FFNNs) inherently capable of estimating the structure of a feature space in the form of fuzzy sets. The hidden units of these networks develop quantized representations of the sample information provided by the training data set in various graded levels of certainty. Unlike other approaches attempting to merge fuzzy logic and neural networks, QNNs can be used in pattern classification problems without any restricting assumptions such as the availability of a priori knowledge or desired membership profile, convexity of classes, a limited number of classes, etc. Experimental results presented here show that QNNs are capable of recognizing structures in data, a property that conventional FFNNs with sigmoidal hidden units lack.

This paper presents an unsupervised learning scheme for initializing the internal representations of feedforward neural networks, which accelerates the convergence of supervised learning algorithms. It is proposed in this paper that the initial set of internal representations can be formed through a bottom-up unsupervised learning process applied before the top-down supervised training algorithm. The synaptic weights that connect the input of the network with the hidden units can be determined through linear or nonlinear variations of a generalized Hebbian learning rule, known as Oja's rule. Various generalized Hebbian rules were experimentally tested and evaluated in terms of their effect on the convergence of the supervised training process. Several experiments indicated that the use of the proposed initialization of the internal representations significantly improves the convergence of gradient-descent-based algorithms used to perform nontrivial training tasks. The improvement of the convergence becomes significant as the size and complexity of the training task increase.

First, we identify an algorithmic defect of the generalized learning vector quantization (GLVQ) scheme that causes it to behave erratically for a certain scaling of the input data. We show that GLVQ can behave incorrectly because its learning rates are reciprocally dependent on the sum of squares of distances from an input vector to the node weight vectors. Finally, we propose a new family of models-the GLVQ-F family-that remedies the problem. We derive competitive learning algorithms for each member of the GLVQ-F model and prove that they are invariant to all scalings of the data. We show that GLVQ-F offers a wide range of learning models since it reduces to LVQ as its weighting exponent (a parameter of the algorithm) approaches one from above. As this parameter increases, GLVQ-F then transitions to a model in which either all nodes may be excited according to their (inverse) distances from an input or in which the winner is excited while losers are penalized. And as this parameter increases without limit, GLVQ-F updates all nodes equally. We illustrate the failure of GLVQ and success of GLVQ-F with the IRIS data.

This paper presents the development of fuzzy algorithms for learning vector quantization (FALVQ). These algorithms are derived by minimizing the weighted sum of the squared Euclidean distances between an input vector, which represents a feature vector, and the weight vectors of a competitive learning vector quantization (LVQ) network, which represent the prototypes. This formulation leads to competitive algorithms, which allow each input vector to attract all prototypes. The strength of attraction between each input and the prototypes is determined by a set of membership functions, which can be selected on the basis of specific criteria. A gradient-descent-based learning rule is derived for a general class of admissible membership functions which satisfy certain properties. The FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms are developed by selecting admissible membership functions with different properties. The proposed algorithms are tested and evaluated using the IRIS data set. The efficiency of the proposed algorithms is also illustrated by their use in codebook design required for image compression based on vector quantization.

This paper presents the development and evaluation of fuzzy vector quantization algorithms. These algorithms are designed to achieve the quality of vector quantizers provided by sophisticated but computationally demanding approaches, while capturing the advantages of the frequently used in practice k-means algorithm, such as speed, simplicity, and conceptual appeal. The uncertainty typically associated with clustering tasks is formulated in this approach by allowing the assignment of each training vector to multiple clusters in the early stages of the iterative codebook design process. A training vector assignment strategy is also proposed for the transition from the fuzzy mode, where each training vector can be assigned to multiple clusters, to the crisp mode, where each training vector can be assigned to only one cluster. Such a strategy reduces the dependence of the resulting codebook on the random initial codebook selection. The resulting algorithms are used in image compression based on vector quantization. This application provides the basis for evaluating the computational efficiency of the proposed algorithms and comparing the quality of the resulting codebook design with that provided by competing techniques.

This paper presents the development of several efficient learning algorithms for neural networks (ELEANNE). The ELEANNE 1 and ELEANNE 2 are two recursive least-squares learning algorithms, proposed for training single-layered neural networks with analog output. This paper also proposes a new optimization strategy for training single-layered neural networks, which provides the basis for the development of a variety of efficient learning algorithms. This optimization strategy is the source of the ELEANNE 3, a second-order learning algorithm for training single-layered neural networks with binary output. A simplified version of this algorithm, called ELEANNE 4, is also derived on the basis of some simplifying but reasonable assumptions. The two algorithms developed for single-layered neural networks provide the basis for the derivation of ELEANNE 5 and ELEANNE 6, which are proposed for training multilayered neural networks with binary output. The ELEANNE 7 is an efficient algorithm developed for training multilayered neural networks with either binary or analog output.>