A study of the memory characteristics of the brain and the computer prompts the creation of a new neuron architecture for neural computation. We hypothesize that neural responses resemble hysteresis loops. The upper and lower halves of the hysteresis loop are described by two sigmoids. Generalizing the two sigmoids to two families of curves accommodates loops of various sizes. This model, which we call the ;hystery model', is capable of memorizing the entire history of its bipolar inputs in an adaptive fashion, with larger memory for longer sequences. We theorize and prove that the hystery model's response converges asymptotically to hysteresis-like loops. A simple application to temporal pattern discrimination demonstrates the nonlinear short-term memory characteristics of the hystery model. This model may have important applications for time-based computations such as control, signal processing and spatiotemporal pattern recognition, especially if it can take advantage of existing hysteresis phenomena in semiconductor materials.

This paper describes a novel neural network architecture named ClusNet. This network is designed to study the trade-offs between the simplicity of instance-based methods and the accuracy of the more computational intensive learning methods. The features that make this network different from existing learning algorithms are outlined. A simple proof of convergence of the ClusNet algorithm is given. Experimental results showing the convergence of the algorithm on a specific problem is also presented. In this paper, ClusNet is applied to predict the temporal continuation of the Mackey-Glass chaotic time series. A comparison between the results obtained with ClusNet and other neural network algorithms is made. For example, ClusNet requires one-tenth the computing resources of the instance-based local linear method for this application while achieving comparable accuracy in this task. The sensitivity of ClusNet prediction accuracies on specific clustering algorithms is examined for an application. The simplicity and fast convergence of ClusNet makes it ideal as a rapid prototyping tool for applications where on-line learning is required.

A new approach for estimating classification errors is presented. In the model, there are two types of classification error: empirical and generalization error. The first is the error observed over the training samples, and the second is the discrepancy between the error probability and empirical error. In this research, the Vapnik and Chervonenkis dimension (VCdim) is used as a measure for classifier complexity. Based on this complexity measure, an estimate for generalization error is developed. An optimal classifier design criterion (the generalized minimum empirical error criterion (GMEE)) is used. The GMEE criterion consists of two terms: the empirical and the estimate of generalization error. As an application, the criterion is used to design the optimal neural network classifier. A corollary to the Gamma optimality of neural-network-based classifiers is proven. Thus, the approach provides a theoretic foundation for the connectionist approach to optimal classifier design. Experimental results to validate this approach.>

A new algorithm called the self-organizing neural network (SONN) is introduced. Its use is demonstrated in a system identification task. The algorithm constructs a network, chooses the node functions, and adjusts the weights. It is compared to the backpropagation algorithm in the identification of the chaotic time series. The results show that SONN constructs a simpler, more accurate model, requiring less training data and fewer epochs. The algorithm can also be applied as a classifier.

A nonlinear neural framework, called the generalized Hopfield network (GHN), is proposed, which is able to solve in a parallel distributed manner systems of nonlinear equations. The method is applied to the general nonlinear optimization problem. We demonstrate GHNs implementing the three most important optimization algorithms, namely the augmented Lagrangian, generalized reduced gradient, and successive quadratic programming methods. The study results in a dynamic view of the optimization problem and offers a straightforward model for the parallelization of the optimization computations, thus significantly extending the practical limits of problems that can be formulated as an optimization problem and that can gain from the introduction of nonlinearities in their structure (e.g., pattern recognition, supervised learning, and design of content-addressable memories).