A common approach to pattern classification problems is to train a bank of layered perceptrons or other classifiers by clustering the input training data and training each classifier with just the data from a specific cluster. There is no provision in such an approach, however, to assure the component layered perceptron is well suited to learn the training data cluster it is assigned. An alternate method of training, herein proposed, lets a layered perceptron in a classifier bank choose the cluster of inputs it processes on the basis of the perceptron's ability to successfully classify those inputs. During training, data is therefore processed only by the classifier in the bank that best classifies the data or, equivalently, to which the data is most receptive. This allows each classifier to learn a localized subset of data dictated by the classifier's own classification ability. Once each classifier in the bank is trained, a separate independent cluster pointer is trained to recognize to which cluster an input test pattern belongs. The cluster pointer is used in the test mode to identify which classifier in the bank will best classify the problem. The approach, also applicable to regression type problems, is illustrated through application on a simulated Gaussian data set and an active sonar test estimation problem. In both cases, the maximally receptive classifer/regression bank significantly outperforms a single layered perceptron trained on the same data.

We address the problem of determining the optimal number and placement of multistatic sonar sensors to achieve maximal coverage while minimizing the number of required sensors. The use of a computationally expensive environmentally-dependent acoustic model for sonar performance prediction precludes the direct application of population-based techniques. Instead, we present computationally-frugal methods based on particle swarm optimization (PSO). Comparison of the algorithms using a surrogate model for sonar performance prediction based on Cassini ovals against a reference set generated through brute force evaluation of all configurations is presented.

We study the problem of determining optimal pinging strategies in multistatic sonar systems with multiple sources. We are specifically investigating algorithms that determine optimal pinging strategies both for generalized search scenarios, and for holding confirmed target tracks with constraints related to maintaining search performance in the rest of the area. An important part of this work is the development of metrics to be used in the optimization procedures. For maintaining search coverage, we used a "probability of target presence" metric formulation. This formulation utilizes sonar performance prediction and a Bayesian update equation to incorporate negative information (i.e., searching an area but finding no targets). We also discuss strategies that can be used to increase the performance of a multistatic field, such as the use of bandwidth diversity

It is important for orthodontists to predict the treatment outcome prior to establishing a treatment plan. Many studies have been conducted to create a predictive model of the treatment outcome for different orthodontic disorders using traditional regression techniques. This paper investigates viability of applying artificial neural networks in constructing a model for prediction of treatment outcomes of patients with Class II malocclusion. We developed two models to assess the treatment success by estimating the value of the Peer Assessment Rating (PAR) index from initial orthodontic measurements. We evaluated the performance of the neural network models on 205 patients, and the results are compared with previous linear regression models.

Sequential particle swarm optimization (S-PSO) is a modification of PSO suitable for high-dimensional optimization problems. S-PSO iteratively optimizes the objective function over randomly selected subspaces of the parameter search space instead of the entire parameter space at once as in standard PSO. This approach is advantageous since fewer particles are needed to solve the lower-dimension subproblems. S-PSO is applied to the distributed sonar sensor placement problem, where not only the dimensionality issue arises, but also the computational complexity of the objective function increases with the problem size. Simulations show that S-PSO outperforms standard PSO both in terms of convergence and computational efficiency.

A technique for block-loss restoration in block-based image and video coding, dubbed recovery of image blocks using the method of alternating projections (RIBMAP), is developed. The algorithm is based on orthogonal projections onto constraint sets in a Hilbert space. For the recovery of a linear dimension N size block, a total of 8N vectors are extracted from the surrounding area of an N x N missing block. These vectors form a library from which the best matching spatial information for the missing block is extracted. Recovery vectors, including both undamaged and restored damaged pixels, are introduced. The vectors are used to find highly correlated information relating to the lost pixels. To assure continuity with the surrounding undamaged area, three additional convex constraints are formulated. Adherance to these sets is imposed using alternating projections. Simulation results using orthogonal projections demonstrate that RIBMAP recovers spatial structure faithfully. Simulation comparisons with other procedures are presented: Ancis and Giusto's hybrid edge-based average-median interpolation technique, Sun and Kwok's projections onto convex sets-based method, Hemami and Meng's interblock correlation interpolation approach, Shirani et al.'s modified interblock correlation interpolation scheme, and Alkachouh and Bellanger's fast discrete cosine transformation-based spatial domain interpolation algorithm. Characteristic of the results are those of the "Lena" JPEG image when one fourth of periodically spaced blocks in the image have errors. The peak signal-to-noise ratio of the restored image is 28.68, 29.99, 31.86, 31.69, 31.57, and 34.65 dB using that of Ancis and Giusto, Sun and Kwok, Hemami and Meng, Shirani et al., Alkachouh and Bellanger, and RIPMAP, respectively.

We consider the problem of minimum power bidirectional topology optimization in wireless networks with sectored antennas. We first develop a mixed integer linear programming model for optimal solution of the problem with sectored antennas. Subsequently, we discuss a centralized heuristic algorithm which is based on Kruskal's algorithm for the minimum spanning tree problem. However, unlike Kruskal's algorithm which chooses minimum cost edges from a set of edge weights, our algorithm uses an incremental cost mechanism to select edges. This incremental cost mechanism is motivated by the inherently broadcast nature of the wireless medium. We also discuss a simple branch exchange heuristic which can be used to improve the topology generated by the Kruskal-like algorithm. Simulation results indicate that reasonably good solutions can be obtained using the proposed heuristic algorithms.

In this paper the problem of minimum power multicasting in wireless networks with sectored antennas is considered. For omnidirectional antenna systems, a transmission from node i to node j will also reach all nodes which are closer to i than j. Depending on the network geometry, this strategy can be highly power efficient In ideal sectored antenna systems, however, this phenomenon is sector specific; i.e., only those nodes which are located in the same sector as j will receive the transmission implicitly. Though this might seem an apparent disadvantage, the higher gains associated with directional antennas (as opposed to omni-directional antennas) allow for reduced transmission powers without sacrificing the signal-to-interference ratio at the receiver. In this paper, we first develop a mixed integer linear programming model for optimal solution of the minimum power multicast problem with sectored antennas. Subsequently, we discuss a biologically inspired algorithm for solving the problem to near-optimality at a very reasonable computation time. Experiments on randomly generated 10, 20 and 30-node networks indicate that near-optimal solutions can be obtained using the proposed algorithm.

An unsupervised learning system, implemented as an autonomous agent is presented. A simulation of a challenging path planning problem is used to illustrate the agent design and demonstrate its problem solving ability. The agent, dubbed the ORG, employs fuzzy logic and clustering techniques to efficiently represent and retrieve knowledge and uses innovative sensor modeling and attention focus to process a large number of stimuli. Simple initial fuzzy rules (instincts) are used to influence behavior and communicate intent to the agent. Self-reflection is utilized so the agent can learn from its environmental constraints and modify its own state. Speculation is utilized in the simulated environment, to produce new rules and fine-tune performance and internal parameters. The ORG is released in a simulated shallow water environment where its mission is to dynamically and continuously plan a path to effectively cover a specified region in minimal time while simultaneously learning from its environment. Several paths of the agent design are shown, and desirable emergent behavior properties of the agent design are discussed.

Block loss and propagation error due to cell loss or missing packet information during the transmission over lossy networks can cause severe degradation of block and predictive-based video coding. Herein, new fast spatial and temporal methods are presented for block loss recovery. In the spatial algorithm, missing block recovery and edge extention are performed by pixel replacement based on range constraints imposed by surrounding neighborhood edge information and structure. In the temporal algorithm, an adaptive temporal correlation method is proposed for motion vector (MV) recovery. Parameters for the temporal correlation measurement are adaptively changed in accordance to surrounding edge information of a missing macroblock (MB). The temporal technique utilizes pixels in the reference frame as well as surrounding pixels of the lost block. Spatial motion compensation is applied after MV recovery when the reference frame does not have sufficient information for lost MB restoration. Simulations demonstrate that the proposed algorithms recover image information reliably using both spatial and temporal restoration. We compare the proposed algorithm with other procedures with consistently favorable results.

We consider the problem of maximizing the time-to-first-failure, defined as the time till the first node in the network runs out of battery energy, in energy constrained broadcast wireless networks. We discuss a greedy algorithm and prove that it solves the problem optimally for a broadcast application, in polynomial time, provided the complete power matrix and the battery residual capacities are known.

Broadcasting in wireless networks, unlike wired networks, inherently reaches several nodes with a single transmission. For omni-directional wireless broadcast to a node, all nodes closer will also be reached. This property can be used to compute routing trees which minimize the sum of the transmitter powers. It has been shown that this problem is NP-complete. In this paper, we present the r-shrink procedure, a heuristic for improving the solutions obtained using fast sub-optimal algorithms. Specifically, we focus on the low-complexity BIP algorithm and Prim's minimum spanning tree algorithm and show through extensive simulations that better solutions are obtained almost always, with considerably lower tree power, if the proposed procedure is used to improve the trees generated using these algorithms.

Preprocessing of data to be learned by a neural network is typically done to improve neural network performance. Output processing is especially important since it directly affects the influence of error in the hidden layers on the error of the neural network output. Principal component analysis is a commonly used preprocessing method that can improve the network performance by reducing the output dimensionality and reducing the number of parameters in a neural network model. Transforming the principal components of the outputs with an orthonormal matrix prior to scaling can further improve network performance.

Using array historical data, the readings from a sensor array may be shown to contain sufficient redundancy such that the readings from one or more lost sensors may be able to be accurately estimated from those remaining. This interdependency can be established by an neural network encoder. The encoder is also used in the restoration process. In this paper, we give some examples of sensor restoration for vibration sensors on jet engine and computer traffic data.

The neural network autoencoder is a useful tool for the restoration of missing sensors when enough known sensors with some relation to those missing are available. Through the idea of a contraction mapping, this paper provides some insight into the convergence of several iterative methods of sensor restoration using the autoencoder to some unique answer given a specific operating point (i.e., the known sensor values), regardless of how the missing sensor values are initialized.

Given a complicated and computationally intensive underwater acoustic model in which some acoustic measurement is a function of sonar system and environmental parameters, it is computationally beneficial to train a neural network to emulate the properties of that model. Given this neural network model, we now have a convenient means of performing geoacoustic inversion without the computational intensity required when attempting to do so with the actual model. This paper proposes an efficient and reliable method of performing the inversion of a neural network underwater acoustic model to obtain parameters pertaining to the characteristics of the ocean floor, using two different modified version of particle swarm optimization (PSO): two-step (gradient approximation) PSO and hierarchical cluster-based PSO.

Knowledge of the dynamic security border can provide an operator with valuable information on how to safely steer the power system away from vulnerable operating regions. The large set of non-linear differential equations that describe modern large-scale power systems makes it difficult to determine the security border either analytically or numerically in real time. As an alternative, neural networks trained off line on emulator data can provide a commensurate representation of the system transfer function, while significantly decreasing evaluation time. Using neural network inversion, sets of input points corresponding to a fixed output can be evaluated quickly. Different inversion procedures and their properties are reviewed here. We also review various metrics used for determining whether sufficient coverage of the border is achieved. Finally, we illustrate the use of border identification for preventive control when feature selection is initially performed to reduce the dimensionality of the input space.

There are many methods for performing neural network inversion. Multi-element evolutionary inversion procedures are capable of finding numerous inversion points simultaneously. Constrained neural network inversion requires that the inversion solution belong to one or more specified constraint sets. In many cases, iterating between the neural network inversion solution and the constraint set can successfully solve constrained inversion problems. This paper surveys existing methodologies for neural network inversion, which is illustrated by its use as a tool in query-based learning, sonar performance analysis, power system security assessment, control, and generation of codebook vectors.

This paper summarizes BC Hydro's experience in applying neural networks to dynamic security contingency screening and ranking. The idea is to use the information on the prevailing operating condition and directly provide contingency screening and ranking using a trained neural network. To train the two neural networks for the large scale systems of BC Hydro and Hydro Quebec, in total 1691 detailed transient stability simulation were conducted, 1158 for BC Hydro system and 533 for the Hydro Quebec system. The simulation program was equipped with the energy margin calculation module (second kick) to measure the energy margin in each run. The first set of results showed poor performance for the neural networks in assessing the dynamic security. However a number of corrective measures improved the results significantly. These corrective measures included: 1) the effectiveness of output; 2) the number of outputs; 3) the type of features (static versus dynamic); 4) the number of features; 5) system partitioning; and 6) the ratio of training samples to features. The final results obtained using the large scale systems of BC Hydro and Hydro Quebec demonstrates a good potential for neural network in dynamic security assessment contingency screening and ranking.

A training procedure that adapts the weights of a trained layered perceptron artificial neural network to training data originating from a slowly varying nonstationary process is proposed. The resulting adaptively trained neural network (ATNN), based on nonlinear programming techniques, is shown to adapt to new training data that are in conflict with earlier training data without affecting the neural networks' response to data elsewhere. The adaptive training procedure also allows for new data to be weighted in terms of its significance. The adaptive algorithm is applied to the problem of electric load forecasting and is shown to outperform the conventionally trained layered perceptron.

Multilayer perceptrons and trained classification trees are two very different techniques which have recently become popular. Giving enough data and time, both methods are capable of performing arbitrary nonlinear classification. The two techniques have not previously been compared on real-world problems. The authors first consider the important differences between multilayer perceptrons and classification trees and conclude that there is not enough theoretical basis for the clear-cut superiority of one technique over the other. They then present results of a number of empirical tests on quite different problems in power system load forecasting and speaker-independent vowel identification. They compare the performance for classification and prediction in terms of accuracy outside the training set. In all cases, even with various sizes of training sets, the multilayer perceptron performed as well as or better than the trained classification trees. The authors are confident that the univariate version of the trained classification trees do not perform as well as the multilayer perceptron. More studies are needed, however, on the comparative performance of the linear combination version of the classification trees.>