The failure of complex artificial intelligence (AI) systems seems ubiquitous. To provide a model to describe these shortcomings, we define complexity in terms of a system’s sensors and the number of environments or situations in which it performs. The complexity is not looked at in terms of the difficulty of design, but in the final performance of the system as a function of the sensor and environmental count. As the complexity of AI, or any system, increases linearly the contingencies increase exponentially and the number of possible design performances increases as a compound exponential. In this worst case scenario, the exponential increase in contingencies makes the assessment of all contingencies difficult and eventually impossible. As the contingencies grow large, unexpected and undesirable contingencies are all expected to increase in number. This, the worst case scenario, is highly connected, or conjunctive. Contingencies grow linearly with respect to complexity for systems loosely connected, or disjunctive. Mitigation of unexpected outcomes in either case can be accomplished using tools such as design expertise and iterative redesign informed by intelligent testing.

Neural networks have often been described as black boxes. A generic neural network trained to differentiate between kittens and puppies will classify a picture of a kumquat as a kitten or a puppy. An autoencoder watch dog screens trained classifier/regression machine input candidates before processing, e.g. to first test whether the neural network input is a puppy or a kitten. Preliminary results are presented using convolutional neural networks and convolutional autoencoder watchdogs using MNIST images.

Both Shannon and Kolmogorov–Chaitin–Solomonoff (KCS) information models fail to measure meaningful information in images. Pictures of a cow and correlated noise can both have the same Shannon and KCS information, but only the image of the cow has meaning. The application of ‘algorithmic specified complexity’ (ASC) to the problem of distinguishing random images, simple images and content‐filled images is explored. ASC is a model for measuring meaning using conditional KCS complexity. The ASC of various images given a context of a library of related images is calculated. The ‘portable network graphic’ (PNG) file format's compression is used to account for typical redundancies found in images. Images which containing content can thereby be distinguished from those containing simply redundancies, meaningless or random noise.

Algorithmic specified complexity (ASC) measures the degree to which an object is meaningful. Neither fundamental Shannon nor Kolmogorov information models are equipped to do so. ASC uses performance context in an information theoretic framework to measure the degree of specified complexity in bits. To illustrate, we apply ASC to Conway's Game of Life to differentiate patterns designed by programmers from those originating by chance. A variety of machines created by Game of Life hobbyists, as expected, exhibit high ASC thereby corroborating ASC's efficacy.

One of the benefits of using Combs disjunctive control in swarm intelligence is the ability of the swarm to continue to operate even when one or more of its sensors are broken. Instead of failing, the swarm continues to perform based on the inputs of the remaining sensors. In this technical correspondence, a multiobjective scenario is designed for a swarm of agents to complete. Once optimized, the swarm is modified by removing one of its sensors. Maintaining the original objective, the evolution process is continued to allow the swarm to compensate for its handicap. This technical correspondence describes the different emergent behaviors of the swarm to compensate for the loss of function. The solution for the diminished swarm can vary significantly from that of the original.

Non-contiguous OFDM is a promising technique for cognitive radio systems. Nevertheless, the sidelobes of the OFDM-modulated tones cause the out-of-band power (OBP) which can induce large interference to the incumbent communication systems. Another major drawback of OFDM-based systems is the high peak-to-average power ratio (PAPR). In this paper, two algorithms are proposed to jointly reduce the OBP and the PAPR based on the method of alternating projections onto convex sets (POCS). Several OFDM subcarriers are reserved to accommodate the adjusting weights that reduce the OBP and the PAPR. In the first algorithm, the adjusting weights are projected onto two convex sets that are defined according to the OBP and the PAPR limits. Each POCS iteration requires a convex optimization process. In the second algorithm, the frequency-domain OFDM symbol is projected onto multiple convex sets and the adjusting weights are separated into two groups for OBP and PAPR reductions, respectively. The adjusting weights can be obtained in analytical forms so that the computational complexity is lowered. Simulation results show that, within just a few POCS iterations, the proposed algorithms can achieve good performance in the joint reduction of the OBP and the PAPR.

The allocation of resources between tasks within a swarm of agents can be difficult without a centralized controller. Disjunctive control has been shown to be a viable method to control the behavior of a swarm. In this project, a disjunctive fuzzy control system is used to solve the problem of resource management. A multi-state swarm is evolved with an offline learning algorithm to adapt to a dynamic scenario with multiple objectives. Some of the emergent behaviors developed through the evolutionary algorithm were state-switching and recruitment techniques.

In a cognitive radio network, it is indispensable to assign common control channels for group operations of the secondary users of the spectrum. The assignment requires that multiple secondary users establish the least amount of frequency channels among them while each chooses a channel that has minimum interference to its nearby primary users. We model this problem as a strategic game and design its utility function such that the game is a potential game. A set of pure Nash equilibria are found by locating the local optima of the potential function. We develop sequential and asynchronous updates of game players' strategies using the best response dynamic. In order for the search to escape the local optimum and reach the global optimum of the potential function, we adopt simulated annealing in the sequential and asynchronous updates of the strategies. The optimal assignment of the common control channel is obtained accordingly and the convergence property is analyzed for these updating schemes.

Given simple agent rules, a swarm's emergent behavior can be difficult to predict. The inverse problem is even more difficult: Given a desired emergent behavior, what are the rules by which swarm agents should operate? Disjunctive fuzzy control is proposed as a method to model swarm agents. Compared to more commonly used conjunctive fuzzy control such as that proposed by Mamdani, disjunctive fuzzy control is robustly fault tolerant and disjointly connected. Swarms are inherently disjunctive. Instead of agents working in coordination with one another, each swarm agent contributes individually to the result. The disjunctive attribute can also be applied at the sensor level for each individual agent. Disjunctive control allows adaptation of the describing membership function, as is commonly done in conjunctive control. The inversion process is illustrated with numerous simulation examples, including a predator-prey game, gang warfare, and escaping agents. The swarm is instructed what to do but not how to do it. Imposition of fitness constraints and repeated generations of evolutionary molding of agent performance can then result in unexpected emergent behaviors of the swarm, e.g., use of decoys, self-sacrifice, flanking maneuvers, and shielding of the weak.

The dynamics of large decentralized groups of agents, or swarms, can be difficult to characterize due to complex and often unpredictable behaviors that arise from low-level interactions between agents. When designing multiagent systems, these emergent behaviors can have hidden and undesirable implications on the overall operation of the swarm. This paper examines the use of inversion of swarm dynamics to refine individual agents' rules of operation in order to achieve a given collective goal and applies this method to a scenario of tactical relevance: the point defense of a very important person between two attacking and defending swarms. An alternating competitive evolution is used in a toggled behavioral arms race in order to refine tactics and anticipate counteractions. Results include creative solutions with varying levels of success at addressing defensive tactical scenarios, with the attacking swarms evolving behaviors (such as rushing, splitting, and baiting) and the defending swarm evolving proactive and reactive solutions.

Conservation of information (COI) popularized by the no free lunch theorem is a great leveler of search algorithms, showing that on average no search outperforms any other. Yet in practice some searches appear to outperform others. In consequence, some have questioned the significance of COI to the performance of search algorithms. An underlying foundation of COI is Bernoulli's Principle of Insufficient Reason(PrOIR) which imposes of a uniform distribution on a search space in the absence of all prior knowledge about the search target or the search space structure. The assumption is conserved under mapping. If the probability of finding a target in a search space is p, then the problem of finding the target in any subset of the search space is p. More generally, all some-to-many mappings of a uniform search space result in a new search space where the chance of doing better than p is 50-50. Consequently the chance of doing worse is 50-50. This result can be viewed as a confirming property of COI. To properly assess the significance of the COI for search, one must completely identify the precise sources of information that affect search performance. This discussion leads to resolution of the seeming conflict between COI and the observation that some search algorithms perform well on a large class of problems.

According to conservation of information theorems, performance of an arbitrarily chosen search, on average, does no better than blind search. Domain expertise and prior knowledge about search space structure or target location is therefore essential in crafting the search algorithm. The effectiveness of a given algorithm can be measured by the active information introduced to the search. We illustrate this by identifying sources of active information in Avida, a software program designed to search for logic functions using nand gates. Avida uses stair step active information by rewarding logic functions using a smaller number of nands to construct functions requiring more. Removing stair steps deteriorates Avida's performance while removing deleterious instructions improves it. Some search algorithms use prior knowledge better than others. For the Avida digital organism, a simple evolutionary strategy generates the Avida target in far fewer instructions using only the prior knowledge available to Avida.

A common approach to pattern classification problems is to train a bank of layered perceptrons or other classifiers by clustering the input training data and training each classifier with just the data from a specific cluster. There is no provision in such an approach, however, to assure the component layered perceptron is well suited to learn the training data cluster it is assigned. An alternate method of training, herein proposed, lets a layered perceptron in a classifier bank choose the cluster of inputs it processes on the basis of the perceptron's ability to successfully classify those inputs. During training, data is therefore processed only by the classifier in the bank that best classifies the data or, equivalently, to which the data is most receptive. This allows each classifier to learn a localized subset of data dictated by the classifier's own classification ability. Once each classifier in the bank is trained, a separate independent cluster pointer is trained to recognize to which cluster an input test pattern belongs. The cluster pointer is used in the test mode to identify which classifier in the bank will best classify the problem. The approach, also applicable to regression type problems, is illustrated through application on a simulated Gaussian data set and an active sonar test estimation problem. In both cases, the maximally receptive classifer/regression bank significantly outperforms a single layered perceptron trained on the same data.

We present an application of particle swarm techniques to the problem of sparse signal recovery. Although a direct application of particle swarm is straightforward, specifics of the signal recovery problem can be incorporated into particle behavior in a way that substantially improves the quality of the recovered signal. With encouraging results for synthetic signals, we apply this technique to the problem of image compression, where typical image blocks can be expected to exhibit many very small elements under a transformation like the DCT. In this application, we observe that better results are obtained by first forcing image blocks to be sparse rather than compressively sampling blocks that are approximately sparse.

Conservation of information theorems indicate that any search algorithm performs, on average, as well as random search without replacementunlessit takes advantage of problem-specific information about the search target or the search-space structure. Combinatorics shows that even a moderately sized search requires problem-specific information to be successful. Computers, despite their speed in performing queries, are completely inadequate for resolving even moderately sized search problems without accurate information to guide them. We propose three measures to characterize the information required for successful search: 1)endogenous information, which measures the difficulty of finding a target using random search; 2)exogenous information, which measures the difficulty that remains in finding a target once a search takes advantage of problem-specific information; and 3)active information, which, as the difference between endogenous and exogenous information, measures the contribution of problem-specific information for successfully finding a target. This paper develops a methodology based on these information measures to gauge the effectiveness with which problem-specific information facilitates successful search. It then applies this methodology to various search tools widely used in evolutionary search.

Implementation of real-time neural network inversion on the SRC-6e, a computer that uses multiple field-programmable gate arrays (FPGAs) as reconfigurable computing elements, is examined using a sonar application as a specific case study. A feedforward multilayer perceptron neural network is used to estimate the performance of the sonar system (Jung et al., 2001). A particle swarm algorithm uses the trained network to perform a search for the control parameters required to optimize the output performance of the sonar system in the presence of imposed environmental constraints (Fox et al., 2002). The particle swarm optimization (PSO) requires repetitive queries of the neural network. Alternatives for implementing neural networks and particle swarm algorithms in reconfigurable hardware are contrasted. The final implementation provides nearly two orders of magnitude of speed increase over a state-of-the-art personal computer (PC), providing a real-time solution.

Simple rules, when executed by individual agents in a large group, or swarm, can lead to complex behaviors that are often difficult or impossible to predict knowing only the rules. However, aggregate behavior is not always unpredictable-even for swarm models said to be beyond analysis. For the class of swarming algorithms examined herein, we analytically identify several possible emergent behaviors and their underlying causes: clustering, drifting, and explosion. They also analyze the likelihood of these behaviors emerging from randomly selected swarm configurations and present a few examples. The analytic results are illustrated via simulation.

We consider the problem of power controlled minimum frame length scheduling for TDMA wireless networks. Given a set of one-hop transmission requests, our objective is to schedule them in a minimum number of time slots, so that each slot schedule is free of self-interferences and meets desired SINR constraints. Additionally, the transmit power vector corresponding to each slot schedule should be minimal. We consider two different versions of the problem, a per-slot version and a per-frame version, and develop mixed integer linear programming models which can be used for solving the problems optimally. In addition, we propose a heuristic algorithm for the per-slot version.

Real-time computing platforms are ubiquitous in the fields of automation, robotics and control. These are often based on a real-time operating system that offers a high level of determinism to support processes such as discrete-event and feedback control. Using methods derived the theory of dynamic systems on time scales, this paper discusses through modeling and simulation the effects of non-determinism. The paper's conclusion supports the hypothesis that strict determinism is not always required to obtain adequate overall system performance.

Particle swarm inversion of large neural networks is a computationally intensive process. By the implementing a modified particle swarm optimizer and neural network in reconfigurable hardware, many of the computations can be preformed simultaneously, significantly reducing compilation time compared to a conventional computer.

A technique for block-loss restoration in block-based image and video coding, dubbed recovery of image blocks using the method of alternating projections (RIBMAP), is developed. The algorithm is based on orthogonal projections onto constraint sets in a Hilbert space. For the recovery of a linear dimension N size block, a total of 8N vectors are extracted from the surrounding area of an N x N missing block. These vectors form a library from which the best matching spatial information for the missing block is extracted. Recovery vectors, including both undamaged and restored damaged pixels, are introduced. The vectors are used to find highly correlated information relating to the lost pixels. To assure continuity with the surrounding undamaged area, three additional convex constraints are formulated. Adherance to these sets is imposed using alternating projections. Simulation results using orthogonal projections demonstrate that RIBMAP recovers spatial structure faithfully. Simulation comparisons with other procedures are presented: Ancis and Giusto's hybrid edge-based average-median interpolation technique, Sun and Kwok's projections onto convex sets-based method, Hemami and Meng's interblock correlation interpolation approach, Shirani et al.'s modified interblock correlation interpolation scheme, and Alkachouh and Bellanger's fast discrete cosine transformation-based spatial domain interpolation algorithm. Characteristic of the results are those of the "Lena" JPEG image when one fourth of periodically spaced blocks in the image have errors. The peak signal-to-noise ratio of the restored image is 28.68, 29.99, 31.86, 31.69, 31.57, and 34.65 dB using that of Ancis and Giusto, Sun and Kwok, Hemami and Meng, Shirani et al., Alkachouh and Bellanger, and RIPMAP, respectively.

Genomic systems design (GSD) is an outgrowth of the union rule configuration (URC), a propositional logic construct that eliminates the combinatorial problem for rule-based systems. Its architecture is scalable, adaptive and fault-tolerant and is well-suited to multi-criteria decision systems and applications that must deal with sparse and missing data. This novel programming paradigm is similar in architecture to a biological process called symbiogenesis. This biological process is said to facilitate the evolution of new species through the inheritance of genomes from organisms that are participating in symbiotic relationships. This similarity, together with the characteristics of the URC, enables genomic systems design to offer a promising alternative methodology for the design of autonomous agents/robots, fault-tolerant and adaptive control systems, cellular automata and bioinformatics.

Previously (J.J. Weinschenk et al., 2003), we demonstrated that rule explosion is avoidable in fuzzy inference engines by mapping an intersection rule configuration (IRC) to a union rule configuration (URC). Although a conversion between IRC and URC systems is simple for a special, additively separable class of IRC rule tables, the conversion process is more difficult when the IRC rule table is additively inseparable. In a previous paper, we demonstrated that a layered URC architecture can implement these types of systems, but did not provide a method for computing the weights. In this follow-on paper, we provide a Fourier-based algorithm for computation of the weights. We also prove that this technique is optimal in the sense that the mean-square approximation error is minimized for a fixed amount of resources. Finally, through the course of an example, we explore the implementation issues and evaluate the overall effectiveness of the technique.

We consider the problem of minimum power bidirectional topology optimization in wireless networks with sectored antennas. We first develop a mixed integer linear programming model for optimal solution of the problem with sectored antennas. Subsequently, we discuss a centralized heuristic algorithm which is based on Kruskal's algorithm for the minimum spanning tree problem. However, unlike Kruskal's algorithm which chooses minimum cost edges from a set of edge weights, our algorithm uses an incremental cost mechanism to select edges. This incremental cost mechanism is motivated by the inherently broadcast nature of the wireless medium. We also discuss a simple branch exchange heuristic which can be used to improve the topology generated by the Kruskal-like algorithm. Simulation results indicate that reasonably good solutions can be obtained using the proposed heuristic algorithms.

This paper presents a method by which controllers operating on real-time networks such as the Controller Area Network (CAN) can reduce their bandwidth requirements in response to periods of high network traffic from sporadic sources. The method derives from advances in the theory of dynamic systems on time scales.

In this paper the problem of minimum power multicasting in wireless networks with sectored antennas is considered. For omnidirectional antenna systems, a transmission from node i to node j will also reach all nodes which are closer to i than j. Depending on the network geometry, this strategy can be highly power efficient In ideal sectored antenna systems, however, this phenomenon is sector specific; i.e., only those nodes which are located in the same sector as j will receive the transmission implicitly. Though this might seem an apparent disadvantage, the higher gains associated with directional antennas (as opposed to omni-directional antennas) allow for reduced transmission powers without sacrificing the signal-to-interference ratio at the receiver. In this paper, we first develop a mixed integer linear programming model for optimal solution of the minimum power multicast problem with sectored antennas. Subsequently, we discuss a biologically inspired algorithm for solving the problem to near-optimality at a very reasonable computation time. Experiments on randomly generated 10, 20 and 30-node networks indicate that near-optimal solutions can be obtained using the proposed algorithm.

An unsupervised learning system, implemented as an autonomous agent is presented. A simulation of a challenging path planning problem is used to illustrate the agent design and demonstrate its problem solving ability. The agent, dubbed the ORG, employs fuzzy logic and clustering techniques to efficiently represent and retrieve knowledge and uses innovative sensor modeling and attention focus to process a large number of stimuli. Simple initial fuzzy rules (instincts) are used to influence behavior and communicate intent to the agent. Self-reflection is utilized so the agent can learn from its environmental constraints and modify its own state. Speculation is utilized in the simulated environment, to produce new rules and fine-tune performance and internal parameters. The ORG is released in a simulated shallow water environment where its mission is to dynamically and continuously plan a path to effectively cover a specified region in minimal time while simultaneously learning from its environment. Several paths of the agent design are shown, and desirable emergent behavior properties of the agent design are discussed.

Block loss and propagation error due to cell loss or missing packet information during the transmission over lossy networks can cause severe degradation of block and predictive-based video coding. Herein, new fast spatial and temporal methods are presented for block loss recovery. In the spatial algorithm, missing block recovery and edge extention are performed by pixel replacement based on range constraints imposed by surrounding neighborhood edge information and structure. In the temporal algorithm, an adaptive temporal correlation method is proposed for motion vector (MV) recovery. Parameters for the temporal correlation measurement are adaptively changed in accordance to surrounding edge information of a missing macroblock (MB). The temporal technique utilizes pixels in the reference frame as well as surrounding pixels of the lost block. Spatial motion compensation is applied after MV recovery when the reference frame does not have sufficient information for lost MB restoration. Simulations demonstrate that the proposed algorithms recover image information reliably using both spatial and temporal restoration. We compare the proposed algorithm with other procedures with consistently favorable results.

We present a novel mapping whereby a classical fuzzy system, based on an intersection rule configuration (IRC), is converted to a union rule configuration (URC) system. Previous work has demonstrated that URC fuzzy systems avoid rule explosion, where a linear increase in the number of antecedents gives rise to an exponential increase in the number of fuzzy rules. However, there has been some doubt as to the validity of URC systems and previous findings. We resolve lingering questions and prove that any arbitrary IRC system can be converted to a URC system with identical performance. Further, we show that URC systems do avoid rule explosion for many problems. Finally, we note that a URC system is a universal approximator.

We consider the problem of maximizing the time-to-first-failure, defined as the time till the first node in the network runs out of battery energy, in energy constrained broadcast wireless networks. We discuss a greedy algorithm and prove that it solves the problem optimally for a broadcast application, in polynomial time, provided the complete power matrix and the battery residual capacities are known.

Broadcasting in wireless networks, unlike wired networks, inherently reaches several nodes with a single transmission. For omni-directional wireless broadcast to a node, all nodes closer will also be reached. This property can be used to compute routing trees which minimize the sum of the transmitter powers. It has been shown that this problem is NP-complete. In this paper, we present the r-shrink procedure, a heuristic for improving the solutions obtained using fast sub-optimal algorithms. Specifically, we focus on the low-complexity BIP algorithm and Prim's minimum spanning tree algorithm and show through extensive simulations that better solutions are obtained almost always, with considerably lower tree power, if the proposed procedure is used to improve the trees generated using these algorithms.

Preprocessing of data to be learned by a neural network is typically done to improve neural network performance. Output processing is especially important since it directly affects the influence of error in the hidden layers on the error of the neural network output. Principal component analysis is a commonly used preprocessing method that can improve the network performance by reducing the output dimensionality and reducing the number of parameters in a neural network model. Transforming the principal components of the outputs with an orthonormal matrix prior to scaling can further improve network performance.

Using array historical data, the readings from a sensor array may be shown to contain sufficient redundancy such that the readings from one or more lost sensors may be able to be accurately estimated from those remaining. This interdependency can be established by an neural network encoder. The encoder is also used in the restoration process. In this paper, we give some examples of sensor restoration for vibration sensors on jet engine and computer traffic data.

The neural network autoencoder is a useful tool for the restoration of missing sensors when enough known sensors with some relation to those missing are available. Through the idea of a contraction mapping, this paper provides some insight into the convergence of several iterative methods of sensor restoration using the autoencoder to some unique answer given a specific operating point (i.e., the known sensor values), regardless of how the missing sensor values are initialized.

Given a complicated and computationally intensive underwater acoustic model in which some acoustic measurement is a function of sonar system and environmental parameters, it is computationally beneficial to train a neural network to emulate the properties of that model. Given this neural network model, we now have a convenient means of performing geoacoustic inversion without the computational intensity required when attempting to do so with the actual model. This paper proposes an efficient and reliable method of performing the inversion of a neural network underwater acoustic model to obtain parameters pertaining to the characteristics of the ocean floor, using two different modified version of particle swarm optimization (PSO): two-step (gradient approximation) PSO and hierarchical cluster-based PSO.

We introduce a novel layered fuzzy architecture that avoids rule explosion. Unlike a single layer union rule configuration (URC) fuzzy system, a layered URC fuzzy system can approximate any surface without the need of burdensome "corrective" terms. Further, we show that the URC fuzzy system is a generalized layered perceptron - an insight that allows one to choose interconnection weights in an intuitive manner with very basic problem knowledge. In some cases, training may not be necessary. Further, the fuzzy linguistic meaning of variables is preserved throughout the layers of the system. The universal approximation property of this architecture is discussed and we demonstrate how a layered URC fuzzy system solves a simple regression problem.

Knowledge of the dynamic security border can provide an operator with valuable information on how to safely steer the power system away from vulnerable operating regions. The large set of non-linear differential equations that describe modern large-scale power systems makes it difficult to determine the security border either analytically or numerically in real time. As an alternative, neural networks trained off line on emulator data can provide a commensurate representation of the system transfer function, while significantly decreasing evaluation time. Using neural network inversion, sets of input points corresponding to a fixed output can be evaluated quickly. Different inversion procedures and their properties are reviewed here. We also review various metrics used for determining whether sufficient coverage of the border is achieved. Finally, we illustrate the use of border identification for preventive control when feature selection is initially performed to reduce the dimensionality of the input space.

There are many methods for performing neural network inversion. Multi-element evolutionary inversion procedures are capable of finding numerous inversion points simultaneously. Constrained neural network inversion requires that the inversion solution belong to one or more specified constraint sets. In many cases, iterating between the neural network inversion solution and the constraint set can successfully solve constrained inversion problems. This paper surveys existing methodologies for neural network inversion, which is illustrated by its use as a tool in query-based learning, sonar performance analysis, power system security assessment, control, and generation of codebook vectors.

An algorithm for adaptively controlling genetic algorithm parameter (GAP) coding using fuzzy rules is presented. The fuzzy GAP coding algorithm is compared to the dynamic parameter encoding scheme proposed by Schraudolph and Belew. The performance of the algorithm on a hydraulic brake emulator parameter identification problem is investigated. Fuzzy GAP coding control is shown to dramatically increase the rate of convergence and accuracy of genetic algorithms.

EURAL-NETWORK technology has reached a degree of maturity as evidenced by an ever-increasing number of applications. Our experience, however, is that most practitioners of neural networks are familiar with only a handful of cases where neural-network technology has been reduced to practice. The objective of this special issue is presentation of some specific cases of ongoing everyday use of neural networks. Specifically excluded are neural-network applications still in the exploratory stage. While publication of extraordinary exploratory applications papers is within the scope of the IEEE TRANSACTIONS ON NEURAL NETWORKS, this special issue deals only with neural networks used on a regular basis. At minimum, the system must be at the beta test stage. Of the 53 papers received for the special issue, the 14 herein were chosen. In some important cases, papers solicited for submission were unfortunately withheld because developers or licensees wished to not disclose proprietary technology. Nevertheless, the spectrum of the everyday neural-network applications reported herein is a veritable smorgasbord of variety. Applications are reported in telecommunications, control of Publisher Item Identifier S 1045-9227(97)05726-3. steel plants, plasma etching, pattern recognition of cataloged parts, credit card fraud detection, space robot tuning, electric utility load forecasting, railway maintenance, power system security assessment, scanning electron microscope image characterization, cold mill prediction, economic forecasting and, not least, assessment of wine bottle cork quality. This sampling of applications in everyday use is in no way complete. It gives, however, a taste of the impact of neural technology in society. Indeed, impact is the metric by which all technology is ultimately measured.

The generalization performance of feedforward layered perceptrons can, in many cases, be improved either by smoothing the target via convolution, regularizing the training error with a smoothing constraint, decreasing the gain (i.e., slope) of the sigmoid nonlinearities, or adding noise (i.e., jitter) to the input training data, In certain important cases, the results of these procedures yield highly similar results although at different costs. Training with jitter, for example, requires significantly more computation than sigmoid scaling.

A solution to the problem of implementation of the adaptive resonance theory (ART) of neural networks that uses an optical correlator which allows the large body of correlator research to be leveraged in the implementation of ART is presented. The implementation takes advantage of the fact that one ART-based architecture, known as ART1, can be broken into several parts, some of which are better to implement in parallel. The control structure of ART, often regarded as its most complex part, is actually not very time consuming and can be done in electronics. The bottom-up and top-down gated pathways, however, are very time consuming to simulate and are difficult to implement directly in electronics due to the high number of interconnections. In addition to the design, the authors present experiments with a laboratory prototype to illustrate its feasibility and to discuss implementation details that arise in practice. This device can potentially outperform alternative implementations of ART1 by as much as two to three orders of magnitude in problems requiring especially large input fields.

The inversion of snow parameters from passive microwave remote sensing measurements is performed with a neural network trained with a dense-media multiple-scattering model. The input-output pairs generated by the scattering model are used to train the neural network. Simultaneous inversion of three parameters, mean-grain size of ice particles in snow, snow density, and snow temperature from five brightness temperatures, is reported. It is shown that the neural network gives good results for simulated data. The absolute percentage errors for mean-grain size of ice particles and snow density are less than 10%, and the absolute error for snow temperature is less than 3 K. The neural network with the trained weighting coefficients of the three-parameter model is also used to invert SSMI data taken over the Antarctic region.>

An approach is presented for query-based neural network learning. A layered perceptron partially trained for binary classification is considered. The single-output neuron is trained to be either a zero or a one. A test decision is made by thresholding the output at, for example, one-half. The set of inputs that produce an output of one-half forms the classification boundary. The authors adopted an inversion algorithm for the neural network that allows generation of this boundary. For each boundary point, the classification gradient can be generated. The gradient provides a useful measure of the steepness of the multidimensional decision surfaces. Conjugate input pairs are generated using the boundary point and gradient information and presented to an oracle for proper classification. These data are used to refine further the classification boundary, thereby increasing the classification accuracy. The result can be a significant reduction in the training set cardinality in comparison with, for example, randomly generated data points. An application example to power system security assessment is given.

During communication between neurons in a continuous-time analog neural network, propagation skew typically varies from neuron pair to neuron pair. For no dispersion, S. Oh et al. (Opt. Eng., vol.28, p.526-32, 1989) previously demonstrated that the steady-state performance of an iterative neural network is not affected if the combination of the neural network's weights and neural nonlinearity is contractive. This result is extended to the case of dispersive skew. It is shown that, under nearly the same conditions, the same steady-state result will occur in the neural network in the presence of dispersive skew.

A training procedure that adapts the weights of a trained layered perceptron artificial neural network to training data originating from a slowly varying nonstationary process is proposed. The resulting adaptively trained neural network (ATNN), based on nonlinear programming techniques, is shown to adapt to new training data that are in conflict with earlier training data without affecting the neural networks' response to data elsewhere. The adaptive training procedure also allows for new data to be weighted in terms of its significance. The adaptive algorithm is applied to the problem of electric load forecasting and is shown to outperform the conventionally trained layered perceptron.

A novel approach is presented to solve a constrained inverse problem encountered in the design of frequency selective surfaces (FSSs). Due to the many-to-one nonlinear functional relationship between an FSS and its frequency response, there is no closed-form solution directly from the given desired frequency response to the corresponding surface. Therefore, to design an FSS for a given response, one has to search in the knowledge base through a laborious and tedious trial-and-error procedure. The authors' approach adopts an iterative regularized inversion technique, which starts with an inversion algorithm for multilayer perceptrons to generate the corresponding 2-D surface for the given desired frequency response. A constraint-satisfaction mechanism is then used to reshape the 2-D surface to satisfy the constraints, and the resulting surface is used as the initial point for the next inversion algorithm. This procedure is mathematically similar to the projection-onto-convex-set algorithm for constrained optimization problems

A novel approach to query-based neural network learning is presented. A layered perceptron partially trained for binary classification is considered. The single-output neuron is trained to be either a 0 or a 1. A test decision is made by thresholding the output at, for example, 1/2. The set of inputs that produce an output of 1/2 forms the classification boundary. For each boundary point, the classification gradient can be generated. The gradient provides a useful measure of the sharpness of the multidimensional decision surfaces. Conjugate input pair locations are generated using the boundary point and gradient information and are presented to the oracle for proper classification. These new data are used to further refine the classification boundary, thereby increasing the classification accuracy. The result can be a significant reduction in the training set cardinality in comparison with, for example, randomly generated data points. An application example to power security assessment is given

Multilayer perceptrons and trained classification trees are two very different techniques which have recently become popular. Giving enough data and time, both methods are capable of performing arbitrary nonlinear classification. The two techniques have not previously been compared on real-world problems. The authors first consider the important differences between multilayer perceptrons and classification trees and conclude that there is not enough theoretical basis for the clear-cut superiority of one technique over the other. They then present results of a number of empirical tests on quite different problems in power system load forecasting and speaker-independent vowel identification. They compare the performance for classification and prediction in terms of accuracy outside the training set. In all cases, even with various sizes of training sets, the multilayer perceptron performed as well as or better than the trained classification trees. The authors are confident that the univariate version of the trained classification trees do not perform as well as the multilayer perceptron. More studies are needed, however, on the comparative performance of the linear combination version of the classification trees.>

Shannon has shown that one can communicate exactly over a noisy channel. It thus stands to reason that one can compute accurately with an inexact processor. For example, optical processors offer advantages of massive parallelism and great speed but suffer from significant inexactitude. Another example is wafer-scale VLSI processors, which can be inexact due to a distribution of defects. We propose an architecture whereby the accuracy of matched filter processors can be improved significantly at the cost of a modest increase in computational intensity. Errors due to noisy data and/or inexact computing can be detected and in some cases corrected. We show that over one million objects can potentially be detected with single error correction capability using only 25 (composite) matched filters.