We describe a decorrelation network training method for improving the quality of regression learning in 'ensemble' neural networks NNs that are composed of linear combinations of individual NNs. In this method, individual networks are trained by backpropogation not only to reproduce a desired output, but also to have their errors linearly decorrelated with the other networks. Outputs from the individual networks are then linearly combined to produce the output of the ensemble network. We demonstrate the performances of decorrelated network training on learning the 'three-parity' logic function, a noisy sine function and a one-dimensional non-linear function, and compare the results with the ensemble networks composed of independently trained individual networks without decorrelation training . Empirical results show than when individual networks are forced to be decorrelated with one another the resulting ensemble NNs have lower mean squared errors than the ensemble networks having independently trained individual networks. This method is particularly applicable when there is insufficient data to train each individual network on disjoint subsets of training patterns.

This paper presents a technique for image recognition, reconstruction, and processing using a novel massively parallel system. This device is a physical implementation of a Boltzmann machine type of neural network based on the use of magnetic thin films and opto-magnetic control. Images or patterns in the form of pixel arrays are imposed on the magnetic film using a laser in an external magnetic field. These images are learned and can be recalled later when a similar image is presented. A stored image is recallable even when a partial, noisy, or corrupted version of that image is imposed on the film. The system can also be used for feature detection and image compression. The operation and construction of the physical system is described, together with a discussion of the physical basis for its operation. The authors have developed Monte Carlo style computer simulations of the system for a variety of platforms, including serial workstations and hypercube configured parallel systems. They describe here some of the factors involved in computer simulations of the system, which can be fast and relatively simple in implementation. Simulation results are presented and, in particular, the behavior of the model under simulated annealing in the light of statistical physics is discussed. The simulation itself can be used as a neural network model capable of the functions ascribed to the physical device.