The paper reviews implementations of neural networks on parallel digital machines. The connectionist neural networks models are discussed from the point of view of their computational characteristics. The levels of parallelism available in the models and the factors affecting their performance of the models on the parallel machines are presented. Several mapping methodologies applicable to neural networks are described. Selected examples of implementations of neural networks on parallel digital machines are provided.>

Implementations of neural networks on programmable massively parallel computers are addressed. The methods are based on a graph theoretic approach and are applicable to a large class of networks in which the computations can be described by means of matrix and vector operations. A detailed characterization of the target machine is provided. Two mappings are presented. The first is designed for a processor array consisting of a very large number of small processing units. The neurons and the nonzero synaptic weights are assigned to the processors in a predetermined order, one per processor. The data transfers between processors containing neurons and weights are implemented using a novel routing algorithm. The second mapping is designed for the data array of size N*N and a smaller processor array of size P*P, P>

The authors consider parallel implementation of neural network computations of fine grain SIMD machines. The authors show a mapping of a neural network having n nodes and e connections onto a parallel machine having (n+e) PEs arranged in an array of square root n+e* square root n+e PEs such that routing for each update iteration of the recall phase can be performed in 24( square root n+e-1) elemental data shifts. The array uses simple PEs and few local registers to perform the routing and computations. The method is simple and is well suited for implementation of various classes of neural networks on many currently available parallel machines.>