Abstract not provided

Solving real world problems with embedded neural networks requires both training algorithms that achieve high performance and compatible hardware that runs in real time while remaining energy efficient. For the former, deep learning using backpropagation has recently achieved a string of successes across many domains and datasets. For the latter, neuromorphic chips that run spiking neural networks have recently achieved unprecedented energy efficiency. To bring these two advances together, we must first resolve the incompatibility between backpropagation, which uses continuous-output neurons and synaptic weights, and neuromorphic designs, which employ spiking neurons and discrete synapses. Our approach is to treat spikes and discrete synapses as continuous probabilities, which allows training the network using standard backpropagation. The trained network naturally maps to neuromorphic hardware by sampling the probabilities to create one or more networks, which are merged using ensemble averaging. To demonstrate, we trained a sparsely connected network that runs on the TrueNorth chip using the MNIST dataset. With a high performance network (ensemble of $64$), we achieve $99.42\%$ accuracy at $121 \mu$J per image, and with a high efficiency network (ensemble of $1$) we achieve $92.7\%$ accuracy at $0.408 \mu$J per image.

Drawing on neuroscience, we have developed a parallel, event-driven kernel for neurosynaptic computation, that is efficient with respect to computation, memory, and communication. Building on the previously demonstrated highly optimized software expression of the kernel, here, we demonstrate True North, a co-designed silicon expression of the kernel. True North achieves five orders of magnitude reduction in energy to-solution and two orders of magnitude speedup in time-to solution, when running computer vision applications and complex recurrent neural network simulations. Breaking path with the von Neumann architecture, True North is a 4,096 core, 1 million neuron, and 256 million synapse brain-inspired neurosynaptic processor, that consumes 65mW of power running at real-time and delivers performance of 46 Giga-Synaptic OPS/Watt. We demonstrate seamless tiling of True North chips into arrays, forming a foundation for cortex-like scalability. True North's unprecedented time-to-solution, energy-to-solution, size, scalability, and performance combined with the underlying flexibility of the kernel enable a broad range of cognitive applications.

We consider the scenario in which a set of sources generates messages in a network and a receiver node demands an arbitrary linear function of these messages. We formulate an algebraic test to determine whether an arbitrary network can compute linear functions using linear codes. We identify a class of linear functions that can be computed using linear codes in every network that satisfies a natural cut-based condition. Conversely, for another class of linear functions, we show that the cut-based condition does not guarantee the existence of a linear coding solution. For linear functions over the binary field, the two classes are complements of each other.

Marching along the DARPA SyNAPSE roadmap, IBM unveils a trilogy of innovations towards the TrueNorth cognitive computing system inspired by the brain's function and efficiency. The non-von Neumann nature of the TrueNorth architecture necessitates a novel approach to efficient system design. To this end, we have developed a set of abstractions, algorithms, and applications that are natively efficient for TrueNorth. First, we developed repeatedly-used abstractions that span neural codes (such as binary, rate, population, and time-to-spike), long-range connectivity, and short-range connectivity. Second, we implemented ten algorithms that include convolution networks, spectral content estimators, liquid state machines, restricted Boltzmann machines, hidden Markov models, looming detection, temporal pattern matching, and various classifiers. Third, we demonstrate seven applications that include speaker recognition, music composer recognition, digit recognition, sequence prediction, collision avoidance, optical flow, and eye detection. Our results showcase the parallelism, versatility, rich connectivity, spatio-temporality, and multi-modality of the TrueNorth architecture as well as compositionality of the corelet programming paradigm and the flexibility of the underlying neuron model.

We study the use of linear codes for network computing in single-receiver networks with various classes of target functions of the source messages. Such classes include reducible, semi-injective, and linear target functions over finite fields. Computing capacity bounds and achievability are given with respect to these target function classes for network codes that use routing, linear coding, or nonlinear coding.

We study the use of linear codes for network computing in single-receiver networks with various classes of target functions of the source messages. Such classes include reducible, injective, and semi-injective target functions. Computing capacity bounds are given with respect to these target function classes for network codes that use routing, linear coding, or nonlinear coding.

The following network computing problem is considered. Source nodes in a directed acyclic network generate independent messages and a single receiver node computes a target functionfof the messages. The objective is to maximize the average number of timesfcan be computed per network usage, i.e., the “computing capacity”. The network coding problem for a single-receiver network is a special case of the network computing problem in which all of the source messages must be reproduced at the receiver. For network coding with a single receiver, routing is known to achieve the capacity by achieving the network min-cut upper bound. We extend the definition of min-cut to the network computing problem and show that the min-cut is still an upper bound on the maximum achievable rate and is tight for computing (using coding) any target function in multi-edge tree networks. It is also tight for computing linear target functions in any network. We also study the bound's tightness for different classes of target functions. In particular, we give a lower bound on the computing capacity in terms of the Steiner tree packing number and a different bound for symmetric functions. We also show that for certain networks and target functions, the computing capacity can be less than an arbitrarily small fraction of the min-cut bound.

This paper considers the following network computation problem:nnodes are placed on a √n × √n grid, each node is connected to every other node within distancer(n) of itself, and it is assigned an arbitrary input bit. Nodes communicate with their neighbors and a designated sink node computes a functionfof the input bits, wherefis either the identity or a symmetric function. We first consider a model where links are interference and noise-free, suitable for modeling wired networks. Then, we consider a model suitable for wireless networks. Due to interference, only nodes which do not share neighbors are allowed to transmit simultaneously, and when a node transmits a bit, all of its neighbors receive an independent noisy copy of the bit. We present lower bounds on the minimum number of transmissions and on the minimum number of time slots required to computef. We also describe efficient schemes that match both of these lower bounds up to a constant factor and are thus jointly (near) optimal with respect to the number of transmissions and the number of time slots required for computation. At the end of the paper, we extend results on symmetric functions to general network topologies, and obtain a corollary that answers an open question posed by El Gamal in 1987 regarding the computation of the parity function over ring and tree networks.

We study the computation of the arithmetic sum of the q-ary source messages in the reverse butterfly network. Specifically, we characterize the maximum rate at which the message sum can be computed at the receiver and demonstrate that linear coding is suboptimal.

This paper considers the following network computation problem: n nodes are placed on a radic(n)timesradic(n) grid, each node in the network is connected to every other node within distance r(n) of itself, and is given an arbitrary input bit. Connected nodes communicate with each other over independent binary symmetric channels of a given transition probability epsiv ges 0, and an arbitrarily designated node computes a symmetric target function f of the input bits. We characterize up to order the minimum number of transmissions required to compute f with a probability of error less than any given positive constant delta. As a side result, we answer an open question posed by El Gamal in 1987 regarding the number of transmissions required to compute the parity function over ring and tree networks.

Recently Golay complementary sets were shown to exist in the subsets of second-order cosets of a 𝑞-ary generalization of the first-order Reed–Muller (RM) code. We show that mutually orthogonal Golay complementary sets can also be directly constructed from second-order cosets of a 𝑞-ary generalization of the first-orderRMcode. This identification can be used to construct zero correlation zone (ZCZ) sequences directly and it also enables the construction of ZCZ sequences with special subsets.

It is known that linear codes are sufficient to solve the multicast network coding problem when each out-edge of a network node carries its own specific function of the in-edges of the node, i.e. operating in "point-to-point-mode." Alternatively, in "broadcast-mode," a network has the property that for each node, every out-edge of the node carries the same function of the in-edges of the node. Only one transmission is required in order to send the same function on all of the out-edges of a node. The edge functions in broadcast-mode can vary from node to node and each edge can carry an arbitrary number of transmissions, with at most one per time unit. We prove that linear codes are sufficient, in terms of total number of transmissions, for multicast networks in broadcast-mode. That is, we show that for any broadcast-mode solution to a multicast network, there exists a linear broadcast-mode solution over some finite field which does not increase the total number of network transmissions

In this correspondence, new characterizations for the construction of zero correlation zone (ZCZ) sequences from mutually orthogonal Golay complementary sets (MOGCS) is presented. It is shown that the recursive construction of MOGCS is inherent in these characterizations. Previously known constructions of ZCZ sequences and MOGCS are shown to be special cases of this characterization. The notion of mutually orthogonal ZCZ sequence sets is also introduced