In this work, we demonstrate the training, conversion, and implementation flow of an FPGA-based bin-ratio ensemble spiking neural network applied for radioisotope identification. The combination of techniques including learned step quantisation (LSQ) and pruning facilitated the implementation by compressing the network's parameters down to 30% yet retaining the accuracy of 97.04% with an accuracy loss of less than 1%. Meanwhile, the proposed ensemble network of 20 3-layer spiking neural networks (SNNs), which incorporates 1160 spiking neurons, only needs 334 μs for a single inference with the given clock frequency of 100 MHz. Under such optimisation, this FPGA implementation in an Artix-7 board consumes 157 μJ per inference by estimation.

This paper presents a new methodology to alleviate the fundamental trade-off between accuracy and latency in spiking neural networks (SNNs). The approach involves decoding confidence information over time from the SNN outputs and using it to develop a decision-making agent that can dynamically determine when to terminate each inference.The proposed method, Dynamic Confidence, provides several significant benefits to SNNs. 1. It can effectively optimize latency dynamically at runtime, setting it apart from many existing low-latency SNN algorithms. Our experiments on CIFAR-10 and ImageNet datasets have demonstrated an average 40% speedup across eight different settings after applying Dynamic Confidence. 2. The decisionmaking agent in Dynamic Confidence is straightforward to construct and highly robust in parameter space, making it extremely easy to implement. 3. The proposed method enables visualizing the potential of any given SNN, which sets a target for current SNNs to approach. For instance, if an SNN can terminate at the most appropriate time point for each input sample, a ResNet-50 SNN can achieve an accuracy as high as 82.47% on ImageNet within just 4.71 time steps on average. Unlocking the potential of SNNs needs a highly-reliable decision-making agent to be constructed and fed with a high-quality estimation of ground truth. In this regard, Dynamic Confidence represents a meaningful step toward realizing the potential of SNNs. Code is available1.

This paper details FPGA implementation methodology for Convolutional Spiking Neural Networks (CSNN) and applies this methodology to low-power radioisotope identification using high resolution data. A power consumption of 75 mW has been achieved on an FPGA implementation of a CSNN, with the inference accuracy of 90.62% on a synthetic dataset. The chip validation method is presented. Prototyping was accelerated by evaluating SNN parameters using SpiNNaker neuromorphic platform.

Spiking neural networks (SNNs) are promising neural network models to achieve power-efficient and event-based computing on neuromorphic hardware. SNNs inherently contain noise and are robust to noisy inputs as well as noise related to the discrete 1-bit spike. In this paper, we find that SNNs are more robust to Gaussian noise in synaptic weights than artificial neural networks (ANNs) under some conditions. This finding will enhance our understanding of the neural dynamics in SNNs and of the advantages of SNNs compared with ANNs. Our results imply the possibility of using high-performance cutting-edge materials with intrinsic noise as an information storage medium in SNNs.

As the SLAM research area matures and the number of SLAM systems available increases, the need for frameworks that can objectively evaluate them against prior work grows. This new version of SLAMBench moves beyond traditional visual SLAM, and provides new support for scene understanding and non-rigid environments (dynamic SLAM). More concretely for dynamic SLAM, SLAMBench 3.0 includes the first publicly available implementation of DynamicFusion, along with an evaluation infrastructure. In addition, we include two SLAM systems (one dense, one sparse) augmented with convolutional neural networks for scene understanding, together with datasets and appropriate metrics. Through a series of use-cases, we demonstrate the newly incorporated algorithms, visulation aids and metrics (6 new metrics, 4 new datasets and 5 new algorithms).

Neuromorphic chips are used to model biologically inspired Spiking-Neural-Networks (SNNs) where most models are based on differential equations. Equations for most SNN algorithms usually contain variables with one or more excomponents. SpiNNaker is a digital neuromorphic chip that has so far been using pre-calculated look-up tables for exponential function. However this approach is limited because the memory requirements grow as more complex neural models are developed. To save already limited memory resources in the next generation SpiNNaker chip, we are including a fast exponential function in the silicon. In this paper we analyse iterative algorithms for elementary functions and show how to build a single hardware accelerator for exp and natural log, for a neuromorphic chip prototype, to be manufactured in a 22 nm FDSOI process. We present the accelerator that has algorithmic level approximation control, allowing it to trade precision for latency and energy efficiency. As an addition to neuromorphic chip application, we provide analysis of a parameterized elementary function unit that can be tailored for other systems with different power, area, accuracy and latency constraints.

This paper provides a performance evaluation and trade-off analysis of a novel chip architecture for neuromorphic computing, especially focused on the memory subsystems and the Network-On-Chip (NoC). More precisely, we study the performance-related effect of the number of memory modules, as well as that of allowing direct core-to-core communication. Our simulation-based experimental work throws many interesting results on the above aspects and allows to ensure that congestion at the NoC-level is unlikely to degrade performance.

SLAM is becoming a key component of robotics and augmented reality (AR) systems. While a large number of SLAM algorithms have been presented, there has been little effort to unify the interface of such algorithms, or to perform a holistic comparison of their capabilities. This is a problem since different SLAM applications can have different functional and non-functional requirements. For example, a mobile phone-based AR application has a tight energy budget, while a UAV navigation system usually requires high accuracy. SLAMBench2 is a benchmarking framework to evaluate existing and future SLAM systems, both open and close source, over an extensible list of datasets, while using a comparable and clearly specified list of performance metrics. A wide variety of existing SLAM algorithms and datasets is supported, e.g. ElasticFusion, InfiniTAM, ORB-SLAM2, OKVIS, and integrating new ones is straightforward and clearly specified by the framework. SLAMBench2 is a publicly-available software framework which represents a starting point for quantitative, comparable and val-idatable experimental research to investigate trade-offs across SLAM systems.

Speech recognition has become an important task to improve the human-machine interface. Taking into account the limitations of current automatic speech recognition systems, like non-real time cloud-based solutions or power demand, recent interest for neural networks and bio-inspired systems has motivated the implementation of new techniques. Among them, a combination of spiking neural networks and neuromorphic auditory sensors offer an alternative to carry out the human-like speech processing task. In this approach, a spiking convolutional neural network model was implemented, in which the weights of connections were calculated by training a convolutional neural network with specific activation functions, using firing rate-based static images with the spiking information obtained from a neuromorphic cochlea. The system was trained and tested with a large dataset that contains ”left” and ”right” speech commands, achieving 89.90% accuracy. A novel spiking neural network model has been proposed to adapt the network that has been trained with static images to a non-static processing approach, making it possible to classify audio signals and time series in real time.

The SpiNNaker chip is a multi-core processor optimized for neuromorphic applications. Many SpiNNaker chips are assembled to make a highly parallel million core platform. This system can be used for simulation of a large number of neurons in real-time. SpiNNaker is using a general purpose ARM processor that gives a high amount of flexibility to implement different methods for processing spikes. Various libraries and packages are provided to translate a high-level description of Spiking Neural Networks (SNN) to low-level machine language that can be used in the ARM processors. In this paper, we introduce and compare three different methods to implement this intermediate layer of abstraction. We have examined the advantages of each method by various criteria, which can be useful for professional users to choose between them. All the codes that are used in this paper are available for academic propose.

Visual understanding of 3-D environments in real time, at low power, is a huge computational challenge. Often referred to as simultaneous localization and mapping (SLAM), it is central to applications spanning domestic and industrial robotics, autonomous vehicles, and virtual and augmented reality. This paper describes the results of a major research effort to assemble the algorithms, architectures, tools, and systems software needed to enable delivery of SLAM, by supporting applications specialists in selecting and configuring the appropriate algorithm and the appropriate hardware, and compilation pathway, to meet their performance, accuracy, and energy consumption goals. The major contributions we present are: 1) tools and methodology for systematic quantitative evaluation of SLAM algorithms; 2) automated, machine-learning-guided exploration of the algorithmic and implementation design space with respect to multiple objectives; 3) end-to-end simulation tools to enable optimization of heterogeneous, accelerated architectures for the specific algorithmic requirements of the various SLAM algorithmic approaches; and 4) tools for delivering, where appropriate, accelerated, adaptive SLAM solutions in a managed, JIT-compiled, adaptive runtime context.

We present here a learning system using the iCub humanoid robot and the SpiNNaker neuromorphic chip to solve the real-world task of object-specific attention. Integrating spiking neural networks with robots introduces considerable complexity for questionable benefit if the objective is simply task performance. But, we suggest, in a cognitive robotics context, where the goal is understanding how to compute, such an approach may yield useful insights to neural architecture as well as learned behavior, especially if dedicated neural hardware is available. Recent advances in cognitive robotics and neuromorphic processing now make such systems possible. Using a scalable, structured, modular approach, we build a spiking neural network where the effects and impact of learning can be predicted and tested, and the network can be scaled or extended to new tasks automatically. We introduce several enhancements to a basic network and show how they can be used to direct performance toward behaviorally relevant goals. Results show that using a simple classical spike-timing-dependent plasticity (STDP) rule on selected connections, we can get the robot (and network) to progress from poor task-specific performance to good performance. Behaviorally relevant STDP appears to contribute strongly to positive learning: "do this" but less to negative learning: "don't do that." In addition, we observe that the effect of structural enhancements tends to be cumulative. The overall system suggests that it is by being able to exploit combinations of effects, rather than any one effect or property in isolation, that spiking networks can achieve compelling, task-relevant behavior.

Networks-on-chip (NoCs) have become a new chip design paradigm as the size of transistors continues to shrink. Globally-asynchronous locally-synchronous (GALS) on-chip networks are proposed for solving issues such as large clock tree distribution and signal delay variations. More interestingly, for the GALS networks using m-of-n delay-insensitive interconnect, the asynchronous interconnect not only can be used for on-chip interconnection, but also provides a simple, direct and power-saving solution for off-chip interconnection. This paper presents an asynchronous interface FIFO design to improve throughput over asynchronous inter-chip links using 2-of-7 Non-Return-to-Zero (NRZ) encoding in an existing many-core system. The proposed design is suitable for implementation on commodity FPGAs without using the limited global clock buffer resources, but involves using the FPGA to implement asynchronous circuits. The interface FIFO is constructed from the transition detectors themselves rather than by employing a separate buffer in the more conventional fashion. The proposed solution has been demonstrated in an existing system and is suitable for adaptation to other asynchronous m-of-n NRZ coding protocols for high-throughput communication.

We present a dynamic voltage and frequency scaling technique within SoCs for per-core power management: the architecture allows for individual, self triggered performance-level scaling of the processing elements (PEs) within less than 100ns. This technique enables each core to adjust its local supply voltage and frequency depending on its current computational load. A test chip has been implemented in 28nm CMOS technology, as prototype of the SpiNNaker2 neuromorphic many core system, containing 4 PEs which are operational within the range of 1.1V down to 0.7V at frequencies from 666MHz down to 100MHz; The particular domain area of this application specific processor is real-time neuromorphics. Using a standard benchmark - the synfire chain - we show that the total power consumption can be reduced by 45%, with 85% baseline power reduction and a 30% reduction of energy per neuron and synapse computation, all while maintaining biological real-time operation.

We present a dynamic voltage and frequency scaling technique within SoCs for per-core power management: the architecture allows for individual, self triggered performance-level scaling of the processing elements (PEs) within less than 100ns. This technique enables each core to adjust its local supply voltage and frequency depending on its current computational load. A test chip has been implemented in 28nm CMOS technology, as prototype of the SpiNNaker2 neuromorphic many core system, containing 4 PEs which are operational within the range of 1.1V down to 0.7V at frequencies from 666MHz down to 100MHz; the effectiveness of the power management technique is demonstrated using a standard benchmark from the application domain. The particular domain area of this application specific processor is real-time neuromorphics. Using a standard benchmark - the synfire chain - we show that the total power consumption can be reduced by 45%, with 85% baseline power reduction and a 30% reduction of energy per neuron and synapse computation, all while maintaining biological real-time operation.

Many models of spiking neural networks heavily rely on exponential waveforms. On neuromorphic multiprocessor systems like SpiNNaker, they have to be approximated by dedicated algorithms, often dominating the processing load. Here we present a processor extension for fast calculation of exponentials, aimed at integration in the next-generation SpiNNaker system. Our implementation achieves single-LSB precision in a 32bit fixed-point format and 250Mexp/s throughput at 0.44nJ/exp for nominal supply (1.0V), or 0.21nJ/exp at 0.7V supply and 77Mexp/s, demonstrating a throughput multiplication of almost 50 and 98% energy reduction at 2% area overhead per processor on a 28nm CMOS chip.

In this live demonstration we exploit the use of a serial link for fast asynchronous communication in massively parallel processing platforms connected to a DVS for real-time implementation of bio-inspired vision processing on spiking neural networks.

Address-Event-Representation (AER) is a widely extended asynchronous technique for interchanging “neural spikes” among different hardware elements in Neuromorphic Systems. Conventional AER links use parallel physical wires together with a pair of handshaking signals (Request and Acknowledge). Here we present a fully serial implementation using bidirectional SATA connectors with a pair of LVDS (low voltage differential signaling) wires for each direction. The proposed implementation can multiplex a number of conventional parallel AER links per LVDS physical connection. It uses flow control, clock correction, and byte alignment techniques to transmit 32-bit address events reliably over multiplexed serial connections. The setup has been tested using commercial Spartan6 FPGAs reaching a maximum event transmission speed of 75Meps (Mega Events per second) for 32-bit events at 3.0Gbps line data rate.

The biological brain is a highly plastic system within which the efficacy and structure of synaptic connections are constantly changing in response to internal and external stimuli. While numerous models of this plastic behavior exist at various levels of abstraction, how these mechanisms allow the brain to learn meaningful values is unclear. The Neural Engineering Framework (NEF) is a hypothesis about how large-scale neural systems represent values using populations of spiking neurons, and transform them using functions implemented by the synaptic weights between populations. By exploiting the fact that these connection weight matrices are factorable, we have recently shown that static NEF models can be simulated very efficiently using the SpiNNaker neuromorphic architecture. In this paper, we demonstrate how this approach can be extended to efficiently support both supervised and unsupervised learning rules designed to operate on these factored matrices. We then present a heteroassociative memory architecture built using these learning rules and prove that it is capable of learning a human-scale semantic network. Finally we demonstrate a 100 000 neuron version of this architecture running on the SpiNNaker simulator with a speed-up exceeding 150x when compared to the Nengo reference simulator.

This paper presents an efficient strategy to implement parallel and distributed computing for image processing on a neuromorphic platform. We use SpiNNaker, a many-core neuromorphic platform inspired by neural connectivity in the brain, to achieve fast response and low power consumption. Our proposed method is based on fault-tolerant finegrained parallelism that uses SpiNNaker resources optimally for process pipelining and decoupling. We demonstrate that our method can achieve a performance of up to 49.7 MP/J for Sobel edge detector, and can process 1600 × 1200 pixel images at 697 fps. Using simulated Canny edge detector, our method can achieve a performance of up to 21.4 MP/J. Moreover, the framework can be extended further by using larger SpiNNaker machines. This will be very useful for applications such as energy-aware and time-critical-mission robotics as well as very high resolution computer vision systems.

Real-time dense computer vision and SLAM offer great potential for a new level of scene modelling, tracking and real environmental interaction for many types of robot, but their high computational requirements mean that use on mass market embedded platforms is challenging. Meanwhile, trends in low-cost, low-power processing are towards massive parallelism and heterogeneity, making it difficult for robotics and vision researchers to implement their algorithms in a performance-portable way. In this paper we introduce SLAMBench, a publicly-available software framework which represents a starting point for quantitative, comparable and validatable experimental research to investigate trade-offs in performance, accuracy and energy consumption of a dense RGB-D SLAM system. SLAMBench provides a KinectFusion implementation in C++, OpenMP, OpenCL and CUDA, and harnesses the ICL-NUIM dataset of synthetic RGB-D sequences with trajectory and scene ground truth for reliable accuracy comparison of different implementation and algorithms. We present an analysis and breakdown of the constituent algorithmic elements of KinectFusion, and experimentally investigate their execution time on a variety of multicore and GPU-accelerated platforms. For a popular embedded platform, we also present an analysis of energy efficiency for different configuration alternatives.

By building and simulating neural systems we hope to understand how the brain may work and use this knowledge to build neural and cognitive systems to tackle engineering problems. The Neural Engineering Framework (NEF) is a hypothesis about how such systems may be constructed and has recently been used to build the world's first functional brain model, Spaun. However, while the NEF simplifies the design of neural networks, simulating them using standard computer hardware is still computationally expensive - often running far slower than biological real-time and scaling very poorly: problems the SpiNNaker neuromorphic simulator was designed to solve. In this paper we (1) argue that employing the same model of computation used for simulating general purpose spiking neural networks on SpiNNaker for NEF models results in suboptimal use of the architecture, and (2) provide and evaluate an alternative simulation scheme which overcomes the memory and compute challenges posed by the NEF. This proposed method uses factored weight matrices to reduce memory usage by around 90% and, in some cases, simulate 2000 neurons on a processing core - double the SpiNNaker architectural target.

Deep neural networks have become the state-of-the-art approach for classification in machine learning, and Deep Belief Networks (DBNs) are one of its most successful representatives. DBNs consist of many neuron-like units, which are connected only to neurons in neighboring layers. Larger DBNs have been shown to perform better, but scaling-up poses problems for conventional CPUs, which calls for efficient implementations on parallel computing architectures, in particular reducing the communication overhead. In this context we introduce a realization of a spike-based variation of previously trained DBNs on the biologically-inspired parallel SpiNNaker platform. The DBN on SpiNNaker runs in real-time and achieves a classification performance of 95% on the MNIST handwritten digit dataset, which is only 0.06% less than that of a pure software implementation. Importantly, using a neurally-inspired architecture yields additional benefits: during network run-time on this task, the platform consumes only 0.3 W with classification latencies in the order of tens of milliseconds, making it suitable for implementing such networks on a mobile platform. The results in this paper also show how the power dissipation of the SpiNNaker platform and the classification latency of a network scales with the number of neurons and layers in the network and the overall spike activity rate.

This live demonstration shows real-time visual object recognition based on a spiking neural network adaptation of the HMAX model running on a purely event-based computational hardware platform. Visual input to the system is provided by an ATIS spiking silicon retina sensor. A SpiNNaker board processes the event-encoded visual information from the scene. Using a Leaky Integrate-and-Fire (LIF) neuron model implemented on SpiNNaker, an event-driven, multi-layer network is created that performs real-time orientation extraction and recombination. In this demonstration, the network will be tuned to recognize complex objects such as printed characters.

This paper presents a real-time spiking neural network adaptation of the HMAX object recognition model on an event-driven platform. Visual input is provided by a spiking silicon retina, while the SpiNNaker system is used as a computational hardware platform for implementation. We show the implementation of a simple Leaky Integrate-and-Fire (LIF) neuron model on SpiNNaker to create an event driven network, where a neuron only updates when it receives an interrupt indicating that a new input spike has been received. The model output consists of view tuned neurons which respond selectively to a particular view of an object. The network can be used to discriminate between objects, or between the same object at different views. On a 26 class character recognition task, the correct class is always assigned the highest probability (69.42% on average).

The SpiNNaker Hardware platform allows emulating generic neural network topologies, where each neuron-to-neuron connection is defined by an independent synaptic weight. Consequently, weight storage requires an important amount of memory in the case of generic neural network topologies. This is solved in SpiNNaker by encapsulating with each SpiNNaker chip (which includes 18 ARM cores) a 128MB DRAM chip within the same package. However, ConvNets (Convolutional Neural Network) posses "weight sharing" property, so that many neuron-to-neuron connections share the same weight value. Therefore, a very reduced amount of memory is required to define all synaptic weights, which can be stored on local SRAM DTCM (data-tightly-coupled-memory) at each ARM core. This way, DRAM can be used extensively to store traffic data for off-line analyses. We show an implementation of a 5-layer ConvNet for symbol recognition. Symbols are obtained with a DVS camera. Neurons in the ConvNet operate in an event-driven fashion, and synapses operate instantly. With this approach it was possible to allocate up to 2048 neurons per ARM core, or equivalently 32k neurons per SpiNNaker chip.

We demonstrate an interactive handwritten digit recognition system with a spike-based deep belief network running in real-time on SpiNNaker, a biologically inspired many-core architecture. Results show that during the simulation a SpiNNaker chip can deliver spikes in under 1 μs, with a classification latency in the order of tens of milliseconds, while consuming less than 0.3 W.

Simulation of neural behavior on digital architectures often requires the solution of ordinary differential equations (ODEs) at each step of the simulation. For some neural models, this is a significant computational burden, so efficiency is important. Accuracy is also relevant because solutions can be sensitive to model parameterization and time step. These issues are emphasized on fixed-point processors like the ARM unit used in the SpiNNaker architecture. Using the Izhikevich neural model as an example, we explore some solution methods, showing how specific techniques can be used to find balanced solutions. We have investigated a number of important and related issues, such as introducing explicit solver reduction (ESR) for merging an explicit ODE solver and autonomous ODE into one algebraic formula, with benefits for both accuracy and speed; a simple, efficient mechanism for cancelling the cumulative lag in state variables caused by threshold crossing between time steps; an exact result for the membrane potential of the Izhikevich model with the other state variable held fixed. Parametric variations of the Izhikevich neuron show both similarities and differences in terms of algorithms and arithmetic types that perform well, making an overall best solution challenging to identify, but we show that particular cases can be improved significantly using the techniques described. Using a 1 ms simulation time step and 32-bit fixed-point arithmetic to promote real-time performance, one of the second-order Runge-Kutta methods looks to be the best compromise; Midpoint for speed or Trapezoid for accuracy. SpiNNaker offers an unusual combination of low energy use and real-time performance, so some compromises on accuracy might be expected. However, with a careful choice of approach, results comparable to those of general-purpose systems should be possible in many realistic cases.

The human brain is a complex biological neural network characterised by high degrees of connectivity among neurons. Any system designed to simulate large-scale spiking neuronal networks needs to support such connectivity and the associated communication traffic in the form of spike events. This paper investigates how best to generate multicast routes for SpiNNaker, a purpose-built, low-power, massively-parallel architecture. The discussed algorithms are an essential ingredient for the efficient operation of SpiNNaker since generating multicast routes is known to be an NP-complete problem. In fact, multicast communications have been extensively studied in the literature, but we found no existing algorithm adaptable to SpiNNaker. The proposed algorithms exploit the regularity of the two-dimensional triangular torus topology and the availability of selective multicast at hardware level. A comprehensive study of the parameters of the algorithms and their effectiveness is carried out in this paper considering different destination distributions ranging from worst-case to a real neural application. The results show that two novel proposed algorithms can reduce significantly the pressure exerted onto the interconnection infrastructure while remaining effective to be used in a production environment.

SpiNNaker is a multi-core computing engine, with a bespoke and specialised communication infrastructure that supports almost perfect scalability up to a hard limit of 216× 18 = 1,179,648 cores. This remarkable property is achieved at the cost of ignoring memory coherency, global synchronisation and even deterministic message passing, yet it is still possible to perform meaningful computations. Whilst we have yet to assemble the full machine, the scalability properties make it possible to demonstrate the capabilities of the machine whilst it is being assembled; the more cores we connect, the larger the problems become that we are able to attack. Even with isolated printed circuit boards of 864 cores, interesting capabilities are emerging. This paper is the third of a series charting the development trajectory of the system. In the first two, we outlined the hardware build. Here, we lay out the (rather unusual) low-level foundation software developed so far to support the operation of the machine.

Living organisms are capable of autonomously adapting to dynamically changing environments by receiving inputs from highly specialized sensory organs and elaborating them on the same parallel, power-efficient neural substrate. In this paper we present a prototype for a comprehensive integrated platform that allows replicating principles of neural information processing in real-time. Our system consists of (a) an autonomous mobile robotic platform, (b) on-board actuators and multiple (neuromorphic) sensors, and (c) the SpiNNaker computing system, a configurable neural architecture for exploration of parallel, brain-inspired models. The simulation of neurally inspired perception and reasoning algorithms is performed in real-time by distributed, low-power, low-latency event-driven computing nodes, which can be flexibly configured using C or specialized neural languages such as PyNN and Nengo. We conclude by demonstrating the platform in two experimental scenarios, exhibiting real-world closed loop behavior consisting of environmental perception, reasoning and execution of adequate motor actions.

Simulations of biological tissue have been extensively used to replicate phenomena observed by in-vivo and in-vitro experiments as an alternative methodology for explaining how computations could take place in a brain region. Additional benefits of simulated neural networks over in-vivo experiments include greater observability, experimental control and reproducibility. General-purpose supercomputers provide the computational power and parallelism required to implement highly complex neural models, but this comes at the expense of high power requirements and communication overheads. Moreover, there are certain cases where real-time simulation performance is a desirable feature, for example in the field of cognitive robotics where embodied agents need to interact with their environment through biologically inspired asynchronous sensors. The SpiNNaker neuromimetic platform is a scalable architecture that has been designed to enable energy-efficient, large-scale simulations of spiking neurons in biological realtime. This work is based on a recent study which revealed that while they are generally energy efficient, SpiNNaker chips dissipate significant amount of power whilst in the idle state. In this paper we perform a systematic investigation into the overall energy consumption of a SpiNNaker system and propose a number of optimised suspend modes in order to reduce this. The proposed implementation is 60% more energy efficient in the idle state, 50% in the uploading and 52% in the downloading phases, while the power dissipation of the whole simulation is reduced by 52%. For demonstration purposes, we run a neural network simulation comprising thousands of neurons and millions of complex synapses on a 48-chip SpiNNaker board, generating millions of synaptic events per second.

The SpiNNaker (Spiking Neural Network Architecture) project will soon deliver a machine incorporating a million ARM processor cores for real-time modelling of large-scale spiking neural networks. Although the scale of the machine is in the realms of high-performance computing, the technology used to build the machine comes very much from the mobile embedded world, using small integer cores and Network-on-Chip communications both on and between chips. The full machine will use a total of 10 square meters of active silicon area with 57,600 routers using predominantly multicast algorithms to convey real- time spike information through a lightweight asynchronous packet-switched fabric. In this talk I will focus on the NoC aspects, including novel approaches to fault-tolerance and deadlock avoidance.

The spiking neural network architecture (SpiNNaker) project aims to deliver a massively parallel million-core computer whose interconnect architecture is inspired by the connectivity characteristics of the mammalian brain, and which is suited to the modeling of large-scale spiking neural networks in biological real time. Specifically, the interconnect allows the transmission of a very large number of very small data packets, each conveying explicitly the source, and implicitly the time, of a single neural action potential or “spike.” In this paper, we review the current state of the project, which has already delivered systems with up to 2500 processors, and present the real-time event-driven programming model that supports flexible access to the resources of the machine and has enabled its use by a wide range of collaborators around the world.

Recent papers have shown the possibility to implement large scale neural network models that perform complex algorithms in a biologically realistic way. However, such models have been simulated on architectures unable to perform real-time simulations. In previous work we presented the possibility to simulate simple models in real-time on the SpiNNaker neuromimetic architecture. However, such models were “static”: the algorithm performed was defined at design-time. In this paper we present a novel learning rule, that exploits the peculiarities of the SpiNNaker system, enabling models designed with the Neural Engineering Framework (NEF) to learn transfer functions using a supervised framework. We show that the proposed learning rule, belonging to the Prescribed Error Sensitivity (PES) class, is able to learn, effectively, both linear and non-linear functions.

When two or more sound detectors are available, interaural time differences may be used to determine the direction of a sound's origin. This process, known as sound localization, is performed in mammals via the auditory pathways of the head and by computation in the brain. The Jeffress Model successfully describes the mechanism by exploiting coincidence detector neurons in conjunction with delay lines. However, one of the difficulties of using this model on neural simulators is that it requires timing accuracies which are much finer than the typical 1 ms resolution provided by simulation platforms. One solution is clearly to reduce the simulation's time step, but in this paper we also explore the use of population coding to represent more precise timing information without changing the simulation's timing resolution. The implementation of both the Jeffress and population coded models are contrasted, together with their results, which show that population coding is indeed able to provide successful sound localization.

With neuromorphic hardware rapidly moving towards large-scale, possibly immovable systems capable of implementing brain-scale neural models in hardware, there is an emerging need to be able to integrate multi-system combinations of sensors and cortical processors over distributed, multisite configurations. If there were a standard, direct interface allowing large systems to communicate using native signalling, it would be possible to use heterogeneous resources efficiently according to their task suitability. We propose a UDP-based AER spiking interface that permits direct bidirectional spike communications over standard networks, and demonstrate a practical implementation with two large-scale neuromorphic systems, BrainScaleS and SpiNNaker. Internally, the interfaces at either end appear as interceptors which decode and encode spikes in a standardised AER address format onto UDP frames. The system is able to run a spiking neural network distributed over the two systems, in both a side-by-side setup with a direct cable link and over the Internet between 2 widely spaced sites. Such a model not only realises a solution for connecting remote sensors or processors to a large, central neuromorphic simulation platform, but also opens possibilities for interesting automated remote neural control, such as parameter tuning, for large, complex neural systems, and suggests methods to overcome differences in timescale and simulation model between different platforms. With its entirely standard protocol and physical layer, the interface makes large neuromorphic systems a distributed, accessible resource available to all.

SpiNNaker is a computer architecture designed to simulate many millions of neurons in real-time, using very many low-power processors and biologically inspired communications. This paper demonstrates that prototype SpiNNaker hardware correctly and quickly simulates networks of point neurons with respect to established reference simulators. Models of increasing complexity are presented and the simulation results obtained from SpiNNaker, NEST and Brian are shown to correlate. An execution profile is sketched of real-time simulation on SpiNNaker, and it is shown to outperform NEST using similar computational resources on a standard benchmark model.

Simulating large spiking neural networks is non trivial: supercomputers offer great flexibility at the price of power and communication overheads; custom neuromorphic circuits are more power efficient but less flexible; while alternative approaches based on GPGPUs and FPGAs, whilst being more readily available, show similar model specialization. As well as efficiency and flexibility, real time simulation is a desirable neural network characteristic, for example in cognitive robotics where embodied agents interact with the environment using low-power, event-based neuromorphic sensors. The SpiNNaker neuromimetic architecture has been designed to address these requirements, simulating large-scale heterogeneous models of spiking neurons in real-time, offering a unique combination of flexibility, scalability and power efficiency. In this work a 48-chip board is utilised to generate a SpiNNaker power estimation model, based on numbers of neurons, synapses and their firing rates. In addition, we demonstrate simulations capable of handling up to a quarter of a million neurons, 81 million synapses and 1.8 billion synaptic events per second, with the most complex simulations consuming less than 1 Watt per SpiNNaker chip.

SpiNNaker is a biologically-inspired massively-parallel computer designed to model up to a billion spiking neurons in real-time. A full-fledged implementation of a SpiNNaker system will comprise more than 105 integrated circuits (half of which are SDRAMs and half multi-core systems-on-chip). Given this scale, it is unavoidable that some components fail and, in consequence, fault-tolerance is a foundation of the system design. Although the target application can tolerate a certain, low level of failures, important efforts have been devoted to incorporate different techniques for fault tolerance. This paper is devoted to discussing how hardware and software mechanisms collaborate to make SpiNNaker operate properly even in the very likely scenario of component failures and how it can tolerate system-degradation levels well above those expected.

SpiNNaker (a contraction of Spiking Neural Network Architecture) is a million-core computing engine whose flagship goal is to be able to simulate the behavior of aggregates of up to a billion neurons in real time. It consists of an array of ARM9 cores, communicating via packets carried by a custom interconnect fabric. The packets are small (40 or 72 bits), and their transmission is brokered entirely by hardware, giving the overall engine an extremely high bisection bandwidth of over 5 billion packets/s. Three of the principal axioms of parallel machine design (memory coherence, synchronicity, and determinism) have been discarded in the design without, surprisingly, compromising the ability to perform meaningful computations. A further attribute of the system is the acknowledgment, from the initial design stages, that the sheer size of the implementation will make component failures an inevitable aspect of day-to-day operation, and fault detection and recovery mechanisms have been built into the system at many levels of abstraction. This paper describes the architecture of the machine and outlines the underlying design philosophy; software and applications are to be described in detail elsewhere, and only introduced in passing here as necessary to illuminate the description.

One approach to creating a massively-parallel high-performance machine is to use large quantities of power-efficient processors, primarily due to the energy consumption of conventional high-performance computing designs. SpiNNaker is a novel high-performance architecture formed by large numbers of highly-interconnected low-power processing elements, more typically found in embedded systems. This paper presents the results of the implementation of a low-overhead management framework enabled by a universal translation layer: SpiNNmate. SpiNNmate is located between a SpiNNaker machine and the communication protocols of external applications, and we include results from a translation of the standards-based SNMP protocol to SpiNNaker.

SpiNNaker is a hardware-based massively-parallel real-time universal neural network simulator designed to simulate large-scale spiking neural networks. Spikes are distributed across the system using a multicast packet router. Each packet represents an event (spike) generated by a neuron. On the basis of the source of the spike (chip, core and neuron), the routers distribute the network packet across the system towards the destination neuron(s). This paper describes a novel approach to the projection routing problem that shows advantages in both the size of the routing tables generated and the computational complexity for the generation of routing tables. To achieve this, spikes are routed on the basis of the source population, leaving to the destination core the duty to propagate the received spike to the appropriate neuron(s).

Simulation of large-scale networks of spiking neurons has become appealing for understanding the computational principles of the nervous system by producing models based on biological evidence. In particular, networks that can assume a variety of (dynamically) stable states have been proposed as the basis for different behavioural and cognitive functions. This work focuses on implementing the Neural Engineering Framework (NEF), a formal method for mapping attractor networks and control-theoretic algorithms to biologically plausible networks of spiking neurons, on the SpiNNaker system, a massive programmable parallel architecture oriented to the simulation of networks of spiking neurons. We describe how to encode and decode analog values to patterns of neural spikes directly on chip. These methods take advantage of the full programmability of the ARM968 cores constituting the processing base of a SpiNNaker node, and exploit the fast Network-on-chip for spike communication. In this paper we focus on the fundamentals of representing, transforming and implementing dynamics in spiking networks. We show real time simulation results demonstrating the NEF principles and discuss advantages, precision and scalability. More generally, the present approach can be used to state and test hypotheses with large-scale spiking neural network models for a range of different cognitive functions and behaviours.

As models of neural networks scale in concert with increasing computational performance, gaining insight into their operation becomes increasingly important. This paper proposes an efficient and generalised method to access simulation data via in-system aggregation, providing visualised representation at all layers of the network in real-time. Enabling neural networks for real-time visualisation allows a user to gain insight into the network dynamics of their systems as they operate over time. This visibility also permits users (or a computational agent) to determine whether early intervention is required to adjust parameters, or even to terminate operation of experimental networks that are not operating correctly. Conventionally the determination of correctness would occur post-simulation, so with sufficient `in-flight' insight, a significant advantage may be obtained, and compute time minimised. For this paper we apply the real-time visualisation platform to the SpiNNaker programmable neuromimetic system and a variety of neural network models. The visualisation platform is shown to be capable across a range of diverse simulations, and at supporting differing layers of network abstraction, requiring minimal configuration to represent each model. The resulting general-purpose visualisation platform for neural networks, is effective at presenting data to users in order to aid their comprehension of the network dynamics during operation, and scales from small to biologically-significant network sizes.

While new neural hardware is increasingly emphasizing spiking neural models, there will still be a need to model “classical” neural networks like the multilayer perceptron (MLP) for the foreseeable future. Given that the trend in new chips is towards a “neuromimetic” design that specialises the hardware for neural networks but does not hardwire the model, it is worth examining whether it is possible to implement the MLP on such hardware and realise performance gains over conventional simulation on general-purpose computers. Using the SpiNNaker chip as a demonstration platform, we show that it is possible to find efficient mappings to improve both the performance and the scalability of the MLP network, allowing for much larger models than possible in software. These mappings, however require a careful consideration of how to transform the “timeless” MLP model into an event-driven implementation. Examination of the hardware performance also reveals the importance of distributing processing and traffic load so that local congestion does not end up crippling the simulation. These considerations being solved, not only does the hardware demonstrate the potential for significant performance improvement, it also illustrates important general techniques and methods for translating nonspiking models onto the emerging generation of spiking neural hardware. The results suggest both the form that models might take and the architectures that future neural hardware could adopt for optimum generality and performance.

Moore's Law continues to deliver ever-more transistors on an integrated circuit, but discontinuities in the progress of technology mean that the future isn't simply an extrapolation of the past. For example, design cost and complexity constraints have recently caused the microprocessor industry to switch to multi-core architectures, even though these parallel machines present programming challenges that are far from solved. Moore's Law now translates into ever-more processors on a multi-, and soon many-core chip. The software challenge is compounded by the need for increasing fault-tolerance as near-atomic-scale variability and robustness problems bite harder. We look beyond this transitional phase to a future where the availability of processor resource is effectively unlimited and computations must be optimised for energy usage rather than load balancing, and we look to biology for examples of how such systems might work. Conventional concerns such as synchronisation and determinism are abandoned in favour of real-time operation and adapting around component failure with minimal loss of system efficacy.

The scaling of MOSFETs has improved performance and lowered the cost per function of CMOS integrated circuits and systems over the last 40 years, but devices are subject to increasing amounts of statistical variability within the deca-nano domain. The causes of these statistical variations and their effects on device performance have been extensively studied, but there have been few systematic studies of their impact on circuit performance. This paper describes a method for modelling the impact of random intra-die statistical variations on digital circuit timing and power consumption. The method allows the variation modelled by large-scale statistical transistor simulations to be propagated up the design flow to the circuit level, by making use of commercial STA and standard cell characterisation tools. The method provides circuit designers with the information required to analyse power, performance and yield trade-offs when fabricating a design, while removing the large levels of pessimism generated by traditional Corner Based Analysis.

Spike Timing Dependent Plasticity (STDP) is a well known paradigm for learning in neural networks. In this paper we propose a new approach to this problem based on the standard STDP algorithm, with modifications and approximations, that relate the membrane potential with the LTP (Long Term Potentiation) part of the basic STDP rule. On the other side we use the standard STDP rule for the LTD (Long Term Depression) part of the algorithm. We show that on the basis of the membrane potential [5] it is possible to make a statistical prediction of the time needed by the neuron to reach the threshold, and therefore the LTP part of the STDP algorithm can be triggered when the neuron receives a spike.We present results that show the efficacy of this algorithm using one or more input patterns repeated over the whole time of the simulation. Through the approximations we suggest in this paper we introduce a learning rule that is easy to implement in simulators and reduces the execution time if compared with the standard STDP rule.

The introduction of axonal delays in networks of spiking neurons has enhanced the representational capabilities of neural networks, whilst also providing more biological realism. Approaches in neural coding such as rank order coding and polychronization have exploited the precise timing of action potential observed in real neurons. In a rank order code information is coded in the order of firing of a pool of neurons; on the other hand with polychronization it is the time of arrival of different spikes at the postsynaptic neuron which triggers different post-synaptic responses, with the axonal delays compensating for different timings in the afferents. In this paper we propose a model in which rank order coding is used to represent an arbitrary symbol, and a polychronous layer is used to decode, represent and recall that symbol. To prove that the polychronous layer is able to do this a detector neuron is trained with a supervised learning strategy and associated with a single code. According to this premise the detector neuron only fires on the appearance of the associated code, even in the presence of noise. Tests prove that rank order coding and polychronization can be coupled to code and decode information such as intensity or significance using timing information in spiking neural networks in an effective way.

Neural networks present a fundamentally different model of computation from conventional sequential hardware, making it inefficient for very-large-scale models. Current neuromorphic devices do not yet offer a fully satisfactory solution even though they have improved simulation performance, in part because of fixed hardware, in part because of poor software support. SpiNNaker introduces a different approach, the “neuromimetic” architecture, that maintains the neural optimisation of dedicated chips while offering FPGA-like universal configurability. Central to this parallel multiprocessor is an asynchronous event-driven model that uses interrupt-generating dedicated hardware on the chip to support real-time neural simulation. In turn this requires an event-driven software model: a rethink as fundamental as that of the hardware. We examine this event-driven software model for an important hardware subsystem, the previously-introduced virtual synaptic channel. Using a scheduler-based system service architecture, the software can “hide” low-level processes and events from models so that the only event the model sees is “spike received”. Results from simulation on-chip demonstrate the robustness of the system even in the presence of extremely bursty, unpredictable traffic, but also expose important model-evel tradeoffs that are a consequence of the physical nature of the SpiNNaker chip. This event-driven subsystem is the first component of a library-based development system that allows the user to describe a model in a high-level neural description environment and be able to rely on a lower layer of system services to execute the model efficiently on SpiNNaker. Such a system realises a general-purpose platform that can generate an arbitrary neural network and run it with hardware speed and scale.

SpiNNaker is a massively-parallel neuromorphic computing architecture designed to model very large, biologically plausible spiking neural networks in real-time. A SpiNNaker machine consists of up to 216homogeneous eighteen-core multiprocessor chips, each with an on-board router which forms links with neighbouring chips for packet-switched interprocessor communications. The architecture is designed for dynamic reconfiguration and optimised for transmission of neural activity data, which presents a challenge for machine configuration, program loading and simulation monitoring given a lack of globally-shared memory resources, intrinsic addressing mode or sideband configuration channel. We propose distributed software mechanisms to address these problems and present experiments which demonstrate the necessity of this approach in contrast to centralised mechanisms.

The design and implementation of globally asynchronous locally synchronous systems-on-chip is a challenging activity. The large size and complexity of the systems require the use of computer-aided design (CAD) tools but, unfortunately, most tools do not work adequately with asynchronous circuits. This article describes the successful design and implementation of SpiNNaker, a GALS multicore system-on-chip. The process was completed using commercial CAD tools from synthesis to layout. A hierarchical methodology was devised to deal with the asynchronous sections of the system, encapsulating and validating timing assumptions at each level. The crossbar topology combined with a pipelined asynchronous fabric implementation allows the on-chip network to meet the stringent requirements of the system. The implementation methodology constrains the design in a way that allows the tools to complete their tasks successfully. A first test chip, with reduced resources and complexity was taped-out using the proposed methodology. Test chips were received in December 2009 and were fully functional. The methodology had to be modified to cope with the increased complexity of the SpiNNaker SoC. SpiNNaker chips were delivered in May 2011 and were also fully operational, and the interconnect requirements were met.

A new hardware scheme for computing the transition and control matrix of a parallel cyclic redundancy checksum is proposed. This opens possibilities for parallel high-speed cyclic redundancy checksum circuits that reconfigure very rapidly to new polynomials. The area requirements are lower than those for a realization storing a precomputed matrix. An additional simplification arises as only the polynomial needs to be supplied. The derived equations allow the width of the data to be processed in parallel to be selected independently of the degree of the polynomial. The new design has been simulated and outperforms a recently proposed architecture significantly in speed, area, and energy efficiency.

This paper presents the algorithm and software developed for parallel simulation of spiking neural networks on multiple SpiNNaker universal neuromorphic chips. It not only describes approaches to simulating neural network models, such as dynamics, neural representations, and synaptic delays, but also presents the software design of loading a neural application and initial a simulation on the multi-chip SpiNNaker system. A series of sub-issues are also investigated, such as neuron-processor allocation, synapses distribution, and route planning. The platform is verified by running spiking neural applications on both the SoC Designer model and the physical SpiNNaker Test Chip. This work sums the problems we have solved and highlights those requiring further investigations, and therefore it forms the foundation of the software design on SpiNNaker, leading the future development towards a universal platform for real-time simulations of extreme large-scale neural systems.

This paper presents an efficient approach for implementing spike-timing-dependent plasticity (STDP) on the SpiNNaker neuromorphic hardware. The event-address mapping and the distributed synaptic weight storage schemes used in parallel neuromorphic hardware such as SpiNNaker make the conventional pre-post-sensitive scheme of STDP implementation inefficient, since STDP is triggered when either a pre- or post-synaptic neuron fires. An alternative pre-sensitive scheme approach is presented to solve this problem, where STDP is triggered only when a pre-synaptic neuron fires. An associated deferred event-driven model is developed to enable the pre-sensitive scheme by deferring the STDP process until there are sufficient history spike timing records. The paper gives detailed description of the implementation as well as performance estimation of STDP on multi-chip SpiNNaker machine, along with the discussion on some issues related to efficient STDP implementation on a parallel neuromorphic hardware.

Large-scale neural hardware systems are trending increasingly towards the “neuromimetic” architecture: a general-purpose platform that specialises the hardware for neural networks but allows flexibility in model choice. Since the model is not hard-wired into the chip, exploration of different neural and synaptic models is not merely possible but provides a rich field for research: the possibility to use the hardware to establish useful abstractions of biological neural dynamics that could lead to a functional model of neural computation. Two areas of neural modelling stand out as central: 1) What level of detail in the neurodynamic model is necessary to achieve biologically realistic behaviour? 2) What is role and effect of different types of synapses in the computation? Using a universal event-driven neural chip, SpiNNaker, we develop a simple model, the Leaky-Integrate-and-Fire (LIF) neuron, as a tool for exploring the second of these questions, complementary to the existing Izhikevich model which allows exploration of the first of these questions. The LIF model permits the development of multiple synaptic models including fast AMPA/GABA-A synapses with or without STDP learning, and slow NMDA synapses, spanning a range of different dynamic time constants. Its simple dynamics make it possible to expand the complexity of synaptic response, while the general-purpose design of SpiNNaker makes it possible if necessary to increase the neurodynamic accuracy with Izhikevich (or even Hodgkin-Huxley) neurons with some tradeoff in model size. Furthermore, the LIF model is a universally-accepted “standard” neural model that provides a good basis for comparisons with software simulations and introduces minimal risk of obscuring important synaptic effects due to unusual neurodynamics. The simple models run thus far demonstrate the viability of both the LIF model and of various possible synaptic models on SpiNNaker and illustrate how it can be used as a platform for model exploration. Such an architecture provides a scalable system for high-performance large-scale neural modelling with complete freedom in model choice.

This paper demonstrates the feasibility and evaluates the performance of using the SpiNNaker neuromorphic hardware to simulate traditional non-spiking multi-layer perceptron networks with the back propagation learning rule. In addition to investigating the mapping of checker-boarding partitioning scheme onto SpiNNaker, we propose a new algorithm called pipelined checker-boarding partitioning which introduces a pipelined mode and captures the parallelism within each partition of the weight matrix, allowing the overlapping of communication and computation. Not only does the proposed algorithm localize communication, but it can also hide a part of or even all the communication. The performance is evaluated with SpiNNaker configurations up to 1000 nodes (20000 cores).

A multicast routing infrastructure is proposed as a core feature of SpiNNaker, a massively parallel computer for the real-time simulation of large-scale spiking neural networks. The infrastructure is implemented using a communications router, based on an event-driven routing scheme, on each multicore processing node in the system. The design considerations emphasize the difference between the requirements of neural network communications and those of conventional computer networks and on-chip networks. The focus of the design is on neural modelling flexibility, power-efficiency, fault-tolerance and the communication throughput of the router.

In this paper, we present biologically inspired means to enhance perceptually important information retrieval from rank-order encoded images. Validating a retinal model proposed by VanRullen and Thorpe, we observe that on average only up to 70% of the available information can be retrieved from rank-order encoded images. We propose a biologically inspired treatment to reduce losses due to a high correlation of adjacent basis vectors and introduce a filter-overlap correction algorithm (FoCal) based on the lateral inhibition technique used by sensory neurons to deal with data redundancy. We observe a more than 10% increase in perceptually important information recovery. Subsequently, we present a model of the primate retinal ganglion cell layout corresponding to the foveal-pit. We observe that information recovery using the foveal-pit model is possible only if FoCal is used in tandem. Furthermore, information recovery is similar for both the foveal-pit model and VanRullen and Thorpe's retinal model when used with FoCal. This is in spite of the fact that the foveal-pit model has four ganglion cell layers as in biology while VanRullen and Thorpe's retinal model has a 16-layer structure.

SpiNNaker is a massively parallel architecture designed to model large-scale spiking neural networks in (biological) real-time. Its design is based around ad-hoc multi-core System-on-Chips which are interconnected using a two-dimensional toroidal triangular mesh. Neurons are modeled in software and their spikes generate packets that propagate through the on- and inter-chip communication fabric relying on custom-made on-chip multicast routers. This paper models and evaluates large-scale instances of its novel interconnect (more than 65 thousand nodes, or over one million computing cores), focusing on real-time features and fault-tolerance. The key contribution can be summarized as understanding the properties of the feasible topologies and establishing the stable operation of the SpiNNaker under different levels of degradation. First we derive analytically the topological characteristics of the network, which are later confirmed by experimental work. With the computational model developed, we investigate the topology of SpiNNaker, and compare it with a standard 3-dimensional torus. The novel emergency routing mechanism, implemented within the routers, allows the topology of SpiNNaker to be more robust than the 3-dimensional torus, regardless of the latter having better topological characteristics. Furthermore, we obtain optimal values of two router parameters related with livelock and deadlock avoidance mechanisms.

We propose a model of the primate retinal ganglion cell layout corresponding to the foveal-pit to test rank-order codes as a means of sensory information transmission in primate vision. We use the model for encoding images in rank-order. We observe that the model is functional only when the lateral inhibition technique is used to remove redundancy from the sampled data. Further, more than 80% of the input information can be decoded by the time only up to 10% of the ganglion cells of our model have fired their first spikes.

High-speed asynchronous hardware makes it possible to virtualise neural networks' temporal dynamics as well as their structure. Through SpiNNaker, a dedicated neural chip multiprocessor, we introduce a real-time modelling architecture that makes the neural model run on the device independent of the hardware specifics. The central features of this modelling architecture are: native concurrency, ability to support very large (Gt 109neurons) networks, and decoupling of the temporal and spatial characteristics of the model from those of the hardware. It circumvents a virtually fatal tradeoff in large-scale neural hardware between model support limitations or scalability limitations, without imposing a synchronous timing model. The chip itself combines an array of general-purpose processors with a configurable asynchronous interconnect and memory fabric to achieve true on- and off-chip parallelism, universal network architecture support, and programmable temporal dynamics. An HDL-like concurrent configuration software model using libraries of templates, allows the user to embed the neural model onto the hardware, mapping the virtual network structure and time dynamics into physical on-chip components and delay specifications. Initial modelling experiments demonstrate the ability of the processor to support real-time neural processing using 2 different neural models. The complete system is therefore an environment able, within a wide range of model characteristics, to model real-time dynamic neural network behaviour on dedicated hardware.

In large neural networks, partial connectivity is both biologically plausible and a matter of necessity when targetting a hardware implementation. We are using the SpiNNaker neural chip multiprocessor to model such networks as a drop-in replacement for the Lens network simulator. For the popular MLP network, a theoretical model of the relation between connectivity, network size and gain in the activation function provides a method to set these parameters to near-optimal values. Using the model, we run a series of network simulations in Lens, permuting the parameters to explore the effects in 2 networks of different size and application. Initial test results show a clear connectivity-gain relation and a benefit to partial connectivity in both networks, with optimal hidden-output connectivity values ranging from ~10%-~30% depending on the network type. We show that optimal connectivity-gain settings reduce training time, minimising error oscillations during learning. Preliminary analysis also suggests that while very low connectivities may improve error they may also result in decreased adaptivity to new inputs or component failure. These results in combination with the theoretical relation give a method for determining reasonable initial connectivity and gain valuesatdesigntimefor an MLP network, allowing more efficient use of hardware resources such as SpiNNaker and faster simulations in any software environment. They also suggest a different way of considering the problem of MLP network design: rather than specify a fixed number of neurons, specify a fixed number of connections and vary the number of neurons to reach optimal connectivity.

Configuring a million-core parallel system at boot time is a difficult process when the system has neither specialised hardware support for the configuration process nor a preconfigured default state that puts it in operating condition. SpiNNaker is a parallel chip multiprocessor (CMP) system for neural network (NN) simulation. Where most large CMP systems feature a sideband network to complete the boot process, SpiNNaker has a single homogeneous network interconnect for both application inter-processor communications and system control functions such as boot load and run-time user-system interaction. This network improves fault tolerance and makes it easier to support dynamic run-time reconfiguration, however, it requires a boot process that is transaction-level compatible with the application's communications model. Since SpiNNaker uses event-driven asynchronous communications throughout, the loader operates with purely local control: there is no global synchronisation, state information, or transition sequence. A novel two-stage ldquounfoldingrdquo boot-up process efficiently configures the SpiNNaker hardware and loads the application using a high-speed flood-fill technique with support for run-time re-configuration. SystemC simulation of a multi-CMP SpiNNaker system indicates an error-free CMP configuration time of 1.3 ms, while a high-level simulation of a full-scale system (64 K CMPs) indicates a mean application-loading time of ~20 ms (for a 100 KB application), which is virtually independent of the size of the system. We verified the CMP configuration process with hardware-level Verilog simulation.

A Token-ManagedAdmission Control (TMAC) mechanism is introduced in order to provide efficient Quality-of-Service (QoS) support for different types of application on a best-effort Globally-Asynchronous Locally-Synchronous (GALS) interconnect fabric. The mechanism is applied at the ingress edges of the fabric using tokens to allocate dynamic network resources and prevent network congestion. The degree of fairness is controllable, in order to balance the desired throughput and data transfer resource allocation appropriately for a particular application. The simulation and analysis presented here shows efficient QoS provision. Our detailed implementation and analysis show that TMAC provides service guarantees on the network while using a modest physical area because of the simplicity of the control logic.

We propose a system based on the Izhikevich model running on a scalable chip multiprocessor - SpiNNaker - for large-scale spiking neural network simulation. The design takes into account the requirements for processing, storage, and communication which are essential to the efficient modelling of spiking neural networks. To gain a speedup of the processing as well as saving storage space, the Izhikevich model is implemented in 16-bit fixed-point arithmetic. An approach based on using two scaling factors is developed, making the precision comparable to the original. With the two scaling factors scheme, all of the firing patterns by the original model can be reproduced with a much faster execution speed. To reduce the communication overhead, rather than sending synaptic weights on communicating, we only send out event packets to indicate the neuron firings while holding the synaptic weights in the memory of the post-synaptic neurons, which is so-called event-driven algorithm. The communication based on event packets can be handled efficiently by the multicast system supported by the SpiNNaker machine. We also describe a system level model for spiking neural network simulation based on the schemes above. The model has been functionally verified and experimental results are included. An analysis of the performance of the whole system is presented at the end of the paper.

SpiNNaker is a novel chip - based on the ARM processor - which is designed to support large scale spiking neural networks simulations. In this paper we describe some of the features that permit SpiNNaker chips to be connected together to form scalable massively-parallel systems. Our eventual goal is to be able to simulate neural networks consisting of 109neurons running in dasiareal timepsila, by which we mean that a similarly sized collection of biological neurons would run at the same speed. In this paper we describe the methods by which neural networks are mapped onto the system, and how features designed into the chip are to be exploited in practice. We will also describe the modelling and verification activities by which we hope to ensure that, when the chip is delivered, it will work as anticipated.

Given the limited current understanding of the neural model of computation, hardware neural network architectures that impose a specific relationship between physical connectivity and model topology are likely to be overly restrictive. Here we introduce, in the SpiNNaker chip, an alternative approach: a mappable virtual topology using an asynchronous network-on-chip (NoC) that decouples the “logical” connectivity map from the physical wiring. Borrowing the established digital RAM model for synapses, we develop a concurrent memory access channel optimised for neural processing that allows each processing node to perform its own synaptic updates as if the synapses were local to the node. The highly concurrent nature of interconnect access, however, requires careful design of intermediate buffering and arbitration. We show here how a locally buffered, one-transaction-per-node model with multiple synapse updates per transaction enables the local node to offload continuous burst traffic from the NoC, allowing for a hardware-efficient design that supports biologically realistic speeds. The design not only presents a flexible model for neural connectivity but also suggests an ideal form for general-purpose high-performance on-chip interconnect.

Progress in computer technology over the last four decades has been spectacular, driven by Moore's law which, though initially an observation, has become a self-fulfilling prophecy and a boardroom planning tool. Although Gordon Moore expressed his vision of progress simply in terms of the number of transistors that could be manufactured economically on an integrated circuit, the means of achieving this progress was based principally on shrinking transistor dimensions, and with that came collateral gains in performance, power-efficiency and, last but not least, cost. The semiconductor industry appears to be confident in its ability to continue to shrink transistors, at least for another decade or so, but the game is already changing. We can no longer assume that smaller circuits will go faster, or be more power-efficient. As we approach atomic limits, device variability is beginning to hurt, and design costs are going through the roof. These are impacting the economics of design in ways that will affect the entire computing and communications industries. For example, on the desktop there is a trend away from high-speed uniprocessors towards multi-core processors, despite the fact that general-purpose parallel programming remains one of the greatest unsolved problems of computer science. If computers are to benefit from future advances in technology then there are major challenges ahead, involving understanding how to build reliable systems on increasingly unreliable technology and how to exploit parallelism increasingly effective, not only to improve performance, but also to mask the consequences of component failure. Biological systems demonstrate many of the properties we aspire to incorporate into our engineered technology, so perhaps that suggests a possible source of ideas that we could seek to incorporate into future novel computation systems?

A variant of a sparse distributed memory (SDM) is shown to have the capability of storing and recalling patterns containing rank-order information. These are patterns where information is encoded not only in the subset of neuron outputs that fire, but also in the order in which that subset fires. This is an interesting companion to several recent works in the neuroscience literature, showing that human memories may be stored in terms of neural spike timings. In our model, the ordering is stored in static synaptic weights using a Hebbian single-shot learning algorithm, and can be reliably recovered whenever the associated input is supplied. It is shown that the memory can operate using only unipolar binary connections throughout. The behavior of the memory under noisy input conditions is also investigated. It is shown that the memory is capable of improving the quality of the data that passes through it. That is, under appropriate conditions the output retrieved from the memory is less noisy than the input used to retrieve it. Thus, this memory architecture could be used as a component in a complex system with stable noise properties and, we argue, it can be implemented using spiking neurons.

The resources available on a chip continue to grow, following Moore's Law. However, the major process by which the benefits of Moore's Law accrue, which is the continuing reduction in feature size, is predicted to bring with it disadvantages in terms of device reliability and parameter variability. The problems that this will bring are underlined by the predictions from an Intel commentator: within a decade we will see 100 billion transistor chips. That is the good news. The bad news is that 20 billion of those transistors will fail in manufacture and a further 10 billion will fail in the first year of operation.

The real-time modeling of large systems of spiking neurons is computationally very demanding in terms of processing power, synaptic weight memory requirements and communication throughput. We propose to build a high-performance computer for this purpose with a multicast communications infrastructure inspired by neurobiology. The core component is a chip multiprocessor incorporating some tens of small embedded processors, interconnected by a NoC that carries spike events between processors on the same or different chips. The design emphasizes modeling flexibility, power-efficiency, and fault-tolerance, and is intended to yield a general-purpose platform for the real-time simulation of large-scale spiking neural systems

This paper presents the design of an associative memory with feedback that is capable of on-line temporal sequence learning. A framework for on-line sequence learning has been proposed, and different sequence learning models have been analysed according to this framework. The network model is an associative memory with a separate store for the sequence context of a symbol. A sparse distributed memory is used to gain scalability. The context store combines the functionality of a neural layer with a shift register. The sensitivity of the machine to the sequence context is controllable, resulting in different characteristic behaviours. The model can store and predict on-line sequences of various types and length. Numerical simulations on the model have been carried out to determine its properties.

The CHAIN self-timed network-on-chip (NoC) architecture provides a flexible, clock-independent solution to the problems of system-on-chip (SoC) interconnect. In this paper we look at the use of CHAIN in a low-performance, smartcard chip to connect two self-timed processors and a range of memories and peripherals. Key design-time advantages provided by the use of CHAIN in this design included the ability to operate a very-narrow, high-frequency network fabric using serial communication without the need for high frequency clocking, rapid assembly in the final stages of the design and the avoidance of the need to perform timing analysis or validation on the SoC interconnect. Additionally we describe a bare port that provided direct access to the CHAIN fabric which was instrumental in testing and debugging the smartcard chip.

In this paper we investigate the statistics of multiplier operands and identify two characteristics of their distribution that have important consequences for the design of low power multipliers: most inputs are positive, and most inputs have a small number of significant bits. These characteristics are exploited in the design of a multiplier that employs three techniques to minimize power consumption: asynchronous control, a radix-2 algorithm, and split registers. The power savings resulting from the use of these techniques are 55%, 23 % and 12 % respectively when compared to a synchronous multiplier using a radix-4 modified Booth’s algorithm with unified registers. The results are derived from HSPICE simulations using input vectors from benchmark programs. A high-level software model is also used to compare the numbers of transitions in the various models.

Communication-centric design is a key paradigm for systems-on-chips (SoCs), where most computing blocks are predesigned IP cores. Due to the problems with distributing a clock across a large die, future system designs are more asynchronous or self-timed. For portable, battery-run applications, power and pin efficiency is an important property of a communication system where the cost of a signal transition on a global interconnect is much greater than for internal wires in logic blocks. We address this issue by designing an asynchronous communication system aimed at power and pin efficiency. Another important issue of SoC design is design productivity. It demands new methods and tools, particularly for designing communication protocols and interconnects. The design of a self-timed communication system is approached employing formal techniques supported by verification and synthesis tools. The protocol is formally specified and verified with respect to deadlock-freedom and delay-insensitivity using a Petri-net-based model-checking tool. A protocol controller has been synthesized by a direct mapping of the Petri net model derived from the protocol specification. The logic implementation was analyzed using the Cadence toolkit. The results of SPICE simulation show the advantages of the direct mapping method compared to logic synthesis.

This paper presents a new approach to an on-chip asynchronous transmission system suitable for next generation asynchronous on-chip networks. It implements multivalued logic to reduce the number of wires and a low-voltage swing for lower dynamic power dissipation. Furthermore, the transmission system described here enjoys fully static design and has zero static power consumption. Two versions of the transmitter circuit and the receiver are described. The proposed signaling scheme is compared to a classical dual-rail signaling system with regard to speed, power consumption, and reliability. The simulation results show that the asynchronous ternary logic signaling (ATLS) system delivers over 70% higher bandwidth per wire and consumes over 50% less power than the dual-rail signaling system on 10-mm-long on-chip interconnection.

Memory bandwidth is a limiting factor with many modem microprocessors and it is usual to include a cache to reduce the amount of memory traffic. Of the two commonly used cache write-policies, the copy-back approach is better than the write-through approach in this respect. The performance of both approaches can be further aided by the inclusion of a small buffer in the path of outgoing writes to the main memory, especially if this buffer is capable of forwarding its contents back into the main cache if they are needed again before they are emptied from the buffer This is what is known as a victim cache. For an asynchronous microprocessor it is logical that the cache system should be asynchronous as well; since a large degree of the flexibility of an asynchronous microprocessor would be lost if it were to use a standard synchronous memory interface. However implementing a forwarding mechanism in an asynchronous system is more difficult because the data to be forwarded is flowing in a manner unsynchronised to the process which requires it. This paper presents an architecture for a victim cache to resolve forwarding in a totally asynchronous environment. The resultant structure forms a key part of an asynchronous copy-back cache system for the Amulet3, a third generation asynchronous implementation of the ARM processor.

The AMULET group at the University of Manchester has spent a decade designing asynchronous microprocessors of increasing performance and complexity. Getting these large designs (almost) right first time is very challenging, and any verification or validation methodologies that can be brought to bear can help improve the design quality as measured by the number and severity of bugs in the final silicon. This paper presents the methodologies used on the first three AMULET processors and describes the problems that escaped detection. The potential and need for improved tools is discussed, as are looming problems with near-future process technologies which will make right-first-time design much more difficult to achieve.

AMULET3 is a 32-bit asynchronous processor core that is fully instruction set compatible with the clocked ARM cores. It represents the culmination of ten years of research and development into asynchronous processor design at the University of Manchester, and is the first step into commercial use for this technology. AMULET3 shows that asynchronous technology is commercially viable, and is competitive in terms of performance, area and power-efficiency with clocked design. In addition, asynchronous design offers significant advantages in terms of reduced electromagnetic interference and unique power management capabilities.

AMULET2e is an embedded system chip incorporating a 32-bit ARM-compatible asynchronous processor core, a 4-Kb pipelined cache, a flexible memory interface with dynamic bus sizing, and assorted programmable control functions. Many on-chip performance-enhancing and power-saving features are switchable, enabling detailed experimental analysis of their effectiveness. AMULET2e silicon demonstrates competitive performance and power efficiency, ease of system design, and it includes innovative features that exploit its asynchronous operation to advantage in applications that require low standby power and/or freedom from the electromagnetic interference generated by system clocks.

This paper presents a straightforward approach for synthesizing a standard VHDL description of an asynchronous circuit from a behavioural VHDL description. The asynchronous circuit style is based on 'micropipelines', a style currently used to develop asynchronous microprocessors at Manchester University. The rules of partition and conversion which are used to implement the synthesizer are also described. The synthesizer greatly reduces the design time of a complex micropipeline circuit.

AMULET3 is a fully asynchronous implementation of ARM architecture v4T and was designed at the University of Manchester between 1996 and 1998. It is the third generation asynchronous ARM, and is aimed at a significantly higher performance level than its predecessors. Achieving this higher performance has required significant enhancements to the internal micro-architecture, such as the introduction of a reorder buffer to support efficient forwarding while retaining exact exception handling. In this paper we present an overview of the AMULET3 core, highlighting the issues which arise when striving for high performance and instruction set compatibility in an asynchronous design framework.

An asynchronous implementation of the ARM microprocessor has been developed using an approach based on Sutherland's Micropipelines. The design allows considerable internal asynchronous concurrency. This paper presents the rationale for the work, the organization of the chip, and the characteristics of the prototype silicon. The design displays unusual properties such as nondeterministic (but bounded) prefetch depth beyond a branch instruction, a data dependent throughput, and employs a novel register locking mechanism. This work demonstrates the feasibility of building complex asynchronous systems and gives an indication of the costs and benefits of the Micropipeline approach.

There is now compelling evidence that asynchronous design is, indeed, feasible on CMOS VLSI, since several complex asynchronous chips have been built and shown to work. The work at Philips amply demonstrates power savings (at least within a particular application domain) and their elegant Tangpam synthesis tools support the concurrency claim. However, much more needs to be done to validate all these claims. Perhaps more convincing is the increasing industrial interest. Among the established players, Philips and Sun have invested considerable internal resource in asynchronous technology, and recently Intel has shown active interest. Whenever a significant technology change is foreseen, start-up companies form to exploit the inertia of the multinationals, and asynchronous logic can now claim at least two such new companies.

Standard micropipelines use simple two-phase control circuits. The latches employed on AMULET1 are level sensitive, so two- to four-phase converters are required in each latch controller. To avoid this overhead an investigation has been carried out into four-phase micropipeline control circuits; this has thrown up several design issues relating to cost, performance and safety, and forms a useful illustration of asynchronous design techniques.

A method to design and test asynchronous sequential circuits (ASCs) based on the micropipeline design style is presented in this paper. According to the proposed scan test approach the combinational block is tested separately by scanning the test vectors in and shifting the responses out of the state registers. This provides for the detection of all single stuck-at and delay faults in the ASC under test. The complexity of the test procedure of such a testable ASC is reduced to that of the combinational circuit. Tests for the combinational circuit and state holding elements can be derived using standard test generation techniques.

The micropipeline approach to designing asynchronous VLSI circuits has successfully been used in the AMULET1 microprocessor. A method to design and test micropipelines is presented in this paper. The test strategy is based on the scan test technique. It allows the separate testing of all the data processing blocks by scanning the test patterns in and shifting the responses out of the stage registers. The proposed test approach provides for the detection of all single stuck-at and delay faults in the micropipeline. Tests for the combinational processing logic and state holding elements can be derived using standard test generation techniques.

AMULET1 is a fully asynchronous implementation of the ARM microprocessor which was designed at Manchester University between 1991 and 1993. First silicon arrived in April 1994 and was found to be functional, demonstrating that asynchronous design of complex circuits is feasible with present day CAD tools. This paper presents the motivation for the work, some of the design choices which were made, the problems which were encountered during the development of the design and the characteristics of the device itself. The future potential for asynchronous circuits is also discussed.>

A high-performance register bank is a central component of a RISC processor. A novel register bank design has been developed, as an integral part of a self-timed implementation of a commercial RISC microprocessor, to address the problem of register interlocking in an asynchronous micropipelined execution unit. The problem in an asynchronous design is to maintain coherent register operation while allowing concurrent read and write accesses with arbitrary timing. The solution presented here includes a novel arbiter-free locking mechanism which enables efficient read operations in the presence of multiple pending write operations.>