The advent of hybrid memristor-CMOS technologies opens an exciting alternative to implement compact neuromorphic hardware with dense layers of neurons interconnected via memristive synapses for low-power energy-efficient high-speed inference applications and online learning. However, before implementing large multi-layer networks with higher number of neurons, many practical issues should be overcome, both associated to the memristive devices and the CMOS architecture. For that, we have designed a configurable and scalable multi-core architecture based on a computing core formed by 64 pre- and 64 post-synaptic neurons densely interconnected through 4k memristors. In this work, we present a chip including 16 cores, which allow for several different combinations with a total number of 64k memristors. A test infrastucture has been implemented to configure different architectures and run experiments from a processor. A new iterative self-calibration algorithm has been proposed to compensate for mismatch variations of the critical CMOS part, and an inference experiment with larger-resolution binary images has been demonstrated as a proof of concept.

The emergence of hybrid memristor-CMOS technologies open a promising way to implement compact neuromorphic systems with dense layers of neurons interconnected through memristive synapses for high-speed and low-power inference applications and online learning. However, there are still many practical issues which should be overcome, especially related to the memristive devices, but also to the CMOS circuitry. In particular, transistor mismatch can produce very different behaviors between neurons, reducing dramatically the performance of the system. In this work, we propose a mismatch calibration strategy to compensate this effect by performing a post-fabrication characterization of neurons behavior and applying proportional threshold voltages at the comparator which activates the generation of output events. We have implemented the proposed strategy on a CMOL-like memristor-CMOS neuromorphic chip with 64 input neurons, 64 output neurons and 4096 1T1R synapses, fabricated in 130nm CMOS with 200nmsized Ti/HfOx/TiN memristors on top, obtaining a performance improvement from 49% to 80% in an inference experiment.

This paper describes the design of a monolithic CMOS-memristive neuromorphic chip performing vector matrix multiplication between spike coded input vectors and the synaptic weights stored in the memristive array. A computing core including a$64\times 64$memristive array connecting 64 input and 64 output neurons has been fabricated. A Spiking Neural Network with a memristive synaptic layer exhibiting Stochastic Binary Spike-Time-Dependent-Plasticity has been experimentally demonstrated with the fabricated core. The CMOS-memristive neuromorphic processor is designed following a compact pseudo-CMOL design style that results in a modular and scalable computing core with a synaptic density of 22Ksynapses/mm2. A single core has been fabricated in CEA-LETI 130nm CMOS-RRAM technology and its operation has been experimentally characterized. A multicore architecture with reconfigurable connectivity, where cores can be interconnected to either share pre-synaptic neurons and expand post-synaptic neurons, or vice versa, share post-synaptic neurons and expand pre-synaptic neurons, is proposed and presented here.

We address the problem of testing artificial intelligence (AI) hardware accelerators implementing spiking neural networks (SNNs). We define a metric to quickly rank available samples for training and testing based on their fault detection capability. The metric measures the interclass spike count difference of a sample for the fault-free design. In particular, each sample is assigned a score equal to the spike count difference between the first two top classes. The hypothesis is that samples with small scores achieve high fault coverage because they are prone to misclassification, i.e., a small perturbation in the network due to a fault will result in these samples being misclassified with high probability. We show that the proposed metric correlates with the per-sample fault coverage and that retaining a set of high-ranked samples in the order of ten achieves near-perfect fault coverage for critical faults that affect the SNN accuracy. The proposed test generation approach is demonstrated on two SNNs modeled in Python and on actual neuromorphic hardware. We discuss fault modeling and perform an analysis to reduce the fault space so as to speed up test generation time.

Despite the parallelism and sparsity in neural network models, their transfer into hardware unavoidably makes them susceptible to hardware-level faults. Hardware-level faults can occur either during manufacturing, such as physical defects and process-induced variations, or in the field due to environmental factors and aging. The performance under fault scenarios needs to be assessed so as to develop cost-effective fault-tolerance schemes. In this work, we assess the resilience characteristics of a hardware accelerator for Spiking Neural Networks (SNNs) designed in VHDL and implemented on an FPGA. The fault injection experiments pinpoint the parts of the design that need to be protected against faults, as well as the parts that are inherently fault-tolerant.

The error-resiliency of Artificial Intelligence (AI) hardware accelerators is a major concern, especially when they are deployed in mission-critical and safety-critical applications. In this paper, we propose a neuron fault tolerance strategy for Spiking Neural Networks (SNNs). It is optimized for low area and power overhead by leveraging observations made from a large-scale fault injection experiment that pinpoints the critical fault types and locations. We describe the fault modeling approach, the fault injection framework, the results of the fault injection experiment, the fault-tolerance strategy, and the fault-tolerant SNN architecture. The idea is demonstrated on two SNNs that we designed for two SNN-oriented datasets, namely the N-MNIST and IBM's DVS128 gesture datasets.

This paper gives an overview of Cognitive-Radio (CR) circuits and systems, that will enable the implementation of new technology paradigms such as software-defined electronics and Artificial Intelligence (AI) managed Internet-of-Things (IoT). A survey of the state of the art, trends and design challenges is presented from a top-down perspective - from system-level to circuit and chip implementation. As an application, special emphasis is put on analog/digital interfaces as one of the key building blocks in CR-based devices. Cutting-edge architectures - mostly based on ΣΔ Modulators (ΣΔMs) - are discussed, as well as the best candidate circuit strategies to implement CR- based digitizers in deep nanometer CMOS.

The emergence of nano-scale memristive devices encouraged many different research areas to exploit their use in multiple applications. One of the proposed applications was to implement synaptic connections in bio-inspired neuromorphic systems. Large-scale neuromorphic hardware platforms are being developed with increasing number of neurons and synapses, having a critical bottleneck in the online learning capabilities. Spike-timing-dependent plasticity (STDP) is a widely used learning mechanism inspired by biology which updates the synaptic weight as a function of the temporal correlation between pre- and post-synaptic spikes. In this work, we demonstrate experimentally that binary stochastic STDP learning can be obtained from a memristor when the appropriate pulses are applied at both sides of the device.

The deployment of Artificial Intelligence (AI) hardware accelerators in a variety of applications, including safety-critical ones, requires assessing their inherent reliability to hardware-level faults and developing cost-effective fault tolerance techniques. This entails performing large-scale fault simulation experiments. However, transistor-level fault simulation is prohibitive and fault simulation should be carried out at a higher abstraction level. In this work, we focus on spiking neural networks (SNNs), and we follow a bottom-up approach starting from transistor-level simulations for developing a neuron behavioral-level fault model that can be readily employed for performing behavioral-level fault simulation of deep SNNs.

“Fail often to succeed sooner” is a common mantra that we are told is the secret to success. When reporting research results, however, scholars rarely write about their failed attempts and only focus on the successful ones. Perhaps the source of this disconnect between what we preach and what we do can be found in the underlying assumption that published work is meant to move the field forward and failed attempts supposedly do not. The goal of the confessions presented in this paper is to show that even failed attempts are genuine and valuable contributions to our field provided that we learn from our mistakes and correct them. The 27 confessions span from planning oversights, digital and analog design errors, misunderstanding of devices, overlooked parasitics, LVS errors, and troubles in testing.

Reading several ReRAMs simultaneously in a neuromorphic circuit increases power consumption and limits scalability. Applying small inference read pulses is a vain attempt when offset voltages of the read-out circuit are decisively more. This paper presents an experimental validation of a three-stage calibration scheme to calibrate the DC offset voltage across the rows of the memristive crossbar. The proposed method is based on biasing the body terminal of one of the differential pair MOSFETs of the buffer through a series of cascaded resistor banks arranged in three stages-coarse, fine and finer stages. The circuit is designed in a 130 nm CMOS technology, where the OxRAM-based binary memristors are built on top of it. A dedicated PCB and other auxiliary boards have been designed for testing the chip. Experimental results validate the presented approach, which is only limited by mismatch and electrical noise.

We have developed a fully configurable event-driven convolutional module with refractory period mechanism that can be used to implement arbitrary Convolutional Neural Networks (ConvNets) on FPGAs following a 2D array structure. Using this module, we have implemented in a Spartan6 FPGA a 4-layer ConvNet with 22 convolutional modules trained for poker card symbol recognition. It has been tested with a stimulus where 40 poker cards were observed by a Dynamic Vision Sensor (DVS) in 1s time. A traffic control mechanism is implemented to down-sample high speed input stimuli while keeping spatio-temporal correlation. For slow stimulus play back, a 96% recognition rate is achieved with a power consumption of 0.85mW. At maximum play back speed, the recognition rate is still above 63% when less than 20% of the input events are processed.

Object tracking is a major problem for many computer vision applications, but it continues to be computationally expensive. The use of bio-inspired neuromorphic event-driven dynamic vision sensors (DVSs) has heralded new methods for vision processing, exploiting reduced amount of data and very precise timing resolutions. Previous studies have shown these neural spiking sensors to be well suited to implementing single-sensor object tracking systems, although they experience difficulties when solving ambiguities caused by object occlusion. DVSs have also performed well in 3-D reconstruction in which event matching techniques are applied in stereo setups. In this paper, we propose a new event-driven stereo object tracking algorithm that simultaneously integrates 3-D reconstruction and cluster tracking, introducing feedback information in both tasks to improve their respective performances. This algorithm, inspired by human vision, identifies objects and learns their position and size in order to solve ambiguities. This strategy has been validated in four different experiments where the 3-D positions of two objects were tracked in a stereo setup even when occlusion occurred. The objects studied in the experiments were: 1) two swinging pens, the distance between which during movement was measured with an error of less than 0.5%; 2) a pen and a box, to confirm the correctness of the results obtained with a more complex object; 3) two straws attached to a fan and rotating at 6 revolutions per second, to demonstrate the high-speed capabilities of this approach; and 4) two people walking in a real-world environment.

The recently developed Dynamic Vision Sensors (DVS) sense dynamic visual information asynchronously and code it into trains of events with sub-micro second temporal resolution. This high temporal precision makes the output of these sensors especially suited for dynamic 3D visual reconstruction, by matching corresponding events generated by two different sensors in a stereo setup. This paper explores the use of Gabor filters to extract information about the orientation of the object edges that produce the events, applying the matching algorithm to the events generated by the Gabor filters and not to those produced by the DVS. This strategy provides more reliably matched pairs of events, improving the final 3D reconstruction.

We present a novel method to generate realistic simulations of extracellular recordings. The simulations were obtained by superimposing the activity of neurons placed randomly in a cube of brain tissue. Detailed models of individual neurons were used to reproduce the extracellular action potentials of close-by neurons. To reduce the computational load, the contributions of neurons further away were simulated using previously recorded spikes with their amplitude normalized by the distance to the recording electrode. For making the simulations more realistic, we also considered a model of a finite-size electrode by averaging the potential along the electrode surface and modeling the electrode-tissue interface with a capacitive filter. This model allowed studying the effect of the electrode diameter on the quality of the recordings and how it affects the number of identified neurons after spike sorting. Given that not all neurons are active at a time, we also generated simulations with different ratios of active neurons and estimated the ratio that matches the signal-to-noise values observed in real data. Finally, we used the model to simulate tetrode recordings.

This paper summarizes how Convolutional Neural Networks (ConvNets) can be implemented in hardware using Spiking neural network Address-Event-Representation (AER) technology, for sophisticated pattern and object recognition tasks operating at mili second delay throughputs. Although such hardware would require hundreds of individual convolutional modules and thus is presently not yet available, we discuss methods and technologies for implementing it in the near future. On the other hand, we provide precise behavioral simulations of large scale spiking AER convolutional hardware and evaluate its performance, by using performance figures of already available AER convolution chips fed with real sensory data obtained from physically available AER motion retina chips. We provide simulation results of systems trained for people recognition, showing recognition delays of a few miliseconds from stimulus onset. ConvNets show good up scaling behavior and possibilities for being implemented efficiently with new nano scale hybrid CMOS/nonCMOS technologies.

This paper summarizes how Convolutional Neural Networks (ConvNets) can be implemented in hardware using Spiking neural network Address-Event-Representation (AER) technology, for sophisticated pattern and object recognition tasks operating at mili second delay throughputs. Although such hardware would require hundreds of individual convolutional modules and thus is presently not yet available, we discuss methods and technologies for implementing it in the near future. On the other hand, we provide precise behavioral simulations of large scale spiking AER convolutional hardware and evaluate its performance, by using performance figures of already available AER convolution chips fed with real sensory data obtained from physically available AER motion retina chips. We provide simulation results of systems trained for people recognition, showing recognition delays of a few miliseconds from stimulus onset. ConvNets show good up scaling behavior and possibilities for being implemented efficiently with new nano scale hybrid CMOS/nonCMOS technologies.

Address-event representation (AER) is an emergent hardware technology which shows a high potential for providing in the near future a solid technological substrate for emulating brain-like processing structures. When used for vision, AER sensors and processors are not restricted to capturing and processing still image frames, as in commercial frame-based video technology, but sense and process visual information in a pixel-level event-based frameless manner. As a result, vision processing is practically simultaneous to vision sensing, since there is no need to wait for sensing full frames. Also, only meaningful information is sensed, communicated, and processed. Of special interest for brain-like vision processing are some already reported AER convolutional chips, which have revealed a very high computational throughput as well as the possibility of assembling large convolutional neural networks in a modular fashion. It is expected that in a near future we may witness the appearance of large scale convolutional neural networks with hundreds or thousands of individual modules. In the meantime, some research is needed to investigate how to assemble and configure such large scale convolutional networks for specific applications. In this paper, we analyze AER spiking convolutional neural networks for texture recognition hardware applications. Based on the performance figures of already available individual AER convolution chips, we emulate large scale networks using a custom made event-based behavioral simulator. We have developed a new event-based processing architecture that emulates with AER hardware Manjunath's frame-based feature recognition software algorithm, and have analyzed its performance using our behavioral simulator. Recognition rate performance is not degraded. However, regarding speed, we show that recognition can be achieved before an equivalent frame is fully sensed and transmitted.

This paper describes CAVIAR, a massively parallel hardware implementation of a spike-based sensing-processing-learning-actuating system inspired by the physiology of the nervous system. CAVIAR uses the asychronous address-event representation (AER) communication framework and was developed in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45k neurons (spiking cells), up to 5M synapses, performs 12G synaptic operations per second, and achieves millisecond object recognition and tracking latencies.

We present a neuromorphic fully digital convolution microchip for Address Event Representation (AER) spike-based processing systems. This microchip computes 2-D convolutions with a programmable kernel in real time. It operates on a pixel array of size 32 x 32, and the kernel is programmable and can be of arbitrary shape and size up to 32 x 32 pixels. The chip receives and generates data in AER format, which is asynchronous and digital. The paper describes the architecture of the chip, the test setup, and experimental results obtained from a fabricated prototype.