[EN] This study presents a comprehensive circuit-based nanopore model for emulating memristive and synaptic behaviors in neuromorphic systems. A set of governing mathematical equations describing the voltage- and time-dependent ion transport dynamics within the nanopore were derived and implemented in Verilog-A, enabling seamless integration into circuit simulators such as Cadence Virtuoso for hardware-level analysis. The model reproduces key electronic characteristics, including nonlinear current¿voltage (I¿V) hysteresis and voltage-controlled resistive switching, confirming its memristive nature. Beyond conventional memristive behavior, the nanopore model exhibits biologically inspired learning mechanisms such as spike-duration-, spike-voltage-, and spike-frequency-dependent plasticity (SDDP, SVDP, SFDP), as well as paired-pulse facilitation (PPF), closely mimicking the short-term and long-term adaptation observed in biological synapses. Simulation results demonstrate how the nanopore¿s conductance dynamically evolves with variations in pulse width, amplitude, and frequency, revealing tunable memory effects and temporal learning capabilities. These findings establish the nanopore-based memristor as a promising candidate for neuromorphic computing, bio-signal processing, and cognitive hardware architectures, providing a scalable bridge between nanoscale ionic transport physics and large-scale circuit-level implementation.

Epilepsy, affecting more than 50 million individuals globally, represents a significant clinical challenge, as numerous cases exhibit resistance to pharmacological treatment. Neural implants with advanced algorithms for on-device processing are transforming epilepsy management, enabling precise seizure detection and control. On-device algorithms must optimize computation to lower power use and minimize transmission bandwidth, ensuring long-term viability. We propose a new algorithm that combines a Z-test-based outlier detection approach with a heuristic decision tree to differentiate between interictal and ictal events. This algorithm operates in real-time to segment local field potentials (LFP) recorded via microelectrode arrays (MEA), detecting seizure onsets with minimal latency and low power requirements. Tested on an MEA dataset from brain slices, our algorithm achieved 98% sensitivity, 93% precision, 92% accuracy, and a 6% false detection rate (FDR), achieving a seizure detection latency of 0.5 ± 0.6 seconds. This work shows a robust, low-computation, low-latency approach resilient to noise requiring minimal configuration, advancing brain implantable devices for epilepsy treatment.

The advent of hybrid memristor-CMOS technologies opens an exciting alternative to implement compact neuromorphic hardware with dense layers of neurons interconnected via memristive synapses for low-power energy-efficient high-speed inference applications and online learning. However, before implementing large multi-layer networks with higher number of neurons, many practical issues should be overcome, both associated to the memristive devices and the CMOS architecture. For that, we have designed a configurable and scalable multi-core architecture based on a computing core formed by 64 pre- and 64 post-synaptic neurons densely interconnected through 4k memristors. In this work, we present a chip including 16 cores, which allow for several different combinations with a total number of 64k memristors. A test infrastucture has been implemented to configure different architectures and run experiments from a processor. A new iterative self-calibration algorithm has been proposed to compensate for mismatch variations of the critical CMOS part, and an inference experiment with larger-resolution binary images has been demonstrated as a proof of concept.

Event cameras form a fundamental foundation for visual perception in scenes characterized by high speed and a wide dynamic range. Although deep learning techniques have achieved remarkable success in estimating event-based optical flow, existing methods have not adequately addressed the significance of temporal information in capturing spatiotemporal features. Due to the dynamics of spiking neurons in SNNs, which preserve important information while forgetting redundant information over time, they are expected to outperform analog neural networks (ANNs) with the same architecture and size in sequential regression tasks. In addition, SNNs on neuromorphic hardware achieve advantages of extremely low power consumption. However, present SNN architectures encounter issues related to limited generalization and robustness during training, particularly in noisy scenes. To tackle these problems, this study introduces an innovative spike-based self-supervised learning algorithm known as SeLHIB, which leverages the information bottleneck theory. By utilizing event-based camera inputs, SeLHIB enables robust estimation of optical flow in the presence of noise. To the best of our knowledge, this is the first proposal of a self-supervised information bottleneck learning strategy based on SNNs. Furthermore, we develop spike-based self-supervised algorithms with nonlinear and high-order information bottleneck learning that employs nonlinear and high-order mutual information to enhance the extraction of relevant information and eliminate redundancy. We demonstrate that SeLHIB significantly enhances the generalization ability and robustness of optical flow estimation in various noise conditions. In terms of energy efficiency, SeLHIB achieves 90.44% and 45.70% cut down of energy consumption compared to its counterpart ANN and counterpart SNN models, while attaining 33.78% lower AEE (MVSEC), 5.96% lower RSAT (ECD) and 6.21% lower RSAT (HQF) compared to the counterpart ANN implementations with the same sizes and architectures.

This paper provides an overview of the ongoing work to enable novel modalities of passive monocular neuromorphic vision in the NimbleAI sensing-processing architecture; namely, foveated and light-field event-driven vision with selective visual attention. The latter vision modality encodes 3D visual surroundings as sparse visual events in a 4D spatiotemporal domain, adding depth to current representation of visual information delivered by Dynamic Vision Sensors (DVS). The NimbleAI architecture implements hardware support for efficient execution of mainstream computer vision algorithms and AI models using these visual inputs. The architecture is designed to harness the latest advancements in 3D silicon integration, making it possible to squeeze sensing and spiking circuitry, memory, and processing engines into a miniature silicon volume.

The emergence of hybrid memristor-CMOS technologies open a promising way to implement compact neuromorphic systems with dense layers of neurons interconnected through memristive synapses for high-speed and low-power inference applications and online learning. However, there are still many practical issues which should be overcome, especially related to the memristive devices, but also to the CMOS circuitry. In particular, transistor mismatch can produce very different behaviors between neurons, reducing dramatically the performance of the system. In this work, we propose a mismatch calibration strategy to compensate this effect by performing a post-fabrication characterization of neurons behavior and applying proportional threshold voltages at the comparator which activates the generation of output events. We have implemented the proposed strategy on a CMOL-like memristor-CMOS neuromorphic chip with 64 input neurons, 64 output neurons and 4096 1T1R synapses, fabricated in 130nm CMOS with 200nmsized Ti/HfOx/TiN memristors on top, obtaining a performance improvement from 49% to 80% in an inference experiment.

Configurable delays are a basic feature in many neuromorphic neural network hardware accelerators. However, they have been rarely used in model implementations, despite their promising impact on performance and efficiency in tasks that exhibit complex dynamics, as it has been unclear how to optimize them. In this work, we propose a framework to train and deploy in digital neuromorphic hardware highly performing spiking neural networks (SNNs) where apart from the synaptic weights, the delays are also co-optimized. We consider synaptic (i.e. per-synapse) delays and evaluate them in two neuromorphic digital hardware platforms: Intel's Loihi and Imec's Seneca. Leveraging spike-based back-propagation-through-time, the training process accounts for both platform constraints, such as synaptic weight precision and the total number of parameters per core, as a function of the network size. In addition, a delay pruning technique is used to reduce memory footprint with a low cost in performance. The evaluated benchmark involves several models for solving the SHD (Spiking Heidelberg Digits) classification task, where minimal accuracy degradation during the transition from software to hardware is demonstrated. To our knowledge, this is the first work show-casing how to train and deploy hardware-aware models parameterized with synaptic delays, on multicore neuromorphic hardware accelerators.

The brain possesses the remarkable ability to seamlessly integrate perception with decision making within a dynamically changing environment in a fault-tolerant, end-to-end manner. This extraordinary capability offers a compelling solution for brain-inspired intelligence, replete with the advantages of end-to-end decision making: robustness, high accuracy, real-time responsiveness, autonomous intelligence, and a high degree of biological plausibility. Neuromorphic computing stands as a promising avenue for brain-inspired intelligence through the harmonious co-design of algorithms and hardware, aimed at unlocking full potential. This article introduces a comprehensive neuromorphic computing framework for end-to-end intelligence. It introduces the quadruplet spike-timing-dependent plasticity, which serves as a cornerstone for perceptual to decision-making tasks. A fault-tolerant neuromorphic routing strategy is presented to fortify the framework’s robustness. Empirical results underscore its impressive attributes with high accuracy, robustness, fault tolerance, and minimal computational latency when orchestrating end-to-end decision-making alongside visual perception. This study marks a pioneering effort in unified, fault-tolerant neuromorphic framework engineered for brain-inspired end-to-end intelligent tasks, merging visual perception with adaptive decision making. Such an endeavor is profoundly meaningful, as it propels the development of artificial general intelligence, holding vast implications for the field’s advancement.

The NimbleAI Horizon Europe project leverages key principles of energy-efficient visual sensing and processing in biological eyes and brains, and harnesses the latest advances in$\mathbf{33D}$stacked silicon integration, to create an integral sensing-processing neuromorphic architecture that efficiently and accurately runs computer vision algorithms in area-constrained endpoint chips. The rationale behind the NimbleAI architecture is: sense data only with high information value and discard data as soon as they are found not to be useful for the application (in a given context). The NimbleAI sensing-processing architecture is to be specialized after-deployment by tunning system-level trade-offs for each particular computer vision algorithm and deployment environment. The objectives of NimbleAI are: (1)$\mathbf{100x}$performance per mW gains compared to state-of-the-practice solutions (i.e., CPU/GPUs processing frame-based video); (2)$\mathbf{50x}$processing latency reduction compared to CPU/GPUs; (3) energy consumption in the order of tens of mWs; and (4) silicon area of approx. 50 mm2.

In this paper, we present an event-based control framework for the efficient tracking of contour-based areas, such as road pavements, using a multirotor aerial vehicle equipped with a bio-inspired Dynamic Vision Sensor (DVS). Concerning the detection part, the DVS camera captures events, which are asynchronously fed into a Neuromorphic Hough Transform algorithm running on a SpiNN-3 board and implemented as a Spiking Neural Network (SNN). Next, the asynchronous output of the detection module is fed into an analytically formulated event-based Partitioned Visual Servoing (PVS) algorithm, running on conventional processing hardware, which allows the multirotor to autonomously track and navigate along the detected contour. The proposed architecture achieves efficient tracking of contour-based areas, while constantly maintaining the latter inside the DVS camera's field of view. A set of real-time experiments in various settings employing an octorotor equipped with a downward-looking DVS and a SpiNN-3 board demonstrate the effectiveness of the suggested framework.

This paper describes the design of a monolithic CMOS-memristive neuromorphic chip performing vector matrix multiplication between spike coded input vectors and the synaptic weights stored in the memristive array. A computing core including a$64\times 64$memristive array connecting 64 input and 64 output neurons has been fabricated. A Spiking Neural Network with a memristive synaptic layer exhibiting Stochastic Binary Spike-Time-Dependent-Plasticity has been experimentally demonstrated with the fabricated core. The CMOS-memristive neuromorphic processor is designed following a compact pseudo-CMOL design style that results in a modular and scalable computing core with a synaptic density of 22Ksynapses/mm2. A single core has been fabricated in CEA-LETI 130nm CMOS-RRAM technology and its operation has been experimentally characterized. A multicore architecture with reconfigurable connectivity, where cores can be interconnected to either share pre-synaptic neurons and expand post-synaptic neurons, or vice versa, share post-synaptic neurons and expand pre-synaptic neurons, is proposed and presented here.

The role of axonal synaptic delays in the efficacy and performance of artificial neural networks has been largely unexplored. In step-based analog-valued neural network models (ANNs), the concept is almost absent. In their spiking neuroscience-inspired counterparts, there is hardly a systematic account of their effects on model performance in terms of accuracy and number of synaptic operations. This paper proposes a methodology for accounting for axonal delays in the training loop of deep Spiking Neural Networks (SNNs), intending to efficiently solve machine learning tasks on data with rich temporal dependencies. We then conduct an empirical study of the effects of axonal delays on model performance during inference for the Adding task [1]–[3], a benchmark for sequential regression, and for the Spiking Heidelberg Digits dataset (SHD) [4], commonly used for evaluating event-driven models. Quantitative results on the SHD show that SNNs incorporating axonal delays instead of explicit recurrent synapses achieve state-of-the-art, over 90% test accuracy while needing less than half trainable synapses. Additionally, we estimate the required memory in terms of total parameters and energy consumption of accomodating such delay-trained models on a modern neuromorphic accelerator [5], [6]. These estimations are based on the number of synaptic operations and the reference GF-22nm FDX CMOS technology. As a result, we demonstrate that a reduced parameterization, which incorporates axonal delays, leads to approximately 90% energy and memory reduction in digital hardware implementations for a similar performance in the aforementioned task.

Dynamic Vision Sensors (DVS) are an unconventional type of camera that produces sparse and asynchronous event data, which has recently led to a strong increase in its use for computer vision tasks namely in robotics. Embedded systems face limitations in terms of energy resources, memory, computational power, and communication bandwidth. Hence, this application calls for a way to reduce the amount of data to be processed while keeping the relevant information for the task at hand. We thus believe that a formal definition of event data reduction methods will provide a step further towards sparse data processing.The contributions of this paper are twofold: we introduce two complementary neuromorphic methods based on Spiking Neural Networks for DVS data spatial reduction, which is to best of our knowledge the first proposal of neuromorphic event data reduction; then we study for each method the trade-off between the amount of information kept after reduction, the performance of gesture classification after reduction and their capacity to handle events in real time. We demonstrate here that the proposed SNN-based methods outperform existing methods in a classification task for most dividing factors and are significantly better at handling data in real time, and make therefore the optimal choice for fully-integrated energy-efficient event data reduction running dynamically on a neuromorphic platform. Our code is publicly available online at: https://github.com/amygruel/EvVisu.

Analog implementation of Oscillatory Neural Networks (ONNs) has the potential to implement fast and ultra-low-power computing capabilities. One of the drawbacks of analog implementation is component mismatches which cause desynchronization and instability in ONNs. Emerging devices like memristors and VO2are particularly prone to variations. In this paper, we study the effect of component mismatches on the performance of differential ONNs (DONNs). Mismatches were considered in two main blocks: differential oscillatory neurons and synaptic circuits. To measure DONN tolerance to mismatches in each block, performance was evaluated with mismatches being present separately in each block. Memristor-bridge circuits with four memristors were used as the synaptic circuits. The differential oscillatory neurons were based on VO2-devices. The simulation results showed that DONN performance was more vulnerable to mismatches in the components of the differential oscillatory neurons than to mismatches in the synaptic circuits. DONNs were found to tolerate up to 20% mismatches in the memristance of the synaptic circuits. However, mismatches in the differential oscillatory neurons resulted in non-uniformity of the natural frequencies, causing desynchronization and instability. Simulations showed that 0.5% relative standard deviation (RSD) in natural frequencies can reduce DONN performance dramatically. In addition, sensitivity analyses showed that the high threshold voltage of VO2-devices is the most sensitive parameter for frequency non-uniformity and desynchronization.

To investigate the diversified technologies in Internet of Vehicles (IoVs) under intelligent edge computing, brain-inspired computing techniques are proposed in this study, which is a promising biologically inspired method by using brain cognition mechanism for various applications. A neuromorphic approach in a scalable and fault-tolerant framework is presented, targeting to realize the navigation function for the edge computing in IoV applications. A novel fault-tolerant address event representation approach is proposed for the spike information routing, which makes the presented model both scalable and fault-tolerant. Experimental results reveal that the proposed approaches can enhance the communication distance, the load balancing and the maximum throughput of the neuromorphic system accordingly. Based on the proposed neuromorphic model, the effects of the dopamine level are investigated. Besides, the results show that the proposed work can realize the accurate obstacle avoidance for the edge IoV computing, and the performance of the proposed network is superior to the network without the proposed scalable and fault-tolerant design. Therefore, the proposed IoV model provides an experimental basis for the improvement of the IoV system.

Despite the parallelism and sparsity in neural network models, their transfer into hardware unavoidably makes them susceptible to hardware-level faults. Hardware-level faults can occur either during manufacturing, such as physical defects and process-induced variations, or in the field due to environmental factors and aging. The performance under fault scenarios needs to be assessed so as to develop cost-effective fault-tolerance schemes. In this work, we assess the resilience characteristics of a hardware accelerator for Spiking Neural Networks (SNNs) designed in VHDL and implemented on an FPGA. The fault injection experiments pinpoint the parts of the design that need to be protected against faults, as well as the parts that are inherently fault-tolerant.

This paper presents a spiking neural network for pattern recognition. The network synapses are realized by resistive switching random access memory (ReRAM) cells, which are a stack of Au/Ti/C/Ti/HfO2/Pt. These cells are connected to an array of NMOS transistors (fabricated in a CMOS 180nm technology) to form a 4by4 1T1R crossbar between pre and postsynaptic circuitries. The pre-synaptic part contains conditioning circuits to reshape the inputs before applying them to the memristive crossbar. The post-synaptic section includes current attenuators that allowed the memristor domain currents to be mapped to neuron domain currents, as well as physiologically realistic neuron circuits fabricated in a CMOS 180nm technology. As a demonstrator, the network is trained with one-shot winner-takes-all method to differentiate four input patterns in its inference mode.

Neurophysiological observations confirm that the brain not only is able to detect the impaired synapses (in brain damage) but also it is relatively capable of repairing faulty synapses. It has been shown that retrograde signaling by astrocytes leads to the modulation of synaptic transmission and thus bidirectional collaboration of astrocyte with nearby neurons is an important aspect of self-repairing mechanism. Specifically, the retrograde signaling via astrocyte can increase the transmission probability of the healthy synapses linked to the neuron. Motivated by these findings, in the present research, a CMOS neuromorphic circuit with self-repairing capabilities is proposed based on astrocyte signaling. In this way, the computational model of self-repairing process is hired as a basis for designing a novel analog integrated circuit in the 180-nm CMOS technology. It is illustrated that the proposed analog circuit is able to successfully recompense the damaged synapses by appropriately modifying the voltage signals of the remaining healthy synapses in the wide range of frequency. The proposed circuit occupies 7500- [Formula: see text] silicon area and its power consumption is about [Formula: see text]. This neuromorphic fault-tolerant circuit can be considered as a key candidate for future silicon neuronal systems and implementation of neurorobotic and neuro-inspired circuits.

Brain-inspired computing employs devices and architectures that emulate biological functions for more adaptive and energy-efficient systems. Oscillatory neural networks (ONNs) are an alternative approach in emulating biological functions of the human brain and are suitable for solving large and complex associative problems. In this work, we investigate the dynamics of coupled oscillators to implement such ONNs. By harnessing the complex dynamics of coupled oscillatory systems, we forge a novel computation model-information is encoded in the phase of oscillations. Coupled interconnected oscillators can exhibit various behaviors due to the strength of the coupling. In this article, we present a novel method based on subharmonic injection locking (SHIL) for controlling the oscillatory states of coupled oscillators that allow them to lock in frequency with distinct phase differences. Circuit-level simulation results indicate SHIL effectiveness and its applicability to large-scale oscillatory networks for pattern recognition.

Neuromorphic computing is a promising technology that realizes computation based on event-based spiking neural networks (SNNs). However, fault-tolerant on-chip learning remains a challenge in neuromorphic systems. This study presents the first scalable neuromorphic fault-tolerant context-dependent learning (FCL) hardware framework. We show how this system can learn associations between stimulation and response in two context-dependent learning tasks from experimental neuroscience, despite possible faults in the hardware nodes. Furthermore, we demonstrate how our novel fault-tolerant neuromorphic spike routing scheme can avoid multiple fault nodes successfully and can enhance the maximum throughput of the neuromorphic network by 0.9%-16.1% in comparison with previous studies. By utilizing the real-time computational capabilities and multiple-fault-tolerant property of the proposed system, the neuronal mechanisms underlying the spiking activities of neuromorphic networks can be readily explored. In addition, the proposed system can be applied in real-time learning and decision-making applications, brain-machine integration, and the investigation of brain cognition during learning.

A novel programmable single pulse generator for producing a pulse in different time scales and amplitudes is presented in this paper to drive integrate-and-fire neuron circuits. The proposed circuit generates pulses with controlled variable width from 1.3ns up to several milliseconds that coincide with a wide range of binary and analog memristor applications. The designed pulse generator is intended as a tool to precisely control the amount of charge injected in memristors devices so that precise characterization of the memristors can be done without using external controlling circuitry. Furthermore, a much finer precision in the control of total injected charge can be achieved using the proposed technique. The proposed single pulse generator has been designed in XFAB 0.35μm technology to characterize the properties of memristors.

The error-resiliency of Artificial Intelligence (AI) hardware accelerators is a major concern, especially when they are deployed in mission-critical and safety-critical applications. In this paper, we propose a neuron fault tolerance strategy for Spiking Neural Networks (SNNs). It is optimized for low area and power overhead by leveraging observations made from a large-scale fault injection experiment that pinpoints the critical fault types and locations. We describe the fault modeling approach, the fault injection framework, the results of the fault injection experiment, the fault-tolerance strategy, and the fault-tolerant SNN architecture. The idea is demonstrated on two SNNs that we designed for two SNN-oriented datasets, namely the N-MNIST and IBM's DVS128 gesture datasets.

The emergence of nano-scale memristive devices encouraged many different research areas to exploit their use in multiple applications. One of the proposed applications was to implement synaptic connections in bio-inspired neuromorphic systems. Large-scale neuromorphic hardware platforms are being developed with increasing number of neurons and synapses, having a critical bottleneck in the online learning capabilities. Spike-timing-dependent plasticity (STDP) is a widely used learning mechanism inspired by biology which updates the synaptic weight as a function of the temporal correlation between pre- and post-synaptic spikes. In this work, we demonstrate experimentally that binary stochastic STDP learning can be obtained from a memristor when the appropriate pulses are applied at both sides of the device.

Human gestures recognition is a complex visual recognition task where motion across time distinguishes the type of action. Automatic systems tackle this problem using complex machine learning architectures and training datasets. In recent years, the use and success of robust deep learning techniques was compatible with the availability of a great number of these sets. This paper presents SL-Animals-DVS, an event-based action dataset captured by a Dynamic Vision Sensor (DVS). The DVS records humans performing sign language gestures of various animals as a continuous spike flow at very low latency. This is especially suited for sign language gestures which are usually made at very high speeds. We also benchmark the recognition performance on this data using two state-of-the-art Spiking Neural Networks (SNN) recognition systems. SNNs are naturally compatible to make use of the temporal information that is provided by the DVS where the information is encoded in the spike times. The dataset has about 1100 samples of 58 subjects performing 19 sign language gestures in isolation at different scenarios, providing a challenging evaluation platform for this emerging technology.

The deployment of Artificial Intelligence (AI) hardware accelerators in a variety of applications, including safety-critical ones, requires assessing their inherent reliability to hardware-level faults and developing cost-effective fault tolerance techniques. This entails performing large-scale fault simulation experiments. However, transistor-level fault simulation is prohibitive and fault simulation should be carried out at a higher abstraction level. In this work, we focus on spiking neural networks (SNNs), and we follow a bottom-up approach starting from transistor-level simulations for developing a neuron behavioral-level fault model that can be readily employed for performing behavioral-level fault simulation of deep SNNs.

“Fail often to succeed sooner” is a common mantra that we are told is the secret to success. When reporting research results, however, scholars rarely write about their failed attempts and only focus on the successful ones. Perhaps the source of this disconnect between what we preach and what we do can be found in the underlying assumption that published work is meant to move the field forward and failed attempts supposedly do not. The goal of the confessions presented in this paper is to show that even failed attempts are genuine and valuable contributions to our field provided that we learn from our mistakes and correct them. The 27 confessions span from planning oversights, digital and analog design errors, misunderstanding of devices, overlooked parasitics, LVS errors, and troubles in testing.

Reading several ReRAMs simultaneously in a neuromorphic circuit increases power consumption and limits scalability. Applying small inference read pulses is a vain attempt when offset voltages of the read-out circuit are decisively more. This paper presents an experimental validation of a three-stage calibration scheme to calibrate the DC offset voltage across the rows of the memristive crossbar. The proposed method is based on biasing the body terminal of one of the differential pair MOSFETs of the buffer through a series of cascaded resistor banks arranged in three stages-coarse, fine and finer stages. The circuit is designed in a 130 nm CMOS technology, where the OxRAM-based binary memristors are built on top of it. A dedicated PCB and other auxiliary boards have been designed for testing the chip. Experimental results validate the presented approach, which is only limited by mismatch and electrical noise.

This paper presents a new current attenuator circuit to scale down the inference currents in memristor based crossbars that drive integrate-and-fire neurons, which subsequently allows to reduce the size of integrating capacitors by several orders of magnitude, making IC integration possible. The proposed circuit uses a linear switch to divide the inference current and scale it down by a factor of about 104. The proposed attenuator has been designed in 130nm CMOS technology. Simulation results considering noise, process and temperature variations are shown to validate the presented approach.

We have developed a fully configurable event-driven convolutional module with refractory period mechanism that can be used to implement arbitrary Convolutional Neural Networks (ConvNets) on FPGAs following a 2D array structure. Using this module, we have implemented in a Spartan6 FPGA a 4-layer ConvNet with 22 convolutional modules trained for poker card symbol recognition. It has been tested with a stimulus where 40 poker cards were observed by a Dynamic Vision Sensor (DVS) in 1s time. A traffic control mechanism is implemented to down-sample high speed input stimuli while keeping spatio-temporal correlation. For slow stimulus play back, a 96% recognition rate is achieved with a power consumption of 0.85mW. At maximum play back speed, the recognition rate is still above 63% when less than 20% of the input events are processed.

Neuromorphic Computing or Spiking (also called Event-Driven) Neural Systems are becoming of high interest as they potentially allow for lower power hardware computing platforms, where power consumption is data driven. Traditional approaches (both in software and in hardware), which are not data driven, rely on generic system state updates, consuming a fixed amount of computing resources at each step, independent on the data itself. In neuromorphic spiking or (event-driven) computing systems power is consumed (in principle) if new data is transferred, either at the system input, system output, or internally between computing nodes. One such neuromorphic event-driven computing platform is the scalable SpiNNaker system, which is aimed for a million ARM core platform, capable of emulating in the order of a billion neurons in real time. An important practical drawback of the platform is the long time it takes to download to the hardware a given computational architecture. This step has to be repeated even if one wants to update a set of parameters. Here we present a method for updating internal parameters without downloading again the full architecture, by adding special neurons into the computing architecture which when they spike change given parameters. This allows to download the computing architecture only once to the SpiNNaker platform, and then take advantage of its highly efficient communication network to command specific parameter changes. This allows for intensive parameter searches in a more efficient manner.

Interest in event-based vision sensors has proliferated in recent years, with innovative technology becoming more accessible to new researchers and highlighting such sensors' potential to enable low-latency sensing at low computational cost. These sensors can outperform frame-based vision sensors regarding data compression, dynamic range, temporal resolution and power efficiency. However, available mature frame-based processing methods by using Artificial Neural Networks (ANNs) surpass Spiking Neural Networks (SNNs) in terms of accuracy of recognition. In this paper, we introduce a Hybrid Neural Network which is an intermediate solution to exploit advantages of both event-based and frame-based processing. We have implemented this network in FPGA and benchmarked its performance by using different event-based versions of MNIST dataset. HDL codes for this project are available for academic purpose upon request.

The SpiNNaker chip is a multi-core processor optimized for neuromorphic applications. Many SpiNNaker chips are assembled to make a highly parallel million core platform. This system can be used for simulation of a large number of neurons in real-time. SpiNNaker is using a general purpose ARM processor that gives a high amount of flexibility to implement different methods for processing spikes. Various libraries and packages are provided to translate a high-level description of Spiking Neural Networks (SNN) to low-level machine language that can be used in the ARM processors. In this paper, we introduce and compare three different methods to implement this intermediate layer of abstraction. We have examined the advantages of each method by various criteria, which can be useful for professional users to choose between them. All the codes that are used in this paper are available for academic propose.

Object tracking is a major problem for many computer vision applications, but it continues to be computationally expensive. The use of bio-inspired neuromorphic event-driven dynamic vision sensors (DVSs) has heralded new methods for vision processing, exploiting reduced amount of data and very precise timing resolutions. Previous studies have shown these neural spiking sensors to be well suited to implementing single-sensor object tracking systems, although they experience difficulties when solving ambiguities caused by object occlusion. DVSs have also performed well in 3-D reconstruction in which event matching techniques are applied in stereo setups. In this paper, we propose a new event-driven stereo object tracking algorithm that simultaneously integrates 3-D reconstruction and cluster tracking, introducing feedback information in both tasks to improve their respective performances. This algorithm, inspired by human vision, identifies objects and learns their position and size in order to solve ambiguities. This strategy has been validated in four different experiments where the 3-D positions of two objects were tracked in a stereo setup even when occlusion occurred. The objects studied in the experiments were: 1) two swinging pens, the distance between which during movement was measured with an error of less than 0.5%; 2) a pen and a box, to confirm the correctness of the results obtained with a more complex object; 3) two straws attached to a fan and rotating at 6 revolutions per second, to demonstrate the high-speed capabilities of this approach; and 4) two people walking in a real-world environment.

In this live demonstration we exploit the use of a serial link for fast asynchronous communication in massively parallel processing platforms connected to a DVS for real-time implementation of bio-inspired vision processing on spiking neural networks.

Address-Event-Representation (AER) is a widely extended asynchronous technique for interchanging “neural spikes” among different hardware elements in Neuromorphic Systems. Conventional AER links use parallel physical wires together with a pair of handshaking signals (Request and Acknowledge). Here we present a fully serial implementation using bidirectional SATA connectors with a pair of LVDS (low voltage differential signaling) wires for each direction. The proposed implementation can multiplex a number of conventional parallel AER links per LVDS physical connection. It uses flow control, clock correction, and byte alignment techniques to transmit 32-bit address events reliably over multiplexed serial connections. The setup has been tested using commercial Spartan6 FPGAs reaching a maximum event transmission speed of 75Meps (Mega Events per second) for 32-bit events at 3.0Gbps line data rate.

We present live demonstration of a hardware that can learn visual features on-line and in real-time during presentation of objects. Input Spikes are coming from a bio-inspired silicon retina or Dynamic Vision Sensor (DVS) and are processed in a Spiking Convolutional Neural Network (SCNN) that is equipped with a Spike Timing Dependent Plasticity (STDP) learning rule implemented on FPGA.

We present a highly hardware friendly STDP (Spike Timing Dependent Plasticity) learning rule for training Spiking Convolutional Cores in Unsupervised mode and training Fully Connected Classifiers in Supervised Mode. Examples are given for a 2-layer Spiking Neural System which learns in real time features from visual scenes obtained with spiking DVS (Dynamic Vision Sensor) Cameras.

The SpiNNaker Hardware platform allows emulating generic neural network topologies, where each neuron-to-neuron connection is defined by an independent synaptic weight. Consequently, weight storage requires an important amount of memory in the case of generic neural network topologies. This is solved in SpiNNaker by encapsulating with each SpiNNaker chip (which includes 18 ARM cores) a 128MB DRAM chip within the same package. However, ConvNets (Convolutional Neural Network) posses "weight sharing" property, so that many neuron-to-neuron connections share the same weight value. Therefore, a very reduced amount of memory is required to define all synaptic weights, which can be stored on local SRAM DTCM (data-tightly-coupled-memory) at each ARM core. This way, DRAM can be used extensively to store traffic data for off-line analyses. We show an implementation of a 5-layer ConvNet for symbol recognition. Symbols are obtained with a DVS camera. Neurons in the ConvNet operate in an event-driven fashion, and synapses operate instantly. With this approach it was possible to allocate up to 2048 neurons per ARM core, or equivalently 32k neurons per SpiNNaker chip.

This paper introduces an event-driven feedforward categorization system, which takes data from a temporal contrast address event representation (AER) sensor. The proposed system extracts bio-inspired cortex-like features and discriminates different patterns using an AER based tempotron classifier (a network of leaky integrate-and-fire spiking neurons). One of the system's most appealing characteristics is its event-driven processing, with both input and features taking the form of address events (spikes). The system was evaluated on an AER posture dataset and compared with two recently developed bio-inspired models. Experimental results have shown that it consumes much less simulation time while still maintaining comparable performance. In addition, experiments on the Mixed National Institute of Standards and Technology (MNIST) image dataset have demonstrated that the proposed system can work not only on raw AER data but also on images (with a preprocessing step to convert images into AER events) and that it can maintain competitive accuracy even when noise is added. The system was further evaluated on the MNIST dynamic vision sensor dataset (in which data is recorded using an AER dynamic vision sensor), with testing accuracy of 88.14%.

The recently developed Dynamic Vision Sensors (DVS) sense dynamic visual information asynchronously and code it into trains of events with sub-micro second temporal resolution. This high temporal precision makes the output of these sensors especially suited for dynamic 3D visual reconstruction, by matching corresponding events generated by two different sensors in a stereo setup. This paper explores the use of Gabor filters to extract information about the orientation of the object edges that produce the events, applying the matching algorithm to the events generated by the Gabor filters and not to those produced by the DVS. This strategy provides more reliably matched pairs of events, improving the final 3D reconstruction.

Nowadays spike-based brain processing emulation is taking off. Several EU and others worldwide projects are demonstrating this, like SpiNNaker, BrainScaleS, FACETS, or NeuroGrid. The larger the brain process emulation on silicon is, the higher the communication performance of the hosting platforms has to be. Many times the bottleneck of these system implementations is not on the performance inside a chip or a board, but in the communication between boards. This paper describes a novel modular Address-Event-Representation (AER) FPGA-based (Spartan6) infrastructure PCB (the AER-Node board) with 2.5Gbps LVDS high speed serial links over SATA cables that offers a peak performance of 32-bit 62.5Meps (Mega events per second) on board-to-board communications. The board allows back compatibility with parallel AER devices supporting up to x2 28-bit parallel data with asynchronous handshake. These boards also allow modular expansion functionality through several daughter boards. The paper is focused on describing in detail the LVDS serial interface and presenting its performance.

State-of-the-art image sensors suffer from significant limitations imposed by their very principle of operation. These sensors acquire the visual information as a series of “snapshot” images, recorded at discrete points in time. Visual information gets time quantized at a predetermined frame rate which has no relation to the dynamics present in the scene. Furthermore, each recorded frame conveys the information from all pixels, regardless of whether this information, or a part of it, has changed since the last frame had been acquired. This acquisition method limits the temporal resolution, potentially missing important information, and leads to redundancy in the recorded image data, unnecessarily inflating data rate and volume. Biology is leading the way to a more efficient style of image acquisition. Biological vision systems are driven by events happening within the scene in view, and not, like image sensors, by artificially created timing and control signals. Translating the frameless paradigm of biological vision to artificial imaging systems implies that control over the acquisition of visual information is no longer being imposed externally to an array of pixels but the decision making is transferred to the single pixel that handles its own information individually. In this paper, recent developments in bioinspired, neuromorphic optical sensing and artificial vision are presented and discussed. It is suggested that bioinspired vision systems have the potential to outperform conventional, frame-based vision systems in many application fields and to establish new benchmarks in terms of redundancy suppression and data compression, dynamic range, temporal resolution, and power efficiency. Demanding vision tasks such as real-time 3-D mapping, complex multiobject tracking, or fast visual feedback loops for sensory-motor action, tasks that often pose severe, sometimes insurmountable, challenges to conventional artificial vision systems, are in reach using bioinspired vision sensing and processing techniques.

This paper presents a new DVS sensor with one order of magnitude improved contrast sensitivity over previous reported DVSs. This sensor has been applied to a bio-inspired event-based binocular system that performs 3D event-driven reconstruction of a scene. Events from two DVS sensors are matched by using precise timing information of their ocurrence. To improve matching reliability, satisfaction of epipolar geometry constraint is required, and simultaneously available information on the orientation is used as an additional matching constraint.

Event-driven visual sensors have attracted interest from a number of different research communities. They provide visual information in quite a different way from conventional video systems consisting of sequences of still images rendered at a given "frame rate." Event-driven vision sensors take inspiration from biology. Each pixel sends out an event (spike) when it senses something meaningful is happening, without any notion of a frame. A special type of event-driven sensor is the so-called dynamic vision sensor (DVS) where each pixel computes relative changes of light or "temporal contrast." The sensor output consists of a continuous flow of pixel events that represent the moving objects in the scene. Pixel events become available with microsecond delays with respect to "reality." These events can be processed "as they flow" by a cascade of event (convolution) processors. As a result, input and output event flows are practically coincident in time, and objects can be recognized as soon as the sensor provides enough meaningful events. In this paper, we present a methodology for mapping from a properly trained neural network in a conventional frame-driven representation to an event-driven representation. The method is illustrated by studying event-driven convolutional neural networks (ConvNet) trained to recognize rotating human silhouettes or high speed poker card symbols. The event-driven ConvNet is fed with recordings obtained from a real DVS camera. The event-driven ConvNet is simulated with a dedicated event-driven simulator and consists of a number of event-driven processing modules, the characteristics of which are obtained from individually manufactured hardware modules.

This paper proposes an algorithm for feedforward categorization of objects and, in particular, human postures in real-time video sequences from address-event temporal-difference image sensors. The system employs an innovative combination of event based hardware and bio-inspired software architecture. An event-based temporal difference image sensor is used to provide input video sequences, while a software module extracts size and position invariant line features inspired by models of the primate visual cortex. The detected line features are organized into vectorial segments. After feature extraction, a modified line segment Hausdorff distance classifier combined with on-the-fly cluster-based size and position invariant categorization. The system can achieve about 90 percent average success rate in the categorization of human postures, while using only a small number of training samples. Compared to state-of-the-art bio-inspired categorization methods, the proposed algorithm requires less hardware resource, reduces the computation complexity by at least five times, and is an ideal candidate for hardware implementation with event-based circuits.

People rarely put in their papers the things that didn't work, the mistakes they made, and how they found out what went wrong. Such confessions can help others learn how to avoid similar mistakes. Twenty-six confessions were collected to form the bulk of this paper. Themes that arise are errors that result from not understanding the limitations of simulation tools in modeling physical reality, chip verification errors that result from lack of clear communication between designers, and projects that are considered in their own isolated environment of technical challenges rather than the broader context of their environment or application.

This paper presents a voltage-mode high speed driver to transmit serial AER data in scalable multi-chip AER systems. To take advantage of the asynchronous nature of AER (Address Event Representation) streams, this implementation al- lows an energy efficient burst-mode operation. This is achieved by switching on/off the driver in data pauses to reduce static power consumption. Impedance matching is calibrated continuously to track temperature variations, obtaining an optimal performance without degrading the data rate. Power management techniques for switching drivers are discussed and an internally compensated high speed regulator is presented. The system has been designed in a 0.35μm CMOS technology to transmit data rates up to 500Mbps using Manchester enconding. Layout extracted simulation results are presented, which include all interconnect parasitics. Estimated peak rate is 15Meps for 32 bit events. Simulated power consumption of transmitter and receiver at peak rate is 33.2mW, while below 100 Keps is 1.3mW.

In this paper we propose the first bio-inspired six layer convolutional network (ConvNet) non-frame based that can be implemented with already physically available spike-based electronic devices. The system was designed to recognize people in three different positions: standing, lying or up-side down. The inputs were spikes obtained with a motion retina chip. We provide simulation results showing recognition delays of 16 milliseconds from stimulus onset (time-to-first spike) with a recognition rate of 94%. The weight sharing property in ConvNets and the use of AER protocol allow a great reduction in the number of both trainable parameters and connections (only 748 trainable parameters and 123 connections in our AER system (out of 506998 connections that would be required in a frame-based implementation).

This paper summarizes how Convolutional Neural Networks (ConvNets) can be implemented in hardware using Spiking neural network Address-Event-Representation (AER) technology, for sophisticated pattern and object recognition tasks operating at mili second delay throughputs. Although such hardware would require hundreds of individual convolutional modules and thus is presently not yet available, we discuss methods and technologies for implementing it in the near future. On the other hand, we provide precise behavioral simulations of large scale spiking AER convolutional hardware and evaluate its performance, by using performance figures of already available AER convolution chips fed with real sensory data obtained from physically available AER motion retina chips. We provide simulation results of systems trained for people recognition, showing recognition delays of a few miliseconds from stimulus onset. ConvNets show good up scaling behavior and possibilities for being implemented efficiently with new nano scale hybrid CMOS/nonCMOS technologies.

This paper summarizes how Convolutional Neural Networks (ConvNets) can be implemented in hardware using Spiking neural network Address-Event-Representation (AER) technology, for sophisticated pattern and object recognition tasks operating at mili second delay throughputs. Although such hardware would require hundreds of individual convolutional modules and thus is presently not yet available, we discuss methods and technologies for implementing it in the near future. On the other hand, we provide precise behavioral simulations of large scale spiking AER convolutional hardware and evaluate its performance, by using performance figures of already available AER convolution chips fed with real sensory data obtained from physically available AER motion retina chips. We provide simulation results of systems trained for people recognition, showing recognition delays of a few miliseconds from stimulus onset. ConvNets show good up scaling behavior and possibilities for being implemented efficiently with new nano scale hybrid CMOS/nonCMOS technologies.

The four chips presented in the special session on "Activity-driven, event-based vision sensors" quickly output compressed digital data in the form of events. These sensors reduce redundancy and latency and increase dynamic range compared with conventional imagers. The digital sensor output is easily interfaced to conventional digital post processing, where it reduces the latency and cost of post processing compared to imagers. The asynchronous data could spawn a new area of DSP that breaks from conventional Nyquist rate signal processing. This paper reviews the rationale and history of this event-based approach, introduces sensor functionalities, and gives an overview of the papers in this session. The paper concludes with a brief discussion on open questions.

Reported AER (Address Event Representation) contrast retinae perform a contrast computation based on the ratio between a pixel's local light intensity and a spatially weighted average of its neighbourhood. This results in compact circuits, but with the penalty of all pixels generating output signals even if they sense no contrast. In this paper we present a spatial contrast retina with bipolar output: contrast is computed as the relative normalized difference (not the ratio) between a pixel's local light and its weighted spatial average, normalized to average light. As a result, contrast includes a sign, is ambient light independent, and the output will be zero if there is no contrast. Furthermore, an adjustable thresholding mechanism has been included, such that pixels remain silent until they sense an absolute contrast above the adjustable threshold. The pixel contrast computation circuit is based on Boahen's Biharmonic operator contrast circuit, which has been improved to include mismatch calibration and adaptive current based biasing. As a result, the contrast computation circuit shows much less mismatch, is almost insensitive to ambient light illumination, and biasing is much less critical than in the original voltage biasing scheme. The retina also includes an optional TFS (Time-to-First-Spike) integration mode. A full AER retina version has been fabricated and tested. In the present paper we provide preliminary experimental results.

Neuromorphic circuits and systems techniques have great potential for exploiting novel nanotechnology devices, which suffer from great parametric spread and high defect rate. In this paper we explore some potential ways of building neural network systems for sophisticated pattern recognition tasks using memristors. We will focus on spiking signal coding because of its energy and information coding efficiency, and concentrate on Convolutional Neural Networks because of their good scaling behavior, both in terms of number of synapses and temporal processing delay. We propose asynchronous architectures that exploit memristive synapses with specially designed neurons that allow for arbitrary scalability as well as STDP learning. We present some behavioral simulation results for small neural arrays using electrical circuit simulators, and system level spike processing results on human detection using a custom made event based simulator.

Address-event representation (AER) is an emergent hardware technology which shows a high potential for providing in the near future a solid technological substrate for emulating brain-like processing structures. When used for vision, AER sensors and processors are not restricted to capturing and processing still image frames, as in commercial frame-based video technology, but sense and process visual information in a pixel-level event-based frameless manner. As a result, vision processing is practically simultaneous to vision sensing, since there is no need to wait for sensing full frames. Also, only meaningful information is sensed, communicated, and processed. Of special interest for brain-like vision processing are some already reported AER convolutional chips, which have revealed a very high computational throughput as well as the possibility of assembling large convolutional neural networks in a modular fashion. It is expected that in a near future we may witness the appearance of large scale convolutional neural networks with hundreds or thousands of individual modules. In the meantime, some research is needed to investigate how to assemble and configure such large scale convolutional networks for specific applications. In this paper, we analyze AER spiking convolutional neural networks for texture recognition hardware applications. Based on the performance figures of already available individual AER convolution chips, we emulate large scale networks using a custom made event-based behavioral simulator. We have developed a new event-based processing architecture that emulates with AER hardware Manjunath's frame-based feature recognition software algorithm, and have analyzed its performance using our behavioral simulator. Recognition rate performance is not degraded. However, regarding speed, we show that recognition can be achieved before an equivalent frame is fully sensed and transmitted.

Address event representation (AER) is an emergent technology for assembling modular multi-blocks bio-inspired sensory and processing systems. Visual sensors (retinae) are among the first AER modules to be reported since the introduction of the technology. Spatial contrast AER retinae are of special interest since they provide highly compressed data flow without reducing the relevant information required for performing recognition. Reported AER contrast retinae perform a contrast computation based on the ratio between a pixel's local light intensity and a spatially weighted average of its neighbourhood. This resulted in compact circuits, but with the penalty of all pixels generating output signals even if they sensed no contrast. In this paper we present a spatial contrast retina with bipolar output: contrast is computed as the relative difference between a pixel's local light and its weighted spatial average. As a result, contrast includes a sign and the output will be zero if there is no contrast. Furthermore, an adjustable thresholding mechanism has been included, such that pixels remain silent until they sense an absolute contrast above the adjustable threshold. The pixel contrast computation circuit is based on Boahen's biharmonic operator contrast circuit, which has been improved to include mismatch calibration and adaptive current based biasing. As a result, the contrast computation circuit shows much less mismatch, is almost insensitive to ambient light illumination, and biasing is much less critical than in the original voltage biasing scheme. A full AER retina version has been submitted for fabrication. In the present paper we provide simulation results.

This paper presents the design and simulation of an OTA-C oscillator intended to be used as on-chip frequency reference. This reference will be part of the high speed clock generation circuit for Manchester serial LVDS-AER links. A Manchester LVDS receiver can adapt its operation in a limited range of frequencies, so the most important specification is the frequency stability over temperature and process variations. A novel design methodology is presented to design two oscillators in a 90 nm technology using transistors with 2.5 V supply voltage. Intensive simulations with temperature, process, supply voltage variations and mismatch effects were performed in order to analyze the validity of this approach, obtaining Delta ap 7%.

This paper describes CAVIAR, a massively parallel hardware implementation of a spike-based sensing-processing-learning-actuating system inspired by the physiology of the nervous system. CAVIAR uses the asychronous address-event representation (AER) communication framework and was developed in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45k neurons (spiking cells), up to 5M synapses, performs 12G synaptic operations per second, and achieves millisecond object recognition and tracking latencies.

In this paper we briefly summarize the fundamental properties of spike events processing applied to artificial vision systems. This sensing and processing technology is capable of very high speed throughput, because it does not rely on sensing and processing sequences of frames, and because it allows for complex hierarchically structured cortical-like layers for sophisticated processing. The paper describes briefly cortex-like spike event vision processing principles, and the AER (Address Event Representation) technique used in hardware spiking systems. Then a texture-based image retrieval using the AER technique is proposed. Realistic behavioral simulations based on existing hardware characteristics, reveal that the application, although processing large kernel convolutions, is capable of performing recognition in less than 10 ms.

We present a neuromorphic fully digital convolution microchip for Address Event Representation (AER) spike-based processing systems. This microchip computes 2-D convolutions with a programmable kernel in real time. It operates on a pixel array of size 32 x 32, and the kernel is programmable and can be of arbitrary shape and size up to 32 x 32 pixels. The chip receives and generates data in AER format, which is asynchronous and digital. The paper describes the architecture of the chip, the test setup, and experimental results obtained from a fabricated prototype.

Low current applications, like neuromorphic circuits, where operating currents can be as low as few nano amps or less, suffer from huge transistor mismatches, resulting in around or less than 1-bit precision. Here we present a new calibration approach based on individually calibrated current sources made with MOS transistors of digitally adjustable length, which require only N unit transistors. The scheme includes a translinear circuit based tuning scheme, which allows to expand the operating range of the calibrated circuits with graceful precision degradation, over 4 decades of operating currents. Experimental results are provided for 5-bit resolution DACs operating at 20 nA.

In this paper we briefly summarize the fundamental properties of spikes processing applied to artificial vision systems. This sensing and processing technology is capable of very high speed throughput, because it does not rely on sensing and processing sequences of frames, and because it allows for complex hierarchically structured cortical-like layers for sophisticated processing. The paper describes briefly cortex-like spiking vision processing principles, and the AER (address event representation) technique used in hardware spiking systems. Afterwards an example application is described, which is a simplification of Fukushima's Neocognitron. Realistic behavioral simulations based on existing AER hardware characteristics, reveal that the simplified neocognitron, although it processes 52 large kernel convolutions, is capable of performing recognition in less than 10 mus.

This paper presents the design and simulation of a serial AER LVDS communication link. It converts data from classical AER parallel bus with a 4-phase handshaking protocol into a bit stream which is transmitted serially into a single LVDS wire. At the receiver side data from the LVDS cable are transformed back to a parallel AER bus and handshaking signals are also properly managed. The link has been designed in a 90 nms technology. Extensive simulations have been performed demonstrating that the link can operate at a speed of 1 Gbps for all the technology corners, exhibiting a power consumption of 27.8 mW for the transmitter and 12.3 mW for the receiver. In the simulation the transmission channel was modelled as a 50 cm cat5E UTP cable, connected to the AER chip through 5 cm PCB traces modelled as a coupled microstrip transmission line. The design has been completed up to the layout level and has been submitted for fabrication. The transmitter and the receiver take up an area of 311times148 mum2and 300x148 mum2respectively.

In this paper, a chip that performs real-time image convolutions with programmable kernels of arbitrary shape is presented. The chip is a first experimental prototype of reduced size to validate the implemented circuits and system level techniques. The convolution processing is based on the address–event-representation (AER) technique, which is a spike-based biologically inspired image and video representation technique that favors communication bandwidth for pixels with more information. As a first test prototype, a pixel array of 16$\,\times\,$16 has been implemented with programmable kernel size of up to 16$\, \times\,$16. The chip has been fabricated in a standard 0.35-${\mu }$m complimentary metal–oxide–semiconductor (CMOS) process. The technique also allows to process larger size images by assembling 2-D arrays of such chips. Pixel operation exploits low-power mixed analog–digital circuit techniques. Because of the low currents involved (down tonanoamperesor evenpicoamperes), an important amount of pixel area is devoted to mismatch calibration. The rest of the chip uses digital circuit techniques, both synchronous and asynchronous. The fabricated chip has been thoroughly tested, both at the pixel level and at the system level. Specific computer interfaces have been developed for generating AER streams from conventional computers and feeding them as inputs to the convolution chip, and for grabbing AER streams coming out of the convolution chip and storing and analyzing them on computers. Extensive experimental results are provided. At the end of this paper, we pro

The paper presents a contrast retina microchip that provides its output as an AER (Address Event Representation) stream. Contrast is computed as the ratio between pixel photocurrent and a local average between neighboring pixels obtained with a diffusive network. This current based computation produces a large mismatch between neighboring pixels, because the currents can be as low as a few pico amperes. Consequently, a compact calibration circuitry has been included to calibrate each pixel. The paper describes the design of the pixel with its contrast computation and calibration sections. Experimental results are provided for a prototype fabricated in a standard 0.35μm CMOS process.

In 1989 Pelgrom et al. published a mismatch model for MOS transistors, where the standard quadratic deviation of the mismatch in a parameter between two identical transistors, is given by two independent terms: (1) a transistor size-dependent term; and (2) an inter-transistor distance-dependent term. To include the distance term, some researchers have developed CAD tools based on the so called σ-Space Methodology, which result in very computationally expensive algorithms. Such algorithms become non-viable even for circuits with a reduced number of transistors. On the other hand, by understanding and interpreting correctly the physical origin of Pelgrom's model distance term, one can implement in a straight forward manner this mismatch contribution in a CAD tool. Furthermore, the computational cost results negligible and viable for any number of transistors.

In this paper we briefly summarize the fundamental properties of spike events processing applied to artificial vision systems. This sensing and processing technology is capable of very high speed throughput, because it does not rely on sensing and processing sequences of frames, and because it allows for complex hierarchically structured cortical-like layers for sophisticated processing. The paper includes a few examples that have demonstrated the potential of this technology for high-speed vision processing, such as a multilayer event processing network of 5 sequential cortical-like layers, and a recognition system capable of discriminating propellers of different shape rotating at 5000 revolutions per second (300000 revolutions per minute).

In 1989, Pelgrom et al. published a mismatch model for MOS transistors, where the variation of parameter mismatch between two identical transistors is given by two independent terms: a size-dependent term and a distance-dependent term. Some CAD tools based on a nonphysical interpretation of Pelgrom's distance term result in excessive computationally expensive algorithms, which become nonviable even for circuits with a reduced number of transistors. Furthermore, some researchers are reporting new variations on the original nonphysically interpreted algorithms, which may render false results. The purpose of this paper is to clarify the physical interpretation of the distance term of Pelgrom and indicate how to model it efficiently in prospective CAD tools.

Address-event-representation (AER) is a communication protocol for transferring asynchronous events between VLSI chips, originally developed for bio-inspired processing systems (for example, image processing). Such systems may consist of a complicated hierarchical structure with many chips that transmit data among them in real time, while performing some processing (for example, convolutions). To develop AER based systems for image processing it is very convenient to have available some kind of tool for generating AER streams from on-computer stored images. In this paper we present a hardware method for generating AER streams with Poisson statistics in real time from a sequence of images stored in a computer's memory. We quantify that the events generated follow a Poisson distribution using the Kolmogorov-Smirnov test. We have developed a USB-AER board, based on the Xilinx Spartan II FPGA and the Cygnal 8051 microcontroller, developed by our RTCAR group have been used for the analysis

A high speed sample image processing application using AER-based components is presented. The setup objective is to distinguish between two propellers of different shape rotating at high speed (around 1000 revolutions/sec) to show event-based systems capabilities in high speed applications. Event-based schemes allow the most relevant information to propagate faster through the system layers. So image processing is sped up because a rough result may be available when only a little part of the input has arrived. This setup is much faster than the conventional frame-based image processing systems because they would need to process more than 10kFrames/s to do the same task proposed here, whereas only few events are required with the event based technique

A chip that performs real-time image convolutions with programmable kernels of arbitrary shape is presented. This is a first prototype of reduced size (16 times 16 pixels) to validate system level techniques. It has been fabricated in AMS-0.35 mum, 2-poly, 3-metal technology. Chip inputs and outputs are coded using address event representation (AER). This is an emergent neuromorphic interchip communication protocol that allows for real-time virtual massive connectivity between huge numbers of pixels located on different chips. Pixels generate 'events' according to their activity levels. More active pixels generate more events per unit time and access the interchip communication channel more frequently, whereas pixels with low activity consume less communication bandwidth. This allows communicating more relevant information in a very short time. Specific PCI boards have been developed to feed images into the chip and to read images out of it

This paper addresses the problem of converting a conventional video stream based on sequences of frames into the spike event-based representation known as the address-event-representation (AER). In this paper we concentrate on rate-coded AER. The problem is addressed as an algorithmic problem, in which different methods are proposed, implemented and tested through software algorithms. The proposed algorithms are comparatively evaluated according to different criteria. Emphasis is put on the potential of such algorithms for a) doing the frame-based to event-based representation in real time, and b) that the resulting event streams ressemble as much as possible those generated naturally by rate-coded address-event VLSI chips, such as silicon AER retinae. It is found that simple and straightforward algorithms tend to have high potential for real time but produce event distributions that differ considerably from those obtained in AER VLSI chips. On the other hand, sophisticated algorithms that yield better event distributions are not efficient for real time operations. The methods based on linear-feedback-shift-register (LFSR) pseudorandom number generation is a good compromise, which is feasible for real time and yield reasonably well distributed events in time. Our software experiments, on a 1.6-GHz Pentium IV, show that at 50% AER bus load the proposed algorithms require between 0.011 and 1.14 ms per 8 bit-pixel per frame. One of the proposed LFSR methods is implemented in real time hardware using a prototyping board that includes a VirtexE 300 FPGA. The demonstration hardware is capable of transforming frames of 64 x 64 pixels of 8-bit depth at a frame rate of 25 frames per second, producing spike events at a peak rate of 10(7) events per second.

This paper presents a very large scale integration (VLSI) implementation of a low-power current-mode fuzzy-adaptive resonance theory (ART) cell. The cell is based on a compact new current source multibit memory cell with online learning capability. A small prototype of the designed cell and its peripheral block has been fabricated in the AustriaMicroSystems (AMS)-0.35-microm technology. The cell occupies a total area of 44 x 34 microm2 and consumes a maximum current of 22 nA.

A 5-layer neuromorphic vision processor whose components communicate spike events asychronously using the address-event- representation (AER) is demonstrated. The system includes a retina chip, two convolution chips, a 2D winner-take-all chip, a delay line chip, a learning classifier chip, and a set of PCBs for computer interfacing and address space remappings. The components use a mixture of analog and digital computation and will learn to classify trajectories of a moving object. A complete experimental setup and measurements results are shown.

This paper considers the viability of compact low-resolution low-power mini digital-to-analog converters (mini-DACs) for use in large arrays of neural type cells, where programmable weights are required. Transistors are biased in weak inversion in order to yield small currents and low power consumptions, a necessity when building large size arrays. One important drawback of weak inversion operation is poor matching between transistors. The resulting effective precision of a fabricated array of 50 DACs turned out to be 47% (1.1 bits), due to transistor mismatch. However, it is possible to combine them two by two in order to build calibrated DACs, thus compensating for inter-DAC mismatch. It is shown experimentally that the precision can be improved easily by a factor of 10 (4.8% or 4.4 bits), which makes these DACs viable for low-resolution applications such as massive arrays of neural processing circuits. A design methodology is provided, and illustrated through examples, to obtain calibrated mini-DACs of a given target precision. As an example application, we show simulation results of using this technique to calibrate an array of digitally controlled integrate-and-fire neurons.

We have identified a second-order reaction-diffusion differential equation able to reproduce through parameter setting different complex spatio-temporal behaviors. We have designed a log-domain hardware that implements the spatially discretized version of the selected reaction-diffusion equation. The logarithmic compression of the state variables allows several decades of variation of these state variables within subthreshold operation of the MOS transistors. Furthermore, as all the equation parameters are implemented as currents, they can be adjusted several decades. As a demonstrator, we have designed a chip containing a linear array of ten second-order dynamics coupled cells. Using this hardware, we have experimentally reproduced two complex spatio-temporal phenomena: the propagation of travelling waves and of trigger waves, as well as isolated oscillatory cells.

A neural architecture that implements a programmable 2D image filter has been presented. The architecture allows to implement any 2D filter F(p,q) decomposable into x-axis and y-axis components F(p,q) = H(p)V(q) such that the product can be approximated by a signed minimum. Positive and negative values of H(p) and V(q) can be programmed. The architecture requires an address even representation (AER) input. This allows to rotate the 2D convolution kernel any angle. Circuit simulation results of critical components were given. System-level behavioral simulations of a 128x128 array have been included which validate the proposed approach.

This paper presents a methodology for mismatch parameter extraction and mismatch simulation using conventional electrical simulators, like HSpice. A measurement and extraction procedure has been carefully designed to be able to obtain reliable measurements of the mismatch parameters of a given technology. The correctness of this extraction procedure method has been checked through three different validation methods. We also present two methods for performing mismatch simulation with conventional circuit simulators (like HSpice) using the extracted parameters.

A new 5-parameter MOS transistor mismatch model is introduced capable of predicting transistor mismatch with very high accuracy for ohmic and saturation strong inversion regions, including short channel transistors. The new model is based on splitting the contribution of the mobility degradation parameter mismatch into two components, and modulating them as the transistor transitions from ohmic to saturation regions. The model is tested for a wide range of transistor sizes (30), and shows excellent precision, never reported before for such a wide range of transistor sizes, including short channel transistors.

An architecture is proposed for the realization of real-time edge-extraction filtering operation in an Address-Event-Representation (AER) vision system. Furthermore, the approach is valid for any 2D filtering operation as long as the convolutional kernel F(p,q) is decomposable into an x-axis and a y-axis component, i.e. F(p,q)=H(p)V(q), for some rotated coordinate system [p,q]. If it is possible to find a coordinate system [p,q], rotated with respect to the absolute coordinate system a certain angle, for which the above decomposition is possible, then the proposed architecture is able to perform the filtering operation for any angle we would like the kernel to be rotated. This is achieved by taking advantage of the AER and manipulating the addresses in real time. The proposed architecture, however, requires one approximation: the product operation between the horizontal component H(p) and vertical component V(q) should be able to be approximated by a signed minimum operation without significant performance degradation. It is shown that for edge-extraction applications this filter does not produce performance degradation. The proposed architecture is intended to be used in a complete vision system known as the Boundary-Contour-System and Feature-Contour-System Vision Model, proposed by Grossberg and collaborators. The present paper proposes the architecture, provides a circuit implementation using MOS transistors operated in weak inversion, and shows behavioral simulation results at the system level operation and electrical simulation and experimental results at the circuit level operation of some critical subcircuits.

Recently, a real-time clustering microchip neural engine based on the ART1 architecture has been reported. However, that chip rendered an extremely high silicon area consumption of 1 cm(2), and consequently an extremely low yield of 6%. Redundant circuit techniques can be introduced to improve yield performance at the cost of further increasing chip size. In this paper we present an improved ART1 chip prototype based on a different approach to implement the most area consuming circuit elements of the first prototype: an array of several thousand current sources which have to match within a precision of around 1%. Such achievement was possible after a careful transistor mismatch characterization of the fabrication process (ES2-1.0 mum CMOS). A new prototype chip has been fabricated which can cluster 50-b input patterns into up to ten categories. The chip has 15 times less area, shows a yield performance of 98%, and presents the same precision and speed than the previous prototype. Due to its higher robustness multichip systems are easily assembled. As a demonstration we show results of a two-chip ART1 system, and of an ARTMAP system made of two ART1 chips and an extra interfacing chip.

This paper presents an analog current-mode VLSI implementation of an unsupervised clustering algorithm. The clustering algorithm is based on the popular ART1 algorithm, but has been modified resulting in a more VLSI-friendly algorithm that allows a more efficient hardware implementation with simple circuit operators, little memory requirements, modular chip assembly capability, and higher speed figures. The chip described in this paper implements a network that can cluster 100 binary pixel input patterns into up to 18 different categories. Modular expansibility of the system is directly possible by assembling a V/spl times/M array of chips without any extra interfacing circuitry, so that the maximum number of clusters is 18/spl times/M and the maximum number of bits of the input pattern is N/spl times/100. Pattern classification and learning is performed in 1.8 /spl mu/s, which is an equivalent computing power of 4.4/spl times/10/sup 9/ connections per second plus connection-updates per second. The chip has been fabricated in a standard low cost 1.6 /spl mu/m double-metal single-poly CMOS process, has a die area of 1 cm/sup 2/, and is mounted in a 120-pin PGA package. Although internally the chip is analog in nature, it interfaces to the outside world through digital signals, and thus has a true asynchronous digital behavior. Experimental chip test results are available, obtained through digital chip test equipment. Fault tolerance at the system level operation is demonstrated through the experimental testing of faulty chips.

In this paper we present a Winner-Take-All (WTA) circuit realized using current-mode circuit design techniques. The operation of the WTA is based on current comparators and current mirrors. Speed and precision is determined primarily by the characteristics of the current mirror used. Using special current mirrors, resolutions below 1% and settling times below 100 ns were simulated. The circuit remains operative with reasonable speed and precision for an input-signal range wider than two decades. It requires current input signals and provides voltage output signals.>

We describe an analog VLSI implementation of the ARTI algorithm (Carpenter, 1987). A prototype chip has been fabricated in a standard low cost 1.5~m double-metal single-poly CMOS process. It has a die area of lcm2 and is mounted in a 12O-pins PGA package. The chip realizes a modified version of the original ARTI architecture. Such modification has been shown to preserve all computational properties of the original algorithm (Serrano, 1994a), while being more appropriate for VLSI realizations. The chip implements an ARTI network with 100 F 1 nodes and 18 F2 nodes. It can therefore cluster 100 binary pixels input patterns into up to 18 different categories. Modular expansibility of the system is possible by assembling an NxM array of chips without any extra interfacing circuitry, resulting in an F 1 layer with l00xN nodes, and an F2 layer with 18xM nodes. Pattern classification is performed in less than 1.8~s, which means an equivalent computing power of 2.2x109 connections and connection-updates per second. Although internally the chip is analog in nature, it interfaces to the outside world through digital signals, thus having a true asynchrounous digital behavior. Experimental chip test results are available, which have been obtained through test equipments for digital chips.

The transconductance-mode (T-mode) approach is extended to implement analog continuous-time neural network hardware systems to include on-chip Hebbian learning and on-chip analog weight storage capability. The demonstration vehicle used is a 5+5-neuron bidirectional associative memory (BAM) prototype fabricated in a standard 2-mum double-metal double-polysilicon CMOS process. Mismatches and nonidealities in learning neural hardware are not supposed to be critical if on-chip learning is available, because they will be implicitly compensated. However, mismatches in the learning circuits themselves cannot always be compensated. This mismatch is specially important if the learning circuits use transistors operating in weak inversion. The authors estimate the expected mismatch between learning circuits in the BAM network prototype and evaluate its effect on the learning performance, using theoretical computations and Monte Carlo HSPICE simulations. These theoretical predictions are verified using experimentally measured results on the test vehicle prototype.