In robotics, one of the most common tasks involves executing trajectories to reach a specific target in space; but, such movements, and consequently the trajectories, frequently exhibit oscillatory or non-uniform behavior around the target point within a continuous trajectory. Neuromorphic engineering seeks to integrate the computational mechanisms observed in animal brains into contemporary technological systems. Neuromorphic engineering has been successfully applied in fields such as robotics, autonomous systems, edge computing, and healthcare. In robotics, it efficiently addresses challenges like dynamic control in path planning. By adopting this strategy, in this work, we present a space-time smooth control mechanism for closed-loop spiking robot arms. The system is based on a spiking neural network inspired by the structure and function of the nervous system. The proposed neural network has been implemented on the mixed analog-digital signal special purpose hardware platform DYNAP-SE2. A set of experiments has been conducted to test the system on both forward and reverse reference trajectories for different waiting times in the interpolation carried out by the FPGA-based control. A tunable smoothing of the trajectory has been achieved for one joint of the robotic arm. Given the temporal limitations of the hardware setup, the optimal interval between interpolated points is 8 ms.

Neuromorphic engineering aims to incorporate the computational principles found in animal brains, into modern technological systems. Following this approach, in this work we propose a closed-loop neuromorphic control system for an event-based robotic arm. The proposed system consists of a shifted Winner-Take-All spiking network for interpolating a reference trajectory and a spiking comparator network responsible for controlling the flow continuity of the trajectory, which is fed back to the actual position of the robot. The comparator model is based on a differential position comparison neural network, which governs the execution of the next trajectory points to close the control loop between both components of the system. To evaluate the system, we implemented and deployed the model on a mixed-signal analog-digital neuromorphic platform, the DYNAP-SE2, to facilitate integration and communication with the ED-Scorbot robotic arm platform. Experimental results on one joint of the robot validate the use of this architecture and pave the way for future neuro-inspired control of the entire robot.

In this study, a recurrent spiking neural network (RSNN) has been implemented on an accelerator deployed on a reconfigurable circuit (FPGA) to predict the Condenser Fouling Factor (CFF) in a chiller as a predictive maintenance (PdM) measure. This system operates through a low-power device utilising edge computing, geared towards AIoT/EdgeAI applications. It can accurately identify CFF levels that exceed 25% based on several pressure and temperature sensor data distributed in the chiller, achieving an accuracy greater than 90% with an architecture of 256 parallel and recurrent neurons in the hidden layer. The model was trained using a proprietary dataset that recorded sensor states during controlled experiments where the condenser was manually obstructed at varying coverage percentages. The primary advantage of employing RSNN techniques lies in their dual capability: first, they are designed to detect temporal signal patterns, and second, their trainability allows for adaptation to various applications across different contexts. The training for the particular accelerator used in this work is done on the FPGA, not requiring power-hungry machines.

Neuromorphic engineering endeavors to integrate the computational prowess and efficiency inherent in biological neuronal systems, such as the brain, into contemporary technological systems, primarily through the deployment of spiking neural networks. This research delineates the development and implementation of a bio-inspired sequential hippocampus memory model, which can effectively learn and sequentially recall memories, within a robotic infrastructure. The hippocampus memory model, implemented on the SpiNNaker platform, has been tactically utilized to control a 4-joint event-based robot arm, the ED-ScorBot, by learning and then recalling trajectories via a sequence of memories regarding joint positions. The conveyed spiking information from SpiNNaker is interpreted by an FPGA in real-time to command the event-driven motors of the robotic arm, integrating learned trajectories into physical robotic movement. An empirical exploration validates the model’s capability to govern the robotic arm’s trajectory with precision and dependability while simultaneously demonstrating the potential for incorporating spike-based memory models in robotic applications. This synergistic convergence of neuromorphic engineering and robotics illustrates a viable pathway towards sophisticated, efficient, and adaptable robotic systems capable of learning and reproducing complex tasks, with significant implications for future developments in autonomous robotic applications.

Prostate cancer is the second most frequent cancer in men worldwide after lung cancer. Its diagnosis is based on the identification of the Gleason score that evaluates the abnormality of cells in glands through the analysis of the different Gleason patterns within tissue samples. The recent advancements in computational pathology, a domain aiming at developing algorithms to automatically analyze digitized histopathology images, lead to a large variety and availability of datasets and algorithms for Gleason grading and scoring. However, there is no clear consensus on which methods are best suited for each problem in relation to the characteristics of data and labels. This paper provides a systematic comparison on nine datasets with state-of-the-art training approaches for deep neural networks (including fully-supervised learning, weakly-supervised learning, semi-supervised learning, Additive-MIL, Attention-Based MIL, Dual-Stream MIL, TransMIL and CLAM) applied to Gleason grading and scoring tasks. The nine datasets are collected from pathology institutes and openly accessible repositories. The results show that the best methods for Gleason grading and Gleason scoring tasks are fully supervised learning and CLAM, respectively, guiding researchers to the best practice to adopt depending on the task to solve and the labels that are available.

This paper presents a sensory fusion neuromorphic dataset collected with precise temporal synchronization using a set of Address-Event-Representation sensors and tools. The target application is the lip reading of several keywords for different machine learning applications, such as digits, robotic commands, and auxiliary rich phonetic short words. The dataset is enlarged with a spiking version of an audio-visual lip reading dataset collected with frame-based cameras. LIPSFUS is publicly available and it has been validated with a deep learning architecture for audio and visual classification. It is intended for sensory fusion architectures based on both artificial and spiking neural network algorithms.

Biological nervous systems typically perform the control of numerous degrees of freedom for example in animal limbs. Neuromorphic engineers study these systems by emulating them in hardware for a deeper understanding and its possible application to solve complex problems in engineering and robotics. Central-Pattern-Generators (CPGs) are part of neuro-controllers, typically used at their last steps to produce rhythmic patterns for limbs movement. Different patterns and gaits typically compete through winner-take-all (WTA) circuits to produce the right movements. In this work we present a WTA circuit implemented in a Spiking-Neural-Network (SNN) processor to produce such patterns for controlling a robotic arm in real-time. The robot uses spike-based proportional-integrative-derivative (SPID) controllers to keep a commanded joint position from the winner population of neurons of the WTA circuit. Experiments demonstrate the feasibility of robotic control with spiking circuits following brain-inspiration.

In this work, an all-in-one neuromorphic controller system with reduced latency and power consumption for a robotic arm is presented. Biological muscle movement consists of stretching and shrinking fibres via spike-commanded signals that come from motor neurons, which in turn are connected to a central pattern generator neural structure. In addition, biological systems are able to respond to diverse stimuli rather fast and efficiently, and this is based on the way information is coded within neural processes. As opposed to human-created encoding systems, neural ones use neurons and spikes to process the information and make weighted decisions based on a continuous learning process. The Event-Driven Scorbot platform (ED-Scorbot) consists of a 6 Degrees of Freedom (DoF) robotic arm whose controller implements a Spiking Proportional-Integrative-Derivative algorithm, mimicking in this way the previously commented biological systems. In this paper, we present an infrastructure upgrade to the ED-Scorbot platform, replacing the controller’s hardware, which was comprised of two Spartan Field Programmable Gate Arrays (FPGAs) and a barebone computer, with an edge device, the Xilinx Zynq-7000 SoC (System on Chip) which reduces the response time, power consumption and overall complexity.

Neuromorphic systems are a viable alternative to conventional systems for real-time tasks with constrained resources. Their low power consumption, compact hardware realization, and low-latency response characteristics are the key ingredients of such systems. Furthermore, the event-based signal processing approach can be exploited for reducing the computational load and avoiding data loss due to its inherently sparse representation of sensed data and adaptive sampling time. In event-based systems, the information is commonly coded by the number of spikes within a specific temporal window. However, the temporal information of event-based signals can be difficult to extract when using rate coding. In this work, we present a novel digital implementation of the model, called time difference encoder (TDE), for temporal encoding on event-based signals, which translates the time difference between two consecutive input events into a burst of output events. The number of output events along with the time between them encodes the temporal information. The proposed model has been implemented as a digital circuit with a configurable time constant, allowing it to be used in a wide range of sensing tasks that require the encoding of the time difference between events, such as optical flow-based obstacle avoidance, sound source localization, and gas source localization. This proposed bioinspired model offers an alternative to the Jeffress model for the interaural time difference estimation, which is validated in this work with a sound source lateralization proof-of-concept system. The model was simulated and implemented on a field-programmable gate array (FPGA), requiring 122 slice registers of hardware resources and less than 1 mW of power consumption.

The brain cortex processes visual information to classify it following a scheme that has been mimicked by Convolutional Neural Networks (CNN). Specialised hardware accelerators are currently used as CPU co-processors for mobile applications. These accelerators are getting closer to the sensors for an edge computation of its output towards a faster and lower power consumption improvements. In this demonstration we use a dynamic vision sensor (inspired in the retina neural cells) as a visual source of the NullHop CNN accelerator deployed on a MPSoC FPGA and placed into a mobile robot for edge-computing the visual information and classify it to properly command a Summit-XL mobile robot for a target destiny. The reduced latency of the used CNN accelerator allows to process several histograms before taking a movement decision. A distance sensor mounted on the robot ensures that the direction change is done at the right distance for a proper path following.

The brain is able to solve complex tasks in real time by combining different sensory cues with previously acquired knowledge. Inspired by the brain, we designed a neuromorphic demonstrator which combines auditory and visual input to find an obstacle free direction closest to the sound source. The system consists of two event-based sensors (the eDVS for vision and the NAS for audition) mounted onto a pan-tilt unit and a spiking neural network implemented on the SpiNNaker platform. By combining the different sensory information, the demonstrator is able to point at a sound source direction while avoiding obstacles in real time.

Spiking convolutional neural networks have become a novel approach for machine vision tasks, due to the latency to process an input stimulus from a scene, and the low power consumption of these kind of solutions. Event-based systems only perform sum operations instead of sum of products of frame-based systems. In this work an upgrade of a neuromorphic event-based convolution accelerator for SCNN, which is able to perform multiple layers with different kernel sizes, is presented. The system has a latency per layer from 1.44 us to 9.98us for kernel sizes from 1x1 to 7x7.

Glioma is a type of brain tumor that causes mortality in many cases. Early diagnosis is an important factor. Typically, it is detected through MRI and then either a treatment is applied, or it is removed through surgery. Deep-learning techniques are becoming popular in medical applications and image-based diagnosis. Convolutional Neural Networks are the preferred architecture for object detection and classification in images. In this paper, we present a study to evaluate the efficiency of using CNNs for diagnosis aids in glioma detection and the improvement of the method when using a clustering method (Fuzzy C-means) for preprocessing the input MRI dataset. Results offered an accuracy improvement from 0.77 to 0.81 when using Fuzzy C-Means.

Deep Learning tools are widely used for medical image segmentation. The results produced by these techniques depend to a great extent on the data sets used to train the used network. Nowadays many cloud service providers offer the required resources to train networks and deploy deep learning networks. This makes the idea of segmentation as a cloud-based service attractive. In this paper we study the possibility of training, a generalized configurable, Keras U-Net to test the feasibility of training with images acquired, with specific instruments, to perform predictions on data from other instruments. We use, as our application example, the segmentation of Optic Disc and Cup which can be applied to glaucoma detection. We use two publicly available data sets (RIM-One V3 and DRISHTI) to train either independently or combining their data.

Breast cancer is one of the most frequent causes of mortality in women. For the early detection of breast cancer, the mammography is used as the most efficient technique to identify abnormalities such as tumors. Automatic detection of tumors in mammograms has become a big challenge and can play a crucial role to assist doctors in order to achieve an accurate diagnosis. State-of-the-art Deep Learning algorithms such as Faster Regional Convolutional Neural Networks are able to determine the presence of an object and also its position inside the image in a reduced computation time. In this work, we evaluate these algorithms to detect tumors in mammogram images and propose a detection system that contains: (1) a preprocessing step performed on mammograms taken from the Digital Database for Screening Mammography (DDSM) and (2) the Neural Network model, which performs feature extraction over the mammograms in order to locate tumors within each image and classify them as malignant or benign. The results obtained show that the proposed algorithm has an accuracy of 97.375%. These results show that the system could be very useful for aiding physicians when detecting tumors from mammogram images.

Along with the proliferation of mobile devices and wireless signal coverage, IoT devices, such as smart wristbands \nfor monitoring its owner’s activity or sleep patterns, get great popularity. Wearable technology in human \nlife has become quite useful due to the information given (sleep hours, heart rate, etc). However, wearables \nfor animals does not give information about behaviour directly: they collect raw data that is sent to a server \nwhere, after a post-processing step, the behaviour is known. In this work, we present a smart IoT device that \nclassifies different animal behaviours from the information obtained from on-board sensors using an embedded \nneural network running in the device. This information is uploaded to a server through a wireless sensor \nnetwork based on Zigbee communication. The architecture of the device allows an easy assembly in a reduced \ndimension wearable case. The firmware allows a modular functionality by activating or deactivating \nmodules independently, which improve the power efficiency of the device. The power consumption has been \nanalyzed, allowing the 1Ah battery to work the system during several days. A novel localization and distance \nestimation technique (for 802.15.4 networks) is presented to recover a lost device in Do˜nana National Park \nwith unidirectional antennas and log-normalization distance estimation over RSSI.

Falls are one of the most frequent causes of injuries in elderly people. Wearable Fall Detection Systems provided a ubiquitous tool for monitoring and alert when these events happen. Recurrent Neural Networks (RNN) are algorithms that demonstrates a great accuracy in some problems analyzing sequential inputs, such as temporal signal values. However, their computational complexity are an obstacle for the implementation in IoT devices. This work shows a performance analysis of a set of RNN architectures when trained with data obtained using different sampling frequencies. These architectures were trained to detect both fall and fall hazards by using accelerometers and were tested with 10-fold cross validation, using the F1-score metric. The results obtained show that sampling with a frequency of 25Hz does not affect the effectiveness, based on the F1-score, which implies a substantial increase in the performance in terms of computational cost. The architectures with two RNN layers and without a first dense layer had slightly better results than the smallest architectures. In future works, the best architectures obtained will be integrated in an IoT solution to determine the effectiveness empirically.

This live demonstration presents an audio-guided neuromorphic robot: from a Neuromorphic Auditory Sensor (NAS) to locomotion using Spiking Central Pattern Generators (sCPGs). Several gaits are generated by sCPGs implemented on a SpiNNaker board. The output of these sCPGs is sent in a real-time manner to an Field Programmable Gate Array (FPGA) board using an AER-to-SpiNN interface. The control of the hexapod robot joints is performed by the FPGA board. The robot behavior can be changed in real-time by means of the NAS. The audio information is sent to the SpiNNaker board which classifies it using a Spiking Neural Network (SNN). Thus, the input sound will activate a specific gait pattern which will eventually modify the behavior of the robot.

In this demonstration a spiking neural network architecture for vision recognition using an FPGA spiking convolution processor, based on leaky integrate and fire neurons (LIF) and a SpiNNaker board is presented. The network has been trained with Poker-DVS dataset in order to classify the four different card symbols. The spiking convolution processor extracts features from images in form of spikes, computes by one layer of 64 convolutions. These features are sent to an OKAERtool board that converts from AER to 2-7 protocol to be classified by a spiking neural network deployed on a SpiNNaker platform.

Convolutional neural networks (CNNs) have become the dominant neural network architecture for solving many state-of-the-art (SOA) visual processing tasks. Even though graphical processing units are most often used in training and deploying CNNs, their power efficiency is less than 10 GOp/s/W for single-frame runtime inference. We propose a flexible and efficient CNN accelerator architecture called NullHop that implements SOA CNNs useful for low-power and low-latency application scenarios. NullHop exploits the sparsity of neuron activations in CNNs to accelerate the computation and reduce memory requirements. The flexible architecture allows high utilization of available computing resources across kernel sizes ranging from 1×1 to 7×7. NullHop can process up to 128 input and 128 output feature maps per layer in a single pass. We implemented the proposed architecture on a Xilinx Zynq field-programmable gate array (FPGA) platform and presented the results showing how our implementation reduces external memory transfers and compute time in five different CNNs ranging from small ones up to the widely known large VGG16 and VGG19 CNNs. Postsynthesis simulations using Mentor Modelsim in a 28-nm process with a clock frequency of 500 MHz show that the VGG19 network achieves over 450 GOp/s. By exploiting sparsity, NullHop achieves an efficiency of 368%, maintains over 98% utilization of the multiply-accumulate units, and achieves a power efficiency of over 3 TOp/s/W in a core area of 6.3 mm2. As further proof of NullHop's usability, we interfaced its FPGA implementation with a neuromorphic event camera for real-time interactive demonstrations.

Neural networks algorithms are commonly used to recognize patterns from different data sources such as audio or vision. In image recognition, Convolutional Neural Networks are one of the most effective techniques due to the high accuracy they achieve. This kind of algorithms require billions of addition and multiplication operations over all pixels of an image. However, it is possible to reduce the number of operations using other computer vision techniques rather than frame-based ones, e.g. neuromorphic frame-free techniques. There exists many neuromorphic vision sensors that detect pixels that have changed their luminosity. In this study, an event-based convolution engine for FPGA is presented. This engine models an array of leaky integrate and fire neurons. It is able to apply different kernel sizes, from 1x1 to 7x7, which are computed row by row, with a maximum number of 64 different convolution kernels. The design presented is able to process 64 feature maps of 7x7 with a latency of 8.98μs.

Several studies have focused on classifying behavioral patterns in wildlife and captive species to monitor their activities and so to understanding the interactions of animals and control their welfare, for biological research or commercial purposes. The use of pattern recognition techniques, statistical methods and Overall Dynamic Body Acceleration (ODBA) are well known for animal behavior recognition tasks. The reconfigurability and scalability of these methods are not trivial, since a new study has to be done when changing any of the configuration parameters. In recent years, the use of Artificial Neural Networks (ANN) has increased for this purpose due to the fact that they can be easily adapted when new animals or patterns are required. In this context, a comparative study between a theoretical research is presented, where statistical and spectral analyses were performed and an embedded implementation of an ANN on a smart collar device was placed on semi-wild animals. This system is part of a project whose main aim is to monitor wildlife in real time using a wireless sensor network infrastructure. Different classifiers were tested and compared for three different horse gaits. Experimental results in a real time scenario achieved an accuracy of up to 90.7%, proving the efficiency of the embedded ANN implementation.

In this demonstration we present a spiking neural network architecture for audio samples classification using SpiNNaker. The network consists of different leaky integrate-and-fire neuron layers. The connections between them are trained using firing rate based algorithms. Tests use sets of pure tones with frequencies that range from 130.813 to 1396.91 Hz. Audio signals coming from the computer are converted to spikes using a Neuromorphic Auditory Sensor and, after that, this information is sent to the SpiNNaker board through a PCB that translates from AER to 2-of-7 protocol. The classification output obtained in the spiking neural network deployed on SpiNNaker is then shown in the computer screen. Different levels of random noise are added to the original audio signals in order to test the robustness of the classification system.

Despite being well established in robotics, classical motor controllers have several disadvantages: they pose a high computational load, therefore requiring powerful devices, they are not easy to tune and they are not suited for neuroprosthetics. In contrast, bio-inspired controller do not transform the output of the controller therefore no delays are introduced and a smooth response is achieved; they also have a high scalability. Finally, the most important feature of bio-inspired controllers is that they could integrate learning features to make them adaptable to new tasks within the same hardware robotic platform. We present the model and simulation of a spiking neural network for low-level motor control. The proposed neural network acts as a motor controller and produces pulsed signals which can be directly interfaced with commercial DC motors. The simulated network is compatible with neuromorphic VLSI implementation and paves the way to the implementation bio-inspired motor controller which are compact, low power, scalable and compatible with neuroprosthetic. The network presented is inspired by the current knowledge about biological motor control: it comprises alpha motoneuron for driving the motor and spindle populations to provide the feedback and close the loop. The spikes from the motoneuron population are time lengthen to a fixed amount of time and supplied to the simulated motor: Pulse Frequency Modulation (PFM) modulation is used. This paper presents the software simulations using the Brian simulator for a position controller. Our controller is a first step toward a novel bio-inspired motor control approach suitable for robotics as well as neuroprosthetic.

In this live demonstration we exploit the use of a serial link for fast asynchronous communication in massively parallel processing platforms connected to a DVS for real-time implementation of bio-inspired vision processing on spiking neural networks.

Address-Event-Representation (AER) is a widely extended asynchronous technique for interchanging “neural spikes” among different hardware elements in Neuromorphic Systems. Conventional AER links use parallel physical wires together with a pair of handshaking signals (Request and Acknowledge). Here we present a fully serial implementation using bidirectional SATA connectors with a pair of LVDS (low voltage differential signaling) wires for each direction. The proposed implementation can multiplex a number of conventional parallel AER links per LVDS physical connection. It uses flow control, clock correction, and byte alignment techniques to transmit 32-bit address events reliably over multiplexed serial connections. The setup has been tested using commercial Spartan6 FPGAs reaching a maximum event transmission speed of 75Meps (Mega Events per second) for 32-bit events at 3.0Gbps line data rate.

This paper presents a new architecture, design flow, and field-programmable gate array (FPGA) implementation analysis of a neuromorphic binaural auditory sensor, designed completely in the spike domain. Unlike digital cochleae that decompose audio signals using classical digital signal processing techniques, the model presented in this paper processes information directly encoded as spikes using pulse frequency modulation and provides a set of frequency-decomposed audio information using an address-event representation interface. In this case, a systematic approach to design led to a generic process for building, tuning, and implementing audio frequency decomposers with different features, facilitating synthesis with custom features. This allows researchers to implement their own parameterized neuromorphic auditory systems in a low-cost FPGA in order to study the audio processing and learning activity that takes place in the brain. In this paper, we present a 64-channel binaural neuromorphic auditory system implemented in a Virtex-5 FPGA using a commercial development board. The system was excited with a diverse set of audio signals in order to analyze its response and characterize its features. The neuromorphic auditory system response times and frequencies are reported. The experimental results of the proposed system implementation with 64-channel stereo are: a frequency range between 9.6 Hz and 14.6 kHz (adjustable), a maximum output event rate of 2.19 Mevents/s, a power consumption of 29.7 mW, the slices requirements of 11141, and a system clock frequency of 27 MHz.

This paper describes the software and FPGA implementation of a Retinal Ganglion Cell model which detects moving objects. It is shown how this processing, in conjunction with a Dynamic Vision Sensor as its input, can be used to extrapolate information about object position. Software-wise, a system based on an array of these of RGCs has been developed in order to obtain up to two trackers. These can track objects in a scene, from a still observer, and get inhibited when saccadic camera motion happens. The entire processing takes on average 1000 ns/event. A simplified version of this mechanism, with a mean latency of 330 ns/event, at 50 MHz, has also been implemented in a Spartan6 FPGA.

This demonstration shows how object detection and tracking are possible thanks to a new implementation which takes inspiration from the visual processing of a particular type of ganglion cell in the retina.

In this paper, we explore the capabilities of a sound classification system that combines both a novel FPGA cochlear model implementation and a bio-inspired technique based on a trained convolutional spiking network. The neuromorphic auditory system that is used in this work produces a form of representation that is analogous to the spike outputs of the biological cochlea. The auditory system has been developed using a set of spike-based processing building blocks in the frequency domain. They form a set of band pass filters in the spike-domain that splits the audio information in 128 frequency channels, 64 for each of two audio sources. Address Event Representation (AER) is used to communicate the auditory system with the convolutional spiking network. A layer of convolutional spiking network is developed and trained on a computer with the ability to detect two kinds of sound: artificial pure tones in the presence of white noise and electronic musical notes. After the training process, the presented system is able to distinguish the different sounds in real-time, even in the presence of white noise.

This paper describes an open-source implementation of an event-based dynamic and active pixel vision sensor (DAVIS) for racing human vs. computer on a slot car track. The DAVIS is mounted in "eye-of-god" view. The DAVIS image frames are only used for setup and are subsequently turned off because they are not needed. The dynamic vision sensor (DVS) events are then used to track both the human and computer controlled cars. The precise control of throttle and braking afforded by the low latency of the sensor output enables consistent outperformance of human drivers at a laptop CPU load of <;3% and update rate of 666Hz. The sparse output of the DVS event stream results in a data rate that is about 1000 times smaller than from a frame-based camera with the same resolution and update rate. The scaled average lap speed of the 1/64 scale cars is about 450km/h which is twice as fast as the fastest Formula 1 lap speed. A feedbackcontroller mode allows competitive racing by slowing the computer controlled car when it is ahead of the human. In tests of human vs. computer racing the computer still won more than 80% of the races.

Dynamic vision sensors (DVS) are frame-free sensors with an asynchronous variable-rate output that is ideal for hard real-time dynamic vision applications under power and latency constraints. Post-processing of the digital sensor output can reduce sensor noise, extract low level features, and track objects using simple algorithms that have previously been implemented in software. In this paper we present an FPGA-based framework for event-based processing that allows uncorrelated-event noise removal and real-time tracking of multiple objects, with dynamic capabilities to adapt itself to fast or slow and large or small objects. This framework uses a new hardware platform based on a Lattice FPGA which filters the sensor output and which then transmits the results through a super-speed Cypress FX3 USB microcontroller interface to a host computer. The packets of events and timestamps are transmitted to the host computer at rates of 10 Mega events per second. Experimental results are presented that demonstrate a low latency of 10us for tracking and computing the center of mass of a detected object.

A key requirement for modern large scale neuromorphic systems is the ability to detect and diagnose faults and to explore self-correction strategies. In particular, to perform this under area-constraints which meet scalability requirements of large neuromorphic systems. A bio-inspired online fault detection and self-correction mechanism for neuro-inspired PID controllers is presented in this paper. This strategy employs a fault detection unit for online testing of the PID controller; uses a fault detection manager to perform the detection procedure across multiple controllers, and a controller selection mechanism to select an available fault-free controller to provide a corrective step in restoring system functionality. The novelty of the proposed work is that the fault detection method, using synapse models with excitatory and inhibitory responses, is applied to a robotic spike-based PID controller. The results are presented for robotic motor controllers and show that the proposed bio-inspired self-detection and self-correction strategy can detect faults and re-allocate resources to restore the controller's functionality. In particular, the case study demonstrates the compactness (~1.4% area overhead) of the fault detection mechanism for large scale robotic controllers.

Multisensory integration is commonly used in various robotic areas to collect much more information from an environment using different and complementary types of sensors. This demonstration presents a scenario where the motor rotation frequency is obtained using an AER DVS128 retina chip (Dynamic Vision Sensor) and a frequency decomposer auditory system on a FPGA that mimics a biological cochlea. Both of them are spike-based sensors with Address-Event-Representation (AER) outputs. A new AER monitor hardware interface, based on a Spartan-6 FPGA, allows two operational modes: real-time (up to 5 Mevps through USB2.0) and off-line mode (up to 20 Mevps and 33.5 Mev stored in DDR RAM). The sensory integration allows the bio-inspired cochlea limit to provide a concrete range of rpm approaches, which are obtained by the silicon retina.

Nowadays spike-based brain processing emulation is taking off. Several EU and others worldwide projects are demonstrating this, like SpiNNaker, BrainScaleS, FACETS, or NeuroGrid. The larger the brain process emulation on silicon is, the higher the communication performance of the hosting platforms has to be. Many times the bottleneck of these system implementations is not on the performance inside a chip or a board, but in the communication between boards. This paper describes a novel modular Address-Event-Representation (AER) FPGA-based (Spartan6) infrastructure PCB (the AER-Node board) with 2.5Gbps LVDS high speed serial links over SATA cables that offers a peak performance of 32-bit 62.5Meps (Mega events per second) on board-to-board communications. The board allows back compatibility with parallel AER devices supporting up to x2 28-bit parallel data with asynchronous handshake. These boards also allow modular expansion functionality through several daughter boards. The paper is focused on describing in detail the LVDS serial interface and presenting its performance.

Spike-based motor control is very important in the field of robotics and also for the neuromorphic engineering community to bridge the gap between sensing / processing devices and motor control without losing the spike philosophy that enhances speed response and reduces power consumption. This paper shows an accurate neuro-inspired spike-based system composed of a DVS retina, a visual processing system that detects and tracks objects, and a SVITE motor control, where everything follows the spike-based philosophy. The control system is a spike version of the neuroinspired open loop VITE control algorithm implemented in a couple of FPGA boards: the first one runs the algorithm and the second one drives the motors with spikes. The robotic platform is a low cost arm with four degrees of freedom.

This paper presents a new DVS sensor with one order of magnitude improved contrast sensitivity over previous reported DVSs. This sensor has been applied to a bio-inspired event-based binocular system that performs 3D event-driven reconstruction of a scene. Events from two DVS sensors are matched by using precise timing information of their ocurrence. To improve matching reliability, satisfaction of epipolar geometry constraint is required, and simultaneously available information on the orientation is used as an additional matching constraint.

Distance calculation is always one of the most important goals in a digital stereoscopic vision system. In an AER system this goal is very important too, but it cannot be calculated as accurately as we would like. This demonstration shows a first approximation in this field, using a disparity algorithm between both retinas. The system can make a distance approach about a moving object, more specifically, a qualitative estimation. Taking into account the stereo vision system features, the previous retina positioning and the very important Hold&Fire building block, we are able to make a correlation between the spike rate of the disparity and the distance.

Neuromorphic engineers study models and implementations of systems that mimic neurons behavior in the brain. Neuro-inspired systems commonly use spikes to represent information. This representation has several advantages: its robustness to noise thanks to repetition, its continuous and analog information representation using digital pulses, its capacity of pre-processing during transmission time, ..., Furthermore, spikes is an efficient way, found by nature, to codify, transmit and process information. In this paper we propose, design, and analyze neuro-inspired building blocks that can perform spike-based analog filters used in signal processing. We present a VHDL implementation for FPGA. Presented building blocks take advantages of the spike rate coded representation to perform a massively parallel processing without complex hardware units, like floating point arithmetic units, or a large memory. Those low requirements of hardware allow the integration of a high number of blocks inside a FPGA, allowing to process fully in parallel several spikes coded signals.

This paper presents a first approach for implementations which fuse the Address-Event-Representation (AER) processing with the Cellular Automata using FPGA and AER-tools. This new strategy applies spike-based convolution filters inspired by Cellular Automata for AER vision processing. Spike-based systems are neuro-inspired circuits implementations traditionally used for sensory systems or sensor signal processing. AER is a neuromorphic communication protocol for transferring asynchronous events between VLSI spike-based chips. These neuro-inspired implementations allow developing complex, multilayer, multichip neuromorphic systems and have been used to design sensor chips, such as retinas and cochlea, processing chips, e.g. filters, and learning chips. Furthermore, Cellular Automata is a bio-inspired processing model for problem solving. This approach divides the processing synchronous cells which change their states at the same time in order to get the solution.

This demonstration shows how a new bio-inspired processing cascade architecture is used for simultaneous objects tracking.

This paper presents a cascade architecture for bio-inspired information processing. We use AER (Address Event Representation) for transmitting and processing visual information provided by an asynchronous temporal contrast silicon retina. Using this architecture, we also present a multiple objects tracking algorithm; this algorithm is described in VHDL and implemented in a FPGA (Spartan II), which is part of the USB-AER platform developed by some of the authors.

Correcting digital images tilt needs huge quantities of memory, high computational resources, and use to take a considerable amount of time. This demonstration shows how a spikes-based silicon retina dynamic vision sensor (DVS) tilt can corrected in real time using a commercial accelerometer. DVS output is a stream of spikes codified using the address-event representation (AER). Event-based processing is focused on change in real time DVS output addresses. Taking into account this DVS feature, we present an AER based layer able to correct in real time the DVS tilt, using a high speed algorithmic mapping layer and introducing a minimum latency in the system. A co-design platform (the AER-Robot platform), based into a Xilinx Spartan 3 FPGA and an 8051 USB microcontroller, has been used to implement the system.

Neuromorphic engineering tries to mimic biological information processing. Address-Event-Representation (AER) is an asynchronous protocol for transferring the information of spiking neuro-inspired systems. Currently AER systems are able sense visual and auditory stimulus, to process information, to learn, to control robots, etc. In this paper we present an AER based layer able to correct in real time the tilt of an AER vision sensor, using a high speed algorithmic mapping layer. A co-design platform (the AER-Robot platform), with a Xilinx Spartan 3 FPGA and an 8051 USB microcontroller, has been used to implement the system. Testing it with the help of the USBAERmini2 board and the jAER software.

Image convolution operations in digital computer systems are usually very expensive operations in terms of resource consumption (processor resources and processing time) for an efficient Real-Time application. In these scenarios the visual information is divided into frames and each one has to be completely processed before the next frame arrives in order to warranty the real-time. A spike-based philosophy for computing convolutions based on the neuro-inspired Address-Event-Representation (AER) is achieving high performances. In this paper we present two FPGA implementations of AER-based convolution processors for relatively small Xilinx FPGAs (Spartan-II 200 and Spartan-3 400), which process 64×64 images with 11×11 convolution kernels. The maximum equivalent operation rate that can be reached is 163.51 MOPS for 11×11 kernels, in a Xilinx Spartan 3 400 FPGA with a 50MHz clock. Formulations, hardware architecture, operation examples and performance comparison with frame-based convolution processors are presented and discussed.

Neuromorphic engineering tries to mimic biological information processing. Address-event representation (AER) is a neuromorphic communication protocol for spiking neurons between different chips. We present a new way to drive robotic platforms using spiking neurons. We have simulated spiking control models for DC motors, and developed a mobile robot (Eddie) controlled only by spikes. We apply AER to the robot control, monitoring and measuring the spike activity inside the robot. The mobile robot is controlled by the AER-Robot tool, and the AER information is sent to a PC using the USBAERmini2 interface.

Neuromorphic engineering tries to mimic biology in information processing. Address-event representation (AER) is a neuromorphic communication protocol for spiking neurons between different layers. AER bio-inspired image sensor are called ldquoretinardquo. This kind of sensors measure visual information not based on frames from real life and generates corresponding events. In this paper we provide an alternative, based on cheap FPGA, to this image sensors that takes images provided by an analog video source (video composite signal), digitalizes it and generates AER streams for testing purposes.

In recent implementations of neuromorphic spike-based sensors, multi-neuron processors, and actuators; the spike traffic between devices is coded in the form of asynchronous spike streams following the address-event-representation protocol. This spike information can be modified during the transmission from one device to another by using a mapper device. In this paper we present a mapper implementation which transforms event addresses and can also delay events in time. We discuss two different architectures for implementing the time delays on an FPGA board (USB-AER), and we present an example of the use of the time delay feature in the mapper in an implementation of a visual elementary motion detection model based on the spike outputs of a temporal contrast retina.

This paper describes CAVIAR, a massively parallel hardware implementation of a spike-based sensing-processing-learning-actuating system inspired by the physiology of the nervous system. CAVIAR uses the asychronous address-event representation (AER) communication framework and was developed in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45k neurons (spiking cells), up to 5M synapses, performs 12G synaptic operations per second, and achieves millisecond object recognition and tracking latencies.

Address-Event-Representation (AER) is an \nasynchronous protocol for transferring the information of \nspiking neuro-inspired systems. Actually AER systems are able \nto see, to ear, to process information, and to learn. Regarding to \nthe actuation step, the AER has been used for implementing \nCentral Pattern Generator algorithms, but not for controlling \nthe actuators in a closed-loop spike-based way. In this paper we \nanalyze an AER based model for a real-time neuro-inspired \nclosed-loop control system. We demonstrate it into a differential \ncontrol system for a two-wheel vehicle using feedback AER \ninformation. PFM modulation has been used to power the DC \nmotors of the vehicle and translation into AER of encoder \ninformation is also presented for the close-loop. A codesign \nplatform (called AER-Robot), based into a Xilinx Spartan 3 \nFPGA and an 8051 USB microcontroller, with power stages for \nfour DC motors has been used for the demonstrator.

In this demo we propose a method for computing real time convolution on AER images. For that we use signed events. The AER events produced on an AER retina or an image/video to AER conversor, are processed using a probabilistic multi event mapper that produces more than one event for each incoming event according to an assigned probability. Kernel convolution size are limited by mapping tables size (on board RAM) and AER bus bandwidth. On reconstruction signed events needs to be simplified (subtracted) to get final convolved image. For that two different methods are proposed.

In this paper, a chip that performs real-time image convolutions with programmable kernels of arbitrary shape is presented. The chip is a first experimental prototype of reduced size to validate the implemented circuits and system level techniques. The convolution processing is based on the address–event-representation (AER) technique, which is a spike-based biologically inspired image and video representation technique that favors communication bandwidth for pixels with more information. As a first test prototype, a pixel array of 16$\,\times\,$16 has been implemented with programmable kernel size of up to 16$\, \times\,$16. The chip has been fabricated in a standard 0.35-${\mu }$m complimentary metal–oxide–semiconductor (CMOS) process. The technique also allows to process larger size images by assembling 2-D arrays of such chips. Pixel operation exploits low-power mixed analog–digital circuit techniques. Because of the low currents involved (down tonanoamperesor evenpicoamperes), an important amount of pixel area is devoted to mismatch calibration. The rest of the chip uses digital circuit techniques, both synchronous and asynchronous. The fabricated chip has been thoroughly tested, both at the pixel level and at the system level. Specific computer interfaces have been developed for generating AER streams from conventional computers and feeding them as inputs to the convolution chip, and for grabbing AER streams coming out of the convolution chip and storing and analyzing them on computers. Extensive experimental results are provided. At the end of this paper, we pro

This paper describes a high-speed USB2.0 address-event representation (AER) interface that allows simultaneous monitoring and sequencing of precisely timed AER data. This low-cost (<$100), two chip, bus powered interface can achieve sustained AER event rates of 5 megaevents per second (Meps). Several boards can be electrically synchronized, allowing simultaneous synchronized capture from multiple devices. It has three parallel AER ports, one for sequencing, one for monitoring and one for passing through the monitored events. This paper also describes the host software infrastructure that makes the board usable for a heterogeneous mixture of AER devices and that allows recording and playback of recorded data.

Address-event-representation (AER) is a communication protocol for transferring asynchronous events between VLSI chips, originally developed for bio-inspired processing systems (for example, image processing). The event information in an AER system is transferred using a high-speed digital parallel bus. This paper presents an experiment using AER for sensing, processing and finally actuating a robot. The AER output of a silicon cochlea is processed by an AER filter implemented on a FPGA to produce rhythmic walking in a humanoid robot (Redbot). We have implemented both the AER rhythm detector and the central pattern generator (CPG) on a Spartan II FPGA which is part of a USB-AER platform developed by some of the authors

The address-event representation (AER) is a neuromorphic communication protocol for transferring asynchronous events between VLSI chips. The event information is transferred using a high speed digital parallel bus. This paper present an experiment based on AER for visual sensing, processing and finally actuating a robot. The AER output of a silicon retina is processed by an AER filter implemented into a FPGA to produce a mimicking behaviour in a humanoid robot (The RoboSapiens V2). We have implemented the visual filter into the Spartan II FPGA of the USB-AER platform and the central pattern generator (CPG) into the Spartan 3 FPGA of the AER-Robot platform, both developed by authors.

Address-event-representation (AER) is a communication protocol for transferring asynchronous events between VLSI chips, originally developed for bio-inspired processing systems (for example, image processing). Such systems may consist of a complicated hierarchical structure with many chips that transmit data among them in real time, while performing some processing (for example, convolutions). The event information is transferred using a high speed digital parallel bus (typically 16 bits and 20ns-40ns per event). This paper presents a testing platform for AER systems that allows analysing a LVDS serial AER link produced by a Spartan 3 FPGA, or by a commercial LVDS transceiver. The interface allows up to 0.728 Gbps (~40Mev/s, 16 bits/ev). The eye diagram ensures that the platform could support 1.2 Gbps.

In this paper we briefly summarize the fundamental properties of spike events processing applied to artificial vision systems. This sensing and processing technology is capable of very high speed throughput, because it does not rely on sensing and processing sequences of frames, and because it allows for complex hierarchically structured cortical-like layers for sophisticated processing. The paper includes a few examples that have demonstrated the potential of this technology for high-speed vision processing, such as a multilayer event processing network of 5 sequential cortical-like layers, and a recognition system capable of discriminating propellers of different shape rotating at 5000 revolutions per second (300000 revolutions per minute).

Address-event-representation (AER) is a communication protocol for transferring asynchronous events between VLSI chips, originally developed for neuro-inspired processing systems (for example, image processing). Such systems may consist of a complicated hierarchical structure with many chips that transmit data among them in real time, while performing some processing (for example, convolutions). The information transmitted is a sequence of spikes coded using high speed digital buses. These multi-layer and multi-chip AER systems perform actually not only image processing, but also audio processing, filtering, learning, locomotion, etc. This paper present an AER interface for controlling an anthropomorphic robotic hand with a neuro-inspired system.

Address-event-representation (AER) is a communications protocol for transferring spikes between bio-inspired chips. Such systems may consist of a hierarchical structure with several chips that transmit spikes among them in real time, while performing some processing. To develop and test AER based systems it is convenient to have a set of instruments that would allow to: generate AER streams, monitor the output produced by neural chips and modify the spike stream produced by an emitting chip to adapt it to the requirements of the receiving elements. In this paper we present a set of tools that implement these functions developed in the CAVIAR EU project

Address-event-representation (AER) is a communication protocol for transferring asynchronous events between VLSI chips, originally developed for bio-inspired processing systems (for example, image processing). Such systems may consist of a complicated hierarchical structure with many chips that transmit data among them in real time, while performing some processing (for example, convolutions). To develop AER based systems for image processing it is very convenient to have available some kind of tool for generating AER streams from on-computer stored images. In this paper we present a hardware method for generating AER streams with Poisson statistics in real time from a sequence of images stored in a computer's memory. We quantify that the events generated follow a Poisson distribution using the Kolmogorov-Smirnov test. We have developed a USB-AER board, based on the Xilinx Spartan II FPGA and the Cygnal 8051 microcontroller, developed by our RTCAR group have been used for the analysis

Address event representation (AER) is a neuromorphic interchip communication protocol that allows for real-time connectivity between huge number neurons located on different chips. By exploiting high speed digital communication circuits (nano-seconds), synaptic neural connections can be time multiplexed (mili-seconds). When building multi-chip muti-layered AER systems it is absolutely necessary to have a computer interface that allows: (a) to read AER interchip traffic; and (b) inject a sequence of events to the AER structure. This paper presents a PCI to AER interface, that dispatches a sequence of events with timing information. It is able to recovery the possible delays introduced by AER bus. It has been implemented in real time hardware using VHDL and tested in a PCI-AER board, developed by authors, that currently capable to send and receive events at a peak rate of 16 Mev/sec, and a typical rate of 10 Mev/sec

A high speed sample image processing application using AER-based components is presented. The setup objective is to distinguish between two propellers of different shape rotating at high speed (around 1000 revolutions/sec) to show event-based systems capabilities in high speed applications. Event-based schemes allow the most relevant information to propagate faster through the system layers. So image processing is sped up because a rough result may be available when only a little part of the input has arrived. This setup is much faster than the conventional frame-based image processing systems because they would need to process more than 10kFrames/s to do the same task proposed here, whereas only few events are required with the event based technique

This paper addresses the problem of converting a conventional video stream based on sequences of frames into the spike event-based representation known as the address-event-representation (AER). In this paper we concentrate on rate-coded AER. The problem is addressed as an algorithmic problem, in which different methods are proposed, implemented and tested through software algorithms. The proposed algorithms are comparatively evaluated according to different criteria. Emphasis is put on the potential of such algorithms for a) doing the frame-based to event-based representation in real time, and b) that the resulting event streams ressemble as much as possible those generated naturally by rate-coded address-event VLSI chips, such as silicon AER retinae. It is found that simple and straightforward algorithms tend to have high potential for real time but produce event distributions that differ considerably from those obtained in AER VLSI chips. On the other hand, sophisticated algorithms that yield better event distributions are not efficient for real time operations. The methods based on linear-feedback-shift-register (LFSR) pseudorandom number generation is a good compromise, which is feasible for real time and yield reasonably well distributed events in time. Our software experiments, on a 1.6-GHz Pentium IV, show that at 50% AER bus load the proposed algorithms require between 0.011 and 1.14 ms per 8 bit-pixel per frame. One of the proposed LFSR methods is implemented in real time hardware using a prototyping board that includes a VirtexE 300 FPGA. The demonstration hardware is capable of transforming frames of 64 x 64 pixels of 8-bit depth at a frame rate of 25 frames per second, producing spike events at a peak rate of 10(7) events per second.

A 5-layer neuromorphic vision processor whose components communicate spike events asychronously using the address-event- representation (AER) is demonstrated. The system includes a retina chip, two convolution chips, a 2D winner-take-all chip, a delay line chip, a learning classifier chip, and a set of PCBs for computer interfacing and address space remappings. The components use a mixture of analog and digital computation and will learn to classify trajectories of a moving object. A complete experimental setup and measurements results are shown.

We present an analytical model for single-slave Bluetooth piconets when the coexistence of multiple interfering devices produces collisions. Closed-form expressions for the channel throughput and the mean packet delay are obtained, including the time spent waiting on the node's queue. The effect of propagation losses and asynchronous piconets are also discussed. The results of the analysis are validated through simulation.

Address-event-representation (AER) is a communication protocol that emulates the nervous system's neurons communication, and that is typically used for transferring images between chips. It was originally developed for bio-inspired and real-time image processing systems. Such systems may consist of a complicated hierarchical structure with many chips that transmit images among them in real time, while performing some processing. In this paper several software methods for generating AER streams from images stored in a computer's memory are presented. A hardware version that works in real-time is also being studied. All of them have been evaluated and compared.