Event-based vision sensors achieve up to three orders of magnitude better speed vs. power consumption trade off in high-speed control of UAVs compared to conventional image sensors. Event-based cameras produce a sparse stream of events that can be processed more efficiently and with a lower latency than images, enabling ultra-fast vision-driven control. Here, we explore how an event-based vision algorithm can be implemented as a spiking neuronal network on a neuromorphic chip and used in a drone controller. We show how seamless integration of event-based perception on chip leads to even faster control rates and lower latency. In addition, we demonstrate how online adaptation of the SNN controller can be realised using on-chip learning. Our spiking neuronal network on chip is the first example of a neuromorphic vision-based controller on chip solving a high-speed UAV control task. The excellent scalability of processing in neuromorphic hardware opens the possibility to solve more challenging visual tasks in the future and integrate visual perception in fast control loops.

Neuromorphic hardware offers computing platforms for the efficient implementation of spiking neural networks (SNNs) that can be used for robot control. Here, we present such an SNN on a neuromorphic chip that solves a number of tasks related to simultaneous localization and mapping (SLAM): forming a map of an unknown environment and, at the same time, estimating the robot's pose. In particular, we present an SNN mechanism to detect and estimate errors when the robot revisits a known landmark and updates both the map and the path integration speed to reduce the error. The whole system is fully realized in a neuromorphic device, showing the feasibility of a purely SNN-based SLAM, which could be efficiently implemented in a small form-factor neuromorphic chip.

Spiking Neuronal Networks (SNNs) realized in neuromorphic hardware lead to low-power and low-latency neuronal computing architectures. Neuromorphic computing systems are most efficient when all of perception, decision making, and motor control are seamlessly integrated into a single neuronal architecture that can be realized on the neuromorphic hardware. Many neuronal network architectures address the perception tasks, while work on neuronal motor controllers is scarce. Here, we present an improved implementation of a neuromorphic PID controller. The controller was realized on Intel's neuromorphic research chip Loihi and its performance tested on a drone, constrained to rotate on a single axis. The SNN controller is built using neuronal populations, in which a single spike carries information about sensed and control signals. Neuronal arrays perform computation on such sparse representations to calculate the proportional, derivative, and integral terms. The SNN PID controller is compared to a PID controller, implemented in software, and achieves a comparable performance, paving the way to a fully neuromorphic system in which perception, planning, and control are realized in an on-chip SNN.

We present a spiking neural network (SNN) for visual pattern recognition with on-chip learning on neuromorphic hardware. We show how this network can learn simple visual patterns composed of horizontal and vertical bars sensed by a Dynamic Vision Sensor, using a local spike-based plasticity rule. During recognition, the network classifies the pattern's identity while at the same time estimating its location and scale. We build on previous work that used learning with neuromorphic hardware in the loop and demonstrate that the proposed network can properly operate with on-chip learning, demonstrating a complete neuromorphic pattern learning and recognition setup. Our results show that the network is robust against noise on the input (no accuracy drop when adding 130% noise) and against up to 20% noise in the neuron parameters.

In this paper, we investigate the use of ultra low-power, mixed signal analog/digital neuromorphic hardware for implementation of biologically inspired neuronal path integration and map formation for a mobile robot. We perform spiking network simulations of the developed architecture, interfaced to a simulated robotic vehicle. We then port the neuronal map formation architecture on two connected neuromorphic devices, one of which features on-board plasticity, and demonstrate the feasibility of a neuromorphic realization of simultaneous localization and mapping (SLAM).