The NimbleAI Horizon Europe project leverages key principles of energy-efficient visual sensing and processing in biological eyes and brains, and harnesses the latest advances in$\mathbf{33D}$stacked silicon integration, to create an integral sensing-processing neuromorphic architecture that efficiently and accurately runs computer vision algorithms in area-constrained endpoint chips. The rationale behind the NimbleAI architecture is: sense data only with high information value and discard data as soon as they are found not to be useful for the application (in a given context). The NimbleAI sensing-processing architecture is to be specialized after-deployment by tunning system-level trade-offs for each particular computer vision algorithm and deployment environment. The objectives of NimbleAI are: (1)$\mathbf{100x}$performance per mW gains compared to state-of-the-practice solutions (i.e., CPU/GPUs processing frame-based video); (2)$\mathbf{50x}$processing latency reduction compared to CPU/GPUs; (3) energy consumption in the order of tens of mWs; and (4) silicon area of approx. 50 mm2.

In this paper, we present SPLEAT, a SPiking Low-power Event-based ArchiTecture for the hardware deployment of Spiking Neural Networks (SNN). SPLEAT is designed as a configurable architecture that allows integrating several hardware modules representing different neural layers and, therefore, gives the ability to deploy a wide range of deep convolutional SNN topologies. Thanks to its event-based structure, SPLEAT takes advantage of the asynchronous spiking behavior to realize prediction efficiently. SPLEAT has been synthesized on a Cyclone V Field-Programmable Gate Array (FPGA) embedded on OPSSAT, a 3U nano-satellite launched on December 18, 2019 by the European Space Agency (ESA). SPLEAT has been used to perform in-orbit binary cloud classification on images provided by a 50 meters resolution sensor, which is still operational to this date. A second goal of the in-orbit experiment was to confront bio-inspired AI with classical deep learning. Consequently, in addition to SNNs executed using SPLEAT, standard Convolutional Neural Networks (CNNs) have been deployed on the same hardware target and evaluated on the same satellite images. The comparison between these neural network accelerators reveal a notable gain in terms of resources occupation for SPLEAT, reducing them by a factor of 6.62 for equivalent classification performances and an FPGA power consumption reduced by a factor of 5.91. To our best knowledge, this in-orbit experiment constitutes a world premiere in the fields of bio-inspired neural networks and aerospace.

Spiking neural networks are expected to bring high resources, power, and energy efficiency to machine learning hardware implementations. In this regard, they could facilitate the integration of Artificial Intelligence in highly constrained embedded systems, such as image classification in drones or satellites. If their logic resource efficiency is widely accepted in the literature, their energy efficiency still remains debated. In this article, a novel high-level metric is used to characterize the expected energy efficiency gain when using Spiking Neural Networks (SNN) instead of Formal Neural Networks (FNN) for hardware implementation: Synaptic Activity Ratio (SAR). This metric is applied to a selection of classification tasks including images and 1D signals. Moreover, a high-level estimator for logic resources, power usage, execution time, and energy is introduced for neural network hardware implementations on FPGA, based on four existing accelerator architectures covering both sequential and parallel implementation paradigms for both spiking and formal coding domains. This estimator is used to evaluate the reliability of the Synaptic Activity Ratio metric to characterize spiking neural network energy efficiency gain on the proposed dataset benchmark. This study led to the conclusion that spiking domain offers significant power and energy savings in sequential implementations. This study also shows that synaptic activity is a critical factor that must be taken into account when addressing low-energy systems.

Recent literature considers that Spiking Neural Networks are now a serious alternative to Formal Neural Networks for embedded artificial intelligence. The changes in the information coding and the elementary neural computation make them more efficient than FNNs in terms of power consumption and chip surface occupation. However, these results are often based on simple neural network topologies with basic data-sets. In this paper, we study the behavior of Spiking Convolutional Neural Networks when applied to two different classification tasks. To do so, we analyze the spiking activity on both MNIST and GTSRB data-sets using different rate-based and temporal coding schemes. Notably, the Spike Select method is confronted to First Spike and Jittered Periodic methods in terms of prediction accuracy and spiking activity. Finally, we conclude about spike generation within spiking CNNs for embedded hardware classification.

Inspired from the brain, neuromorphic computing would be the right alternative to traditional Von-Neumann architecture computing that knows its end of growth as predicted by Moore's law. In this paper, we explore bio-inspired neural networks as an AI-accelerator for embedded systems. To do so, we first map neural networks from formal to spiking domain, then choose the information coding method resulting in better performances. Afterwards, we present the design of two different hardware architectures: time-multiplexed and fully-parallel. Finally, we compare their performances and their hardware cost to select at the end the adequate architecture and conclude about spike-based neural networks as a potential solution for embedded artificial intelligence applications.

Artificial neural networks are experiencing today an unprecedented interest thanks to two main changes: the explosion of open data that is necessary for their training, and the increasing computing power of today's computers that makes the training part possible in a reasonable time. The recent results of deep neural networks on image classification has given neural networks the leading role in machine learning algorithms and artificial intelligence research. However, most applications such as smart devices or autonomous vehicles require an embedded implementation of neural networks. Their implementation in CPU/GPU remains too expensive, mostly in energy consumption, due to the non-adaptation of the hardware to the computation model, which becomes a limit to their use. It is therefore necessary to design neuromorphic architectures, i.e. hardware accelerators that fit to the parallel and distributed computation paradigm of neural networks for reducing their hardware cost implementation. We mainly focus on the optimization of energy consumption to enable integration in embedded systems. For this purpose, we implement two models of artificial neural networks coming from two different scientific domains: the multi- layer perceptron derived from machine learning and the spiking neural network inspired from neuroscience. We compare the performances of both approaches in terms of accuracy and hardware cost to find out the most attractive architecture for the design of embedded artificial intelligence.