This paper explores the combination of neural network quantization and entropy coding for memory footprint minimization. Edge deployment of quantized models is hampered by the harsh Pareto frontier of the accuracy-to-bitwidth tradeoff, causing dramatic accuracy loss below a certain bitwidth. This accuracy loss can be alleviated thanks to mixed precision quantization, allowing for more flexible bitwidth allocation. However, standard mixed precision benefits remain limited due to the 1-bit frontier, that forces each parameter to be encoded on at least 1 bit of data. This paper introduces an approach that combines mixed precision, zero-point quantization and entropy coding to push the compression boundary of Resnets beyond the 1-bit frontier with an accuracy drop below 1% on the ImageNet benchmark. From an implementation standpoint, a compact decoder architecture features reduced latency, thus allowing for inference-compatible decoding.

Post-training quantization of neural networks consists in quantizing a model without retraining nor hyperparameter search, while being fast and data frugal. In this paper, we propose LatticeQ, a novel post-training weight quantization method designed for deep convolutional neural networks (DC-NNs). Contrary to scalar rounding widely used in state-of-the-art quantization methods, LatticeQ uses a quantizer based on lattices - discrete algebraic structures. LatticeQ exploits the inner correlations between the model parameters to the benefit of minimizing quantization error. We achieve state-of-the-art results in post-training quantization. In particular, we achieve ImageNet classification results close to full precision on Resnet-18/50, with little to no accuracy drop for 4-bit models. Our code is available here, and a more thorough version of the paper here.

In the field of Continual Learning, the objective is to learn several tasks one after the other without access to the data from previous tasks. Several solutions have been proposed to tackle this problem but they usually assume that the user knows which of the tasks to perform at test time on a particular sample, or rely on small samples from previous data and most of them suffer of a substantial drop in accuracy when updated with batches of only one class at a time. In this article, we propose a new method, OvA-INN, which is able to learn one class at a time and without storing any of the previous data. To achieve this, for each class, we train a specific Invertible Neural Network to extract the relevant features to compute the likelihood on this class. At test time, we can predict the class of a sample by identifying the network which predicted the highest likelihood. With this method, we show that we can take advantage of pretrained models by stacking an Invertible Network on top of a feature extractor. This way, we are able to outperform state-of-the-art approaches that rely on features learning for the Continual Learning of MNIST and CIFAR-100 datasets. In our experiments, we reach 72% accuracy on CIFAR-100 after training our model one class at a time.

Image Sensors will get more and more pervasive into their environment. In the context of Automotive and IoT, low cost image sensors, with high quality pixels, will embed more and more smart functions, such as the regular low level image processing but also object recognition, movement detection, light detection, etc. 3D technology is a key enabler technology to integrate into a single device the pixel layer and associated acquisition layer, but also the smart computing features and the required amount of memory to process all the acquired data. More computing and memory within the 3D Smart Image Sensors will bring new features and reduce the overall system power consumption. Advanced 3D technology with ultra-fine pitch vertical interconnect density will pave the way towards new architectures for 3D Smart Image Sensors, allowing local vertical communication between pixels, and the associated computing and memory structures. The presentation will give an overview of recent 3D technologies solutions, such as Hybrid Bonding technology and the Monolithic 3D CoolCube™ technology, with respective 3D interconnect pitch in the order of 1 μm and l00nm. Recent 3D Image Sensors will be presented, showing the capability of 3D technology to implement fine grain pixel acquisition and processing with ultra-high speed image acquisition and tile-based processing. As further perspectives, multi-layer 3D image sensor based on events and spiking will reduce power consumption with new detection and learning processing capabilities.

The increasing need for intelligent sensors in a wide range of everyday objects requires the existence of low power information processing systems which can operate autonomously in their environment. In particular, merging and processing the outputs of different sensors efficiently is a necessary requirement for mobile agents with cognitive abilities. In this work, we present a multi-layer spiking neural network for inference of relations between stimuli patterns in dedicated neuromorphic systems. The system is trained with a new version of the backpropagation algorithm adapted to on-chip learning in neuromorphic hardware: Error gradients are encoded as spike signals which are propagated through symmetric synapses, using the same integrate-and-fire hardware infrastructure as used during forward propagation. We demonstrate the strength of the approach on an arithmetic relation inference task and on visual XOR on the MNIST dataset. Compared to previous, biologically-inspired implementations of networks for learning and inference of relations, our approach is able to achieve better performance with less neurons. Our architecture is the first spiking neural network architecture with on-chip learning capabilities, which is able to perform relational inference on complex visual stimuli. These features make our system interesting for sensor fusion applications and embedded learning in autonomous neuromorphic agents.

Artificial intelligence and especially Machine Learning recently gained a lot of interest from the industry. Indeed, new generation of neural networks built with a large number of successive computing layers enables a large amount of new applications and services implemented from smart sensors to data centers. These Deep Neural Networks (DNN) can interpret signals to recognize objects or situations to drive decision processes. However, their integration into embedded systems remains challenging due to their high computing needs. This paper presents PNeuro, a scalable energy-efficient hardware accelerator for the inference phase of DNN processing chains. Simple programmable processing elements architectured in SIMD clusters perform all the operations needed by DNN (convolutions, pooling, non-linear functions, etc.). An FDSOI 28 nm prototype shows an energy efficiency of 700 GMACS/s/W at 800 MHz. These results open important perspectives regarding the development of smart energy-efficient solutions based on Deep Neural Networks.

We introduce a deep spiking convolutional neural network of integrate-and-fire (IF) neurons, which extracts hierarchical features from a stream of event-based vision data in an unsupervised fashion and online. Our network operates with a simple spike-timing dependent plasticity (STDP) rule, which does not require the definition of an input timescale. We demonstrate how our network is able to learn translational invariant features from the event-based N-MNIST dataset, while preserving dynamic information in the data. We demonstrate how these features can be used to perform online classification. Our network is the first neuromorphic system which can be used to learn complex hierarchical features unsupervised from a continuous stream of address-event-representation (AER) data, operating outside of a database framework and on multiple timescales. Additionally, all the mechanisms we use are simple and generic. This could open the possibility to implement our system on current neuromorphic hardware, to build a real-world fully adaptive event-based vision system.

In this work, we will focus on the role that non-volatile resistive memory technologies (RRAM) can play for modeling key features of biological synapses. We will present an architecture and a reading/programming strategy to emulate both Short and Long Term Plasticity (STP, LTP) rules using non-volatile OxRAM arrays. A visual-pattern extraction application is discussed using spiking neural networks. We demonstrated that Long-Term plasticity allows the neural networks to learn patterns and the Short Term plasticity allows to improve accuracy (reduction of the false positive events generated by white noise in the input data) in presence of significant background noise in the input data.

An innovative approach for decoding of brain signals based on Spiking Neural Networks is presented in this paper. Synapses are implemented by BEOL compatible oxide resistive RAM (OxRAM) devices providing low programming voltages (<;2.5V) and currents (~30μA). Spike-timing-dependent plasticity enables the network for autonomous online spike sorting of measured biological signals. Ultra-low synaptic power consumption in the range of 10nW, recognition rates around 90% and real-time functionality bear high potential for future healthcare applications.

In this work, we will focus on the role that new nonvolatile resistive memory technologies can play in emerging fields of application, such as non-volatile logic circuits or neuromorphic circuits, to save energy and increase performance. Concerning the introduction of non-volatile functionalities at the logic level, we will demonstrate hybrid CMOS logic plus ReRAM (specifically CBRAM and OXRAM) circuits for ultra low power FPGA and fixed-logic IC design, as Non Volatile Flip-Flops. Concerning neuromorphic circuits, we will focus on the emulation of synaptic plasticity effects with resistive memory synapses. We will present large-scale energy efficient neuromorphic systems based on ReRAM as stochastic-binary synapses. Prototype applications such as complex visual- and auditory-pattern extraction will be also discussed using feedforward spiking neural networks.

International audience

This paper aims at presenting how new technologies can overcome classical implementation issues of Neural Networks. Resistive memories such as Phase Change Memories and Conductive-Bridge RAM can be used for obtaining low-area synapses thanks to programmable resistance also called Memristors. Similarly, the high capacitance of Through Silicon Vias can be used to greatly improve analog neurons and reduce their area. The very same devices can also be used for improving connectivity of Neural Networks as demonstrated by an application. Finally, some perspectives are given on the usage of 3D monolithic integration for better exploiting the third dimension and thus obtaining systems closer to the brain.

Spin-transfer torque magnetic memory (STT-MRAM) is currently under intense academic and industrial development, since it features nonvolatility, high write and read speed and high endurance. In this work, we show that when used in an original regime, it can additionally act as a stochastic memristive device, appropriate to implement a “synaptic” function. We introduce basic concepts relating to STT-MRAM cell behavior and its possible use to implement learning-capable synapses. System-level simulations on a problem of car counting highlight the potential of the technology for learning systems. Monte Carlo simulations show its robustness to device variations. These results open the way for unexplored applications of STT-MRAM in robust, low power, cognitive-type systems.

Spiking neural networks are naturally asynchronous and use pulses to carry information. In this paper, we consider implementing such networks on a digital chip. We used an event-based simulator and we started from a previously established simulation, which emulates an analog spiking neural network, that can extract complex and overlapping, temporally correlated features. We modified this simulation to allow an easier integration in an embedded digital implementation. We first show that a four bits synaptic weight resolution is enough to achieve the best performance, although the network remains functional down to a 2 bits weight resolution. Then we show that a linear leak could be implemented to simplify the neurons leakage calculation. Finally, we demonstrate that an order-based STDP with a fixed number of potentiated synapses as low as 200 is efficient for features extraction. A simulation including these modifications, which lighten and increase the efficiency of digital spiking neural network implementation shows that the learning behavior is not affected, with a recognition rate of 98% in a cars trajectories detection application.

In this letter, we present an original demonstration of an associative learning neural network inspired by the famous Pavlov's dogs experiment. A single nanoparticle organic memory field effect transistor (NOMFET) is used to implement each synapse. We show how the physical properties of this dynamic memristive device can be used to perform low-power write operations for the learning and implement short-term association using temporal coding and spike-timing-dependent plasticity-based learning. An electronic circuit was built to validate the proposed learning scheme with packaged devices, with good reproducibility despite the complex synaptic-like dynamic of the NOMFET in pulse regime.

In this paper, we present a novel approach to extract complex and overlapping temporally correlated features directly from spike-based dynamic vision sensors. A spiking neural network capable of performing multilayer unsupervised learning through Spike-Timing-Dependent Plasticity is introduced. It shows exceptional performances at detecting cars passing on a freeway recorded with a dynamic vision sensor, after only 10 minutes of fully unsupervised learning. Our methodology is thoroughly explained and first applied to a simpler example of ball trajectory learning. Two unsupervised learning strategies are investigated for advanced features learning. Robustness of our network to synaptic and neuron variability is assessed and virtual immunity to noise and jitter is demonstrated.

We propose a design methodology to exploit adaptive nanodevices (memristors), virtually immune to their variability. Memristors are used as synapses in a spiking neural network performing unsupervised learning. The memristors learn through an adaptation of spike timing dependent plasticity. Neurons' threshold is adjusted following a homeostasis-type rule. System level simulations on a textbook case show that performance can compare with traditional supervised networks of similar complexity. They also show the system can retain functionality with extreme variations of various memristors' parameters, thanks to the robustness of the scheme, its unsupervised nature, and the power of homeostasis. Additionally the network can adjust to stimuli presented with different coding schemes.

Emerging synapse-like nanoscale devices such as memristive devices and synaptic transistors are of great interest to inspire new circuits or systems and promise adaptability, high density and robustness. We reported recently the Nanoparticle-Organic Memory FET transistor (NOMFET), which exhibits interesting behaviors similar to a biological spiking synapse in neural network. A functional model of NOMFET is presented in this paper, which agrees well with the experimental results. It allows then the reliable conception and simulation of hybrid Nano/CMOS circuits. The model is developed in Verilog-A language and implemented on Cadence Virtuoso platform with Spectre 5.1.41 simulator. An iterative physical model and a number of experimental parameters have been integrated to improve the simulation accuracy.