This paper presents new kinds of image sensors based on TFS (Time to First Spike) pixels and DVS (Dynamic Vision Sensor) pixels, which take advantage of non-uniform sampling and redundancy suppression to reduce the data throughput. The DVS pixels only detect a luminance variation, while TFS pixels quantized luminance by measuring the required time to cross a threshold. Such image sensors output requests through an Address Event Representation (AER), which helps to reduce the data stream The resulting event bitstream is composed by time, position, polarity, and magnitude information. Such a bitstream offers new possibilities for image processing such as event-by-event object tracking. In particular, we propose some processing to cluster events, filter noise and extract other useful features, such as a velocity estimation.

Adjusting the quantization according to the data or to the model loss seems mandatory to enable a high accuracy in the context of quantized neural networks. This work presents Histogram-Equalized Quantization (HEQ), an adaptive framework for linear and symmetric quantization. HEQ automatically adapts the quantization thresholds using a unique step size optimization. We empirically show that HEQ achieves state-of-the-art performances on CFAR-10. Experiments on the STL-10 dataset even show that HEQ enables a proper training of our proposed logic-gated (OR, MUX) residual networks with a higher accuracy at a lower hardware complexity than previous work.

Even if Application-Specific Integrated Circuits (ASIC) have proven to be a relevant choice for integrating inference at the edge, they are often limited in terms of applicability. In this paper, we demonstrate that an ASIC neural network accelerator dedicated to image processing can be applied to multiple tasks of different levels: image classification and compression, while requiring a very limited hardware. The key component is a reconfigurable, mixed-precision (3b/2b/1b) encoder that takes advantage of proper weight and activation quantizations combined with convolutional layer structural pruning to lower hardware-related constraints (memory and computing). We introduce an automatic adaptation of linear symmetric quantizer scaling factors to perform quantized levels equalization, aiming at stabilizing quinary and ternary weights training. In addition, a proposed layer-shared Bit-Shift Normalization significantly simplifies the implementation of the hardware-expensive Batch Normalization. For a specific configuration in which the encoder design only requires 1Mb, the classification accuracy reaches 87.5% on CIFAR-10. Besides, we also show that this quantized encoder can be used to compress image patch-by-patch while the reconstruction can performed remotely, by a dedicated full-frame decoder. This solution typically enables an end-to-end compression almost without any block artifacts, outperforming patch-based state-of-the-art techniques employing a patch-constant bitrate.

Event-based imagers are bio-inspired sensors presenting intrinsic High Dynamic Range and High Acquisition Speed properties. However, noisy pixels and asynchronous readout result in poor energy-efficiency and excessively large output data rates.In this work, we use Convolutional Spiking Neural Network filters to compensate these drawbacks and reduce output bandwidth by 10x.We designed a neuromorphic core as a distributable block that benefits from 3D integration technology with direct and parallel access to 32x32 pixels, enabling reduced frequency operation. Post-layout simulations depict a peak energy efficiency with 2.83pJ per Synaptic Operation (equivalent to 0.093fJ/event/pix) at the nominal literature input event rate.

Image Sensors will get more and more pervasive into their environment. In the context of Automotive and IoT, low cost image sensors, with high quality pixels, will embed more and more smart functions, such as the regular low level image processing but also object recognition, movement detection, light detection, etc. 3D technology is a key enabler technology to integrate into a single device the pixel layer and associated acquisition layer, but also the smart computing features and the required amount of memory to process all the acquired data. More computing and memory within the 3D Smart Image Sensors will bring new features and reduce the overall system power consumption. Advanced 3D technology with ultra-fine pitch vertical interconnect density will pave the way towards new architectures for 3D Smart Image Sensors, allowing local vertical communication between pixels, and the associated computing and memory structures. The presentation will give an overview of recent 3D technologies solutions, such as Hybrid Bonding technology and the Monolithic 3D CoolCube™ technology, with respective 3D interconnect pitch in the order of 1 μm and l00nm. Recent 3D Image Sensors will be presented, showing the capability of 3D technology to implement fine grain pixel acquisition and processing with ultra-high speed image acquisition and tile-based processing. As further perspectives, multi-layer 3D image sensor based on events and spiking will reduce power consumption with new detection and learning processing capabilities.

This paper presents a new compressive sensing acquisition scheme well adapted for highly constrained hardware implementations. The proposed sensing model being basically designed to meet both theoretical (i.e., Restricted Isometry Property) and hardware requirements (i.e., power consumption, silicon footprint), is highly suitable for image sensors applications addressing both image rendering and embedded decision making tasks. In fact, for a pixels array, the proposed framework consists in applying for each row a random modulation ±1 and a random permutation of the pixels, and then averaging the outputs by column to extract a compressed vector. This model is shown to be relevant as it has the same theoretical performance as a randomly generated sensing scheme as well as a low silicon footprint for physical implementation. Various numerical results and a discussion on possible implementations will be presented to show the robustness and the efficiency of the proposed model.

This work presents a compact VGA (480 × 640) CMOS Image Sensor (CIS) architecture with dedicated end-of-column Compressive Sensing (CS) scheme allowing embedded object recognition. The architecture takes advantage of a low-footprint pseudo-random data mixing circuit and a first order incremental Sigma-Delta (ΣΔ) Analog to Digital Converter (ADC) to extract compressed features. The proposed CIS achieves an object recognition accuracy of ≃ 93% on the Georgia Tech face recognition database (GIT, 10 classes out of 50) thanks to a linear Support Vector Machine (SVM) classifier implemented by an optimized Digital Signal Processing (DSP). We stress that the signal independent dimensionality reduction performed by our dedicated CS scheme (1/480) allows to dramatically reduce memory requirements (≈ 32 kbit) related -in our case- to the ex-situ learned affine function of the linear SVM.

This paper explores hierarchical clustering methods to learn a hierarchical multi-class classifier on compressed measurements in the context of highly constrained hardware (e.g., always-on ultra low power vision systems). In contrast to the popular multi-class classification approaches based on multiple binary classifiers (i.e., one-vs.-all and one-vs.one [1]), a hierarchical classifier requires only O(log2C) binary classifiers in a decision tree. In this work, we investigate three clustering methods used to construct balanced clusters at each node thus reducing the depth of the decision tree in order to lower hardware requirements to its minimum. A binary Support Vector Machine (SVM) [2] classifier is then learned on Compressive Sensing measurements [3] at each node of the hierarchical tree. Our results, based on two object recognition databases (AT&T and COIL-100 databases), show the competitiveness of hierarchical classification in terms of hardware requirements (lower memory and computational complexity) as well as its classification accuracy.

Today, high performance and low power consumption are important requirements for the embedded SoCs. The variation in transistors characteristics is increasing as CMOS transistors are scaled to nanometer sizes. Indeed, the MIPS per Watt ratio are more and more an important requirement for digital systems. This makes the power consumption constraint a relevant design criterion. This paper illustrates a new architecture based on an asynchronous approach able to easily reduce the power consumption without performance degradation on an existing design. The evaluation results demonstrate the effectiveness of the proposed technique. This new technique can be considered as generic for systems based on busses or NoCs. Experimentation has been done on the industrial AXI bus.

Ultra-low voltage is now a well-known solution for energy constrained applications designed using nanometric process technologies. This work is focused on setting up an automated methodology to enable the design of ultra-low voltage digital circuits exclusively using standard EDA tools. To achieve this goal, a 0.35V energy-delay optimized library was developed. This library, fully compliant with standard library design flow and characterization, was verified through the design and fabrication of a BCH decoder circuit, following a standard front-end to back-end flow. At 0.33V, it performs at 600 kHz with a dynamic energy consumption reduced by a factor 14x from nominal 1.1V. Based on this design, experiments, and preliminary silicon results, two additional libraries were developed in order to enhance future ultra-low voltage circuit performance.

The C-element is a fundamental component in asynchronous circuits and quite used in synchronous circuits to mitigate transient faults. This work evaluates the transient-fault effects on the traditional dynamic, conventional, weak feedback, and symmetric C-element's implementations. An evaluation methodology is developed by means of fault-injection simulations at transistor level. Unlike existing methods, the methodology in this work is able to deal with the C-element function's particularities. In addition, C-element cells in different transient-fault robust versions are designed by using techniques based on sizing and transistor insertion. Results in terms of delay, power consumption, area, and fault-transient robustness show the best C-element options for the design of more robust systems.

Today signal processing systems uniformly sample analog signals without taking advantage of their intrinsic properties. For instance, temperature, pressure, electrocardiograms, speech signals significantly vary only during short moments. The digitizing system does not take into account this specificity and furthermore is highly constrained by the Shannon theory which fixes the sampling frequency at least twice the input signal frequency bandwidth. It has been proved that Analog-to-digital Converters (ADCs) using a non equi-repartition in time of samples leads to interesting power savings compared to Nyquist ADCs. A new class of ADCs called A-ADCs (for Asynchronous ADCs) based on level-crossing sampling (which produces non-uniform samples in time) and asynchronous technology has been developed. This article will present a fully non-uniform filtering technique associated to such an ADC which is able to drastically reduce the power consumption.

A methodology to evaluate transient-fault effects on synchronous and asynchronous is presented in this work. It is developed by means of fault-injection simulation campaigns on gate-level circuit implementations. The methodology is able to deal with the particularities of asynchronous circuits. Unlike previous works, it permits to compare the sensitivity of circuits designed by synchronous and asynchronous logics. The resultant metrics allow identifying at high-level abstraction what is the logic that makes the circuit more transient-fault sensitive. As a case study, a crypto-processor in versions synchronous and asynchronous was evaluated.

Ultra-low voltage is now a well known solution for energy constrained applications designed using nanometric process technologies. This work is focused on setting-up an automated methodology to enable the design of ultra-low voltage digital circuits exclusively using standard EDA tools. To achieve this goal, a 0.35V energy-delay optimized library was developed. This library, fully compliant with standard library design flow and characterization, was verified through the design and fabrication of a BCH decoder circuit, following a standard front-end to back-end flow. It performs at 457 kHz, with a total energy consumption of 2.9fJ per cycle.

In this paper we present the experimental evaluation of a built-in-self-test (BIST) principle for the detection of defective pixels of a CMOS imager. The pixel BIST technique aims at an structural test based on electrical stimuli. Simple electrical test measures are considered. Test limits are set in order to minimize pixel false acceptance and false rejection under mismatch deviations. The pixel BIST is next evaluated by considering the fault coverage obtained with catastrophic and single parametric faults. Finally, test metrics obtained by simulation for mismatch deviations are compared with experimental data.

This paper evaluates a new Built-In-Self-Test (BIST) technique for CMOS imagers. The test stimuli are based on applying electrical pulses at the pixel photodiode anode in order to carry out a purely electrical test. The aim of this work is to eliminate some, if not all, optical tests of the pixel matrix to reduce time and cost during production testing at a wafer level. The quality of the BIST technique is evaluated by computing test metrics such as fault coverage for catastrophic and single parametric faults, and pixel fault acceptance and fault rejection under process deviations for two different pixel architectures.

The production test of CMOS image sensors is complicated and expensive as an electrical and an optical test must be executed for the pixel matrix. In this paper we study a built-in-self-test (BIST) technique for the pixels. The technique is based on applying a voltage stimulus at the photosensitive element of the image sensor. The aim of this work is to avoid light stimuli to realise an only electrical test to determine if a pixel is functional or not. This will then reduce test time and test cost. To quantify the quality of this test approach, test metrics such as fault rejection and fault acceptance are estimated. Catastrophic and parametric faults are taken into consideration for the estimation of the test quality