Oscillatory neural network (ONN) is an emerging neuromorphic architecture composed of oscillators that implement neurons and are coupled by synapses. ONNs exhibit rich dynamics and associative properties, which can be used to solve problems in the analog domain according to the paradigm let physics compute. For example, compact oscillators made of VO2 material are good candidates for building low-power ONN architectures dedicated to AI applications at the edge, like pattern recognition. However, little is known about the ONN scalability and its performance when implemented in hardware. Before deploying ONN, it is necessary to assess its computation time, energy consumption, performance, and accuracy for a given application. Here, we consider a VO2-oscillator as an ONN building block and perform circuit-level simulations to evaluate the ONN performances at the architecture level. Notably, we investigate how the ONN computation time, energy, and memory capacity scale with the number of oscillators. It appears that the ONN energy grows linearly when scaling up the network, making it suitable for large-scale integration at the edge. Furthermore, we investigate the design knobs for minimizing the ONN energy. Assisted by technology computer-aided design (TCAD) simulations, we report on scaling down the dimensions of VO2 devices in crossbar (CB) geometry to decrease the oscillator voltage and energy. We benchmark ONN versus state-of-the-art architectures and observe that the ONN paradigm is a competitive energy-efficient solution for scaled VO2 devices oscillating above 100 MHz. Finally, we present how ONN can efficiently detect edges in images captured on low-power edge devices and compare the results with Sobel and Canny edge detectors.

The increasing volume of smart edge devices, like smart cameras, and the growing amount of data to treat incited the development of light edge Artificial Intelligence (AI) solutions with neuromorphic computing. Oscillatory Neural Network (ONN) is a promising neuromorphic computing approach which uses networks of coupled oscillators, and their inherent parallel synchronization to compute. Also, ONN phase computing allows to limit voltage amplitude and reduce power consumption. Low-power, fast, and parallel computation properties make ONN attractive for edge AI. In state-of-the-art, ONN is built with a fully-connected architecture, with coupling defined from unsupervised learning to perform auto-associative memory tasks, like with Hopfield Networks. However, to allow ONN to solve beyond associative memory applications, there is a need to explore further ONN architectures. In this work, we propose a novel architecture of cascaded analog fully-connected ONNs interconnected with an analog feedforward majority gate layer. In particular, we show this architecture can solve image edge detection task using two fully-connected ONN layers. This is, to our best knowledge, a first analog-based solution to cascade two fully-connected ONNs.

This work proposes a digital implementation of an Oscillatory Neural Network (ONN) in a Field-Programmable Gate Array (FPGA), demonstrating excellent associative memory capabilities. This work goes beyond previous implementations by enabling on-chip learning directly in the FPGA. More specifically, we implement on-chip Hebbian learning, and we compare three different design strategies. The first strategy takes advantage of a System-on-Chip (SoC) composed of a Processing System (PS) and Programmable Logic resources (PL) to integrate Hebbian learning in PS. The two other strategies implement the Hebbian learning directly in PL. We compare the three different design strategies on a digit recognition task in terms of accuracy, utilization, execution time, and maximum frequency. We show that implementing Hebbian learning in PL gives more advantages in terms of resource utilization and latency than implementing Hebbian in PS with several orders of magnitude because the weight matrix computation is performed in hardware. Moreover, we develop an application interface to demonstrate the pattern learning and recognition capabilities of our digital ONN implementation.

Advanced computing systems have long been enablers for breakthroughs in Machine Learning (ML) algorithms either through sheer computational power or form-factor miniaturization. However, as ML algorithms become more complex and the size of datasets increase, existing computing platforms are no longer sufficient to bridge the gap between algorithmic innovation and hardware design. With the rising needs of advanced algorithms for large-scale data analysis and data-driven discovery, and significant growth in emerging applications from the edge to the cloud, we need energy-efficient, low-cost, high- performance, and reliable computing systems targeted for these applications. This paper presents the latest developments in oscillatory neural networks, optical computing, and memristive processing-in-memory (PIM) to address the various challenges in designing efficient computing systems specifically targeting ML applications.

Abstract This paper investigates how to solve image classification with Hopfield neural networks (HNNs) and oscillatory neural networks (ONNs). This is a first attempt to apply ONNs for image classification. State-of-the-art image classification networks are multi-layer models trained with supervised gradient back-propagation, which provide high-fidelity results but require high energy consumption and computational resources to be implemented. On the contrary, HNN and ONN networks are single-layer, requiring less computational resources, however, they necessitate some adaptation as they are not directly applicable for image classification. ONN is a novel brain-inspired computing paradigm that performs low-power computation and is attractive for edge artificial intelligence applications, such as image classification. In this paper, we perform image classification with HNN and ONN by exploiting their auto-associative memory (AAM) properties. We evaluate precision of HNN and ONN trained with state-of-the-art unsupervised learning algorithms. Additionally, we adapt the supervised equilibrium propagation (EP) algorithm to single-layer AAM architectures, proposing the AAM-EP. We test and validate HNN and ONN classification on images of handwritten digits using a simplified MNIST set. We find that using unsupervised learning, HNN reaches 65.2%, and ONN 59.1% precision. Moreover, we show that AAM-EP can increase HNN and ONN precision up to 67.04% for HNN and 62.6% for ONN. While intrinsically HNN and ONN are not meant for classification tasks, to the best of our knowledge, these are the best-reported precisions of HNN and ONN performing classification of images of handwritten digits.

International audience

Brain-inspired computing employs devices and architectures that emulate biological functions for more adaptive and energy-efficient systems. Oscillatory neural networks (ONNs) are an alternative approach in emulating biological functions of the human brain and are suitable for solving large and complex associative problems. In this work, we investigate the dynamics of coupled oscillators to implement such ONNs. By harnessing the complex dynamics of coupled oscillatory systems, we forge a novel computation model-information is encoded in the phase of oscillations. Coupled interconnected oscillators can exhibit various behaviors due to the strength of the coupling. In this article, we present a novel method based on subharmonic injection locking (SHIL) for controlling the oscillatory states of coupled oscillators that allow them to lock in frequency with distinct phase differences. Circuit-level simulation results indicate SHIL effectiveness and its applicability to large-scale oscillatory networks for pattern recognition.