Hardware spiking neural networks that co-integrate analog silicon neurons with memristive synaptic crossbar arrays are promising candidates to achieve low-power processing of event-based data. Learning patterns with real-world timescales, often exceeding the millisecond range, is however difficult with fully analog systems. In this work, we propose to overcome this challenge by introducing mixed-signal strategies to implement hardware-friendly learning rules derived from Spike Timing-Dependent Plasticity. By system-level simulation means, we illustrate the potential of this concept for both unsupervised and reward-modulated learning. In particular, we investigate how such learning rules and their tuning impact the overall system recognition rate depending on different characteristics of event-based inputs or synapses. This work provides useful insights for building versatile energy-efficient event-based neuromorphic systems with online learning capability.

As the end of Moore's law seems to be imminent, emerging technologies that enable high performance neuromorphic hardware systems are attracting increasing attention. A very promising approach is to utilize memristors, programmable nonvolatile memory devices, as synaptic weights in neuromorphic circuits. One of the challenges for memristive hardware with integrated learning capabilities is prohibitively larger number of write cycles that might be required during learning process. In this work we propose a memristive neuromorphic hardware implementation for reinforcement learning based on temporal difference actor-critic algorithm. As a case study, we consider a task of balancing an inverted pendulum, a classical problem in both reinforcement learning and control theory. We introduce training techniques that significantly reduce the number of weight updates and are suitable for efficient in-situ learning hardware implementations. We believe that this study shows the promise of using memristor-based hardware neural networks for handling complex tasks through in-situ reinforcement learning.

Spin transfer torque magnetic random access memory (STT-MRAM) is a major breakthrough for embedded and standalone memory applications. Its basic cell, the magnetic tunnel junction, can also be used in a low-energy stochastic regime and implement a "synaptic" function. It can then be the basic element for learning-capable neuromorphic chips that do not separate logic and memory and exploit the magnetic tunnel junctions with an optimum energy efficiency. Implementing this vision, however, raises challenges at the circuit level. Proper addressing of the junctions can perturb their synaptic function. In this work, we investigate several architectures for a system based on stochastic synapses, and compare them in terms of reliability and energy efficiency. These results show the high potential of this technology, and pinpoint some main design challenges and tradeoff.

Memristive nanodevices offer new frontiers for computing systems that unite arithmetic and memory operations on-chip. Here, we explore the integration of electrochemical metallization cell (ECM) nanodevices with tunable filamentary switching in nanoscale learning systems. Such devices offer a natural transition between short-term plasticity (STP) and long-term plasticity (LTP). In this work, we show that this property can be exploited to efficiently solve noisy classification tasks. A single crossbar learning scheme is first introduced and evaluated. Perfect classification is possible only for simple input patterns, within critical timing parameters, and when device variability is weak. To overcome these limitations, a dual-crossbar learning system partly inspired by the extreme learning machine (ELM) approach is then introduced. This approach outperforms a conventional ELM-inspired system when the first layer is imprinted before training and testing, and especially so when variability in device timing evolution is considered: variability is therefore transformed from an issue to a feature. In attempting to classify the MNIST database under the same conditions, conventional ELM obtains 84% classification, the imprinted, uniform device system obtains 88% classification, and the imprinted, variable device system reaches 92% classification. We discuss benefits and drawbacks of both systems in terms of energy, complexity, area imprint, and speed. All these results highlight that tuning and exploiting intrinsic device timing parameters may be of central interest to future bio-inspired approximate computing systems.

International audience

Spin-transfer torque magnetic memory (STT-MRAM) is currently under intense academic and industrial development, since it features nonvolatility, high write and read speed and high endurance. In this work, we show that when used in an original regime, it can additionally act as a stochastic memristive device, appropriate to implement a “synaptic” function. We introduce basic concepts relating to STT-MRAM cell behavior and its possible use to implement learning-capable synapses. System-level simulations on a problem of car counting highlight the potential of the technology for learning systems. Monte Carlo simulations show its robustness to device variations. These results open the way for unexplored applications of STT-MRAM in robust, low power, cognitive-type systems.