Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i.e. RNNs that are fed by a static input x and settle to a steady state. Training convergent RNNs consists in adjusting the weights until the steady state of output neurons coincides with a target y. Convergent RNNs can also be trained with the more conventional Backpropagation Through Time (BPTT) algorithm. In its original formulation EP was described in the case of real-time neuronal dynamics, which is computationally costly. In this work, we introduce a discrete-time version of EP with simplified equations and with reduced simulation time, bringing EP closer to practical machine learning tasks. We first prove theoretically, as well as numerically that the neural and weight updates of EP, computed by forward-time dynamics, are step-by-step equal to the ones obtained by BPTT, with gradients computed backward in time. The equality is strict when the transition function of the dynamics derives from a primitive function and the steady state is maintained long enough. We then show for more standard discrete-time neural network dynamics that the same property is approximately respected and we subsequently demonstrate training with EP with equivalent performance to BPTT. In particular, we define the first convolutional architecture trained with EP achieving ∼ 1% test error on MNIST, which is the lowest error reported with EP. These results can guide the development of deep neural networks trained with EP.

Many emerging alternative models of computation require massive numbers of random bits, but their generation at low energy is currently a challenge. The superparamagnetic tunnel junction, a spintronic device based on the same technology as spin torque magnetoresistive random access memory has recently been proposed as a solution, as this device naturally switches between two easy to measure resistance states, due only to thermal noise. Reading the state of the junction naturally provides random bits, without the need of write operations. In this work, we evaluate a circuit solution for reading the state of superparamagnetic tunnel junction. We see that the circuit may induce a small read disturb effect for scaled superparamagnetic tunnel junctions, but this effect is naturally corrected in the whitening process needed to ensure the quality of the generated random bits. These results suggest that superparamagnetic tunnel junctions could generate truly random bits at 20 fJ/bit, including overheads, orders of magnitudes below CMOS-based solutions.

Coupled oscillator-based networks are an attractive approach for implementing hardware neural networks based on emerging nanotechnologies. However, the readout of the state of a coupled oscillator network is a difficult challenge in hardware implementations, as it necessitates complex signal processing to evaluate the degree of synchronization between oscillators, possibly more complicated than the coupled oscillator network itself. In this work, we focus on a coupled oscillator network particularly adapted to emerging technologies, and evaluate two schemes for reading synchronization patterns that can be readily implemented with basic CMOS circuits. Through simulation of a simple generic coupled oscillator network, we compare the operation of these readout techniques with a previously proposed full statistics evaluation scheme. Our approaches provide results nearly identical to the mathematical method, but also show better resilience to moderate noise, which is a major concern for hardware implementations. These results open the door to widespread realization of hardware coupled oscillator-based neural systems.

Bioinspired hardware holds the promise of low-energy, intelligent, and highly adaptable computing systems. Applications span from automatic classification for big data management, through unmanned vehicle control, to control for biomedical prosthesis. However, one of the major challenges of fabricating bioinspired hardware is building ultra-high-density networks out of complex processing units interlinked by tunable connections. Nanometer-scale devices exploiting spin electronics (or spintronics) can be a key technology in this context. In particular, magnetic tunnel junctions (MTJs) are well suited for this purpose because of their multiple tunable functionalities. One such functionality, non-volatile memory, can provide massive embedded memory in unconventional circuits, thus escaping the von-Neumann bottleneck arising when memory and processors are located separately. Other features of spintronic devices that could be beneficial for bioinspired computing include tunable fast nonlinear dynamics, controlled stochasticity, and the ability of single devices to change functions in different operating conditions. Large networks of interacting spintronic nanodevices can have their interactions tuned to induce complex dynamics such as synchronization, chaos, soliton diffusion, phase transitions, criticality, and convergence to multiple metastable states. A number of groups have recently proposed bioinspired architectures that include one or several types of spintronic nanodevices. In this paper, we show how spintronics can be used for bioinspired computing. We review the different approaches that have been proposed, the recent advances in this direction, and the challenges toward fully integrated spintronics complementary metal-oxide-semiconductor (CMOS) bioinspired hardware.

Spintronic oscillators are nanodevices that are serious candidates for CMOS integration due to their compactness and easy frequency tunability. Among them vortex-based oscillators appear as one of the most promising technology because of their lower power supply and higher quality factors. To assess their potential in circuits and systems, compact models describing their behavior are necessary. In this work, we propose an implementation of a spintronic nano-oscillator (STNO) model for integrated circuit (IC) architectures design. The modeled device is a vortex-based magnetic oscillator demonstrating self-sustained magnetization oscillations under current bias, inducing alternating voltage across the device. This model describes the coupled electrical and magnetic behavior of the device, taking into account phase and amplitude noises associated with thermal fluctuations. Compatibility with commercial CMOS design kits is demonstrated, and an implementation in a CMOS circuit is proposed for AC signal generation. These results will allow to develop and evaluate innovative hybrid STNO/CMOS systems and their potential to efficiently complement existing full-CMOS technologies.

Since memristor came out in 2008, neuromorphic designers investigated the possibility of using memristors as plastic synapses due to their intrinsic properties of plasticity and weight storage. In this paper we will present a silicon neuron compatible with memristive synapses in order to build analog neural network. This neuron mainly includes current conveyor (CCII) for driving memristor as excitatory or inhibitory synapses and spike generator whose waveform is dedicated to synaptic plasticity algorithm based on Spike Timing Dependent Plasticity (STDP). This silicon neuron has been fabricated, characterized and finally connected with a ferroelectric memristor to validate the synaptic weight updating principle.