Pushing the frontiers of time-series information processing in the ever-growing domain of edge devices with stringent resources has been impeded by the systems’ ability to process information and learn locally on the device. Local processing and learning of time-series information typically demand intensive computations and massive storage as the process involves retrieving information and tuning hundreds of parameters back in time. In this work, we developed a memristor-based echo state network accelerator that features efficient temporal data processing and in situ online learning. The proposed design is benchmarked using various datasets involving real-world tasks, such as forecasting the load energy consumption and weather conditions. The experimental results illustrate that the hardware model experiences a marginal degradation in performance as compared to the software counterpart. This is mainly attributed to the limited precision and dynamic range of network parameters when emulated using memristor devices. The proposed system is evaluated for lifespan, robustness, and energy-delay product. It is observed that the system demonstrates reasonable robustness for device failure below 10%, which may occur due to stuck-at faults. Furthermore, 247× reduction in energy consumption is achieved when compared to a custom CMOS digital design implemented at the same technology node.

Neural network models catastrophically forget previously learned information while acquiring new knowledge, requiring a fundamental change in learning models and architectures. These enhancements to architecture structures and training mechanisms lead to an increase in memory and computational resources, making it difficult to deploy models on resource-constrained edge devices. To enhance both memory and computational efficiency, we propose a model compression approach for spiking continual learning models, where the model parameters are quantized with varying precision according to their weight distribution.

Metaplasticity, the activity-dependent modification of synaptic plasticity, is an important technique for mitigating catastrophic forgetting in neural networks. Often, continual learning models with metaplasticity require compute-intensive training. In this research, we propose a probabilistic metaplastic synapse with discrete hidden states that alleviates the computational cost. We implement a digital architecture of the network with on-chip training to achieve further power savings. Results show upto ~ 22% and ~ 21% improvement in mean accuracy for Split-MNIST and sequential MNIST-FMNIST benchmarks respectively, compared to previous metaplasticity models. Simulations of the full digital architecture show ~ 53× lower power consumption per weight update with similar accuracy as gradient-based network counterparts.

Recent studies have shown that metaplastic synapses can retain information longer than simple binary synapses and are beneficial for continual learning. In this paper, we explore the multistate metaplastic synapse characteristics in the context of high retention and reception of information. Inherent behavior of a memristor emulating the multistate synapse is employed to capture the metaplastic behavior. An integrated neural network study for learning and memory retention is performed by integrating the synapse in a 5 × 3 crossbar at the circuit level and 128 × 128 network at the architectural level. An on-device training circuitry ensures the dynamic learning in the network. In the 128 × 128 network, it is observed that the number of input patterns the multistate synapse can classify is ≃ 2.1× that of a simple binary synapse model, at a mean accuracy of ≥ 75%.

Neuromorphic systems that learn and predict from streaming inputs hold significant promise in pervasive edge computing and its applications. In this article, a neuromorphic system that processes spatio-temporal information on the edge is proposed. Algorithmically, the system is based on hierarchical temporal memory that inherently offers online learning, resiliency, and fault tolerance. Architecturally, it is a full custom mixed-signal design with an underlying digital communication scheme and analog computational modules. Therefore, the proposed system features reconfigurability, real-time processing, low power consumption, and low-latency processing. The proposed architecture is benchmarked to predict on real-world streaming data. The network's mean absolute percentage error on the mixed-signal system is 1.129 X lower compared to its baseline algorithm model. This reduction can be attributed to device non-idealities and probabilistic formation of synaptic connections. We demonstrate that the combined effect of Hebbian learning and network sparsity also plays a major role in extending the overall network lifespan. We also illustrate that the system offers 3.46 X reduction in latency and 77.02 X reduction in power consumption when compared to a custom CMOS digital design implemented at the same technology node. By employing specific low power techniques, such as clock gating, we observe 161.37 X reduction in power consumption.

This paper proposes a neuromemristive multi-layer neural network with on-device learning. The proposed system is studied within the context of a feedforward multi-layer random projection network, where the core learning is modeled by a stochastic gradient descent simplified for memristor crossbar integration. Two random projection network topologies are explored for binomial and multinomial datasets. A detailed study on the resiliency of the networks in the presence of device failure is performed. The topology with softmax output layer exhibits stability and better resiliency in performance after experiencing a device failure. It is shown that this topology can regain full performance after experiencing 30% stuck-at-faults, with 2x increase in the hidden layer neurons.

Hierarchical temporal memory (HTM) is a biomimetic sequence memory algorithm that holds promise for invariant representations of spatial and spatio-temporal inputs. This article presents a comprehensive neuromemristive crossbar architecture for the spatial pooler (SP) and the sparse distributed representation classifier, which are fundamental to the algorithm. There are several unique features in the proposed architecture that tightly link with the HTM algorithm. A memristor that is suitable for emulating the HTM synapses is identified and a new Z-window function is proposed. The architecture exploits the concept of synthetic synapses to enable potential synapses in the HTM. The crossbar for the SP avoids dark spots caused by unutilized crossbar regions and supports rapid on-chip training within two clock cycles. This research also leverages plasticity mechanisms such as neurogenesis and homeostatic intrinsic plasticity to strengthen the robustness and performance of the SP. The proposed design is benchmarked for image recognition tasks using Modified National Institute of Standards and Technology (MNIST) and Yale faces datasets, and is evaluated using different metrics including entropy, sparseness, and noise robustness. Detailed power analysis at different stages of the SP operations is performed to demonstrate the suitability for mobile platforms.

In this paper, a memristor spiking neuron and synaptic trace circuits for efficient on device learning are presented. A key feature of these circuits is the use of memristors to emulate the membrane potential of spiking neurons, as opposed to the conventional use of a capacitor. The circuits are designed in IBM 65nm technology node and validated on a small-scale spiking neural network. It was observed that a 3×3 spiking neural network consumes 19.1 μW of power at 100 MHz.

On-device intelligence is gaining significant attention recently as it offers local data processing and low power consumption. In this research, an on-device training circuitry for threshold-current memristors integrated in a crossbar structure is proposed. Furthermore, alternate approaches of mapping the synaptic weights into fully trained and semi-trained crossbars are investigated. In a semi-trained crossbar, a confined subset of memristors are tuned and the remaining subset of memristors are not programmed. This translates to optimal resource utilization and power consumption, compared to a fully programmed crossbar. The semi-trained crossbar architecture is applicable to a broad class of neural networks. System level verification is performed with an extreme learning machine for binomial and multinomial classification. The total power for a single 4 × 4 layer network, when implemented in IBM 65nm node, is estimated to be ≈42.16μW and the area is estimated to be 26.48μm × 22.35μm.

Extreme learning machine is an emerging neural network architecture that offers fast learning and generalization for multiple tasks. In this work, a scalable digital architecture for multi-classifier extreme learning machine (MT-ELM) is proposed. The proposed architecture performs multiple classification tasks without reconfiguring the network. The design is validated with MNIST dataset and it is shown that the proposed model achieves an accuracy of 91.7% for classifying numbers in the MNIST dataset and an accuracy of 90.35% for categorizing number parity. The design is synthesized on a TSMC-65nm technology node and the power dissipation is 13.6 mW for MT-ELM network with 80 hidden neurons and 12 output neurons.

Memristor crossbars support efficient realizations of spiking and non-spiking neural networks designs. In most of these designs off-chip/ex-situ training is used to set/update the state of the memrisitve devices. However, there is a growing need to design an efficient on-chip/in-situ learning for mobile autonomous systems. In this research, we propose an on-chip learning accelerator, known as Ziksa, that is integrated with the memristor crossbars. We demonstrate how regression and back-propagation in multi-level networks can be realized through Ziksa. The proposed accelerator is evaluated on a fabricated TiN-TaOx-TaTiN memristor crossbar. A 3-layer feedforward network was tested using Ziksa for classification. An accuracy of 95.3% was achieved on Wisconsin breast cancer dataset. The proposed learning accelerator can be envisioned as a core building block in a wide-range of cognitive algorithms that rely on on-chip online learning.