Memristive crossbars based on Non-volatile Memory (NVM) technologies such as RRAM, have recently shown great promise for accelerating Deep Neural Networks (DNNs). They achieve this by performing efficient Matrix-Vector-Multiplications (MVMs) while offering dense on-chip storage and minimal off-chip data movement. However, their analog nature of computing introduces functional errors due to non-ideal RRAM devices, significantly degrading the application accuracy. Further, RRAMs suffer from low endurance and high write costs, hindering on-chip trainability. To alleviate these limitations, we propose HyperX, a hybrid RRAM-SRAM system that leverages the complementary benefits of NVM and CMOS technologies. Our proposed system consists of a fixed RRAM block offering area and energy-efficient MVMs and an SRAM block enabling on-chip training to recover the accuracy drop due to the RRAM non-idealities. The improvements are reported in terms of energy and product of latency and area${\left(ms\,\times \,mm^{2}\right)}$, termed as area-normalized latency. Our experiments on CIFAR datasets using ResNet-20 show up to 2.88 × and 10.1 × improvements in inference energy and area-normalized latency, respectively. In addition, for a transfer learning task from ImageNet to CIFAR datasets using ResNet-18, we observe up to 1.58 × and 4.48 × improvements in energy and area-normalized latency, respectively. These improvements are with respect to an all-SRAM baseline.

Spiking neural networks (SNNs) operating with sparse binary signals (spikes) implemented on event-driven hardware can potentially be more energy -efficient than traditional artificial neural networks (ANNs). However, SNNs perform computations over time, and the neuron activation function does not have a well-defined derivative leading to unique training challenges. In this paper, we discuss the various spike representations and training mechanisms for deep SNN s. Additionally, we review applications that go beyond classification, like gesture recognition, motion estimation, and sequential learning. The unique features of SNNs, such as high activation sparsity and spike-based computations, can be leveraged in hardware implementations for energy-efficient processing. To that effect, we discuss various SNN implementations, both using digital ASICs as well as analog in-memory computing primitives. Finally, we present an outlook on future applications and open research areas for both SNN algorithms and hardware implementations.

Embedded non-volatile memory (eNVM) based crossbars have emerged as energy-efficient building blocks for machine learning accelerators. However, the analog computations in crossbars introduce errors due to several non-idealities. Moreover, since communications between crossbars are usually done in the digital domain, the energy and area costs are dominated by the Analog-to-Digital Converters (ADC). Among the eNVM technologies, Resistive Random-Access-Memory (RRAM) and Phase-Change Memory (PCM) devices suffer from poor endurance, Write variability and conductance drift. Whereas magneto-resistive technologies provide superior endurance, write stability and reliability. To that effect, we propose sparsity-aware device/circuit co-design of robust crossbars using Spin-Orbit-Torque Magnetic Tunnel Junctions (SOT-MTJs). Note, standard MTJs have low $\mathrm{R}_{\mathrm{O}\mathrm{F}\mathrm{F}}/\mathrm{R}_{\mathrm{O}\mathrm{N}}$ and low $\mathrm{R}_{\mathrm{O}\mathrm{N}}$, making them unsuitable for crossbars. In this work, we first demonstrate SOT-MTJs as crossbar elements With high $\mathrm{R}_{\mathrm{O}\mathrm{N}}$ and high $\mathrm{R}_{\mathrm{O}\mathrm{F}\mathrm{F}}/\mathrm{R}_{\mathrm{O}\mathrm{N}}$ by allowing the read-path to have thicker tunneling-barrier, leaving the write path undisturbed. Second, through extensive simulations, we quantitatively assess the impact of various device-circuit parameters such as $\mathrm{R}_{\mathrm{O}\mathrm{N}}, \mathrm{R}_{\mathrm{O}\mathrm{F}\mathrm{F}}/\mathrm{R}_{\mathrm{O}\mathrm{N}}$ ratio, crossbar size, along With input and weight sparsity, on both circuit and application level accuracy and energy consumption. We evaluate system accuracy for Resnet-20 inference on CIFAR-10 dataset and show that leveraging sparsity allows reduced ADC precision, Without degrading accuracy. Our results show that an SOT-MTJ $(\mathrm{R}_{\mathrm{O}\mathrm{N}}=200\mathrm{k}\Omega$ and $\mathrm{R}_{\mathrm{O}\mathrm{F}\mathrm{F}}/\mathrm{R}_{\mathrm{O}\mathrm{N}}=7)$ crossbar array of size 32×32 could achieve near-software accuracy. The 64×64 and 128×128 crossbars show an accuracy degradation of 2% and 9.8%, respectively, from the software accuracy and an energy improvement of upto 3.8× and 6.3× compared to a 32×32 array with 4bit-ADC.

This overview article describes Magnetoresistive Random Access Memory (MRAM) from a circuits and systems perspective. We discuss various tradeoffs and design challenges of MRAM in three broad application areas: 1) embedded non-volatile memory (eNVMs), 2) crossbar-based analog in-memory computing, and 3) stochastic computing. Certain MRAM characteristics, such as high retention, high endurance and fast read and write operations, make them ideal for replacing the standard CMOS memories for last-level cache applications with future scaling. However, various tradeoffs in power, performance and area pose conflicting requirements on MRAM design. We explore these challenges and various circuit techniques that have been developed to mitigate them. Further, we present various requirements of memristive crossbar arrays for accelerating matrix-vector-multiplication (MVM) operations in light of MRAM devices, and highlight various challenges, design considerations, and applicability of MRAM as crossbar arrays. Finally, we will elaborate on how inherent stochasticity of MRAM devices can be leveraged for implementing energy-efficient true random number generators (TRNGs) and stochastic units for performing certain tasks, such as developing fast solvers for combinatorial optimization, and stochastic neural networks.

The saturating scaling trends of CMOS technology have fuelled the exploration of emerging non-volatile memory (NVM) technologies as a promising alternative for accelerating data intensive Machine Learning (ML) workloads. To that effect, researchers have explored special-purpose accelerators based on NVM crossbar primitives. NVM crossbars have high storage density and can efficiently per-form massively parallel in-situ Matrix Vector Multiplication (MVM) operations, the key computation in ML workloads, helping over-come the memory bottleneck faced by von Neumann architectures. Despite the promises, analog computing nature of NVM crossbars can lead to functional errors due to device and circuit non-idealities such as parasitic resistances and device non-linearities. Moreover, NVM crossbars need high cost peripheral circuitry to be integrated in large scale systems. Hence, there is a need to study different levels of the design stack to realize the potential of this technology.In this paper, we present an overview of in-memory computing in NVM crossbars for ML workloads. We discuss the basic anatomy of NVM crossbars and highlight the challenges faced at the primitive level. Next, we present how the high storage density of NVM crossbars can enable spatially distributed architectures. Further, we present various modeling and evaluation tools which can effectively help us study the functionality as well as performance of NVM crossbar systems. Finally, we provide an outlook on the future research directions in this field.

Memory-Augmented Neural Networks (MANNs) have been shown to outperform Recurrent Neural Networks (RNNs) in terms of long-term dependencies. Since MANNs are equipped with an external memory, they can store and retrieve more data through longer periods of time. A MANN generally consists of a network controller and an external memory. Unlike conventional memory having read/write operations to specific addresses, a differentiable memory has soft read and write operations involving all the data stored in the memory. Such soft read and write operations present new computational challenges for hardware implementation of MANNs. In this work, we present a novel in-memory computing primitive to accelerate the differentiable memory operations of MANNs in SRAMs. We propose a 9T SRAM macro capable of performing both Hamming similarity and dot products (crucial for soft read/write and addressing mechanisms in MANNs). Regarding Hamming similarity, we operate the 9T cell in analog Content-Addressable Memory (CAM) mode by applying the key at the bitlines (RBLs/RBLBs) in each column, and reading out the analog output at the sourceline (SL). To perform dot product operation, the input data is applied at the wordlines, and the current passing through RBLs represents the dot product between the input data and the stored bits. The proposed SRAM array performs computations that reliably match the operations required for a differentiable memory, thereby leading to energy-efficient on-chip acceleration of MANNs. Compared to standard GPU systems, the proposed scheme achieves 43x and 85x performance and energy improvements respectively, for computing the differentiable memory operations.

Traditional computing systems based on the von Neumann architecture are fundamentally bottlenecked by data transfers between processors and memory. The emergence of data-intensive workloads, such as machine learning (ML), creates an urgent need to address this bottleneck by designing computing platforms that utilize the principle of colocated memory and processing units. Such an approach, known as “in-memory computing,” can potentially eliminate data movement costs by computing inside the memory array itself. Crossbars based on resistive nonvolatile memory (NVM) devices have shown immense promise in serving as the building blocks of in-memory computing systems for ML workloads. This is because their high density can lead to higher on-chip storage capacity, while they can also perform massively parallel, in situ matrix-vector multiplication (MVM) operations, thereby accelerating the main computational kernel of ML workloads. However, resistive crossbar-based analog computing is inherently approximate due to the device- and circuit-level nonidealities. Furthermore, the area and energy costs of peripheral circuits for conversions between the analog and digital domains can greatly diminish the intrinsic efficiency of crossbar-based MVM computation. We present a comprehensive overview of the emerging paradigm of computing using NVM crossbars for accelerating ML workloads. We describe the design principles of resistive crossbars, including the devices and associated circuits that constitute them. We discuss intrinsic approximations arising from the device and circuit characteristics and study their functional impact on the MVM operation. Next, we present an overview of spatial architectures that exploit the high storage density of NVM crossbars. Furthermore, we elaborate on software frameworks that effectively capture device-circuit-architecture characteristics to evaluate the performance of large-scale deep neural networks (DNNs) using resistive crossbar-based hardware. Finally, we discuss open challenges and future research directions that need to be explored in order to realize the vision of resistive crossbars as the building blocks of future computing platforms.

In this era of nanoscale technologies, the inherent characteristics of some nonvolatile devices, such as resistive random access memory (ReRAM), phase-change material (PCM), and spintronics, can emulate stochastic functionalities. Traditionally, these devices have been engineered to suppress the stochastic switching behavior as it poses reliability concerns for memory storage and logic applications. However, leveraging stochasticity in such devices led to a renewed interest in hardware-software codesign of stochastic algorithms since the CMOS-based implementations of stochastic algorithms involve cumbersome circuitry to generate “stochastic bits.” In this article, we consider two classes of problems: deep neural networks (DNNs) and combinatorial optimization. The rapidly growing demands of artificial intelligence (AI) have sparked an interest in energy-efficient implementations of large DNNs, with binary representations of synaptic weights and neuronal activities. Stochasticity plays an important role in leveraging the benefits of these binary representations, leading to model compression and optimization during training. In combinatorial optimization, such as graph coloring or traveling salesman problems, stochastic algorithms, such as the Ising computing model, have been shown to be effective. These problems require exhaustive computational procedures, and the Ising model uses a natural annealing agent to achieve near-optimal solutions in a reasonable timescale, without getting stuck in “local minima.” In this article, we present a broad review of stochastic computing utilizing the stochastic switching characteristics of devices based on nanoscale nonvolatile technologies. We show how to codesign of the devices and algorithms that can enable optimal solutions for both combinatorial problems and binary neural networks for local learning and inference. Directly mapping the nonvolatile device characteristics to the stochastic algorithms without the need for storing the bits in a separate memory leads to efficient use of hardware.

No abstract available.

Nano-scale neuro-mimetic devices have recently gained wide research interest in the quest to enable brain-like energy-efficiency with cognitive computing abilities. Traditionally, neuromorphic devices have exploited deterministic nano-scale devices for emulating the intrinsic neuronal and synaptic behavior. However, of particular interest are stochastic neuromorphic devices owing to - 1) availability of nano-scale devices that are inherently stochastic based on intrinsic device physics 2) various neuroscience experiments have demonstrated that cortical neurons are stochastic in nature. In this paper, we focus on spin orbit torque based Magnetic Tunnel Junction (SOT-MTJ) that exhibit stochastic sigmoid behavior with respect to the switching process. We first discuss the modeling framework that was used to study the effect of dimensional variations in SOT-MTJs and the resulting changes in the stochastic sigmoid behavior. Our model is based on the well-known stochastic-Landau-Lifshitz-Gilbert-Slonczewski equation under mono-domain approximation. Subsequently, we abstract the sigmoid characteristic of the device into a behavioral model and study the effect of variations in sigmoid characteristics on a deep binary network. Our results show that the variations in the sigmoidal neuron behavior results in a minimal loss in accuracy (for CIFAR 10 dataset). Additionally, the degradation in accuracy monotonically increases with increase in induced variations. This highlights the robustness of stochastic neural networks based on SOT-MTJs in presence of dimensional variations.

From the little we know about the human brain, the inherent cognitive mechanism is very different from the de facto state-of-the-art computing platforms. The human brain uses distributed, yet integrated memory and computation units, unlike the physically separate memory and computation cores in typical von Neumann architectures. Despite huge success of artificial intelligence, hardware systems running these algorithms consume orders of magnitude higher energy compared to the human brain, mainly due to heavy data movements between the memory unit and the computation cores. Spiking neural networks (SNNs) built using bio-plausible neuron and synaptic models have emerged as the power efficient choice for designing cognitive applications. These algorithms involve several lookup-table (LUT) based function evaluations such as high-order polynomials and transcendental functions for solving complex neuro-synaptic models, that typically require additional storage and thus, bigger memories. To that effect, we propose `SPARE'-an in-memory, distributed processing architecture built on ROM-embedded RAM technology, for accelerating SNNs. ROM-embedded RAMs allow storage of LUTs (for neuro-synaptic models), embedded within a typical memory array, without additional area overhead. Our proposed architecture consists of a 2-D array of Processing Elements (PEs), wherein each PE has its own ROM-embedded RAM structure and executes part of the SNN computation. Since most of the computations (including multiple math-table evaluations) are done locally within each PE, unnecessary data transfers are restricted, thereby alleviating the problems arising due to physically separate remote memory unit and the computation core. SPARE thus leverages both, the hardware benefits of distributed, in-memory processing, and also the algorithmic benefits of SNNs. We evaluate SPARE for two different ROM-Embedded RAM structures-CMOS based ROM-Embedded SRAMs (R-SRAMs) and STT-MRAM based ROM-Embedded MRAMs (R-MRAMs). Moreover, we analyze trade-offs in terms of energy, area and performance, for using the two technologies on a range of image classification benchmarks. Furthermore, we leverage the additional storage density to implement complex neuro-synaptic functionalities. This enhances the utility of the proposed architecture by provisioning implementation of any neuron/synaptic behavior as necessitated by the application. Our results show up-to~1.75×,~1.95× and~1.95× improvement in energy, iso-storage area, and iso-area performance, respectively, by using neural network accelerators built on ROM-embedded RAM primitives.

Large-scale digital computing almost exclusively relies on the von Neumann architecture, which comprises separate units for storage and computations. The energy-expensive transfer of data from the memory units to the computing cores results in the well-known von Neumann bottleneck. Various approaches aimed toward bypassing the von Neumann bottleneck are being extensively explored in the literature. These include in-memory computing based on CMOS and beyond CMOS technologies, wherein by making modifications to the memory array, vector computations can be carried out as close to the memory units as possible. Interestingly, in-memory techniques based on CMOS technology are of special importance due to the ubiquitous presence of field-effect transistors and the resultant ease of large-scale manufacturing and commercialization. On the other hand, perhaps the most important computation required for applications such as machine learning, etc., comprises the dot-product operation. Emerging nonvolatile memristive technologies have been shown to be very efficient in computing analog dot products in an in situ fashion. The memristive analog computation of the dot product results in much faster operation as opposed to digital vector in-memory bitwise Boolean computations. However, challenges with respect to large-scale manufacturing coupled with the limited endurance of memristors have hindered rapid commercialization of memristive-based computing solutions. In this paper, we show that the standard 8 transistor (8T) digital SRAM array can be configured as an analoglike in-memory multibit dot-product engine (DPE). By applying appropriate analog voltages to the read ports of the 8T SRAM array and sensing the output current, an approximate analog-digital DPE can be implemented. We present two different configurations for enabling multibit dot-product computations in the 8T SRAM cell array, without modifying the standard bit-cell structure. We also demonstrate the robustness of the present proposal in presence of nonidealities such as the effect of line resistances and transistor threshold voltage variations. Since our proposal preserves the standard 8T-SRAM array structure, it can be used as a storage element with standard read-write instructions and also as an on-demand analoglike dot-product accelerator.