Analog non-volatile memory (NVM)-based accelerators for Deep Neural Networks (DNNs) can achieve high-throughput and energy-efficient multiply-accumulate (MAC) operations by taking advantage of massively parallelized analog compute, implemented with Ohm's law and Kirchhoff's current law on arrays of resistive memory devices. Competitive end-to-end DNN accuracies can be obtained, provided that weights are accurately programmed onto NVM devices and MAC operations are sufficiently linear. In this paper, we report architectural and circuit advances for such Analog NVM-based accelerators. We describe a highly heterogeneous and programmable accelerator architecture for DNN inference that combines analog NVM memory-array “Tiles” for weight-stationary, energy-efficient MAC operations, together with heterogeneous special-function Compute-Cores for auxiliary digital computation. Massively parallel vectors of neuron-activation data are exchanged over short distances using a dense and efficient circuit-switched 2D mesh, enabling a wide range of DNN workloads, including CNNs, LSTMs, and Transformers. We also show a 14-nm inference chip consisting of multiple$\mathbf{512}\times \mathbf{512}$arrays of Phase Change Memory (PCM) devices which implements multiple DNN benchmarks using such a circuit-switched 2D mesh.

We introduce a highly heterogeneous and programmable compute-in-memory (CIM) accelerator architecture for deep neural network (DNN) inference. This architecture combines spatially distributed CIM memory array “tiles” for weight-stationary, energy-efficient multiply–accumulate (MAC) operations, together with heterogeneous special-function compute cores for auxiliary digital computation. Massively parallel vectors of neuron activation data are exchanged over short distances using a dense and efficient circuit-switched 2-D mesh, offering full end-to-end support for a wide range of DNN workloads, including CNNs, long-short-term-memory (LSTM), and transformers. We discuss the design of the “analog fabric”—the 2-D grid of tiles and compute cores interconnected by the 2-D mesh—and address the efficiency in both mapping of DNNs onto the hardware and in pipelining of various DNN workloads across a range of batch sizes. We show, for the first time, system-level assessments using projected component parameters for a realistic “analog AI” system, based on dense crossbar arrays of low-power nonvolatile analog memory elements, while incorporating a single common analog fabric design that can scale to large networks by introducing data transport between multiple analog AI chips. Our performance estimates for several networks, including large LSTM and bidirectional encoder representations from transformers (BERT), show highly competitive throughput while offering$40\times $–$140\times $higher energy efficiency than NVIDIA A100—thus illustrating the strong promise of analog AI and the proposed architecture for DNN inference applications.

Analog non-volatile memory (NVM)-based accelerators for deep neural networks perform high-throughput and energy-efficient multiply-accumulate (MAC) operations (e.g., high TeraOPS/W) by taking advantage of massively parallelized analog MAC operations, implemented with Ohm’s law and Kirchhoff’s current law on array-matrices of resistive devices. While the wide-integer and floating-point operations offered by conventional digital CMOS computing are much more suitable than analog computing for conventional applications that require high accuracy and true reproducibility, deep neural networks can still provide competitive end-to-end results even with modest (e.g., 4-bit) precision in synaptic operations. In this paper, we describe a 14-nm inference chip, comprising multiple 512$\times$ 512 arrays of Phase Change Memory (PCM) devices, which can deliver software-equivalent inference accuracy for MNIST handwritten-digit recognition and recurrent LSTM benchmarks, by using compensation techniques to finesse analog-memory challenges such as conductance drift and noise. We also project accuracy for Natural Language Processing (NLP) tasks performed with a state-of-art large Transformer-based model, BERT, when mapped onto an extended version of this same fundamental chip architecture.

By performing parallelized multiply-accumulate operations in the analog domain at the location of weight data, crossbar-array "tiles" of analog non-volatile memory (NVM) devices can potentially accelerate the forward-inference of deep neural networks. To be successful, such systems will need to achieve two related but challenging goals. First is the achievement of high neural network classification accuracies, indistinguishable from those achieved with conventional approaches, despite the difficulties of programming NVM devices accurately in the presence of significant device-to-device variability. Towards this first goal, we describe row-wise Phase-Change Memory (PCM) programming schemes for rapid yet accurate weight- programming. The second goal is highly energy-efficient forward- inference of multi-layer neural networks, requiring efficiency in both the massively-parallel analog-AI operations performed at each tile, as well as efficiency in how the resulting neuron-excitation data vectors get conveyed from tile to tile. Towards this second goal, micro-architectural design ideas including source-follower-based readout, array segmentation, and transmit-by- duration are described.

Cognitive computing describes “systems that learn at scale, reason with purpose, and interact with humans naturally” [1]. In this paper, we review our work towards enabling “next generation” cognitive computing using neuromorphic computational schemes that could potentially outperform present-day CPUs and GPUs. Here we use large arrays of Resistive Non-Volatile Memories (NVM) with device conductance serving as synaptic weight. We focus on training and classification using fully-connected networks based on the backpropagation algorithm, and show that our approach could offer power and speed advantages over conventional Von-Neumann processors. We also propose some circuit approximations that improve network parallelism without significantly degrading classification accuracy. Finally, we explore the requirements for a system implementation of on-chip learning.

This paper proposes a method to reduce the standby leakage current of MOSFET by controlling the voltage of the source node. The method allows the use of a low threshold device for high performance speed and low power dissipation in both active and standby periods. This method can be easily applied for conventional ASIC library circuits because no additional processes, circuits, or devices except slight modification for the body contact cell are required.