The continued scaling of CMOS technology amplifies the challenges of delivering power efficiently to many-core processors. Dynamic voltage and frequency scaling (DVFS) offers substantial energy savings, but its effectiveness depends on the voltage settling time, efficiency, and integration cost of the underlying regulators. This paper presents a system-level evaluation of monolithic 3D (M3D) per-core DC–DC converters for fine-grained DVFS in many-core CPUs. Using workload traces extracted from gem5 and an energy modeling framework that accounts for regulator efficiency, power delivery network (PDN) parasitics, transition overheads, and area, we compare five representative configurations across 8-, 16-, and 32-core systems. Results show that off-chip converters with global DVFS achieve only ~24% average energy reduction, on-chip per-core DVFS improves this metric to ~41%, and M3D per-core DVFS delivers the highest savings at ~43% while also minimizing die area by up to 20% relative to on-chip converters and 38% relative to off-chip solutions. These findings underscore the importance of localized regulation and vertical integration for designing scalable, energy-efficient processors in future generations.

The Last Level Cache (LLC) is the processor’s critical bridge between on-chip and off-chip memory levels - optimized for high density, high bandwidth, and low operation energy. To date, high-density (HD) SRAM has been the conventional device of choice; however, with the slowing of transistor scaling, as reflected in the industry’s almost identical HD SRAM cell size from 5 nm to 3 nm, alternative solutions such as 3D stacking with advanced packaging (i.e., hybrid bonding) are pursued (as demonstrated in AMD’s V-cache). Escalating data demands necessitate ultra-large on-chip caches to decrease costly off-chip memory movement, pushing the exploration of device technology towards monolithic 3D (M3D) integration, where transistors can be stacked in the back-end-of-line (BEOL) at the interconnect level. M3D integration requires fabrication techniques compatible with a low thermal budget (seconds) when used in a gain-cell configuration. This paper examines device, circuit, and system-level tradeoffs made when optimizing BEOL-compatible AOS-based 2-transistor gain cells (2T-GC) for LLC. A cache early-exploration tool, NS-Cache, is developed to model caches in advanced 7 & 3 nm nodes and is integrated with the Gem5 simulator to systematically benchmark the impact of the newfound density/performance when compared to HD-SRAM, MRAM, and 1T1C eDRAM alternatives for LLC.

Approximate nearest neighbor search (ANNS) plays an indispensable role in a wide variety of applications, including recommendation systems, information retrieval, and semantic search. Among the cutting-edge ANNS algorithms, graph-based approaches provide superior accuracy and scalability on massive datasets. However, the best-performing graph-based ANNS solutions incur tens of hundreds of memory footprints as well as costly distance computation, thus hindering their efficient deployment at scale. The 3D NAND flash is emerging as a promising device for data-intensive applications due to its high density and nonvolatility. In this work, we present the near-storage processing (NSP)-based ANNS solution Proxima to accelerate graph-based ANNS with algorithm-hardware co-design in 3D NAND flash. Proxima significantly reduces the complexity of graph search by leveraging the distance approximation and early termination. On top of the algorithmic enhancement, we implement the Proxima search algorithm in 3D NAND flash using the heterogeneous integration technique. To maximize 3D NAND’s bandwidth utilization, we present a customized dataflow and optimized data allocation scheme. Our evaluation results show that, compared to graph ANNS on CPU and GPU, Proxima achieves a magnitude improvement in throughput or energy efficiency. Proxima yields 7× to 13× speedup over existing ASIC designs. Furthermore, Proxima achieves a good balance between accuracy, efficiency, and storage density compared to previous NSP-based accelerators.

Large language models (LLMs), composed of Transformer decoders, have demonstrated unparalleled proficiency in understanding and generating human language. However, efficient LLM inference on resource-constraint embedded devices remains a challenge because of the sheer model size and memory-intensive operations that arise from feedforward network (FFN) and multi-head attention (MHA) layers. Existing accelerations offload LLM inference to heterogeneous computing systems comprising expensive memory and processing units. However, recent studies show that most hardware resources are not used because LLM exhibits significant sparsity during inference. The sparsity of LLMs provides a good opportunity to perform memory-efficient inference. In this work, we propose SLIM, an algorithm and hardware co-design optimized for sparse LLM serving on the edge. SLIM exploits LLM’s sparsity by only fetching activated neurons to significantly reduce data movement. To this end, the efficient inference algorithm based on adaptive thresholding is proposed to support runtime configurable sparsity at the cost of negligible accuracy loss. Then, we present the SLIM heterogeneous hardware architecture that combines the best of both near-storage processing (NSP) and processing-in-memory (PIM). SLIM stores FFN weights in high-density 3D NAND and computes FFN layers in NSP units, alleviating high memory requirements caused by FFN weights. The memory-intensive MHA with low arithmetic density is processed in the PIM module. By leveraging the inherent sparsity observed in LLM operations and integrating NSP with PIM techniques within SSDs, SLIM significantly reduces memory footprint, data movement, and energy consumption. Meanwhile, we present the software support for integrating design into existing SSD system. Our comprehensive analysis and system-level optimization demonstrate the effectiveness of our sparsity-tailored accelerator, offering 13-18× throughput improvements over SSD-GPU system and 9-10× better energy efficiency over DRAM-GPU system while maintaining low latency.

Dynamic 3D Gaussian splatting (3DGS) extends static 3DGS to render dynamic scenes, enabling AR/VR applications with moving objects. However, implementing dynamic 3DGS on edge devices faces challenges: (1) Loading all Gaussian parameters from DRAM for frustum culling incurs high energy costs. (2) Increased parameters for dynamic scenes elevate sorting latency and energy consumption. (3) Limited on-chip buffer capacity with higher parameters reduces buffer reuse, causing frequent DRAM access. (4) Dynamic 3DGS operations are not readily compatible with digital compute-in-memory (DCIM). These challenges hinder real-time performance and power efficiency on edge devices, leading to reduced battery life or requiring bulky batteries. To tackle these challenges, we propose algorithm-hardware co-design techniques. At the algorithmic level, we introduce three optimizations: (1) DRAM-access reduction frustum culling to lower DRAM access overhead, (2) Adaptive tile grouping to enhance on-chip buffer reuse, and (3) Adaptive interval initialization Bucket-Bitonic sort to reduce sorting latency. At the hardware level, we present a DCIM-friendly computation flow that is evaluated using the measured data from a 16 nm DCIM prototype chip. Our experimental results on Large-Scale Real-World Static/Dynamic Datasets demonstrate the ability to achieve high frame rate real-time rendering exceeding 200 frames per second (FPS) with minimal power consumption—merely 0.28 W for static Large-Scale Real-World scenes and 0.63 W for dynamic Large-Scale Real-World scenes. This work successfully addresses the significant challenges of implementing static/dynamic 3DGS technology on resource-constrained edge devices.

Recent developments have introduced Kolmogorov– Arnold networks (KANs), an innovative architectural paradigm capable of replicating conventional deep neural network (DNN) capabilities while utilizing significantly reduced parameter counts through the employment of parameterized B-spline functions incorporating trainable coefficients. Nevertheless, the B-spline functional components inherent to KAN architectures introduce distinct hardware acceleration complexities. While B-spline function evaluation can be accomplished through lookup table (LUT) implementations that directly encode functional mappings, thus minimizing computational overhead, such approaches continue to demand considerable circuit infrastructure, including LUTs, multiplexers, decoders, and associated components. This work presents an algorithm-hardware co-design approach for KAN acceleration. At the algorithmic level, techniques include alignment–symmetry and PowerGap KAN hardware-aware quantization, KAN sparsity-aware mapping strategy, and circuit-level techniques include N:1 time modulation dynamic voltage input generator with analog-compute-in-memory (ACIM) circuits. Furthermore, this work conducts comprehensive evaluations on large-scale KAN networks to validate the proposed methodologies. Nonideality factors, including partial sum deviations arising from process variations, have been evaluated with the statistics measured from the TSMC 22-nm RRAM-ACIM prototype chips. Utilizing optimally determined KAN hyperparameters in conjunction with circuit optimizations implemented and evaluated at the 22-nm technology node, despite the model sizes for large-scale tasks in this work increasing by 435 K$\times $to 756 K$\times $compared to tiny-scale tasks in previous work, the area overhead increases by only 26 K$\times $to 40 K$\times $, with power consumption rising by merely$48\times $to$93\times $, while accuracy degradation remains minimal at 0.11%–0.22%, thereby demonstrating the scaling potential of our proposed architecture.

Recently, a novel model named Kolmogorov-Arnold Networks (KAN) has been proposed with the potential to achieve the functionality of traditional deep neural networks (DNNs) using orders of magnitude fewer parameters by parameterized B-spline functions with trainable coefficients. However, the B-spline functions in KAN present new challenges for hardware acceleration. Evaluating the B-spline functions can be performed by using lookup tables (LUTs) to directly map the B-spline functions, thereby reducing computational resource requirements. However, this method still requires substantial circuit resources (LUTs, MUXs, decoders, etc.). For the first time, this paper employs an algorithm-hardware co-design methodology to accelerate KAN. The proposed algorithm-level techniques include Alignment-Symmetry and PowerGap KAN hardware aware quantization, KAN sparsity aware mapping strategy, and circuit-level techniques include N:1 Time Modulation Dynamic Voltage input generator with analog-CIM (ACIM) circuits. The impact of non-ideal effects, such as partial sum errors caused by the process variations, has been evaluated with the statistics measured from the TSMC 22nm RRAM-ACIM prototype chips. With the best searched hyperparameters of KAN and the optimized circuits implemented in 22 nm node, we can reduce hardware area by 41.78x, energy by 77.97x with 3.03% accuracy boost compared to the traditional DNN hardware.

This work presents a novel monolithic 3D (M3D) FPGA architecture that leverages stackable back-end-of-line (BEOL) transistors to implement configuration memory and pass gates, significantly improving area, latency, and power efficiency. By integrating n-type (W -doped In2O3) and p-type (SnO) amorphous oxide semiconductor (AOS) transistors in the BEOL, Si SRAM configuration bits are substituted with a less leaky equivalent that can be programmed at logic-compatible voltages. BEOL-compatible AOS transistors are currently under extensive research and development in the device community, with investment by leading foundries, from which reported data is used to develop robust physics-based models in TCAD that enable circuit design. The use of AOS pass gates reduces the overhead of reconfigurable circuits by mapping FPGA switch block (SB) and connection block (CB) matrices above configurable logic blocks (CLBs), thereby increasing the proximity of logic elements and reducing latency. By interfacing with the latest Verilog-to-Routing (VTR) suite, an AOS-based M3D FPGA implemented in 7 nm technology is demonstrated with $3.4 \times$ lower area-time squared product ($\mathbf{A T}^{2}$), 27% lower critical path latency, and 26% lower reconfigurable routing block power on benchmarks including hyperdimensional computing and large language models (LLMs).

We present the first security and disturbance study of monolithic 3D (M3D) embedded DRAM (eDRAM) with 2T gain cell using oxide-channel transistors. We explore the Rowhammer/Rowpress vulnerabilities on amorphous indium tungsten oxide (IWO) transistors for eDRAM with standalone 2D integration and memory-on-memory M3D integration. In addition, We examine M3D-specific electrical disturbances from memory-on-logic M3D integration. We evaluate IWO eDRAM's susceptibility to these vulnerabilities/disturbances and discuss the potential impact on M3D integration. We examine physical design and architecture strategies for M3D integration of IWO eDRAM. We provide systematic recommendations to inform security strategies for M3D integration and security of IWO eDRAM. Our results show that limiting the minimum vertical interlayer distance to 300 nm reduces vertical disturbances in memory-on-memory M3D integration. In addition, for memory-on-logic M3D integration, we observed that IWO eDRAM's read bitline is sensitive to crosstalk from high-speed switching logic circuits. In conjunction, we show that IWO eDRAM standalone 2D integration is 30× more resilient to Rowhammer than current state-of-the-art memory because the IWO transistor's$I_{ON}/I_{OFF}$ratio is roughly three orders of magnitude greater than standard memory access transistors.

The rapid expansion of mass spectrometry (MS) data, now exceeding hundreds of terabytes, poses significant challenges for efficient, large-scale library search — a critical component for drug discovery. Traditional processors struggle to handle this data volume efficiently, making in-storage computing (ISP) a promising alternative. This work introduces an ISP architecture leveraging a 3D Ferroelectric NAND (FeNAND) structure, providing significantly higher density, faster speeds, and lower voltage requirements compared to traditional NAND flash. Despite its superior density, the NAND structure has not been widely utilized in ISP applications due to limited throughput associated with row-by-row reads from serially connected cells. To overcome these limitations, we integrate hyperdimensional computing (HDC), a brain-inspired paradigm that enables highly parallel processing with simple operations and strong error tolerance. By combining HDC with the proposed dual-bound approximate matching (D-BAM) distance metric, tailored to the FeNAND structure, we parallelize vector computations to enable efficient MS spectral library search, achieving 43× speedup and 21× higher energy efficiency over state-of-the-art 3D NAND methods, while maintaining comparable accuracy.

Combinatorial optimization problems (COPs) have a wide range of applications. The Ising model-based annealer is gaining attention for its efficiency and speed in finding approximate solutions. However, building an Ising machine that is area- and energy-efficient, scalable, and with low compute latency in CMOS is challenging. In this paper, we present a digital compute-in-memory (DCIM) Ising annealer that uses ferroelectric capacitor (FeCap)-based nvSRAM to solve COPs like the Traveling Salesman Problem (TSP). By using weak recall operations, our design eliminates the need to reload weights, significantly reducing energy consumption and speeding up processing compared to other approaches. Simulations using a 16nm PDK demonstrate that our nvSRAM-based DCIM array maintains accuracy while reducing latency by up to 55.0% and energy by 49.6% compared to prior work implemented with conventional SRAM DCIM array. Algorithm validation further shows that the random noise introduced by weak recall can be effectively utilized in the annealing process.

As Large Language Models (LLMs) continue to evolve, Mixture of Experts (MoE) architecture has emerged as a prevailing design for achieving state-of-the-art performance across a wide range of tasks.MoE models use sparse gating to activate only a handful of expert sub-networks per input, achieving billion-parameter capacity with inference costs akin to much smaller models.However, such models often pose challenges for hardware deployment due to the massive data volume introduced by the MoE layers.To address the challenges of serving MoE models, we propose Stratum, a system-hardware co-design approach that combines the novel memory technology Monolithic 3D-Stackable DRAM (Mono3D DRAM), near-memory processing (NMP), and GPU acceleration.The logic and Mono3D DRAM dies are connected through hybrid bonding, whereas the Mono3D DRAM stack and GPU are interconnected via silicon interposer.Mono3D DRAM offers higher internal bandwidth than HBM thanks to the dense vertical interconnect pitch enabled by its monolithic structure, which supports implementations of higher-performance near-memory processing.Furthermore, we tackle the latency differences introduced by aggressive vertical scaling of Mono3D DRAM along the 𝑧-dimension by constructing internal memory tiers and assigning data across layers based on * Equal contribution

The increasing demand for energy-efficient computing has created the need for advanced power management solutions. Backside power delivery network (BSPDN) has been introduced in the industry for 2-nm node with passive wires. In this work, we propose adding active components (power transistors) to the backside of silicon in a back-end-of-line (BEOL)-compatible fabrication process. The goal is to enable 12–0.7-V voltage downconversion at the backside of silicon (near the point of load, i.e., the frontside logic compute die) to minimize the IR drop and improve overall system-level conversion efficiency. This work leverages a hybrid monolithic 3-D (M3D)dc-dc converter architecture combining switched-capacitor (SC) and synchronous buck converter topologies with BEOL-compatible active and passive devices. The design employs amorphous tungsten-doped indium oxide (IWO) transistors, which offer high breakdown voltage and tunable threshold voltages, supporting both enhancement and depletion modes for efficient switching. With the experimentally calibrated compact models, the simulated hybrid converter design achieves 12–0.7-V conversion with a peak efficiency of 95.6% at a power density of 330 mW/mm2, demonstrating the feasibility of M3D SC dc-dc converters for next-generation power management in high-performance edge devices.

Digital compute-in-memory (DCIM) has emerged as a promising solution to address scalability and accuracy challenges in analog compute-in-memory (ACIM) for next-generation AI hardware acceleration. In this work, we present a comprehensive device-to-system codesign process for the two proposed 3-D DCIM architectures at the projected 5 angstrom (A5) complementary FET (CFET) technology node: 1) 3-D DCIM based on 8T DCIM bit cell and 2) lookup-table (LUT)-based 3-D DCIM. A novel A5 CFET-based 8T DCIM bit cell (6T SRAM +2T AND gate) is proposed to improve total footprint and latency over the conventional 10T DCIM bit cell, and its functionality is verified through technology computer-aided design (TCAD) simulation. For macro- and system-level evaluation of the proposed 3-D DCIM architectures, an extended NeuroSim V1.4 framework is developed, the first compute-in-memory (CIM) benchmark framework enabling CIM simulation at the A5 CFET technology node. We demonstrate that the proposed 3-D DCIM with 8T DCIM bit cell at the A5 CFET technology node can achieve$8.2\times $improvement in figure of merit (FOM) (=TOPS/W$\times $TOPS/mm2) over the state-of-the-art 3-nm FinFET-based DCIM design. The LUT-based 3-D DCIM design is additionally proposed to achieve further power consumption reduction from the 8T DCIM bit-cell-based 3-D DCIM. LUT-based 3-D DCIM achieves a 44% reduction in energy consumption compared to the conventional 10T DCIM bit-cell-based 3-D DCIM. Our findings suggest the significant implications for technology scaling below 1 nm in high-performance DCIM design.

Using non-volatile “capacitive” crossbar arrays for compute-in-memory (CIM) offers higher energy and area efficiency compared to “resistive” crossbar arrays. However, the impact of device-to-device (D2D) variation and temporal noise on the system-level performance has not been explored yet. In this work, we provide an end-to-end methodology that incorporates experimentally measured D2D variation into the design space exploration from capacitive weight cell design, CIM array with peripheral circuits, to the inference accuracy of SwinV2-T vision transformer and ResNet-50 on the ImageNet dataset. Our framework further assesses the system’s power, performance, and area (PPA) by considering cell design, circuit structure, and model selection. We explore the design space using an early stopping algorithm to produce optimal designs while meeting strict inference accuracy requirements. Overall findings suggest that the capacitive CIM system is robust against D2D variation and noise, outperforming its resistive counterpart by $6.95 \times$ and $14.1 \times$ for the optimal design in the figure of merit (TOPS/W $\times {\mathrm {TOPS}}/\mathrm{mm}^{2}$) for ResNet-50 and SwinV2-T respectively.

Combinatorial optimization problems (COP) are NP-hard and intractable to solve using conventional computing. The Ising model-based annealer has gained increasing attention recently due to its efficiency and speed in finding approximate solutions. However, Ising solvers for travelling salesman problems (TSP) usually suffer from a scalability issue due to quadratically increasing number of spins. In this paper, we propose a digital computing-in-memory (CIM) based clustered annealer to solve tens of thousands of city-scale TSP with only a few mega-byte (MB) of static random access memory (SRAM), using hierarchical clustering to solve input sparsity and digital CIM flexibility to solve weight sparsity. The intrinsic process variations between SRAM devices are utilized to generate the noisy bit errors during pseudo-read under reduced supply voltage, realizing the annealing process. The design space of cluster size and programmability is explored to understand the trade-offs of solution quality and hardware cost, for TSP scale ranging from 3080 to 85900 cities. The proposed design speeds up the convergence by >109× with <25% solution quality overhead compared with the CPU baseline. The comparison with state-of-the-art scalable annealers shows a >1013× improvement on functionally normalized area and power.

The rising computational demands of artificial intelligence (AI) models are driving increased data transfers from off-chip DRAM, resulting in increased energy consumption and latency. With the conventional Von-Neumann architecture nearing its limits, Near-Memory-Compute (NMC) and In-Memory-Compute (IMC) have emerged as potential alternatives, aiming to minimize memory access by computing directly within memory. The choice between IMC and NMC is contingent on the specific needs of an application, especially when balancing throughput and accuracy. While conventional SRAM is not optimized for extensive parallel computation due to its large footprint, 2T gain-cell (GC) embedded DRAM (eDRAM) presents a favorable alternative. However, its integration with IMC and NMC poses challenges in terms of chip size and data retention. We propose that monolithic three-dimensional (M3D) 3T transposable GC eDRAM with tungsten-doped indium oxide channel (IWO) back-end-of-line (BEOL) transistors to efficiently mitigate these challenges. The benchmark results show that 3T IWO GC eDRAM has 60 % area reduction compared to 10T Si SRAM. Furthermore, the data retention of the proposed design is ~104times longer than that of 3T Si GC eDRAM. The comprehensive study from technology to architecture highlights the potential of the 3T IWO GC eDRAM as a robust candidate for L4 cache and parallel computing.

MOTIVATION: Genomic distance estimation is a critical workload since exact computation for whole-genome similarity metrics such as Average Nucleotide Identity (ANI) incurs prohibitive runtime overhead. Genome sketching is a fast and memory-efficient solution to estimate ANI similarity by distilling representative k-mers from the original sequences. In this work, we present HyperGen that improves accuracy, runtime performance, and memory efficiency for large-scale ANI estimation. Unlike existing genome sketching algorithms that convert large genome files into discrete k-mer hashes, HyperGen leverages the emerging hyperdimensional computing (HDC) to encode genomes into quasi-orthogonal vectors (Hypervector, HV) in high-dimensional space. HV is compact and can preserve more information, allowing for accurate ANI estimation while reducing required sketch sizes. In particular, the HV sketch representation in HyperGen allows efficient ANI estimation using vector multiplication, which naturally benefits from highly optimized general matrix multiply (GEMM) routines. As a result, HyperGen enables the efficient sketching and ANI estimation for massive genome collections. RESULTS: We evaluate HyperGen's sketching and database search performance using several genome datasets at various scales. HyperGen is able to achieve comparable or superior ANI estimation error and linearity compared to other sketch-based counterparts. The measurement results show that HyperGen is one of the fastest tools for both genome sketching and database search. Meanwhile, HyperGen produces memory-efficient sketch files while ensuring high ANI estimation accuracy. AVAILABILITY AND IMPLEMENTATION: A Rust implementation of HyperGen is freely available under the MIT license as an open-source software project at https://github.com/wh-xu/Hyper-Gen. The scripts to reproduce the experimental results can be accessed at https://github.com/wh-xu/experiment-hyper-gen.

Hyperdimensional (HD) computing is a promising paradigm for large-scale genome sequencing. In prior work, we proposed a 3D NAND-based HD computing engine as an energy-efficient solution for sequencing several gigabytes or terabytes of genomic data. In this work, we introduce an improved HD computing engine for genome sequencing that leverages heterogeneous 3D integration techniques. We employ Cu-Cu hybrid bonding and CMOS under array (CuA) technologies to integrate the digital logic tier with the 3D NAND-based associative memory tier. We benchmark the performance of the proposed hardware design using a dataset of 10,575 microorganism genomes. The results indicate the robustness of the classification accuracy despite device non-idealities. Compared to conventional methods, the proposed design reduces the system-level energy consumption by$1000\times $, provided the data is not offloaded from the solid-state drive to the computing units.

Over the past decade, numerous compute-in-memory (CIM) platforms have been proposed in the literature. While emerging non-volatile memory based analog CIM (ACIM) has been widely studied, its silicon demonstrations are in the mature legacy node (22 nm or above). As an alternative, digital CIM (DCIM) based on static random access memory (SRAM) is recently drawing significant attention, as it enjoys the scaling benefits with the logic process to the leading-edge node (5 nm or below), and does not suffer from the accuracy loss due to process/voltage/temperature (PVT) variations. To assess the potential of DCIM in the future, we release NeuroSim V1.4, a CIM benchmark framework, which supports advanced technology nodes down to 1 nm node. We project the technology parameters (standard cell, transistor and interconnect) using TCAD device simulations, interconnect modeling, and the available industry/IRDS roadmaps. State-of-the-art technology trends such as fin-depopulation, buried power rail, stacked nanosheet, etc are captured in the updated parameters. Technology scaling down to 1 nm enables DCIM to achieve 1.4$\sim 1.8\times $and 44.1$\sim 63.1\times $higher system-level figure of merit than state-of-the-art 7 nm SRAM-based ACIM and 22 nm RRAM-based ACIM, respectively, for representative workloads such as ResNet18 and ResNet34 inference.

Prior hardware accelerator designs primarily focused on single-chip solutions for 10 MB-class computer vision models. The GB-class transformer models for natural language processing (NLP) impose challenges on existing accelerator design due to the massive number of parameters and the diverse matrix multiplication (MatMul) workloads involved. This work proposes a heterogeneous 3D-based accelerator design for transformer models, which adopts an interposer substrate with multiple 3D memory/logic hybrid cubes optimized for accelerating different MatMul workloads. An approximate computing scheme is proposed to take advantage of heterogeneous computing paradigms of mixed-signal compute-in-memory (CIM) and digital tensor processing units (TPU). From the system-level evaluation results, 10 TOPS/W energy efficiency is achieved for the BERT and GPT2 model, which is about 2.6× ∼ 3.1× higher than the baseline with 7 nm TPU and stacked FeFET memory.

Emerging nonvolatile memories as embedded memories offer low leakage power and high memory density, compared to the static random access memory (SRAM) and embedded dynamic random access memory (eDRAM) at the same technology node. However, the emerging memories generally suffer from limited cycling endurance. For read/write intensive applications, the limited endurance could become a bottleneck that limits the lifetime of the overall system. In this work, Intel’s reported prototype 3-D stackable ferroelectric random access memory (FeRAM) is considered as the global buffer memory of a tensor-processing-unit (TPU)-like architecture. An endurance-aware compiler is proposed to evaluate the maximum number of deep neural network (DNN) trainings considering the experimentally measured endurance limit. In addition, the proposed compiler applies two strategies to alleviate the endurance issue. The first strategy is wear leveling, and the second strategy is the dual-mode operation between volatile and nonvolatile modes. The maximum numbers of trainings increase by$6\times $to$300\times $and$4\times $to$58\times $thanks to the wear-leveling and dual-mode operations, respectively. Finally, a guideline of the system endurance (maximum number of trainings) is provided with given memory device endurance to bridge the gap between memory device engineers and system designers.

Heterogeneous 3-D (H3D) integration not only reduces the chip form factor and fabrication cost but also allows the merging of diverse compute paradigms that suit different applications. This is especially attractive when modern algorithms, such as the augmented reality/virtual reality (AR/VR) workloads, consist of mixed machine learning (ML) and non-ML workloads. To date, codesign that considers the thermal, latency, and power constraints of H3D hardware is largely unexplored. In this work, a thermally aware framework for H3D hardware design is developed to evaluate the thermal, latency, and power trade-offs for a heterogeneous system with compute-in-memory (CIM), digital ML cores, and RISC-V cores. The framework solves for runtime tunable operating points described as the optimal speedup factor, the number of activated RISC-V cores, the cooling coefficient, and the activity rate based on user-defined criteria, achieving up to 135 TOPS and 215 TOPS/W under$74~^{\circ }$C for the AR/VR workloads.

As deep neural networks (DNNs) have been success-fully developed in many applications with continuously increasing complexity, the number of weights in DNNs surges, leading to consistent demands for denser memories than SRAMs. RRAM-based in-memory computing (IMC) achieves high density and energy-efficiency for DNN inference, but RRAM programming remains to be a bottleneck due to high write latency and energy consumption. In this work, we present the Progressive-wRite In-memory program-VErify (PRIVE) scheme, which we verify with an RRAM testchip for IMC-based hardware acceleration for DNNs. We optimize the progressive write operations on different bit positions of RRAM weights to enable error compensation and reduce programming latency/energy, while achieving high DNN accuracy. For 5-bit precision DNNs, PRIVE reduces the RRAM programming energy by 1.82×, while maintaining high accuracy of 91.91% (VGG-7) and 71.47% (ResNet-18) on CIFAR-10 and CIFAR-100 datasets, respectively.

After the success of the transformer networks on natural language processing (NLP), the application of transformers to computer vision has followed suit to deliver unprecedented performance gains on vision tasks including image recognition and object detection. The multi-head self-attention (MSA) is the key component in transformers, allowing the models to learn the amount of attention paid to each input position. In particular, hierarchical vision transformers (HVTs) utilize window-based MSA to capture the benefits of the attention mechanism at various scales for further accuracy enhancements. Despite its strong modeling capability, MSA involves complex operations that make transformers prohibitively costly for hardware deployment. Existing hardware accelerators have mainly focused on the MSA workloads in NLP applications, but HVTs involve different parameter dimensions, input sizes, and data reuse opportunities. Therefore, we design the RAWAtten architecture to target the window-based MSA workloads in HVT models. Each w-core in RAWAtten contains near-memory compute engines for linear layers, MAC arrays for intermediate matrix multiplications, and a lightweight reconfigurable softmax. The w-cores can be combined at runtime to perform hierarchical processing to accommodate varying model parameters. Compared to the baseline GPU, RAWAtten at 40nm provides 2.4x average speedup for running the window-MSA workloads in Swin transformer models while consuming only a fraction of GPU power. In addition, RAWAtten achieves 2x area efficiency compared to prior ASIC accelerator for window-MSA.

One of the primary challenges of deploying deep neural networks (DNNs) is ensuring their reliable performance in unpredictable edge environments, which are often disrupted by a variety of uncertainties and variations. Estimating uncertainty is crucial in order to understand the reliability of task predictions and prevent system failures. However, quantifying uncertainty stemming from non-ideal properties of processing hardware has not yet been thoroughly studied. To address this, we present Cross-Layer Uncertainty Estimator (CLUE), which quantifies task uncertainty originating from both sensing/processing hardware variations and DNN algorithm uncertainty. Our experimental results demonstrate that CLUE provides uncertainty with up to 80.4% less calibration error and only 12% of energy overheads compared to using task DNN solely. Furthermore, CLUE is able to detect unreliable tasks that stem from processing hardware variations, which prior uncertainty estimators were unable to achieve. Finally, we demonstrate an adaptive control of processing hardware using CLUE, which allows a dynamic trade-off control between task accuracy and energy consumption.

The states of resistive random-access memory (RRAM) have been shown to drift from read voltage stress. When using RRAM as weight elements in deep neural networks (DNNs), this drift degrades inference accuracy over time. To maintain accuracy, RRAM cells must be frequently reprogrammed back to their initial state. We propose a method for reprogramming RRAM memory arrays without accessing the nominal DNN parameters stored off-chip (e.g., in cloud). The method utilizes techniques in linear algebra to calculate the correct state of the RRAM as needed. Resistance drift of RRAM cells were measured from a 40 nm RRAM compute-in-memory (CIM) macro test chip, and the impact on inference accuracy was simulated. Then, the accuracy of the method was evaluated and the recovery in inference accuracy was recorded. Results suggest the method is capable of correctly calculating RRAM resistance states in the presence of resistance drift and other realistic chip noises. The method was also evaluated at reduced precision analog-to-digital conversion to examine its effectiveness in a practical CIM design.

Compute-in-memory (CIM) is an attractive solution for machine learning hardware acceleration since it merges computation directly into memory arrays, performing parallel multiply-and-accumulate (MAC) operations. The primary challenge in the reported CIM designs is the analog-to-digital converters (ADCs) that digitize analog MAC values for further processing, causing accuracy loss, excessive power dissipation, latency penalty, and area overhead. In this work, we propose ENNA, a novel CIM architecture based on an ADC-free sub-array design, implementing inter-array data processing in an analog manner. A lightweight input encoding scheme based on pulse-width modulation (PWM) is proposed to improve the throughput. We taped-out a prototype macro and validated the proposed ADC-free RRAM array design in TSMC 40nm process. Based on the measured silicon data, we explore the system-level performance with a partition between analog and digital processing at a level higher than the sub-array. The evaluation results show that the proposed accelerator can achieve 73.6~86.4 TOPS/W energy efficiency and 2.3~7 TOPS throughput (normalized to binary operation) tested on various DNN models. Furthermore, we project the proposed design using a heterogeneous 3D integration (H3D) scheme, showing a$3\times \sim 37\times $throughput improvement depending on different tasks and ~50% reduced area overhead compared to 2D design.

With the advancement of deep learning to solve autonomous driving problems, the computation and memory requirements have been growing rapidly. Near-pixel compute-based CMOS image sensors (CIS) have been investigated as a potential candidate to perform the initial computations of workloads close to the pixel and reduce data movement. In this work, we design a near-pixel compute CIS capable of implementing a temporal frame filtering network, which rejects redundant image frames targeting autonomous driving applications. To improve performance and avoid image distortion, 3D-stacked global shutter CIS is proposed. This architecture integrates photodiodes with memory and compute units using Cu-Cu hybrid bonding. We propose to use back-end-of-line (BEOL) compatible Tungsten-doped Indium Oxide Transistors (IWO FETs) based embedded DRAM as buffer memory to achieve refresh-free storage and high bandwidth connections between various components. Near-pixel compute circuit is optimized by including sparsity-aware adder tree and using NOR gates as data buffers. The two-tier system comprises photodiodes on tier-1 in 40 nm node, and near-pixel compute and buffer memory on tier-2 in 22 nm node. We perform simulations in Cadence, obtaining an energy efficiency of 65 TOPS/W and a compute density of 1.04 TOPS/mm2 for$8\times8\text{b}$MAC, with a total latency of 1.15 ms/frame.

The emerging non-volatile memory (eNVM) based mixed-signal Compute-in-Memory (CIM) accelerators are of great interest in today's AI accelerators design due to their high energy efficiency. Various CIM architectures and circuit-level designs have been proposed, showing superior hardware performance for deep neural network (DNN) acceleration. However, hardware-aware quantization strategies for CIM-based accelerators are not systematically explored. Since there are a variety of design options for neural network mapping on CIM systems while improper strategies may narrow the circuit-level design space and further limit the hardware performance, it is important to make a comprehensive early-stage design space exploration and find appropriate quantization/mapping strategies to achieve better hardware performance. In this paper, we provide a joint algorithm-hardware analysis and compare the system-level hardware performance for various design options, including quantization algorithms, data representation methods and analog-to-digital converter (ADC) configurations. This work aims to propose guidelines for choosing more hardware-friendly design options for chip architects. According to our evaluation results for CIFAR-10/100 and ImageNet classification, the properly chosen quantization approach and optimal mapping strategy (dynamic fixed-point quantization + 2’s complement representation/shifted unsigned INT representation + optimized precision ADC) could achieve ∼2 × energy efficiency and 1.2 ∼ 1.6 × throughput with 5%∼25% reduced area overhead, compared to naïve strategy (fixed-point quantization + differential pair number representation + full precision ADC).

After the success of the transformer networks on natural language processing (NLP), the application of transformers to computer vision (CV) has followed suit to deliver unprecedented performance gains on vision tasks, including image recognition and object detection. The multihead self-attention (MHSA) is the key component in transformers, allowing the models to learn the amount of attention paid to each input position. Despite its strong modeling capability, MHSA involves complex operations that make transformers prohibitively costly for hardware deployment. Existing acceleration efforts with conventional hardware platforms are challenged by the memory wall. To alleviate the memory wall problem, compute-in-memory (CIM) is a promising solution by storing all model parameters on-chip in compute-capable memory arrays. The footprint of 2-D CIM designs must, however, expand to accommodate the increasingly larger model sizes. In this work, we present a heterogeneous 3-D integrated (H3D) accelerator to target the MHSA workloads in vision transformers. H3D allows the proposed H3DAtten architecture to combine the merits of resistive random access memory (RRAM)-based analog CIM (ACIM) in 40 nm and static random access memory (SRAM)-based digital CIM (DCIM) in 16 nm. We perform comprehensive signaling and thermal analyses to examine the effects of 3-D stacking on the accelerator. Compared to iso-capacity 2-D baseline designs, the proposed 5-tier H3DAtten accelerator achieves$8.4\times $compute density without experiencing accuracy loss on the ImageNet-1k dataset.

In this article, we propose a hardware accelerator design using ferroelectric transistor (FeFET)-based hybrid precision synapse (HPS) for deep neural network (DNN) on-chip training. The drain erase scheme for FeFET programming is incorporated for both FeFET HPS design and FeFET buffer design. By using drain erase, high-density FeFET buffers can be integrated onchip to store the intermediate input-output activations and gradients, which reduces the energy consuming off-chip DRAM access. Architectural evaluation results show that the energy efficiency could be improved by 1.2× ∼ 2.1×, 3.9× ∼ 6.0× compared to the other HPS-based designs and emerging non-volatile memory baselines, respectively. The chip area is reduced by 19% ∼ 36% compared with designs using SRAM on-chip buffer even though the capacity of FeFET buffer is increased. Besides, by utilizing drain erase scheme for FeFET programming, the chip area is reduced by 11% ∼ 28.5% compared with the designs using body erase scheme.

In this paper, a dual-mode buffer memory based on the CMOS compatible HfZrO2 ferroelectric material is proposed for DNN accelerators. It can operate in both volatile eDRAM mode and non-volatile ferroelectric RAM (FeRAM) mode. The functionality of the proposed dual-mode memory bit-cell design is verified using SPICE simulation with the multi-domain Preisach physical model. A data-lifetime-aware memory mode configuration protocol is proposed to optimize the buffer access energy for both DNN inference and training. Detailed circuitry and architectural support for the dual-mode memory are discussed. For DNN training with ferroelectric-field-effect-transistor (FeFET) and SRAM-based compute-in-memory (CIM) accelerator, the proposed dual-mode buffer design improves the overall energy efficiency by 92.2%~98.7%, 44.1%~47.6%, 12.6%~13.0% compared to baseline designs using SRAM buffer with the same buffer area, eDRAM and FeRAM with the same buffer capacity, respectively. For DNN inference with tensor-processing-unit (TPU)-like systolic array, the energy efficiency during computing is improved by 40.7%~45.6%, 18.4%~29.6% compared to the designs with eDRAM and FeRAM buffer, respectively. By storing the persistent data using the non-volatile mode, the energy efficiency of systolic array is improved by 2.3~5.5 over SRAM-based design when standby is frequent.

This article presents an algorithmic approach to design reliable deep neural networks (DNNs) in the presence of stochastic variations in the network parameters induced by process variations in the bit cells in a processing-in-memory (PIM) architecture. We propose and derive a Hessian-based sensitivity metric that can be computed without computing or storing the full Hessian to identify and protect the “important” network parameters while allowing large variations in unprotected parameters. We also show that this metric can be used to aggressively quantize unprotected network parameters in the PIM for improved inference efficiency and compute density. Experiments on modern DNNs like ResNet, MobileNetv2, and DenseNet on CIFAR10 using measured RRAM device data shows the effectiveness of our approach.

On-device embedded artificial intelligence prefers the adaptive learning capability when deployed in the field, and thus in situ training is required. The compute-in-memory approach, which exploits the analog computation within the memory array, is a promising solution for deep neural network (DNN) on-chip acceleration. Emerging non-volatile memories are of great interest, serving as analog synapses due to their multilevel programmability. However, the asymmetry and nonlinearity in the conductance tuning remain grand challenges for achieving high in situ training accuracy. In addition, analog-to-digital converters at the edge of the memory array introduce quantization errors. In this work, we present an algorithm-hardware co-optimization to overcome these challenges. We incorporate the device/circuit non-ideal effects into the DNN propagation and weight update steps. By introducing the adaptive “momentum” in the weight update rule, in situ training accuracy on CIFAR-10 could approach its software baseline even under severe asymmetry/nonlinearity and analog-to-digital converter quantization error. The hardware performance of the on-chip training architecture and the overhead for adding “momentum” are also evaluated. By optimizing the backpropagation dataflow, 23.59 TOPS/W training energy efficiency (12× improvement compared to naïve dataflow) is achieved. The circuits that handle “momentum” introduce only 4.2% energy overhead. Our results show great potential and more relaxed requirements that enable emerging non-volatile memories for DNN acceleration on the embedded artificial intelligence platforms.

Compute-in-memory (CIM) is an attractive solution to address the “memory wall” challenges for the extensive computation in machine learning hardware accelerators. Prior CIM-based architectures, though can adapt to different neural network models during the design time, they are implemented to different custom chips. Therefore, a specific chip instance is restricted to a specific network during runtime. However, the development cycle of the hardware is normally far behind the emergence of new algorithms. In this paper, a runtime reconfigurable design methodology of CIM-based accelerator is proposed to support a class of convolutional neural networks running on one pre-fabricated chip instance. First, several design aspects are investigated: 1) reconfigurable weight mapping method; 2) input side of data transmission, mainly about the weight reloading; 3) output side of data processing, mainly about the reconfigurable accumulation. Then, system-level performance benchmark is performed for the inference of different models like VGG-8 on CIFAR-10 dataset and AlexNet, GoogLeNet, ResNet-18 and DenseNet-121 on ImageNet dataset to measure the tradeoffs between runtime reconfigurability, chip area, memory utilization, throughput and energy efficiency.

Deep neural network (DNN) training produces a large amount of intermediate data. As off-chip DRAM access is both energy and time consuming, sufficient on-chip buffer is preferred to achieve high energy efficiency for DNN accelerator designs. However, the low integration density and high leakage current of SRAM lead to large area cost and high standby power. The frequent refresh of embedded DRAM (eDRAM) degrades the energy efficiency due to its short refresh interval (40~100µs). In this paper, a dual-mode buffer memory that can operate in both volatile eDRAM mode and non-volatile ferroelectric RAM (FeRAM) mode is proposed, which is based on the CMOS compatible HfZr02 material. The functionality of the proposed dual-mode memory design is verified using SPICE simulation with the multi-domain Preisach model. A data lifetime-aware memory mode configuration protocol is proposed to optimize the buffer access energy. The architectural benchmark for DNN training shows 33.8%, 17.1 % and 109.4% higher energy efficiency than baseline designs with eDRAM, FeRAM and SRAM with the same buffer area, respectively. The chip standby power is reduced by 26.8x~47.5x and 1.5x~10.6x compared with the SRAM and eDRAM baselines. The chip area overhead of the dual-mode buffer design is 5.7%.

Machine learning inference engine is of great interest to smart edge computing. Compute-in-memory (CIM) architecture has shown significant improvements in throughput and energy efficiency for hardware acceleration. Emerging nonvolatile memory (eNVM) technologies offer great potentials for instant on and off by dynamic power gating. Inference engine is typically pre-trained by the cloud and then being deployed to the field. There is a new security concern on cloning of the weights stored on eNVM-based CIM chip. In this paper, we propose a countermeasure to the weight cloning attack by exploiting the process variations of the periphery circuitry. In particular, we use weight fine-tuning to compensate the analog-to-digital converter (ADC) offset for a specific chip instance while inducing significant accuracy drop for cloned chip instances. We evaluate our proposed scheme on a CIFAR-10 classification task using a VGG- 8 network. Our results show that with precisely chosen transistor size on the employed SAR-ADC, we could maintain 88%~90% accuracy for the fine-tuned chip while the same set of weights cloned on other chips will only have 20~40% accuracy on average. The weight fine-tune could be completed within one epoch of 250 iterations. On average only 0.02%, 0.025%, 0.142% of cells are updated for 2-bit, 4-bit, 8-bit weight precisions in each iteration.

Compute-in-memory has received a lot of research interests recently to implement the data-intensive computation in deep neural networks. By performing the computing at the storage location, CIM avoids the excessive data transfer thus improving the energy efficiency. SRAM based CIM is one of the promising candidates for its mature technology availability at advanced technology node. To further speed up for CMOS circuits, cryogenic computing which operates at low temperatures has emerged as an attractive solution for high-performance computing at the data center. In this work, we modified NeuroSim, a device-to-system modelling framework with experimentally calibrated 28nm transistor parameters from room temperature to 4K Then we benchmark the performance of SRAM based CIM for ResNet-18 on ImagNet dataset. The energy-delay-product is compared across the temperature, revealing the performance and energy efficiency boost by cryogenic computing. When the cooling infrastructure cost is considered, the overall energy benefits are overshadowed though.

As AI applications become pervasive on edge device, incrementally learning new tasks is demanded for deep neural network (DNN) models. In this article, we proposed AILC, a compute-in-memory (CIM)-based accelerator for on-chip incremental learning using STT-MRAM technology. On the software side, a network-expansion-based low-precision training algorithm is proposed for incremental learning, where the loss function is modified to handle the unbalanced training dataset. On the hardware side, the detailed CIM accelerator design for incremental learning is illustrated. A workload-aware hardware resources assignment protocol is proposed to improve the throughput when the workload of weight gradient calculation is low. The software simulation results on CIFAR-100 dataset show that the proposed algorithm can effectively support incremental learning despite the device conductance variation exists. System-level benchmark shows that AILC could achieve 147×, 3.7×~28.7×, 2.05×~2.9× higher energy efficiency than Nvidia Titan-V GPU, RRAM-based CIM accelerators and edge TPU/GPU, respectively. Compared to the baselines, the throughput of AILC is improved by 2.0×~2.2× on average with the hardware resources assignment protocol, which results in 4.1×~21.4× higher throughput than edge TPU/GPU.

DNN+NeuroSim is an integrated framework to benchmark compute-in-memory (CIM) accelerators for deep neural networks, with hierarchical design options from device-level, to circuit level and up to algorithm level. A python wrapper is developed to interface NeuroSim with a popular machine learning platform: Pytorch, to support flexible network structures. The framework provides automatic algorithm-to-hardware mapping, and evaluates chip-level area, energy efficiency and throughput for training or inference, as well as training/inference accuracy with hardware constraints. Our prior inference version of DNN+NeuroSim framework available athttps://github.com/neurosim/DNN_NeuroSim_V1.2was developed to estimate the impact of reliability in synaptic devices, and analog-to-digital converter (ADC) quantization loss on the accuracy and hardware performance of an inference engine. In this work, we further investigated the impact of the “analog” emerging nonvolatile memory (eNVM)’s nonideal device properties for on-chip training. By introducing the nonlinearity, asymmetry, device-to-device and cycle-to-cycle variation of weight update into the python wrapper, and peripheral circuits for error/weight gradient computation in NeuroSim core, we benchmarked CIM accelerators based on state-of-the-art SRAM and eNVM devices for VGG-8 on CIFAR-10 dataset, revealing the crucial specs of synaptic devices for on-chip training. The latest training version of the DNN+NeuroSim framework is available athttps://github.com/neurosim/DNN_NeuroSim_V2.1.

To efficiently deploy machine learning applications to the edge, compute-in-memory (CIM) based hardware accelerator is a promising solution with improved throughput and energy efficiency. Instant-on inference is further enabled by emerging non-volatile memory technologies such as resistive random access memory (RRAM). This paper reviews the recent progresses of the RRAM based CIM accelerator design. First, the multilevel states RRAM characteristics are measured from a test vehicle to examine the key device properties for inference. Second, a benchmark is performed to study the scalability of the RRAM CIM inference engine and the feasibility towards monolithic 3D integration that stacks RRAM arrays on top of advanced logic process node. Third, grand challenges associated with in-situ training are presented. To support accurate and fast in-situ training and enable subsequent inference in an integrated platform, a hybrid precision synapse that combines RRAM with volatile memory (e.g. capacitor) is designed and evaluated at system-level. Prospects and future research needs are discussed.

Compute-in-memory (CIM) is an attractive solution to address the “memory wall” challenges for the extensive computation in deep learning hardware accelerators. For custom ASIC design, a specific chip instance is restricted to a specific network during runtime. However, the development cycle of the hardware is normally far behind the emergence of new algorithms. Although some of the reported CIM-based architectures can adapt to different deep neural network (DNN) models, few details about the dataflow or control were disclosed to enable such an assumption. Instruction set architecture (ISA) could support high flexibility, but its complexity would be an obstacle to efficiency. In this article, a runtime reconfigurable design methodology of CIM-based accelerators is proposed to support a class of convolutional neural networks running on one prefabricated chip instance with ASIC-like efficiency. First, several design aspects are investigated: (1) the reconfigurable weight mapping method; (2) the input side of data transmission, mainly about the weight reloading; and (3) the output side of data processing, mainly about the reconfigurable accumulation. Then, a system-level performance benchmark is performed for the inference of different DNN models, such as VGG-8 on a CIFAR-10 dataset and AlexNet GoogLeNet, ResNet-18, and DenseNet-121 on an ImageNet dataset to measure the trade-offs between runtime reconfigurability, chip area, memory utilization, throughput, and energy efficiency.

Compute-in-memory (CIM), where information can be processed and stored at the same locations, is emerging as a promising paradigm to address the memory wall bottleneck in traditional Von Neumann architectures. Static random-access memory (SRAM) has been demonstrated as a mature candidate for CIM accelerator for deep neural networks (DNNs) due to its availability in advanced technology nodes. However, as SRAM is volatile and could not hold weight after power down, the necessity for downloading models from the cloud to inference engine causes potential threats such as model leaking. Also, saving raw weights of the DNN model stationary in the memory cells will increase the vulnerabilities. This work aims at developing a secure inference engine with a lightweight yet effective countermeasure to protect the DNN models in SRAM-based CIM architecture. We propose a secure XOR-CIM engine with a modified reverse secure sketch protocol to enable on-chip authentication and key processing for XOR-based stream cipher encrypted models. In the XOR-CIM core, we modify the six-transistor SRAM bit cell with dual wordlines to implement XOR decryption without sacrificing the parallel computation’s efficiency. The evaluations at 28 nm show that the XOR-CIM could enhance security, achieving comparable energy efficiency and no throughput loss, with negligible area overhead compared with the normal-CIM design without encryption.

Resistive random access memory (RRAM)-based compute-in-memory architecture helps overcome the bottleneck caused by large memory transactions in the convolutional neural network (CNN) accelerators. However, their deployment using 2-D IC technology faces challenges, as today's RRAM cells remain at legacy nodes above 20 nm due to high programming voltages. Besides, power-hungry analog-to-digital converter (ADC) units limit the throughput of RRAM accelerators. In this article, we present the first-ever heterogeneous (multiple technology nodes) mixed-signal monolithic 3-D IC designs of the RRAM CNN accelerator. Our RRAM remains at legacy 40-nm nodes in one tier, but CMOS periphery scales toward advanced 28/16 nm in another tier. Our 3-D designs overcome the bottleneck caused by ADCs and offer up to 4.9× improvement in energy efficiency in TOPS/W and up to 50% reduction in footprint area over 40-nm 2-D IC designs. Compared with existing 2-D works, our 3-D architecture offers up to 28.6× improvement in energy efficiency.

Compute-in-memory (CIM) with emerging non-volatile memories (eNVMs) is time and energy efficient for deep neural network (DNN) inference. However, challenges still remain for in-situ DNN training with eNVMs due to the asymmetric weight update behavior, high programming latency and energy consumption. To overcome these challenges, a hybrid precision synapse combining eNVMs with capacitor has been proposed. It leverages the symmetric and fast weight update in the volatile capacitor, as well as the non-volatility and large dynamic range of the eNVMs. In this paper, in-situ DNN training architecture with hybrid precision synapses is proposed and benchmarked with the modified NeuroSim simulator. First, all the circuit modules required for in-situ training with hybrid precision synapses are designed. Then, the impact of weight transfer interval and limited capacitor retention time on training accuracy is investigated by incorporating hardware properties into Tensorflow simulation. Finally, a system-level benchmark is conducted for hybrid precision synapse compared with baseline design that is solely based on eNVMs.

On-chip training of large-scale deep neural networks (DNNs) is challenging due to computational complexity and resource limitation. Compute-in-memory (CIM) architecture exploits the analog computation inside the memory array to speed up the vectormatrix multiplication (VMM) and alleviate the memory bottleneck. However, existing CIM prototype chips, in particular, SRAM-based accelerators target at implementing low-precision inference engine only. In this work, we propose a two-way SRAM array design that could perform bi-directional in-memory VMM with minimum hardware overhead. A novel solution of signed number multiplication is also proposed to handle the negative input in backpropagation. We taped-out and validated proposed two-way SRAM array design in TSMC 28nm process. Based on the silicon measurement data on CIM macro, we explore the hardware performance for the entire architecture for DNN on-chip training. The experimental data shows that proposed accelerator can achieve energy efficiency of ~3.2 TOPS/W, >1000 FPS and >300 FPS for ResNet and DenseNet training on ImageNet, respectively.

Embedded artificial intelligence (AI) prefers the adaptive learning capability when deployed in the field, thus in- situ training on-chip is required. Emerging non-volatile memories (eNVMs) are of great interests serving as analog synapses in deep neural network (DNN) on-chip acceleration due to its multilevel programmability. However, the asymmetry/nonlinearity in the conductance tuning remains a grand challenge for achieving high in-situ training accuracy. In addition, analog-to-digital converter (ADC) at the edge of the memory array introduces an additional challenge - quantization error for in-memory computing. In this work, we gain new insights and overcome these challenges through an algorithm-hardware co-optimization. We incorporate these hardware non-ideal effects into the DNN propagation and weight update steps. We evaluate on a VGG-like network for CIFAR-10 dataset, and we show that the asymmetry of the conductance tuning is no longer a limiting factor of in-situ training accuracy if exploiting adaptive "momentum" in the weight update rule. Even considering ADC quantization error, in-situ training accuracy could approach software baseline. Our results show much relaxed requirements that enable a variety of eNVMs for DNN acceleration on the embedded AI platforms.

Compute-in-memory (CIM) is a promising approach that exploits the analog computation inside the memory array to speed up the vector-matrix multiplication (VMM) for deep neural network (DNN) inference. SRAM has been demonstrated as a mature candidate for CIM architecture due to its availability in advanced technology node. However, as the weights of the DNN model are stationary in the memory cells, it causes potential threats and vulnerabilities for inference engine such as model leaking. This work aims at developing a lightweight yet effective countermeasure to protect the DNN model in CIM architecture. We modify the 6-transistor SRAM bit cell with dual wordlines to implement XOR cipher without sacrificing the parallel computation's efficiency. The evaluations at 28 nm show that XOR-CIM could provide enhanced security and achieve 1.4× energy efficiency improvement and no throughput loss, with only 2.5% area overhead compared to the normal-CIM design without encryption.

On-chip training of large-scale deep neural networks (DNNs) is challenging. To solve the memory wall problem, compute-in-memory (CIM) is a promising approach that exploits the analog computation inside the memory array to speed up the vector-matrix multiplication (VMM). Challenges for on-chip CIM training include higher weight precision and higher analog-to-digital converter (ADC) resolution. In this work, we propose a mixed-precision RRAM-based CIM architecture that overcomes these challenges and supports on-chip training. In particular, we split the multi-bit weight into the most significant bits (MSBs) and the least significant bits (LSBs). The forward and backward propagations are performed with CIM transposable arrays for MSBs only, while the weight update is performed in regular memory arrays that store LSBs. Impact of ADC resolution on training accuracy is analyzed. We explore the training performance of a convolutional VGG-like network on the CIFAR-10 dataset using this Mixed-precision IN-memory Training architecture, namely MINT, showing that it can achieve ~91% accuracy under hardware constraints and ~4.46TOPS/W energy efficiency. Compared with the baseline CIM architectures based on RRAM, it can achieve 1.35× higher energy efficiency and only 31.9% chip size (~98.86 mm2at 32 nm node).

STT-MRAM is a promising candidate as embedded non-volatile memory (NVM) at 28nm and beyond. Due to its limited on/off ratio, STT-MRAM is often used as digital memory that only allows row-by-row read-out for near-memory computing. This work proposes design strategies to overcome this limitation with a new bit-cell design to enable parallel read-out for in-memory computing, which is of great interests for deep neural network (DNN) acceleration. We consider the non-ideal device properties that degrade inference accuracy including small on/off ratio, cell-to-cell MTJ conductance variation and current sense amplifier (CSA) offset. We propose three techniques to minimize inference accuracy degradation: 1) a 2T-2MTJ bit-cell design with high on/off ratio, 2) redundancy for MSB weights to mitigate the impact of MTJ conductance variations, and 3) a hybrid-layer mapping scheme to reduce column current thus mitigating CSA offset effect. DNN benchmarking results show that on CIFAR-10 dataset, the inference accuracy can be maintained at > 90% in the presence of 10% MTJ conductance variations, and >87.5% after considering CSA offset effect, with minimal 8% energy and 4% chip area overhead.

Rapid development in deep neural networks (DNNs) is enabling many intelligent applications. However, on-chip training of DNNs is challenging due to the extensive computation and memory bandwidth requirements. To solve the bottleneck of the memory wall problem, compute-in-memory (CIM) approach exploits the analog computation along the bit line of the memory array thus significantly speeds up the vector-matrix multiplications. So far, most of the CIM-based architectures target at implementing inference engine for offline training only. In this article, we propose CIMAT, a CIM Architecture for Training. At the bitcell level, we design two versions of 7T and 8T transpose SRAM to implement bi-directional vector-to-matrix multiplication that is needed for feedforward (FF) and backprogpagation (BP). Moreover, we design the periphery circuitry, mapping strategy and the data flow for the BP process and weight update to support the on-chip training based on CIM. To further improve training performance, we explore the pipeline optimization of proposed architecture. We utilize the mature and advanced CMOS technology at 7 nm to design the CIMAT architecture with 7T/8T transpose SRAM array that supports bi-directional parallel read. We explore the 8-bit training performance of ImageNet on ResNet-18, showing that 7T-based design can achieve 3.38× higher energy efficiency (~6.02 TOPS/W), 4.34× frame rate (~4,020 fps) and only 50 percent chip size compared to the baseline architecture with conventional 6T SRAM array that supports row-by-row read only. The even better performance is obtained with 8T-based architecture, which can reach ~10.79 TOPS/W and ~48,335 fps with 74-percent chip area compared to the baseline.

Compute-in-memory (CIM) with emerging non-volatile memories (eNVMs) is time and energy efficient for deep neural network (DNN) inference. However, challenges still remain for DNN in-situ training with eNVMs due to the asymmetric weight update behavior, high programming latency and energy consumption. To overcome these challenges, a hybrid precision synapse combining eNVMs with capacitor has been proposed. It leverages the symmetric and fast weight update in the volatile capacitor, as well as the non-volatility and large dynamic range of the eNVMs. In this article, DNN in-situ training architecture with hybrid precision synapses is proposed and system level benchmarked is conducted. First, the circuit modules required for in-situ training with hybrid precision synapses are designed and the system architecture is proposed. Then, the impact of different weight precision configurations, weight transfer interval and limited capacitor retention time on training accuracy is investigated by incorporating hardware properties into Tensorflow simulation. Finally, the system-level benchmark is conducted at 32nm technology node in the modified NeuroSim simulator for hybrid precision synapse, in comparison with the baseline designs that are solely based on eNVMs or SRAM technology. The benchmark results show that CIM accelerator based on hybrid precision synapse achieves at least 3.07x and 2.89x better energy efficiency for training compared with its eNVM counterparts and SRAM technology at 32nm node, respectively. 227x and 33.8x better energy efficiency are obtained when compared to GPU and TPU. The scaling trend of hybrid precision synapse is projected towards 7nm node and comparison with state-of-the-art 7nm SRAM technology is made.

Compute-in-memory (CIM) is a promising computing paradigm to accelerate the inference of deep neural network (DNN) algorithms due to its high processing parallelism and energy efficiency. Prior CIM-based DNN accelerators mostly consider full custom design, which assumes that all the weights are stored on-chip. For lightweight smart edge devices, this assumption may not hold. In this article, CIM-based DNN accelerators are designed and benchmarked under different chip area constraints. First, a scheduling strategy and dataflow for DNN inference is investigated when only part of the weights can be stored on-chip. Two weight reload schemes are evaluated: 1) reload partial weights and reuse input/output feature maps and 2) load a batch of input and reuse the partial weights on-chip across the batch. Then, system-level performance benchmark is performed for the inference of ResNet-18 on ImageNet data set. The design tradeoffs with different area constraints, dataflow, and device technologies [static random access memory (SRAM) versus ferroelectric field-effect transistor (FeFET)] are discussed.

RRAM based neural-processing-unit (NPU) is emerging for processing general purpose machine intelligence algorithms with ultra-high energy efficiency, while the imperfections of the analog devices and cross-point arrays make the practical application more complicated. In order to improve accuracy and robustness of the NPU, device-circuit-algorithm codesign with consideration of underlying device and array characteristics should outperform the optimization of individual device or algorithm. In this work, we provide a joint device-circuit-algorithm analysis and propose the corresponding design guidelines. Key innovations include: 1) An end-to-end simulator for RRAM NPU is developed with an integrated framework from device to algorithm. 2) The complete design of circuit and architecture for RRAM NPU is provided to make the analysis much close to the real prototype. 3) A large-scale neural network as well as other general-purpose networks are processed for the study of device-circuit interaction. 4) Accuracy loss from non-idealities of RRAM, such as I-V nonlinearity, noises of analog resistance levels, voltage-drop for interconnect, ADC/DAC precision, are evaluated for the NPU design.

Resistive random access memory (RRAM) based array architecture has been proposed for on-chip acceleration of convolutional neural networks (CNNs), where the array could be configured for dot-product computation in a parallel fashion by summing up the column currents. Prior processing-in-memory (PIM) designs unroll each 3D kernel of the convolutional layers into a vertical column of a large weight matrix, where the input data will be accessed multiple times. As a result, significant latency and energy are consumed in interconnect and buffer. In this paper, in order to maximize both weight and input data reuse for RRAM based PIM architecture, we propose a novel weight mapping method and the corresponding data flow which divides the kernels and assign the input data into different processing-elements (PEs) according to their spatial locations. The proposed design achieves ~65% save in latency and energy for interconnect and buffer, and yields overall 2.1× speed up and ~17% improvement in the energy efficiency in terms of TOPS/W for VGG-16 CNN, compared with the prior design based on the conventional mapping method.

We perform SPICE based simulations to identify the possibility and challenges of a sub-threshold read operation in a cross-point one-selector and one-resistive memory (1S-1R) array enabled by an ovonic threshold switch (OTS) with a high threshold voltage (Vth) and strong non-linearity of a sub-threshold regime. Here, it is important to read an on/off ratio below the Vth, where the OTS has not yet been turned on, leading to robust reliability characteristics of the 1S-1R. We show that the on/off ratio can be observed by adjusting the resistance range of the memory close to the threshold resistance of the OTS. Then, increasing the non-linearity in the sub-threshold regime of the OTS improves the on/off ratio. However, reading the obtained on/off ratio even in very small array is difficult due to sneak-path currents. Therefore, a low off-current of the OTS should be lower than 0.1 nA at 1 V to ensure a minimum readable on/off ratio (~2 ×) in the array.

Three-Dimensional NAND flash technology is one of the most competitive integrated solutions for the high-volume massive data storage. So far, there are few investigations on how to use 3-D NAND flash for in-memory computing in the neural network accelerator. In this brief, we propose using the 3-D vertical channel NAND array architecture to implement the vector-matrix multiplication (VMM) with for the first time. Based on the array-level SPICE simulation, the bias condition including the selector layer and the unselected layers is optimized to achieve high computation accuracy of VMM. Since the VMM can be performed layer by layer in a 3-D NAND array, the read-out latency is largely improved compared to the conventional single-cell read-out operation. The impact of device-to-device variation on the computation accuracy is also analyzed.

The impact of selector devices on the inference and training accuracy of a resistive random access memory (RRAM)-based neuromorphic computing system is rarely studied. In this paper, we analyze the weighted sum and weight update functions in a one-selector-one-RRAM (1S-1R)-based crossbar arrays. We first develop a Verilog-A model based on the lateral evolution of the filament to describe analog conductance tuning in the filamentary RRAM. We then perform an array-level SPICE simulation on the 1S-1R arrays, where the exponential and threshold selectors are employed. In the inference stage, the read-out current is vulnerable to the inevitable IR drop caused by the wire resistance. Our finding reveals that the use of a threshold selector allows the 1S-1R device to have a linear I-V relation, improving the immunity to the IR drop. On the other hand, the threshold selector distorts analog RRAM's linear weight update during the training stage. Instead, an introduction of exponential selector enables the desirable properties of analog RRAM to be maintained even in the 1S-1R device. These results indicate that different selectors suitable for each operation mode (inference or training) are preferred in the neuromorphic computing system.

Binary Neural Networks (BNNs) have been recently proposed to improve the area-/energy-efficiency of the machine/deep learning hardware accelerators, which opens an opportunity to use the technologically more mature binary RRAM devices to effectively implement the binary synaptic weights. In addition, the binary neuron activation enables using the sense amplifier instead of the analog-to-digital converter to allow bitwise communication between layers of the neural networks. However, the sense amplifier has intrinsic offset that affects the threshold of binary neuron, thus it may degrade the classification accuracy. In this work, we analyze a fully parallel RRAM synaptic array architecture that implements the fully connected layers in a convolutional neural network with (+1, -1) weights and (+1, 0) neurons. The simulation results with TSMC 65 nm PDK show that the offset of current mode sense amplifier introduces a slight accuracy loss from ~98.5% to ~97.6% for MNIST dataset. Nevertheless, the proposed fully parallel BNN architecture (P-BNN) can achieve 137.35 TOPS/W energy efficiency for the inference, improved by ~20X compared to the sequential BNN architecture (S-BNN) with row-by-row read-out scheme. Moreover, the proposed P-BNN architecture can save the chip area by ~16% as it eliminates the area overhead of MAC peripheral units in the S-BNN architecture.

Recent advances in deep neural networks (DNNs) have shown Binary Neural Networks (BNNs) are able to provide a reasonable accuracy on various image datasets with a significant reduction in computation and memory cost. In this paper, we explore two BNNs: hybrid BNN (HBNN) and XNOR-BNN, where the weights are binarized to +1/-1 while the neuron activations are binarized to 1/0 and +1/-1 respectively. Two SRAM bit cell designs are proposed, namely, 6T SRAM for HBNN and customized 8T SRAM for XNOR-BNN. In our design, the high-precision multiply-and-accumulate (MAC) is replaced by bitwise multiplication for HBNN or XNOR for XNOR-BNN plus bit-counting operations. To parallelize the weighted sum operation, we activate multiple word lines in the SRAM array simultaneously and digitize the analog voltage developed along the bit line by a multi-level sense amplifier (MLSA). In order to partition the large matrices in DNNs, we investigate the impact of sensing bit-levels of MLSA on the accuracy degradation for different sub-array sizes and propose using the nonlinear quantization technique to mitigate the accuracy degradation. With 64×64 sub-array size and 3-bit MLSA, HBNN and XNOR-BNN architectures can minimize the accuracy degradation to 2.37% and 0.88%, respectively, for an inspired VGG-16 network on the CIFAR-10 dataset. Design space exploration of SRAM based synaptic architectures with the conventional row-by-row access scheme and our proposed parallel access scheme are also performed, showing significant benefits in the area, latency and energy-efficiency. Finally, we have successfully taped-out and validated the proposed HBNN and XNOR-BNN designs in TSMC 65 nm process with measured silicon data, achieving energy-efficiency >100 TOPS/W for HBNN and >50 TOPS/W for XNOR-BNN.

Recent advances in deep learning have shown that Binary Neural Networks (BNNs) are capable of providing a satisfying accuracy on various image datasets with significant reduction in computation and memory cost. With both weights and activations binarized to +1 or -1 in BNNs, the high-precision multiply-and-accumulate (MAC) operations can be replaced by XNOR and bit-counting operations. In this work, we propose a RRAM synaptic architecture (XNOR-RRAM) with a bit-cell design of complementary word lines that implements equivalent XNOR and bit-counting operation in a parallel fashion. For large-scale matrices in fully connected layers or when the convolution kernels are unrolled in multiple channels, the array partition is necessary. Multi-level sense amplifiers (MLSAs) are employed as the intermediate interface for accumulating partial weighted sum. However, a low bit-level MLSA and intrinsic offset of MLSA may degrade the classification accuracy. We investigate the impact of sensing offsets on classification accuracy and analyze various design options with different sub-array sizes and sensing bit-levels. Experimental results with RRAM models and 65nm CMOS PDK show that the system with 128×128 sub-array size and 3-bit MLSA can achieve accuracies of 98.43% for MLP on MNIST and 86.08% for CNN on CIFAR-10, showing 0.34% and 2.39% degradation respectively compared to the accuracies of ideal BNN algorithms. The projected energy-efficiency of XNOR-RRAM is 141.18 TOPS/W, showing ~33X improvement compared to the conventional RRAM synaptic architecture with sequential row-by-row read-out.

We investigate the impact of selector device in cross-point resistive switching memory (RRAM) array on weighted sum operation in the neural network. The requirement of selector devices in a neuromorphic system may be different than that in a conventional memory system. In this work, we developed Verilog-A device models to accurately describe current-voltage (I-V) characteristics of one-selector and one-RRAM (1S-1R) devices obtained experimentally, and then performed the array-level SPICE simulations for weighted sum operation. The weighted sum accuracy is benchmarked as a function of selector non-linearity, array size, and wire resistance. Our results reveal that linearity of the I-V curve in the 1S-1R device with respect to input vector plays an important role in precisely reading-out the weighted sum. Finally, we discuss the desired characteristics of the selector device to be used for the inference stage of the neuromorphic systems.

This is the summary of the special session on reliability and vulnerability of neuromorphic computing systems.

This comprehensive review summarizes state of the art, challenges, and prospects of the neuro-inspired computing with emerging nonvolatile memory devices. First, we discuss the demand for developing neuro-inspired architecture beyond today's von-Neumann architecture. Second, we summarize the various approaches to designing the neuromorphic hardware (digital versus analog, spiking versus nonspiking, online training versus offline training) and discuss why emerging nonvolatile memory is attractive for implementing the synapses in the neural network. Then, we discuss the desired device characteristics of the synaptic devices (e.g., multilevel states, weight update nonlinearity/asymmetry, variation/noise), and survey a few representative material systems and device prototypes reported in the literature that show the analog conductance tuning. These candidates include phase change memory, resistive memory, ferroelectric memory, floating-gate transistors, etc. Next, we introduce the crossbar array architecture to accelerate the weighted sum and weight update operations that are commonly used in the neuro-inspired machine learning algorithms, and review the recent progresses of array-level experimental demonstrations for pattern recognition tasks. In addition, we discuss the peripheral neuron circuit design issues and present a device-circuit-algorithm codesign methodology to evaluate the impact of nonideal device effects on the system-level performance (e.g., learning accuracy). Finally, we give an outlook on the customization of the learning algorithms for efficient hardware implementation.

Neuro-inspired architectures based on synaptic memory arrays have been proposed for on-chip acceleration of weighted sum and weight update in machine/deep learning algorithms. In this paper, we developed NeuroSim, a circuit-level macro model that estimates the area, latency, dynamic energy, and leakage power to facilitate the design space exploration of neuro-inspired architectures with mainstream and emerging device technologies. NeuroSim provides flexible interface and a wide variety of design options at the circuit and device level. Therefore, NeuroSim can be used by neural networks (NNs) as a supporting tool to provide circuit-level performance evaluation. With NeuroSim, an integrated framework can be built with hierarchical organization from the device level (synaptic device properties) to the circuit level (array architectures) and then to the algorithm level (NN topology), enabling instruction-accurate evaluation on the learning accuracy as well as the circuit-level performance metrics at the run-time of online learning. Using multilayer perceptron as a case-study algorithm, we investigated the impact of the “analog” emerging nonvolatile memory (eNVM)'s “nonideal” device properties and benchmarked the tradeoffs between SRAM, digital, and analog eNVM-based architectures for online learning and offline classification.

Memristor-based computation provides a promising solution to boost the power efficiency of the neuromorphic computing system. However, a behavior-level memristor-based neuromorphic computing simulator, which can model the performance and realize an early stage design space exploration, is still missing. In this paper, we propose a simulation platform for the memristor-based neuromorphic system, called MNSIM. A hierarchical structure for memristor-based neuromorphic computing accelerator is proposed to provides flexible interfaces for customization. A detailed reference design is provided for large-scale applications. A behavior-level computing accuracy model is incorporated to evaluate the computing error rate affected by interconnect lines and nonideal device factors. Experimental results show that MNSIM achieves over 7000 times speed-up than SPICE simulation. MNSIM can optimize the design and estimate the tradeoff relationships among different performance metrics for users.

With the speed and power bottleneck in the conventional Von Neumann architecture, the interest in the neuromorphic systems has greatly increased in recent years. To create a highly dense communication network between the pre- and post-neurons, RRAM devices are used as synapses in the neuromorphic systems due to many advantages including their small sizes and low-power operations. However, due to RRAM reliability issues, in particular soft-errors, the performance of the RRAM-based neuromorphic systems are significantly degraded. In this article, we propose a novel framework for detecting and resolving the degradation in the system performance due to the RRAM reliability soft-errors. The read and write circuits modifications to implement the framework, and their impact on the delay and energy consumption of the neuromorphic system are also discussed in this article. Using a combination of BRIAN and SPICE simulations, we demonstrate that the proposed framework can restore the accuracy of the example RRAM-based neuromorphic system from 43% back to its target value of 91.6% with a minimal impact on the read (< 0.1% and 1.1% increase in the delay and energy respectively) and write (0% and < 0.1% increase in the delay and energy respectively) operations.

In this paper, we propose a novel design of strong Physical Unclonable Function (PUF) leveraging the sneak paths in the resistive cross-point (X-point) array for device authentication. The entanglement of the sneak paths in the X-point array greatly enhances the entropy of the physical system, thereby increasing the space of challenge-response pairs (CRPs). “Digital” resistance distribution in X-point array is employed to eliminate the undesired collision or diffuseness when “analog” resistance distribution is used. The X-point PUF's characteristics are evaluated on the 128×128 cross-point arrays by SPICE simulation. The simulation results show that the on-state resistance (Ron) activity affects the PUF's performance and 4% is shown as an optimized design in terms of large reference current (Iref) tolerance range. In addition, the X-point PUF presents no correlation between challenges and responses, showing improved security.

In this paper, we study the 1-selector1-resistor (1S1R) cross-point resistive random access memory (ReRAM) array because of its high density, fast access time, and ultralow stand-by power. Specifically, we focus on an access scheme where a data line is parallelly accessed from multiple subarrays with multibits accessed per subarray. A direct implementation of such a scheme has high energy efficiency but lower reliability compared with a single bit per subarray baseline scheme. So this paper proposes a low cost multilayer approach to improve energy-efficiency of multibits per access scheme without compromising reliability. At the cell level, we show how proper choices of bit-line and source-line voltage and SET recovery help reduce error rate by ten times. At the system level, we propose a new rotated multiarray access scheme where the average error rate of every accessed data line is one order of magnitude lower than the worst case, making it possible to achieve block failure rate of 10-10 with a simple Bose, Chaudhuri, and Hocquenghem t = 4 code. We show that for a 1 GB 1S1R ReRAM, the proposed approach can reduce energy by 41% with 2% extra area while maintaining latency and reliability compared with the baseline system.

For Internet of Things (IoT) edge devices, it is very attractive to have the local sensemaking capability instead of sending all the data back to the cloud for information processing. For image pattern recognition, neuro-inspired machine learning algorithms have demonstrated enormous powerfulness. To effectively implement learning algorithms on-chip for IoT edge devices, on-chip synaptic memory architectures have been proposed to implement the key operations such as weighted-sum or matrix-vector multiplication. In this paper, we proposed a low-power design of static random access memory (SRAM) synaptic array for implementing a low-precision ternary neural network. We experimentally demonstrated that the supply voltage (VDD) of the SRAM array could be aggressively reduced to a level, where the SRAM cell is susceptible to bit failures. The testing results from 65-nm SRAM chips indicate that VDD could be reduced from the nominal 1-0.55 V (or 0.5 V) with a bit error rate ~0.23% (or ~1.56%), which only introduced ~0.08% (or ~1.68%) degradation in the classification accuracy. As a result, the power consumption could be reduced by more than 8× (or 10×).

Resistive RAM (RRAM) device has been extensively used as a scalable nonvolatile memory cell in neuromorphic systems due to its several advantages, including its small size and low-power requirements. However, resulting from the stochastic nature of the oxygen vacancies, the RRAM device suffers from reliability soft errors. In this paper, we provide for the first time a modeling framework to compute the effect of those soft errors on the system accuracy. Applying the proposed technique on a case-study system used to recognize the MNIST data set, our simulation results show that the system accuracy can degrade from 91.6% to 43% due to the RRAM reliability soft errors. To overcome this loss in the system performance, various possible adjustments to the parameters of the neuron pulses are analyzed. Furthermore, in this paper, two methodologies are proposed for automatically detecting and fixing the degradation in the system accuracy caused by the RRAM reliability soft errors. Using the suggested methodologies, the system accuracy of our case-study system can be restored back from 43% to 91.6% with small increase in the training cycle duration and with as small as 0.1% increment in the energy consumption of the system.

Inherent stochastic physical mechanisms in emerging nonvolatile memories (NVMs), such as resistive random-access-memory (RRAM), have recently been explored for hardware security applications. Unlike the conventional silicon Physical Unclonable Functions (PUFs) that are solely based on manufacturing process variation, RRAM has some intrinsic randomness in its physical mechanisms that can be utilized as entropy sources; for instance, resistance variation, random telegraph noise, and probabilistic switching behaviors. This paper reviews the challenges and opportunities in building security primitives with emerging devices. In particular, it presents research progress of RRAM-based hardware security primitives, including PUF and True Random Number Generator (TRNG).

The phenomenon of metal-insulator-transition (MIT) in strongly correlated oxides, such as NbO2, have shown the oscillation behavior in recent experiments. In this work, the MIT based two-terminal device is proposed as a compact oscillation neuron for the parallel read operation from the resistive synaptic array. The weighted sum is represented by the frequency of the oscillation neuron. Compared to the complex CMOS integrate-and-fire neuron with tens of transistors, the oscillation neuron achieves significant area reduction, thereby alleviating the column pitch matching problem of the peripheral circuitry in resistive memories. Firstly, the impact of MIT device characteristics on the weighted sum accuracy is investigated when the oscillation neuron is connected to a single resistive synaptic device. Secondly, the array-level performance is explored when the oscillation neurons are connected to the resistive synaptic array. To address the interference of oscillation between columns in simple cross-point arrays, a 2-transistor-1-resistor (2T1R) array architecture is proposed at negligible increase in array area. Finally, the circuit-level benchmark of the proposed oscillation neuron with the CMOS neuron is performed. At single neuron node level, oscillation neuron shows >12.5× reduction of area. At 128×128 array level, oscillation neuron shows a reduction of ∼4% total area, >30% latency, ∼5× energy and ∼40× leakage power, demonstrating its advantage of being integrated into the resistive synaptic array for neuro-inspired computing.

Memory array architectures have been proposed for on-chip acceleration of weighted sum and weight update in the neuro-inspired machine learning algorithms. As the learning algorithms usually operate on a large weight matrix size, an efficient mapping of a large weight matrix on the hardware accelerator may require partitioning the matrix into multiple sub-arrays. In this work, we built a circuit-level macro simulator to evaluate the performance of partitioning a 512×512 weight matrix into the SRAM and RRAM based accelerators. Generally, with more partitioning and finer granularity of the array architecture, the read/write latency and the dynamic read/write energy will decrease due to an increased computation parallelism at the expense of larger area and leakage power, as shown in the case of the SRAM accelerator. However, the RRAM accelerator does not improve the read latency and read energy beyond a certain partition point because the overhead due to multiple intermediate stages of adders and registers will dominate.

Physical Unclonable Functions (PUF) have to be highly reliable especially when it is being used along with cryptographic hash modules for key generation. To achieve ultrahigh reliability, the conventional approach employs error correction codes (ECC) based on helper data input. Such an approach not only increases the hardware overhead of the PUF but also reduces the entropy of the system, resulting in both hardware and software security issues. In this paper we design a compact and highly reliable PUF architecture based on resistive random access memory (RRAM). We propose a new design where the sum of the read-out currents of multiple RRAM cells is used for generating one response bit. This method statistically minimizes any early-lifetime failure due to RRAM retention degradation at high temperature or under voltage stress. We employ a device model that is calibrated with IMEC HfOx RRAM experimental data and show that with 8 cells per bit, we can ensure99.9999% reliability) for a lifetime >10 years at 125°C. We embed the RRAM PUF into SHA-256 and show that the hardware overhead of the proposed RRAM PUF based architecture is significantly lower than one that uses a traditional RRAM PUF with ECC.

The 3-D integration of resistive switching random access memory (RRAM) array is attractive for low-cost and high-density nonvolatile memory application. In this paper, the design tradeoffs of select transistor drivability, RRAM device characteristics, such as switching current (IW), ON/OFF-state resistance (RON/ROFF), and I-V nonlinearity ratio, interconnect material, and write/read scheme are systematically analyzed using a 3-D circuit simulation. The simulation results show that insufficient current drivability of the vertical transistor severely limits the number of 3-D layers. A low switching current (high RON) or a high nonlinearity is beneficial for improving the write margin, while it degrades the read current sense margin. To alleviate this conflict, the read voltage needs to be boosted to the half write voltage. The common RRAM electrode material TiN is not suitable for the interconnect material due to a high resistivity. To improve write energy efficiency, a multiple-bit write scheme is proposed to reduce the write energy consumption per bit and enable a high bandwidth. With RON= 500 kΩ (IW= 6 μA) and nonlinearity ratio = 10, 1-Mb 3-D vertical RRAM subarray is feasible to meet the specified write/read margin with ~2-pJ/bit energy consumption.

The matrix-vector multiplication is the key operation for many computationally intensive algorithms. In recent years, the emerging metal oxide resistive switching random access memory (RRAM) device and RRAM crossbar array have demonstrated a promising hardware realization of the analog matrix-vector multiplication with ultra-high energy efficiency. In this paper, we analyze the impact of nonlinear voltage-current relationship of RRAM devices and the interconnect resistance as well as other crossbar array parameters on the circuit performance and present a design guide. On top of that, we propose a technological exploration flow for device parameter configuration to overcome the impact of nonideal factors and achieve a better trade-off among performance, energy and reliability for each specific application. The simulation results of a support vector machine (SVM) and MNIST pattern recognition dataset show that the RRAM crossbar array-based SVM is robust to the input signal fluctuation but sensitive to the tunneling gap deviation. A further resistance resolution test presents that a 4-bit RRAM device is able to realize a recognition accuracy of ∼ 90%, indicating the physical feasibility of RRAM crossbar array-based SVM. In addition, the proposed technological exploration flow is able to achieve 10.98% improvement of recognition accuracy on the MNIST dataset and 26.4% energy savings compared with previous work.

Unsupervised learning with sparse coding is widely adopted in applications of feature extraction, pattern classification, and compressive sensing. However, even with the state-of-the-art hardware platform of CPUs/GPUs, solving a sparse coding problem is still expensive in computation. In this paper, the resistive cross-point array architecture (CPA) is proposed to achieve on-chip acceleration of sparse coding, especially the matrix/vector operations that are intensively used in the algorithm. Learning and recognition experiments are conducted with the MNIST handwriting dataset. By co-optimizing the algorithm, architecture, circuit, and resistive synaptic devices, SPICE simulation at 65nm demonstrates that the CPA is able to accelerate sparse coding computation by more than 3800X, compared to software running on an 8-core CPU. Furthermore, this work investigates the technological limitations of a realistic resistive CPA, including reduced ON/OFF range of synaptic devices, nonlinearity in programming, spatial and temporal variations, and interconnect parasitics. The results illustrate both enormous opportunities and practical barriers of resistive CPA in real-time learning on a chip.

The scalability of DRAM faces challenges from increasing power consumption and the difficulty of building high aspect ratio capacitors. Consequently, emerging memory technologies including Phase Change Memory (PCM), Spin-Transfer Torque RAM (STT-RAM), and Resistive RAM (ReRAM) are being actively pursued as replacements for DRAM memory. Among these candidates, ReRAM has superior characteristics such as high density, low write energy, and high endurance, making it a very attractive cost-efficient alternative to DRAM. In this paper, we present a comprehensive study of ReRAM-based memory systems. ReRAM's high density comes from its unique crossbar architecture where some peripheral circuits are laid below multiple layers of ReRAM cells. A crossbar architecture introduces special constraints on operating voltages, write latency, and array size. The access latency of a crossbar is a function of the data patterns involved in a write operation. These combined with ReRAM's exponential relationship between its write voltage and switching latency provide opportunities for architectural optimizations. This paper makes several key contributions. First, we study the crossbar architecture and describe trade-offs involving voltage drop, write latency, and data pattern. We then analyze microarchitectural enhancements such as double-sided ground biasing and multiphase reset operations to improve write performance. At the architecture level, a simple compression based data encoding scheme is proposed to further bring down the latency. As the compressibility of a block varies based on its content, write latency is not uniform across blocks. To mitigate the impact of slow writes on performance, we propose and evaluate a novel scheduling policy that makes writing decisions based on latency and activity of a bank. The experimental results show that our architecture improves the performance of a system using ReRAM-based main memory by about 44% over a conservative baseline and 14% over an aggressive baseline on average, and has less than 10% performance degradation compared to an ideal DRAM-only system.

The cross-point array architecture with resistive synaptic devices has been proposed for on-chip implementation of weighted sum and weight update in the training process of learning algorithms. However, the non-ideal properties of the synaptic devices available today, such as the nonlinearity in weight update, limited ON/OFF range and device variations, can potentially hamper the learning accuracy. This paper focuses on the impact of these realistic properties on the learning accuracy and proposes the mitigation strategies. Unsupervised sparse coding is selected as a case study algorithm. With the calibration of the realistic synaptic behavior from the measured experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ∜97 % to ∜65 %. To mitigate this accuracy loss, the proposed strategies include 1) the smart programming schemes for achieving linear weight update; 2) a dummy column to eliminate the off-state current; 3) the use of multiple cells for each weight element to alleviate the impact of device variations. With the improved synaptic behavior by these strategies, the accuracy increases back to ∜95 %, enabling the reliable integration of realistic synaptic devices in the neuromorphic systems.

Resistive RAM (ReRAM) has fast access time, ultra-low stand-by power and high reliability, making it a viable memory technology to replace DRAM for main memory. The 1-transistor-1-resistor (1T1R) ReRAM array has density comparable to that of a DRAM array and the advantages of lower programming energy and higher reliability compared to the ultrahigh density ReRAM cross-point array. In this paper, we show how circuit operation parameters, such as the pulse amplitude and pulse widths of word-line (WL) voltage, bit-line (BL) voltage, and source-line (SL) voltage can be used to lower latency, lower power and improve reliability. SPICE simulation results demonstrate that appropriate choice of voltage settings can be used to reduce the write latency of the 1T1R cell by 29.4% and reduce write energy by 46.7% over the DRAM cell. Next, we show how the endurance of ReRAM cell can be improved by increasing the ratio between OFF and ON resistances and reducing SL voltage. We find that of these, reducing the SL voltage results in significant improvement in endurance with smaller energy overhead. Next, we evaluate the system-level performance of a 1GB ReRAM and DRAM memory system using CACTI and GEM5. Simulation results using SPEC CPU INT 2006 and DaCapo-9.12 benchmarks show that the ReRAM based main memory can improve IPC by 4.2% and energy by up to 77.8% compared to a DRAM system.

The resistive random access memory (RRAM) technology is a promising candidate for the replacement of NAND FLASH at ultra-scaled nodes. In this work, the scalability of a 2-layer RRAM cross-point array architecture is explored towards the 10 nm node. Device-circuit co-design methodologies are employed to optimize the array architecture. The impact of sneak paths, IR drop along the wire interconnect and RRAM device characteristics are investigated by HSPICE simulations and the memory array organization and partition are optimized by NVSim simulations. It is found that increasing the I-V nonlinearity of a memory cell by adding a cell selector helps maintain the write voltage margin at scaled nodes. With the scaling, the maximum sub-array size decreases due to the IR drop constraint thus the memory architecture evolves to a finer granularity. At the 10 nm node, by hiding a portion of the peripheral circuitry underneath the memory array, a 2-layer RRAM bank is projected to achieve ultra-high density ~3.43 Gb/mm2, and can enable fast write bandwith ~ 300 MB/s and read bandwidth ~1 GB/s.

Resistive random access memory (ReRAM) technology is an emerging candidate for next-generation nonvolatile memory (NVM) architecture due to its simple structure, low programming voltage, fast switching speed, high on/off ratio, excellent scalability, good endurance, and great compatibility with silicon CMOS technology. The most attractive of the characteristics of ReRAM is its cross-point structure, which features a 4 F 2 cell size. In a cross-point structure, the existence of sneak current and resulting voltage loss due to the wire's resistance might cause read and write failures if not designed properly. In addition, a robust ReRAM design needs to deal with both soft and hard errors. In this article, we summarize mechanisms of both soft and hard errors of ReRAM cells and propose a unified model to characterize different failure behaviors. We quantitatively analyze the impact of cell failure types on the reliability of the cross-point array. We also propose an error-resilient architecture, which avoids unnecessary writes in the hard error detection unit. Assuming constant soft error rate, our approach can extend the lifetime of ReRAM up to 75% over a design without hard error detection and up to 12% over the design with a “write-verify” detection mechanism. Our approach yields greater significant lifetime improvement when considering postcycling retention degradation.

Resistive Random Access Memory (ReRAM) is one of the most promising emerging non-volatile memory (NVM) candidates due to its fast read/write speed, excellent scalability and low-power operation. Recently proposed 3D vertical cross-point ReRAM (3D-VRAM) architecture attracts a lot of attention because it offers a cost-competitive solution as NAND Flash replacement. In this work, we first develop an array-level model which includes the geometries and properties of all the components in the 3D structure. The model is capable of analyzing the read/write noise margin of a 3D-VRAM array in the presence of the sneak leakage current and voltage drop. Then we build a system-level design tool that is able to explore the design space with specified constraints and find the optimal design points with different targets. We also study the impact of different design parameters on the array size, bit density, and overall cost-per-bit. Compared to the state-of-the-art 3D horizontal ReRAM (3D-HRAM), the 3D-VRAM shows great cost advantage when stacking more than 16 layers.

The transition metal oxide (TMO) resistive random access memory (ReRAM) has been identified as one of the most promising candidates for the next generation non-volatile memory (NVM) technology. Numerous TMO ReRAMs with different materials have been developed and demonstrate attractive characteristics, such as fast read/write speed, low power consumption, high integrated density, and good scalability. Among them, the most attractive characteristic of ReRAM is its cross-point structure which features a 4F2 cell size. However, the existence of sneak current and voltage drop along the wire resistance in a cross-point array brings in extra design challenges. In addition, a robust ReRAM design needs to deal with both soft and hard errors. In this paper, we summarize mechanisms of both soft and hard errors of ReRAM cells and propose a unified model to characterize different failure behaviors. We quantitatively analyze the impact of cell failure modes on the reliability of cross-point array. We also propose an error resilient architecture which avoids unnecessary writes in the hard error detection unit. Experimental results show that our design can extend the lifetime of ReRAM up to 75% over the design without hard error detections and up to 12% over the design with "write-verify" detection mechanism.

Resistive Random Access Memory (ReRAM) has several advantages over current NAND Flash technology, highlighting orders of magnitude lower access latency and higher endurance. Recently proposed 3D vertical cross-point ReRAM (3D-VRAM) architecture is an encouraging development in ReRAM's evolution as a cost-competitive solution, and thus attracts a lot of attention in both industry and academia. In this work, an array-level model to estimate the read/write energy and characterize the vertical access transistor is developed. We use the model to study a range of design trade-offs by tuning the cell-level characteristics and the read/write schemes. The design space exploration addresses several critical issues that are either unique to 3D-VRAM or have substantially different concerns from the 2D cross-point array design. It provides insights on the design optimizations of the array density and access energy, and several important conclusions have been reached. Then we propose multi-directional write driver to mitigate the writer circuitry overhead, and use remote sensing scheme to take full advantage of limited on-die sensing resources. The benefits of these optimizations are evaluated and validated in our macro-architecture model. With trace-based simulations, system-level comparisons between 3D-VRAM and a wide spectrum of memories are performed in mixed aspects of performance, cost, and energy. The results show that our optimized 3D-VRAM design are better than other contenders for storage memory in both performance and energy.

Stacked HfOxbased vertical RRAM with interface engineering for 3D cross-point architecture is fabricated using a cost-effective fabrication process. The excellent performances such as low reset current, fast switching speed, high switching endurance and disturbance immunity, good retention and self-selectivity are demonstrated in the fabricated HfOxbased vertical RRAM devices. The scaling limit and the functionality along with a viable write/read scheme of the presented vertical RRAM are investigated. The experiments show that the pillar electrode thickness and the plane electrode thickness of the vertical RRAM can be scaled down to 3nm and 5nm without significant performance degradation, respectively.

An emerging application for the oxide based resistive random access memory (RRAM) technology is to serve as the synaptic device for the hardware implementation of neuromorphic computing. The gradual resistance modulation capability in RRAM is proposed for emulating analog synapses, and the stochastic switching behavior in RRAM is proposed for emulating binary synapses. In order to evaluate the effectiveness of analog synapses and binary synapses in realizing the competitive learning algorithm, a simulation of winner-take-all network is performed based on the parameters extracted from the experiments. The simulation suggests that the orientation classification can be effectively realized using both analog synapses and binary synapses.

In this paper, the recent progress on the understandings of the switching mechanisms in oxide resistive switching memory (RRAM) is reviewed. Several representative device modeling approaches including numerical discretized models, numerical continuous models and analytical compact models are discussed using HfOxbipolar RRAM as a model system. The future challenges of RRAM modeling are finally discussed.

Design guidelines were proposed to evaluate and optimize the 3D RRAM cross-point architecture by a full-size 3D circuit simulation in SPICE. The performance metrics that were evaluated include the write/read margin, access latency, energy consumption per programming, and the density per bit. Different 3D cross-point architecture including the horizontally stacked or the vertically stacked structure were compared in terms of these metrics, revealing the advantages of the vertical RRAM structure. Then the scaling trend of the vertical RRAM based 3D array with respect to the scaling of lateral feature size, vertical electrode thickness and vertical isolation layer thickness were evaluated. The design parameters that affect the scaling trend include the metal interconnect resistance, RRAM on-state cell resistance (or the nonlinearity of the I-V). The design trade-offs are discussed considering those parameters constraints.

Hardware implementation of neuromorphic computing is attractive as a computing paradigm beyond the conventional digital Boolean computing. Recently, two-terminal emerging memory devices that show electrically-triggered resistance modulation have been proposed as synaptic devices for neuromorphic computing. The synaptic device candidates include phase change memory (PCM), resistive RAM (RRAM) and conductive bridge RAM (CBRAM), etc. In this paper, we discuss the general design considerations of synaptic devices for plasticity and learning. As a rule of thumb for performance metrics assessment, an ideal synaptic device should have characteristics such as dimension, energy consumption, operation frequency, dynamic range, etc. that are scalable to biological systems with comparable complexity.

In this paper, recent progress of binary metal–oxide resistive switching random access memory (RRAM) is reviewed. The physical mechanism, material properties, and electrical characteristics of a variety of binary metal–oxide RRAM are discussed, with a focus on the use of RRAM for nonvolatile memory application. A review of recent development of large-scale RRAM arrays is given. Issues such as uniformity, endurance, retention, multibit operation, and scaling trends are discussed.