This paper presents an ultra-low-energy keyword spotting (KWS) processor based on the charge-mode multi-level resistive random-access memory (ReRAM) bitcell and adaptive analog-to-digital converter (ADC) quantization. Previous ReRAM computing-in-memory (CIM) architectures have suffered several challenges, including excessive computing energy due to direct current branches, computation non-linearity, and throughput degradation resulting from analog-to-digital conversion. The proposed processor addresses these challenges by introducing a charge-mode multi-bit ReRAM bitcell (CRB), which achieves a 61.39% reduction in multiply-and-accumulate (MAC) energy. The CRB also enhances MAC linearity by$1.65\times $. Additionally, the 5-bit additive powers-of-two ADC achieves a 65.42% reduction in analog-to-digital conversion energy, an 11.70 percentage points decrease in ADC accuracy loss, and a$1.33\times $increase in conversion throughput. Furthermore, the pipelined layer fusion clusters reduce intermediate data movement energy to 97.2 nJ and enhance KWS system throughput by$1.35\times $. The proposed processor is designed utilizing 45 nm CMOS technology and compatible ReRAM devices. The processor occupies an area of 0.94 mm2with a 68.5 KB ReRAM cell. The processor also achieves$0.74~\mu $J/decision energy consumption, 22.59 TOPS/W energy efficiency, and 92.7% accuracy on the Google Speech Commands Dataset.

Large language models (LLMs) based on transformer architecture have significantly advanced various AI applications. However, the multihead self-attention (MHSA) in a transformer requires extensive computation and memory bandwidth. Techniques such as sparse attention and attention formula reordering have been explored recently to mitigate these challenges. However, existing processors still struggle to process LLMs efficiently. To overcome these limitations, an energy-efficient compute-in/near-memory (CINM) embedded dynamic random access memory (eDRAM) processor CINELL is proposed, incorporating the following three key features: 1) an attention block fusion computation (ABFC), which maximizes data reuse in the attention map, reduces external memory access (EMA) by 85.86%, and improves hardware utilization to 86.1%; 2) a CINM architecture designed to address the imbalance between memory and computation, while the heterogeneous pipeline reduces the system latency by 77.27%; and 3) a compute-in-memory (CIM) array that supports the cross-read operation (CRO) addresses data direction conflicts, achieving a 98.44% reduction in latency, while dual-row computation (DRC) with reduced adder logic (RAL) improves energy efficiency by$1.58\times $. Designed in 28-nm CMOS technology, the CINELL processor achieves 36.28–58.05 TOPS/W and an F1-score of 92.41% on the SQuAD 1.1v using the BigBird-large model.

A highly energy-efficient embedded DRAM (eDRAM)-based heterogeneous-in/near-memory computing (H-INMC) processor for large language model with extended sparse attention is proposed. The proposed H-INMC processor reduces external memory access (EMA) and enhances macro/system energy efficiency through three key features: 1) attention block fusion computation strategy to maximize input and intermediate data reuse, achieving 85.86% EMA reduction; 2) H-INMC architecture for addressing the imbalances in memory and computation intensity across different computation stages, reducing 77.27% of system latency; and 3) cross-read 3T1C bitcell architecture for mitigating read/write datapath conflictions between CIMs and enhancing the energy efficiency up to 46 TOPS/W with dual-row in-memory-computation. Designed with 28 nm CMOS technology, the proposed H-INMC processor achieves a 92.41% F1 score, evaluated under the Bigbird-large model on the SQuAD 1.1v dataset.

An ultra-low energy keyword spotting (KWS) processor based on multi-level ReRAM computing-in-memory (CIM) architecture is proposed in this paper. Previous ReRAM-CIMs suffered from several challenges, such as considerable computation energy due to voltage-based bitcell computation, significant energy consumption and throughput overhead from analog-to-digital (A-to-D) conversion. The proposed processor addresses these challenges by introducing charge-based multi-bit computation ReRAM bitcell, which achieves a 61.39% reduction in MAC computation energy and enhances MAC linearity by 1.65×. Additionally, the proposed 5-bit additive powers-of-two nonlinear ADC leads to a 65.42% reduction in A-to-D conversion energy, a 93.57% decrease in ADC quantization error, and a 1.33× increase in ADC throughput. Furthermore, the pipelined layer fusion clusters reduce the energy consumption for data movement to below 1% and improve the system throughput by 1.35× during keyword spotting inference. The proposed ReRAM-CIM processor is designed in 45 nm technology and occupies a 0.94 mm2area with a 68.5 KB ReRAM cell. The proposed processor also achieves 0.74 μJ/decision energy consumption with 92.7% accuracy on the Google Speech Commands dataset.

Artificial neural networks have led to a higher computational burden, complicating inference tasks on low-power edge devices. Spiking neural network (SNN), which leverages sparse spikes for computation and data transmission, is an effective energy-efficient computing technique. However, the length of spike sequences in SNN varies significantly depending on the input coding method, among which rate coding still results in substantial data movement. A highly energy-efficient SNN accelerator with a time-domain CIM processor is proposed with three key features: 1) time-domain bitcell array for high linearity with lower energy, reducing 58.6% power consumption compared to inverter-chain architecture, 2) time-domain multi-bit accumulate for assisting multi-bit weights without analog-to-digital converter, achieving 47.2% energy reduction of domain-conversion energy, 3) analog precision reconstruction unit for supporting phase coding. The proposed TS-CIM is designed in 65 nm CMOS technology and achieves 701.7 TOPS/W energy efficiency, marking a$1.58\times $enhancement compared to the state-of-the-art SNN CIM.

This paper presents a content addressable memory (CAM)-based computing-in-memory (C$^{2}$IM) system designed for energy-efficient k-nearest neighbor (k-NN) searching in 3D point clouds. For autonomous driving applications, an essential process for perceiving the mobile robot’s movements in 3D space is k-NN searching. Especially with the limited hardware resources of mobile processors, the 3D point cloud is too large to upload onto the chip, leading to$O(N^{2})$of external memory accesses and distance calculations. The proposed C$^{2}$IM processor enhances energy efficiency and reduces power consumption through three key features: 1) Dilated 1D-CNN prediction enables voxel-based partitioning, reducing the external memory accesses from$O(N^{2})$to$O(N)$; 2) Vertex clustering reorganizes groups of points into evenly distributed clusters based on the underlying data distribution and reduces the number of points of comparisons by 49.8%; and 3) In-memory k-NN searching with CAM achieves high system energy efficiency while minimizing data transactions between memory and computation logic. Designed with 28 nm CMOS technology, the proposed C$^{2}$IM achieves up to 23.08$\times$energy efficiency, and 48.4% reduction in memory footprint compared to previous ASIC accelerators, and a 99.51% reduction in power consumption compared to state-of-the-art processor implemented in FPGA with high-bandwidth memory.

Intelligent 3D Interaction with Wide & Dynamic Surroundings

An energy-efficient, unified convolutional neural network (CNN) accelerator is proposed with a lightweight RGB-D network to achieve real-time, multi-object semantic segmentation in autonomous electric vehicle system. First, a lightweight Depth-fused Trilateral Network (DTN) is proposed to achieve high accuracy and real-time operation for road and multi-object segmentation at the same time. Optimized with various types of convolution layers and limited hardware resources, the DTN achieves 94.73% accuracy on KITTI Road dataset. Second, the unified CNN processor is designed with dual-mode shift-register-based input reconfiguration units and layer fusion architecture with 2-types of processing elements for depth-wise separable convolution (DSC) to support 5 different types of convolution layers including standard convolution, dilated convolution, transposed convolution, point-wise convolution, and DSC. With flexible architecture, it achieves 17.97$\times$higher throughput with DTN and DSC layer fusion architecture reduces 34.7% of overall external memory access. Implemented with 28nm CMOS technology, the unified CNN processor shows 43.6 mW power consumption and 4.94 TOPS/W energy efficiency. As a result, the proposed system with DTN realizes 40.07 frames-per-second (fps) throughputs in multi-object semantic segmentation application with high resolution driving scenes dataset.

A sparsity-aware 3D-convolution neural network (3D-CNN) accelerator is proposed for the real-time mobile hand gesture recognition (HGR) system. The complex computation of 3D-convolution with the video data makes it difficult for real-time operation, especially in a resource-constrained mobile platform. To facilitate real-time implementation of HGR, this paper proposes three key features: 1) Spatio-temporal Variation Encoding and Inter-frame Differential Aware Network for highly sparse and lightweight network, reducing 94.03% parameters with only 2.57% accuracy loss on NvGesture dataset; 2) the ROI-only Computation architecture for utilizing activation sparsity to reduce the number of MAC operations and the external memory bandwidth by 84.3% and 72.3%, respectively; 3) Weight Sparsity-aware PE and Sparsity-distribution-aware Workload Allocation speed up the inference by$19.8\times $. As a result, the low-latency 3D-CNN accelerator utilizes both activation and weight sparsity with data mapping to maximize the reusability of 3D-CNN, achieving$31\times $faster inference than the state-of-the-art. The proposed processor is designed in 65 nm CMOS technology. It consumes 35 mW of power and achieves 46.25 TOPS/W of energy efficiency. As a result, the system realized 1.584 ms inference latency for real-time HGR in a mobile platform.

An energy-efficient convolutional neural network (CNN) accelerator is proposed for real-time segmentation in autonomous electric vehicle (AEV) system. The computation of semantic segmentation with high-resolution images makes it difficult for real-time operation in time-critical and resource-constrained AEV. To facilitate real-time implementation in AEV, this paper proposes two key features: 1) A compressed multi-object Depth-fused Trilateral Network (DTN) with dilated convolution and depthwise separable convolution that reduces 90% of the overall computation of baseline [1] and achieves 94.73% accuracy on KITTI Road dataset; 2) An energy-efficient CNN accelerator, which supports 5 types of CONV’s, achieving 1.33× higher throughput than the previous processor [2]. Finally, the proposed processor is designed in 28 nm CMOS technology. It consumes 65.7 mW of power and achieves 2.91 TOPS/W of energy efficiency. As a result, the system realizes 72.2 and 37 frames-per-second of semantic segmentation for road and multi-objects with high resolution.