Owing to mature fabrication support and high flexibility, static random-access memory (SRAM) has become a very promising candidate for compute-in-memory (CiM) in accelerating deep neural networks (DNNs). However, SRAM-based CiM has low memory density and thus very limited total on-chip capacity, resulting in frequent weights reloading and additional power consumption during end-to-end inference tasks. Analog ROM CiM increases memory density but suffers from low computing density caused by A/D converter (ADC) limitation. To address these challenges, for the first time, a fully digital compute-in-read-only-memory (DCiROM) design approach is proposed in this paper. DCiROM introduces a novel ROM-logic fusion CiM that successfully reduces CiM area by 51% while maintaining high memory density and computing performance. By reusing multiply-and-accumulation (MAC) resources, DCiROM further achieves flexibility with a minimal area cost. We have implemented a DCiROM chip loaded 3024Kb ResNet-56 parameters using 65nm CMOS technology. This macro achieves 10.2x-55.7x higher normalized FoM (memory density x computing density) than the state-of-the-art CiM works. It also reduces 2.9x-9.9x energy consumption per image inference than SRAM CiM works when considering off-chip access.

Compute-In-Memory (CiM) is an emerging architecture designed to address the memory wall issue in deep neural network (DNN) inference. However, both the ADC in analog CiM (ACiM) and the adder trees in digital CiM (DCiM) contribute to significant energy and area overhead. In response to these challenges, binary neural networks (BNNs) have been proposed recently. Nevertheless, accuracy degradation poses a serious challenge to the application of BNNs in CiM due to errors in partial-sum accumulations. Furthermore, post-processing steps involving binary activation, such as ReLU, scaling, and bias addition, introduce redundant computing that cannot be effectively optimized by BNN-CiM.This work proposes a novel software-hardware co-optimization approach aimed at enabling an ADC-free analog CiM design while maintaining accuracy. Multi-Level binary computing fusion techniques comprising redundant load isolation based row fusion, in-array parallelism adaption based block fusion, and high-precision post-process elimination based layer fusion address the serious accuracy issues associated with conventional BNN algorithms. In contrast with past over 10% accuracy lost BNN-CiM on practical dataset CIFAR-10 and ImageNet, this work achieves more than 2.2x energy efficiency and 7.4x memory density than state-of-the-art with only 2% accuracy loss.

The concept of multi-level cell (MLC) enabled by emerging memory device technologies has introduced new opportunities for memory density improvement, including in the cache scenarios with some high-endurance technologies. However, the access latency of different bits within an MLC memory cell is inherently nonuniform, which raises challenges in utilizing the MLC technology for low-latency cache. To exploit the access performance of the MLC cache, the key is identifying the hot data blocks and mapping them to fast MLC bits. Prior works perform the hot/cold data management based on block-wise access patterns with considerable hardware overheads. Inspired by the memory hierarchy, this work proposes a new concept of in-cell hierarchical victim cache as embedded memory and systematically presents the cache architecture, operating mechanism, design space exploration, optimizations, and evaluations. By utilizing the slow bits as the victim buffer, lower hit latency with low implementation overheads is achieved. Based on the in-cell victim cache, two optimization techniques, namely selective victim retrieval, and victim-bypassing write (VBW) are proposed, to further improve performance and prolong cache endurance, respectively. Evaluation results show that the MLC victim cache significantly improves the average system performance by 20.2% over conventional MLC cache and achieves 98% performance of the upper bound implemented with 2x memory cells SLC. The proposed VBW also reduces energy consumption by 21% and improves lifetime by over 80%, showing a new promising dimension for future MLC cache design.