Compound AI systems showcase impressive performance and versatility compared to single AI models by combining large language models (LLMs) with various smaller expert models. The major bottleneck of compound AI lies in frequent data movement due to the massive parameters and dynamic routing mechanisms. Compute-In-Memory (CiM) has demonstrated great potential to mitigate the memory wall. However, constrained by the rigid array structure, existing CiM accelerators struggle to meet more general and diverse model compression demands of compound AI, such as fine-grained pruning for expert models and outlier-aware quantization for LLM-based router models. The lack of support for agile model compression hinders the deployment of compound AI systems on CiM accelerators.To fully unlock the potential of CiM in accelerating compound AI, we present DANCE, a dual-side N:M sparse compressed digital CiM architecture with cross-layer co-optimizations: (i) At the circuit level, DANCE introduces a customized set-associative selection circuit to extract N:M sparse patterns for both weights and activations, maintaining high parallelism; (ii) At the architecture level, DANCE explores a novel design paradigm that integrates fine-grained pruning and outlier-aware quantization into a unified N:M sparsity compression framework. Experimental results show that DANCE achieves up to 4.36× energy efficiency improvement with <1% accuracy loss for ResNet-18 on CIFAR-100, and up to 2.59× energy efficiency improvement with <0.5 perplexity increase for Llama-7B on WikiText-2, compared to the conventional digital CiM baseline.

Deploying a lightweight quantized model in compute-in-memory (CIM) might result in significant accuracy degradation due to reduced signal-noise rate (SNR). To address this issue, this paper presents ZEBRA, a zero-bit robust-accumulation CIM approach, which utilizes bitwise zero patterns to compress computation with ultra-high resilience against noise due to circuit non-idealities, etc. First, ZEBRA provides a cross-level design that successfully exploits value-adaptive zero-bit patterns to improve the performance in robust 8-bit quantization dramatically. Second, ZEBRA presents a multi-level local computing unit circuit design to implement the bitwise sparsity pattern, which boosts the area/energy efficiency by 2x-4x compared with existing CIM works. Experiments demonstrate that ZEBRA can achieve10% accuracy loss. Such robustness leads to much more stable accuracy for high-parallelism inference on large models in practice.

Hyperdimensional computing (HDC) is an emerging paradigm that employs hypervectors (HV s) to emulate cognitive tasks. In HDC, the most time-consuming and power-hungry process is encoding, the first step that maps raw data into HV s. There have been non-volatile memory (NVM) based computing-in-memory (CiM) HDC encoding designs, which exploit the intrinsic HDC characteristics of high parallelism, massive data, and robustness. These NVM-based CiMs have shown great potential in reducing encoding time and power consumption. Among them, the ferroelectric field-effect transistor (FeFET) based designs show ultra-high energy efficiency. However, existing FeFET-based HDC encoding designs face the challenges of energy -consuming current-mode addition, inefficient HV storage, limited endurance, and single encoding method support. These challenges limit the energy efficiency, lifetime, and versatility of the designs. This work proposes an energy-efficient charge-domain FeFET-based in-memory HDC encoder, i.e., CafeHD, with extended lifetime, good versatility, and comparable accuracy. Area-efficient charge-domain computing is proposed in HDC encoding for the first time, which enables CafeHD with ultra-low power and high scalability. An HV merging technique is explored to improve the performance. A low-cost partial MAJ interface is also proposed to reduce writes. Besides, CafeHD also supports two widely used encoding methods. Results show that CafeHD on average achieves 10.9×/12.7×/3.5× speedup and 103.3×/21.9×/6.3× energy effi-ciency with ~84 % write times reduction and similar accuracy compared with the state-of-the-art ReRAM/PCMlFeFET-based CiM design for HDC encoding, respectively.

Nonvolatile memory (NVM)-based computing-in-memory (CiM) has shown promising prospects in deep neural network (DNN) inference at the edge thanks to its nonvolatility and high density. Moreover, most NVMs support multi-level cell (MLC) storage, which can further boost energy efficiency and storage density. However, MLC NVM-based CiMs suffer from degraded accuracy due to device nonidealities, including large variations, nonlinear current distribution, and state drifts. Although prior works have explored various mitigation measures, such as hybrid SLC/MLC, write-and-verify, and local recovery units, the substantial costs from software support, energy, latency, and area still limit the performance. Therefore, the tradeoff between inference accuracy, storage density and compute density has become a vital challenge in NVM-based CiMs.

Tabular data are a widely used format in data science, and tree-based Machine Learning (ML) models are powerful tools and outperform Deep Neural Network (DNN) with higher accuracy for tasks on tabular data. However, computing multiple trees on massive tabular data via conventional von Neumann architectures suffers from irregular memory accesses. Prior work utilizes Analog Content Addressable Memories (ACAMs) to gain great speedup, but the analog matching method is vulnerable to device and voltage variations, and the limited density of 2D memory makes frequent data movement still inevitable for large-scale tabular data.

Ferroelectric Field Effect Transistors (FeFETs) have spurred increasing interest in both memories and computing applications, thanks to their CMOS compatibility, low-power operation, and high scalability. However, new security threats to the FeFET-based memories also arise. A major threat is the power analysis side-channel attack (P-SCA), which exploits the power traces of the memory access to obtain data information. There have been several effective efforts on resistive nonvolatile memories (NVMs), but they fail to meet the requirements for secure FeFET-based memories due to the different capacitive FeFETs load. Directly applying these existing countermeasures to the P-SCA protection for FeFETs induces huge challenges, especially for the balance between power side-channel resistance and corresponding overheads. To address this issue, we leverage the unique features of FeFETs and propose ProtFe , namely the protection methods for FeFET-based memories, including the pipelined multi-step write strategy ( PiMWrite ) and the split array design ( SpA ). PiMWrite is proposed for general FeFET-based memories, and inserts specially designed intermediate states to mitigate information leakage with pipelined steps to reduce overheads. SpA is proposed for custom FeFET-based memories, and simultaneously writes two split portions of the array with shared minimized peripherals to go beyond the balance between security and overheads. Simulation results show that PiMWrite expands the search space of a single power trace to 21× and involves nearly zero hardware penalties. SpA presents 33× search space improvement with negligible latency, 0.6% area, and only 7.1% energy overhead. ProtFe achieves improved balance between security and overheads, compared with the state-of-the-art works.

Multi-level cell (MLC) NVM-based CiM has become a promising candidate in computing-in-memory (CiM) designs because of its non-volatility, high cell density, and improving compatibility with the CMOS process. However, most MLC CiM faces the challenges of non-ideal device limitations, including the low on/off ratio, large device-to-device variations, and read disturbances, which limit the computing accuracy, reliability, and throughput performance. This work proposes Victor, a variation-resilient approach using cell-clustered charge-domain computing for high-density and high-throughput MLC CiM. A cell-clustered-computing with local recovery unit (LRU) design methodology is proposed to improve matrix-vector-multiplication (MVM) reliability and throughput. To showcase the capability of Victor, 2b-4b MLC Resistive RAM (RRAM) is taken as an example for design and evaluation. Results show that Victor reaches 3.56x energy efficiency, 4x variation tolerance compared with the prior ratio-based MLC CiM. In addition, the throughput is improved by 3.1x with less than 1% DNN accuracy loss. Moreover, a dynamic boundary adaption approach is proposed to restore the accuracy loss of state drifting, which in return reduces the energy and latency overhead by 100x and 1.25x, respectively, compared with the conventional write-and-verify approach.

Genome sequence analysis is a powerful tool in medical and scientific research. Considering the inevitable sequencing errors and genetic variations, approximate string matching (ASM) has been adopted in practice for genome sequencing. However, with exponentially increasing bio-data, ASM hardware acceleration is facing severe challenges in improving the throughput and energy efficiency with the accuracy constraint.This paper presents ASMCap, an ASM acceleration approach for genome sequence analysis with hardware-algorithm co-optimization. At the circuit level, ASMCap adopts charge-domain computing based on the capacitive multi-level content addressable memories (ML-CAMs), and outperforms the state-of-the-art ML-CAM-based ASM accelerators EDAM with higher accuracy and energy efficiency. ASMCap also has misjudgment correction capability with two proposed hardware-friendly strategies, namely the Hamming-Distance Aid Correction (HDAC) for the substitution-dominant edits and the Threshold-Aware Sequence Rotation (TASR) for the consecutive indels. Evaluation results show that ASMCap can achieve an average of 1.2x (from 74.7% to 87.6%) and up to 1.8x (from 46.3% to 81.2%) higher F1score (the key metric of accuracy), 1.4x speedup, and 10.8x energy efficiency improvement compared with EDAM. Compared with the other ASM accelerators, including ResMA based on the comparison matrix, and SaVI based on the seeding strategy, ASMCap achieves an average improvement of 174x and 61x speedup, and 8.7e3x and 943x higher energy efficiency, respectively.

As one type of associative memory, content-addressable memory (CAM) has become a critical component in several applications, including caches, routers, and pattern matching. Compared with the conventional CAM that could only deliver a “matched or not-matched” result, emerging multilevel CAM (ML-CAM) is capable of delivering “the degree of match” with multilevel distance calculation. This feature has been desired in applications that need beyond-Boolean matching results. However, existing ML-CAM designs are limited by the bit-cell device discharging current mismatch and vulnerability to the timing of sensing operations for distance calculation. This inherent constraint makes it difficult to further improve the accuracy and scalability toward higher accuracy and higher dimension matching. In this work, we propose CapCAM, a multilevel Capacitive Content Addressable Memory. It could be implemented based on either static random-access memory (SRAM) or emerging technologies, e.g., the ferroelectric field-effect transistor (FeFET). CapCAM could provide linear and stable voltage drop scaled by the match degree and need no strict timing for result sensing, which embraces the high-accuracy and high-scalability search. The inherent enabler of CapCAM is the charge-domain computing mechanism. This article will present the basic concept, operating mechanisms, detailed circuit designs, and circuit-level simulations of CapCAM. Besides, we apply CapCAM to few-shot learning applications and compare CapCAM with the current-domain TCAM designs. Results show 99.2% accuracy for a five-way five-shot classification task with our proposed CapCAM design while considering 1-fF capacitors, 20-domain FeFETs, and 256 columns. In contrast, the prior work based on discharging dynamics requires strict timing controls and suffers from accuracy degradation under the same configuration, which demonstrates CapCAM’s capability of low-power, accurate, and scalable multilevel CAM (ML-CAM) computing.

Ternary content addressable memory (TCAM) has been a critical component in caches, routers, etc., in which density, speed, power efficiency, and reliability are the major design targets. There have been the conventional low-write-power but bulky SRAM-based TCAM design, and also denser but less reliable or higher-write-power TCAM designs using nonvolatile memory (NVM) devices. Meanwhile, some TCAM designs using dynamic memories have been also proposed. Although dynamic design TCAM is denser than CMOS SRAM TCAM and more reliable than NVM TCAM, the conventional row-by-row refresh operations land up with a bottleneck of interference with normal TCAM activities. Therefore, this paper proposes a custom low-power dynamic TCAM using nanoelectromechanical (NEM) relay devices utilizing one-shot refresh to solve the memory refresh problem. By harnessing the unique NEM relay characteristics with a proposed novel cell structure, the proposed TCAM occupies a small footprint of only 3 transistors (with two NEM relays integrated on the top through the back-end-of-line process), which significantly outperforms the density of 16-transistor SRAM-based TCAM. In addition, evaluations show that the proposed TCAM improves the write energy efficiency by 2.31x, 131x, and 13.5x over SRAM, RRAM, and FeFET TCAMs, respectively; The search energy-delay-product is improved by 12.7x, 1.30x, and 2.83x over SRAM, RRAM, and FeFET TCAMs, respectively.

Ternary content addressable memory (TCAM) is one type of associative memory and has been widely used in caches, routers, and many other mapping-aware applications. While the conventional SRAM-based TCAM is high speed and bulky, there have been denser but slower and less reliable nonvolatile TCAMs using nonvolatile memory (NVM) devices. Meanwhile, some CMOS TCAMs using dynamic memories have been also proposed. Although dynamic TCAM could be denser than the 16T SRAM TCAM and more reliable than the nonvolatile TCAMs, CMOS dynamic TCAMs still suffer from the row-by-row refresh energy and time overheads. In this article, we propose dynamic TCAM using nanoelectromechanical (NEM) relays (DyTAN), and utilize one-shot refresh (OSR) to solve the memory refresh problem. By exploiting the unique NEM relay characteristics, DyTAN outperforms the existing works in the balance between density, speed, and power efficiency. Compared with the 16T SRAM-based TCAM, the 5T CMOS dynamic TCAM, the 2T2R TCAM, and the 2FeFET TCAM, evaluations show that the proposed DyTAN reduces the write energy by up to$2.3\times $,$1.3\times $,$131\times $, and$13.5\times $, and improves the search energy-delay-product (EDP) by up to$12.7\times $,$1.7\times $,$1.3\times $, and$2.8\times $, respectively.

This paper proposes low-power design of embedded dynamic random-access memory (eDRAM) using emerging nanoelectromechanical (NEM) relay devices. The motivation of this work is to reduce the standby refresh power consumption through the improvement of retention time of eDRAM cells. In this paper, it is revealed that the tunable beyond-CMOS characteristics of emerging NEM relay devices, especially the ultra-high OFF-state drain-source resistance, open up new opportunities with device-circuit co-design. In addition, the pull- in and pull-out threshold voltages are tilled to fit the operating mechanisms of eDRAM, so as to support low-voltage operations along with long retention time. Excitingly, when low-gate-leakage thick-gate transistors are used together, the proposed NEM-relay- based eDRAM exhibits so significant retention time improvement that it behaves almost "nonvolatile". Even if using thin-gate transistors in a 130nm CMOS, the evaluation of the proposed eDRAM shows up to 63x and 127x retention time improvement at 1.0V and 1.4V supply, respectively. Detailed performance benchmarking analysis, along with the practical CMOS- compatible NEM relay model, the eDRAM design and optimization considerations, is included in this paper.

Making embedded memory symmetric provides the capability of memory access in both rows and columns, which brings new opportunities of significant energy and time savings if only a portion of data in the words need to be accessed. This work investigates the use of ferroelectric field-effect transistors (FeFETs), an emerging nonvolatile, low-power, deeply-scalable, CMOS-compatible transistor technology, and proposes a new 3-transistor/cell symmetric nonvolatile memory (SymNVM). With ~1.67x higher density as compared with the prior FeFET design, significant benefits of energy and latency improvement have been achieved, as evaluated and discussed in depth in this paper.

Photoacoustic imaging has attracted increasing research interest in recent years due to its unique merit of combining light and sound. Enabling deep tissue imaging with high ultrasound spatial resolution and optical absorption contrast, photoacoustic imaging has been applied in various application scenarios including anatomical, functional and molecular imaging. However, the bulky and expensive laser source is one of the key bottlenecks that need to address for further compact system development. Photoacoustic imaging system based on low-cost laser diode (LD) is one of the promising solutions. In this paper, we report a custom-made fingertip laser diode system enabling both pulsed and continuous modulation modes with shortest pulse-width of 40 ns, largest driving current of 13 A, and highest modulation frequency of 3 MHz, which are suitable for both time and frequency domain photoacoustic imaging. To the best of our knowledge, this may be the most compact laser source reported for photoacoustic imaging. Owing to its super-compact size, the proposed LD system could pave the pathway to low-cost photoacoustic sensing and imaging device, even wearable photoacoustic biomedical sensors.

As multicore systems become the dominant mainstream computing technology, one of the most difficult challenges the industry faces is the software. Applications with large amounts of explicit thread-level parallelism naturally scale performance with the number of cores, but single-threaded applications realize little to no gains with additional cores. One solution to this problem is automatic parallelization that frees the programmer from the difficult task of parallel programming and offers hope for handling the vast amount of legacy single-threaded software. There is a long history of automatic parallelization for scientific applications, but the techniques have generally failed in the context of general-purpose software. Thread-level speculation overcomes the problem of memory dependence analysis by speculating unlikely dependences that serialize execution. However, this approach has lead to only modest performance gains. In this paper, we take another look at exploiting loop-level parallelism in single-threaded applications. We show that substantial amounts of loop-level parallelism is available in general-purpose applications, but it lurks beneath the surface and is often obfuscated by a small number of data and control dependences. We adapt and extend several code transformations from the instruction-level and scientific parallelization communities to uncover the hidden parallelism. Our results show that 61% of the dynamic execution of studied benchmarks can be parallelized with our techniques compared to 27% using traditional thread-level speculation techniques, resulting in a speedup of 1.84 on a four core system compared to 1.41 without transformations.

Chip multiprocessors with multiple simpler cores are gaining popularity because they have the potential to drive future performance gains without exacerbating the problems of power dissipation and complexity. Current chip multiprocessors increase throughput by utilizing multiple cores to perform computation in parallel. These designs provide real benefits for server-class applications that are explicitly multi-threaded. However, for desktop and other systems where single-thread applications dominate, multicore systems have yet to offer much benefit. Chip multiprocessors are most efficient at executing coarse-grain threads that have little communication. However, general-purpose applications do not provide many opportunities for identifying such threads, due to frequent use of pointers, recursive data structures, if-then-else branches, small function bodies, and loops with small trip counts. To attack this mismatch, this paper proposes a multicore architecture, referred to as Voltron that extends traditional multicore systems in two ways. First, it provides a dual-mode scalar operand network to enable efficient inter-core communication and lightweight synchronization. Second, Voltron can organize the cores for execution in either coupled or decoupled mode. In coupled mode, the cores execute multiple instruction streams in lock-step to collectively function as a wide-issue VLIW. In decoupled mode, the cores execute a set of fine-grain communicating threads extracted by the compiler. This paper describes the Voltron architecture and associated compiler support for orchestrating bi-modal execution

Application-specific extensions to the computational capabilities of a processor provide an efficient mechanism to meet the growing performance and power demands of embedded applications. Hardware, in the form of new function units (or coprocessors), and the corresponding instructions are added to a baseline processor to meet the critical computational demands of a target application. In this paper, the design of a system to automate the instruction set customization process is presented. A dataflow graph design space exploration engine efficiently identifies computation subgraphs to create custom hardware and a compiler subgraph matching framework seamlessly exploits this hardware. We demonstrate the effectiveness of this system across a range of application domains and study the applicability of the custom hardware across an entire application domain. Generalization techniques are presented which enable the application-specific hardware to be more effectively used across a domain.

Application-specific extensions to the computational capabilities of a processor provide an efficient mechanism to meet the growing performance and power demands of embedded applications. Hardware, in the form of new function units (or co-processors), and the corresponding instructions, are added to a baseline processor to meet the critical computational demands of a target application. The central challenge with this approach is the large degree of human effort required to identify and create the custom hardware units, as well as porting the application to the extended processor. In this paper, we present the design of a system to automate the instruction set customization process. A dataflow graph design space exploration engine efficiently identifies profitable computation subgraphs from which to create custom hardware, without artificially constraining their size or shape. The system also contains a compiler subgraph matching framework that identifies opportunities to exploit and generalize the hardware to support more computation graphs. We demonstrate the effectiveness of this system across a range of application domains and study the applicability of the custom hardware across the domain.