Gate-level fault simulation is a critical step in design for test and functional safety verification of the chip design process, essential to ensuring circuit reliability. As chip complexity grows for mission-critical applications such as autonomous vehicles, medical devices, and military systems, the efficiency of fault simulation increasingly becomes a bottleneck in the chip’s time-to-market. However, existing methods often suffer from computational redundancy, inefficiencies in memory access, or failure to optimize performance for specific CPU hardware platforms. This paper proposes DomSim, a hardware-aware hybrid fault simulation method that combines compiled simulation and event-driven simulation with an optimized computation-to-memory-access ratio. By utilizing circuit information and hierarchical structure provided by dominator trees, DomSim achieves high-quality circuit partitioning, optimizing hardware resource utilization and memory access locality. Furthermore, a parameter adjustment strategy tailored to hardware capabilities and circuit characteristics enables adaptive optimization. Extensive experiments show that DomSim surpasses a commercial tool by 10.29× on average. Further experiments demonstrate that DomSim exhibits good adaptability across different hardware platforms and circuits, highlighting the superiority of our method.

This is a corrigendum for the article “A data-centric chip design agent framework for Verilog code generation” published in ACM Trans. Des. Autom. Electron Syst. 30, 6, Article 93 (October 2025), 27 pages.

In automatic test pattern generation (ATPG), SAT-based methods are typically used to complement structural approaches, especially for addressing hard-to-detect faults. However, as the size and complexity of circuits grow, SAT-based ATPG faces challenges like pattern inflation and excessive runtime, limiting its overall performance. The key problem lies in the fact that current mainstream SAT solvers perform complete assignments for all primary inputs of the fault’s transitive fanin cone without considering the detection of other faults, making test compaction extremely difficult and time consuming. In this paper, a novel SAT solver PA-MiniSat is proposed, which is capable of generating partial assignments for solving variables and significantly reduces the number of specified bits in test cubes. As an extension of MiniSat, it employs a full-literal watching technique and a circuit-adapted heuristic branching strategy, achieving overall improved performance in ATPG. Based on PA-MiniSat, a hybrid ATPG framework PastATPG is proposed for better test compaction, which tightly integrates structural algorithms with the SAT solver into the unified test compaction flow. Experimental results demonstrate that our method outperforms other SAT solvers in pattern compaction and, in some cases, even surpasses commercial ATPG tools in terms of speed. The code is available at https://github.com/sklp-eda-lab/PastATPG.

Deep learning has significantly advanced Electronic Design Automation (EDA), with circuit representation learning emerging as a key area for modeling the relationship between a circuit’s structure and functionality. Existing methods primarily use either Large Language Models (LLMs) for Register Transfer Level (RTL) code analysis or Graph Neural Networks (GNNs) for netlist modeling. While LLMs excel at high-level functional understanding, they struggle with detailed netlist behavior. GNNs, however, face challenges when scaling to larger sequential circuits due to long-range information dependencies and insufficient functional supervision, leading to decreased accuracy and limited generalization. To address these challenges, we propose MOSS, a multimodal framework that integrates GNNs with LLMs for sequential circuit modeling. By enhancing D-type Flip-Flop (DFF) node features with embeddings from fine-tuned LLMs on RTL code, we focus the GNN on critical anchor points, reducing reliance on long-range dependencies. The LLM also provides global circuit embeddings, offering efficient supervision for functionality-related tasks. Additionally, MOSS introduces an adaptive aggregation method and a two-phase propagation mechanism in the GNN to better model signal propagation and sequential feedback within the circuit. Experimental results demonstrate that MOSS significantly improves the accuracy of functionality and performance predictions for sequential circuits compared to existing methods, particularly in larger circuits where previous models struggle. Specifically, MOSS achieves a $\mathbf{9 5. 2 \%}$ accuracy in arrival time prediction.

As functional safety of electronic chips gains importance in autonomous vehicles and aerospace, standards like ISO 26262 mandate high diagnostic coverage, requiring extensive gate-level fault simulations. However, for large-scale industrial sequential circuits, these simulations are time-consuming, creating a significant bottleneck in chip development. Prior approaches have focused on reducing computational complexity and optimizing CPU hardware usage by minimizing redundant computations during fault propagation and leveraging bit-level parallel processing capabilities. Techniques like parallel-pattern and event-driven simulations have improved performance in combinational circuits but face limitations in sequential circuits due to timing dependencies within loops. The challenge lies in parallelizing simulations across different cycles without violating these dependencies, which is exacerbated by the complex feedback structures in SCCs. In this work, we propose a novel parallel-pattern fault simulation framework that combines loop fusion with efficient event traversal to accelerate sequential circuit simulations. By compiling simple loops into larger nodes, we reduce the number of feedback events without introducing excessive redundancy. For larger SCCs, we develop specialized algorithms for selecting loop entrance nodes based on indegree analysis and implement the lazy propagation strategy for internal nodes. This approach minimizes simulation events caused by inaccurate predictions and reduces overhead associated with false event propagation. We integrate these techniques into our simulation framework, EPICS, which strategically mixes compiled and event-driven simulations to optimize performance. Experimental results demonstrate that EPICS achieves a $5.94 \times$ speedup over state-of-the-art commercial tool while maintaining the same fault coverage.

As intelligent computing devices increasingly integrate into human life, ensuring the functional safety of the corresponding electronic chips becomes more critical. A key metric for functional safety is achieving a sufficient fault coverage. To meet this requirement, extensive time-consuming fault simulation of the RTL code is necessary during the chip design phase. The main overhead in RTL fault simulation comes from simulating behavioral nodes (always blocks). Due to the limited fault propagation capacity, fault simulation results often match the good simulation results for many behavioral nodes. A key strategy for accelerating RTL fault simulation is the identification and elimination of redundant simulations. Existing methods detect redundant executions by examining whether the fault inputs to each RTL node are consistent with the good inputs. However, we observe that this input comparison mechanism overlooks a significant amount of implicit redundant execution: although the fault inputs differ from the good inputs, the node's execution results remain unchanged. Our experiments reveal that this overlooked redundant execution constitutes nearly half of the total execution overhead of behavioral nodes, becoming a significant bottleneck in current RTL fault simulation. The underlying reason for this overlooked redundancy is that, in these cases, the true execution paths within the behavioral nodes are not affected by the changes in input values. In this work, we propose a behavior-level redundancy detection algorithm that focuses on the true execution paths. Building on the elimination of redundant executions, we further developed an efficient RTL fault simulation framework, Eraser. Experimental results show that compared to commercial tools, under the same fault coverage, our framework achieves a 3.9 × improvement in simulation performance on average.

The widespread of Large Language Models (LLMs) marks a significant milestone in generative AI. Nevertheless, the increasing context length and batch size in offline LLM inference escalate the memory requirement of the key-value (KV) cache, which imposes a huge burden on the GPU VRAM, especially for resource-constrained scenarios (e.g., edge computing). Several cost-effective solutions leverage host memory or SSDs to reduce storage costs for offline inference scenarios and improve the throughput. Nevertheless, they suffer from significant performance penalties imposed by intensive KV cache accesses due to limited PCIe bandwidth. To address these issues, we propose InstAttention, a novel LLM inference system that offloads the most performance-critical computation (i.e., attention in decoding phase) and data (i.e., KV cache) parts to Computational Storage Drives (CSDs), which minimize the enormous KV transfer overheads. InstAttention designs a dedicated flashaware in-storage attention engine with KV cache management mechanisms to exploit the high internal bandwidths of CSDs instead of being limited by the PCIe bandwidth. The optimized P2P transmission between GPU and CSDs further reduces data migration overheads. Experimental results demonstrate that for a 13B model using an NVIDIA A6000 GPU, InstAttention improves throughput for long-sequence inference by up to $11.1 \times$, compared to existing SSD-based solutions such as FlexGen.

Neural Vector Search (NVS) has exhibited superior search quality over traditional key-based strategies for information retrieval tasks. An effective NVS architecture requires high recall, low latency, and high throughput to enhance user experience and cost-efficiency. However, implementing NVS on existing neural network accelerators and vector search accelerators is sub-optimal due to the separation between the embedding stage and vector search stage at both algorithm and architecture levels. Fortunately, we unveil that Product Quantization (PQ) opens up an opportunity to break separation. However, existing PQ algorithms and accelerators still focus on either the embedding stage or the vector search stage, rather than both simultaneously. Simply combining existing solutions still follows the beaten track of separation and suffers from insufficient parallelization, frequent data access conflicts, and the absence of scheduling, thus failing to reach optimal recall, latency, and throughput. To this end, we propose a unified and efficient NVS accelerator dubbed NeuVSA based on algorithm and architecture co-design philosophy. Specifically, on the algorithm level, we propose a learned PQ-based unified NVS algorithm that consolidates two separate stages into the same computing and memory access paradigm. It integrates an end-to-end joint training strategy to learn the optimal codebook and index for enhanced recall and reduced PQ complexity, thus achieving smoother acceleration. On the architecture level, we customize a homogeneous NVS accelerator based on the unified NVS algorithm. Each sub-accelerator is optimized to exploit all parallelism exposed by unified NVS, incorporating a structured index assignment strategy and an elastic on-chip buffer to alleviate buffer conflicts for reduced latency. All sub-accelerators are coordinated using a hardware-aware scheduling strategy for boosted throughput. Experimental results show that the joint training strategy improves recall by 4.6% over the separated strategy and accuracy by 43.5% over LUT-NN. NeuVSA achieves $2.82 \times$ to $416.17 \times$ lower latency over CPU, GPU, DFX+ANNA, and PQA+ANNA, and up to $49.60 \times$ and $10.57 \times$ higher average throughput over CPU and GPU, respectively. NeuVSA also reduces chip area by 65.2% over PQA+ANNA.

Graph processing suffers from severe locality challenges due to considerable inefficient irregular memory accesses, which mainly originate from random accesses to neighbors of active vertices (a.k.a frontiers). Graph reordering, which assigns continuous IDs to vertices that are more likely to be accessed consecutively, can improve access locality effectively and has demonstrated significant speedups across various architectures and systems. Existing graph reordering methods primarily explore the overlapping intensity of in-neighbor vertices for the data access locality characterization. However, many graph algorithms often activate a fraction of the vertices across the graphs, which vary substantially over different inputs and processing iterations. Many of these vertices are neither connected nor have any shared neighbors, but they are actually processed at the same time and exhibit potential data access locality, which is generally overlooked in prior graph reordering methods. We notice that the data locality between concurrently activated vertices are usually attributed to the overlapped$k$-order in-neighbors. As the number of$k$-order in-neighbors grows explosively, it is unacceptably time-consuming to analyze the overlapping of$k$-order in-neighbors for graph reordering directly. In this case, we propose to replace the overlapping calculation of$k$-order in-neighbors with frontier distribution analysis of a few BFS samplings. Specifically, we profile the frontiers distributed across iterations of different BFS samplings first and build a feature vector based on the activated iteration order of each vertex in the BFS samplings. On top of the feature vectors, we propose FrontOrder, which has a customized distance metric to characterize the locality between different vertices and leverages$K$-means to cluster vertices with high locality to guide graph reordering. In addition, FrontOrder also takes the load balance into consideration by predicting the runtime computing intensity with the learned clusters of vertices. According to our experiments, FrontOrder delivers an average performance speedup of${2.33\times}$and${1.57\times}$on Ligra and GPOP, respectively, and consistently outperforms the state-of-the-art graph reordering methods on a set of representative graph algorithms and datasets with moderate preprocessing overhead.

Out-of-core graph processing systems are severely bottlenecked by I/O to the external storage because of the low compute-to-I/O ratio and the substantial amount of irregular data accesses. In order to alleviate the I/O bottleneck, prior works either focus on improving the bandwidth utilization by converting random I/O requests into sequential ones, or improving the data utilization by fetching only the required data to avoid the I/O redundancy. However, the former usually loads massive unused data, while the latter can induce frequent finegrained I/O requests, wasting the parallelism of the I/O channels and leading to under-utilization of the limited I/O bandwidth. Different from prior works, we systematically explore the use of computational storage devices (CSDs), which offer in-storage computing facilities with higher I/O bandwidth, to improve both the bandwidth utilization and data utilization for higher I/O efficiency. Specifically, we first introduce a graph-semanticaware data organization to enable the loading of only active graph partitions at the granularity of a physical page, reducing redundant I/O and enhancing data utilization. Additionally, we propose to coalesce parallel I/O requests of graph partitions distributed across different flash dies to maximize the parallelism of internal I/O channels, thereby fully utilizing the internal I/O bandwidth of CSDs. In addition, we capture the dynamic status of graph processing tasks across the iterations and partitions at runtime to dynamically offload I/O-intensive workloads into the instorage processors with restricted computing resources but higher I/O bandwidth to further improve the I/O efficiency. With the above techniques, we implement an out-of-core graph processing system prototype, namely TaijiGraph, on an open-channel CSD. According to our experiments on a set of representative graph datasets and algorithms, TaijiGraph achieves average speedups of$2.43 \times, 3.81 \times, 2.21 \times$and$7.89 \times$, respectively, when compared to state-of-the-art out-of-core graph processing systems including GridGraph, LUMOS, Blaze, and GraphSSD.

The occasional nature of I/O bursts in production clusters makes the substantial and expensive SSD internal hardware resources (e.g., computation and memory resources) always underutilized, resulting in cost inefficiency.Open-Channel SSD (OCSSD), as a pioneering solution, removes the SSD internal resources but rather leverages the host-side resources to serve I/O requests.Unfortunately, it faces adoption obstacles due to the heavy resource contention with user applications, hampered host-SSD collaboration, and proprietary firmware leakage risks.Tackling these challenges, we propose XHarvest, a new cost-efficient and high-performance SSD architecture, which harnesses compute express link (CXL) and trusted execution environment (TEE) to facilitate dynamic, efficient, and secure host resource harvesting.It reserves moderate SSD internal resources to isolate SSD internal tasks and applications under regular I/O loads while coping with occasional I/O bursts via dynamic host resource harvesting.To this end, XHarvest executes the firmware within the host-side TEE without disclosing sensitive

Due to hardware customization capabilities, FPGA-based graph processing accelerators achieve significantly higher energy efficiency than many general-purpose computing engines. However, designing these accelerators remains a substantial challenge for high-level users. To overcome the programming barrier, FPGA-based accelerator design frameworks on top of generic graph processing programming models have been developed to automate accelerator generation through pre-built templates. However, they often tightly couple graph processing algorithms, programming models and processing paradigms, and accelerator architectures, which severely limits the expression scope of the algorithms and may also restrict the performance when the generated accelerators fail to suit dynamic processing patterns of the graph processing algorithms.

Precise and rapid performance prediction for dataflow-based accelerators is essential for efficient hardware design and design space exploration.However, existing methods often fall short due to limited generalization across hardware architectures, applications, and input-dependent control flows.Considering the rich program semantic knowledge contained in pre-trained large language models (LLMs), which is used for text and code generation, we propose a progressive numeric modeling paradigm based on pre-trained LLMs.This is an approach to achieve hardware, application, and control flow-sensitive generalization in dataflow accelerator performance prediction.Specifically, to make accurate performance estimates for unseen applications beyond the scope of the training data, we propose a numeric prediction model capable of estimating any performance range.This is achieved by treating the numerical data of the dataflow program as separate tokens and using categorical output for performance values, allowing us to observe confidence at each numerical position.Second, LLMulator supports input-adaptive performance prediction by introducing a reinforcement learning-based dynamic calibration framework, enabling accurate modeling of applications whose control flow varies with input-unlike prior methods that * Corresponding author.

Large language models (LLMs) trained via reinforcement learning with verifiable reward (RLVR) have achieved breakthroughs on tasks with explicit, automatable verification, such as software programming and mathematical problems. Extending RLVR to electronic design automation (EDA), especially automatically generating hardware description languages (HDLs) like Verilog from natural-language (NL) specifications, however, poses three key challenges: the lack of automated and accurate verification environments, the scarcity of high-quality NL-code pairs, and the prohibitive computation cost of RLVR. To this end, we introduce CodeV-R1, an RLVR framework for training Verilog generation LLMs. First, we develop a rule-based testbench generator that performs robust equivalence checking against golden references. Second, we propose a round-trip data synthesis method that pairs open-source Verilog snippets with LLM-generated NL descriptions, verifies code–NL–code consistency via the generated testbench, and filters out inequivalent examples to yield a high-quality dataset. Third, we employ a two-stage "distill-then-RL" training pipeline: distillation for the cold start of reasoning abilities, followed by adaptive DAPO, our novel RLVR algorithm that can reduce training cost by adaptively adjusting sampling rate. The resulting model, CodeV-R1-7B, achieves 68.6 \% and 72.9 \% pass@1 on VerilogEval v2 and RTLLM v1.1, respectively, surpassing prior state-of-the-art by 12$\sim$20 \%, while even exceeding the performance of 671B DeepSeek-R1 on RTLLM. We have released our model, training code, and dataset to facilitate research in EDA and LLM communities.

Graph reordering is an effective technique for improving the access locality of graph processing. However, existing methods often overlook the data access locality among concurrently activated vertices (a.k.a. frontiers). These vertices, while lacking direct connections or shared neighbors, can exhibit significant locality attributed to their overlapped k-order in-neighbors. However, calculating such overlaps directly is computationally prohibitive. We propose to estimate the overlapped k-order in-neighbors through frontier distribution analysis based on a few BFS samples. Our proposed graph reordering method, FrontOrder, constructs feature vectors from the frontier distribution of BFS samples, and employs K-means clustering with a custom distance metric to group vertices with high locality. Additionally, the learned clusters can predict runtime computing intensity, enabling load balancing through vertex reordering. FrontOrder achieves average speedups of 2.65× on Ligra and 1.73× on GPOP, outperforming state-of-the-art methods.

Approximate nearest neighbor search (ANNS) is essential for applications like recommendation systems and retrieval-augmented generation (RAG) but is highly I/O-intensive and memory-demanding. CPUs face I/O bottlenecks, while GPUs are constrained by limited memory. DRAM-based Processing-in-Memory (DRAM-PIM) offers a promising alternative by providing high bandwidth, large memory capacity, and near-data computation. This work introduces DRIM-ANN, the first optimized ANNS engine leveraging UPMEM’s DRAM-PIM. While UPMEM scales memory bandwidth and capacity, it suffers from low computing power because of the limited processor embedded in each DRAM bank. To address this, we systematically optimize ANNS approximation configurations and replace expensive squaring operations with lookup tables to align the computing requirements with UPMEM’s architecture. Additionally, we propose load-balancing and I/O optimization strategies to maximize parallel processing efficiency. Experimental results show that DRIM-ANN achieves a 2.46× speedup over a 32-thread CPU and up to 2.67× over a GPU when deployed on computationally enhanced PIM platforms.

As integrated circuits grow in complexity, test point insertion (TPI) has become vital for enhancing testability and improving reliability in design for test (DFT). Recent studies have shown the effectiveness of deep learning-based TPI using graph neural networks (GNNs) in improving test quality. However, the high cost of collecting training data, incomplete capture of the intrinsic characteristics of circuits, and the vast search space in large circuits hinder the performance of existing intelligent approaches. This paper introduces HighTPI, a two-stage learning approach for TPI to effectively reduce the number of test patterns, which leverages hierarchical graph representation by constructing a hypergraph based on hypernodes in fanout-free regions (FFRs). HighTPI better captures multi-fanout reconvergence information while lowering the cost of obtaining ground-truth labels due to the smaller scale of the FFR-based hypergraph. Two specialized GNNs are designed in stage I to select candidate insertion points for observation and control points, respectively. This integration of expert knowledge through supervised learning helps guide the reinforcement learning process in stage II, mitigating the challenges of sparse rewards and a large decision space. The experimental results demonstrate that HighTPI outperforms other TPI methods in terms of the trade-off between pattern reduction and fault coverage enhancement.

Recent advances in large language models (LLMs) have demonstrated significant potential for automated hardware description language (HDL) code generation from high-level specifications. However, two critical challenges limit further progress in this domain: the scarcity of quality Verilog training data and the inability of current approaches to generate RTL code optimized for power, performance, and area (PPA) metrics. This article presents a comprehensive data-centric framework that addresses these limitations through innovations in both pre-fine-tuning data preparation and after-fine-tuning optimization strategies. In the pre-fine-tuning phase, we tackle the data scarcity problem with an automated design-data augmentation framework that generates high-volume, high-quality natural language specifications aligned with corresponding Verilog code and EDA scripts. Our approach creates a complete RTL-level feedback loop by augmenting EDA scripts, RTL code, and EDA tool feedback. In the after-fine-tuning phase, we focus on generating PPA-aware RTL code through a novel search and prompt framework. Our approach implements iterative filtering and selection of LLM-generated Verilog variants while providing high-quality predefined prompts, including composition and interface specifications. To evaluate the effectiveness of our data augmentation method, we fine-tune Llama 2-13B and Llama 2-7B models using the dataset generated by our augmentation framework. The results demonstrate a significant improvement in the Verilog generation tasks with LLMs. Moreover, the accuracy of Verilog generation surpasses that of the current state-of-the-art open-source Verilog generation model, increasing from 58.8% to 70.6% with the same benchmark. Our 13B model has a pass rate improvement compared with GPT-3.5 in Verilog generation and outperforms in EDA script (i.e., SiliconCompiler) generation with only 200 EDA script data. Additionally, to evaluate the effectiveness of the our agent framework, we compare the PPA on the GPT-3.5, where the results show that the agent refined RTL code can have a better quality than the generated RTL code only with GPT-3.5.

Graph-based deep learning point cloud processing has gained increasing popularity but its performance is dragged by the dominating graph construction (GC) phase with irregular computation and memory access. Existing works that accelerate GC by tailoring architecture for a single GC algorithm fail to maintain efficiency because they neglected the best GC algorithm variation incurred by the point-cloud density variation in changing scenarios. Therefore, we propose APoX, a unified architecture with an adaptive GC scheme that can identify the optimum GC approach according to the point cloud variation. Experiments indicate that APoX achieves higher performance and energy efficiency over existing accelerators.

Recent advances in large language models have demonstrated their potential for automated generation of hardware description language (HDL) code from high-level prompts. Researchers have utilized fine-tuning to enhance the ability of these large language models (LLMs) in the field of Chip Design. However, the lack of Verilog data hinders further improvement in the quality of Verilog generation by LLMs. Additionally, the absence of a Verilog and electronic design automation (EDA) script data augmentation framework significantly increases the time required to prepare the training dataset for LLM trainers. This paper proposes an automated design-data augmentation framework, which generates high-volume and high-quality natural language aligned with Verilog and EDA scripts. For Verilog generation, it translates Verilog files to an abstract syntax tree and then maps nodes to natural language with a predefined template. For Verilog repair, it uses predefined rules to generate the wrong verilog file and then pairs EDA Tool feedback with the right and wrong verilog file. For EDA Script generation, it uses existing LLM(GPT-3.5) to obtain the description of the Script. To evaluate the effectiveness of our data augmentation method, we finetune Llama2--13B and Llama2-7B models using the dataset generated by our augmentation framework. The results demonstrate a significant improvement in the Verilog generation tasks with LLMs. Moreover, the accuracy of Verilog generation surpasses that of the current state-of-the-art open-source Verilog generation model, increasing from 58.8% to 70.6% with the same benchmark. Our 13B model (ChipGPT-FT1) has a pass rate improvement compared with GPT-3.5 in Verilog generation and outperforms in EDA script (i.e., SiliconCompiler) generation with only 200 EDA script data.

Fully Homomorphic Encryption (FHE) is one of the most promising privacy-preserving techniques that has drawn increasing attention from academia and industry due to its ideal security. Chiplet-based designs integrate multiple dies into the package delivering high performance and thereby are embraced by the resources-hungry FHE. Despite the chiplet-based system with various specialized accelerators, it falls short in supporting FHE with the novel polynomial operations. For a chiplet-based system that is not tailored for FHE, one common approach to support FHE is designing a new dedicated accelerator, However, this full design-and-build approach overlooks the existing abundant resources of accelerators in the system and incurs repeated customization and resource waste.

Automatic test pattern generation (ATPG) is a critical technology in integrated circuit testing. It searches for effective test vectors to detect all possible faults in the circuit as entirely as possible, thereby ensuring chip yield and improving chip quality. However, the process of searching for test vectors is NP-complete. At the same time, the large amount of backtracking generated during the search for test vectors can directly affect the performance of ATPG. In this paper, a learning-based ATPG framework, SmartATPG, is proposed to search for high-quality test vectors, reduce the number of backtracking during the search process, and thereby improve the performance of ATPG. SmartATPG utilizes graph convolutional network (GCN) to fully extract circuit feature information and efficiently explore the ATPG search space through reinforcement learning (RL). Experimental results indicate that the proposed SmartATPG outperforms traditional and artificial neural network (ANN)-based heuristic strategies on most benchmark circuits.

The use of cross-scheme fully homomorphic encryption (FHE) in privacy-preserving applications present to be a new challenge to hardware accelerator design. Existing accelerator architectures with customized polynomial-level operator abstraction fail to efficiently handle hybrid FHE schemes due to the mismatch between computational demands and available hardware resources under various parameter settings. In this work, we propose a new accelerator architecture that consists of a novel finer-grained low-level operator, i.e., Meta-OP, that not only mathematically supports a diverse range of polynomial operations, but is also hardware-friendly for accelerator design without complex topological logic. We then design a new slot-based data management scheme to efficiently handle the distinct memory access patterns over the Meta-OP. With a slot-based data management approach, Alchemist can accelerate both arithmetic and logic FHE workloads with high hardware utilization rates. In the experiment, we show that Alchemist is up to 24,829X faster than CPU. For arithmetic FHE, compared with the SOTA ASIC accelerators, Alchemist achieves a 29.4X performance per area improvement on average. For logic FHE, compared with the SOTA ASIC accelerators, Alchemist achieves a 7.0X overall speed up on average.

Graph convolutional network (GCN) has been proven powerful in various tasks for it combines both neural networks and graph processing operators. However, this characteristic makes GCN exhibit hybrid execution patterns, which is unfavorable for CPUs and GPUs. Therefore, designing specialized GCN accelerators is becoming a prevalent paradigm. Unfortunately, as graph scale continues to grow, existing GCN accelerators suffer from significant bandwidth consumption and memory footprint as they neglect the inherent semantic redundancy of vertex features. Although applying Product Quantization to GCN is a promising solution to reduce the sizeable graph data via distilling semantic redundancy, it introduces novel operations with unique patterns that existing GCN accelerators cannot support. In this paper, we propose GPACE, an energy-efficient GCN accelerator that can fully harness the potential of PQ to reduce bandwidth consumption and data movement. GPACE is designed with a lookup-efficient architecture and well-optimized dataflow to support the unique data access and computation pattern of PQ-GCN. In addition to leveraging PQ to distill semantic redundancy, we exploit the operation redundancy and propose a redundancy-aware architecture to detect and reduce types of redundant operations to achieve higher energy efficiency. Evaluations show GPACE achieves high speedup and energy saving compared with CPU, GPU, and specialized GCN accelerators.

An Large Language Model (LLM)-based question-answering (QA) system has gained attention for its conversational ability. However, domain knowledge limitations, time lag, high training costs, and security concerns suggest building on-premise QA systems with embedding techniques. However, deploying embedding-based QA systems on existing GPUs or domain-specific accelerators is sub-optimal as they only address high computation costs and ignore large memory footprint and data movement costs, which impact response latency and user experience. To address these issues, we propose a hybrid near-data processing platform, HyQA, which collaboratively optimizes response latency, memory footprint, and data movement cost by exploiting the benefit of near-memory and near-storage computing simultaneously. First, HyQA analyzes computational patterns of sub-tasks in embedding-based QA systems, tailors domain-specific hardware accelerators, and assigns suitable computational paradigms. Second, these dedicated accelerators are designed to communicate directly with flash memory, avoiding additional data movement. The experiment shows that HyQA significantly improves performance and reduces energy over CPU, GPU, Cognitive SSD, and DeepStore platforms.

Natural language interfaces have exhibited considerable potential in the automation of Verilog generation derived from high-level specifications through the utilization of large language models, garnering significant attention. Nevertheless, this paper elucidates that visual representations contribute essential contextual information critical to design intent for hardware architectures possessing spatial complexity, potentially surpassing the efficacy of natural-language-only inputs. Expanding upon this premise, our paper introduces an open-source benchmark1 for multi-modal generative models tailored for Verilog synthesis from visual-linguistic inputs, addressing both singular and complex modules. Additionally, we introduce an open-source visual and natural language Verilog query language framework to facilitate efficient and user-friendly multi-modal queries. To evaluate the performance of the proposed multi-modal hardware generative AI in Verilog generation tasks, we compare it with a popular method that relies solely on natural language. Our results demonstrate a significant accuracy improvement in the multi-modal generated Verilog compared to queries based solely on natural language. We hope to reveal a new approach to hardware design in the large-hardware-design-model era, thereby fostering a more diversified and productive approach to hardware design.

The advancements in learning-based embedding models have propelled the embedding vector to become a vital data structure for various semantic search applications. The graph-based vector search method, which relies on the k-nearest neighbor (k-NN) graph, has gained immense popularity due to its superior performance in recall and throughput compared to other vector search methods. However, the construction of the k-NN graph is a time-consuming process, which not only affects the efficiency of identifying the optimal k-NN graph but also impacts data updates, especially for large-scale vector datasets. Despite the availability of various software-level optimization methods, such as iterative, refinement, and incremental methods, their peak performance is still limited to the underlying CPU platform. Furthermore, the existing GPU and specialized accelerator solutions only focus on partial k-NN graph construction methods and still encounter low efficiency when executing iterative methods due to irregular memory access, low data reuse, and redundant distance calculations.

Cross-silo federated learning (FL) leverages homomorphic encryption (HE) to obscure the model updates from the clients. However, HE poses the challenges of complex cryptographic computations and inflated ciphertext sizes. As cross-silo FL scales to accommodate larger models and more clients, the overheads of HE can overwhelm a CPU-centric aggregator architecture, including excessive network traffic, enormous data volume, intricate computations, and redundant data movements. Tackling these issues, we propose Flagger, an efficient and high-performance FL aggregator. Flagger meticulously integrates the data processing unit (DPU) with computational storage drives (CSD), employing these two distinct near-data processing (NDP) accelerators as a holistic architecture to collaboratively enhance FL aggregation. With the delicate delegation of complex FL aggregation tasks, we build Flagger-DPU and Flagger-CSD to exploit both in-network and in-storage HE acceleration to streamline FL aggregation. We also implement Flagger-Runtime, a dedicated software layer, to coordinate NDP accelerators and enable direct peer-to-peer data exchanges, markedly reducing data migration burdens. Our evaluation results reveal that Flagger expedites the aggregation in FL training iterations by ${436\%}$ on average, compared with traditional CPU-centric aggregators.

Deploying advanced large language models on edge devices, such as smartphones and robotics, is a growing trend that enhances user data privacy and network connectivity resilience while preserving intelligent capabilities. However, such a task exhibits single-batch computing with incredibly low arithmetic intensity, which poses the significant challenges of huge memory footprint and bandwidth demands on limited edge resources. To address these issues, we introduce Cambricon-LLM, a chiplet-based hybrid architecture with NPU and a dedicated NAND flash chip to enable efficient on-device inference of 70B LLMs. Such a hybrid architecture utilizes both the high computing capability of NPU and the data capacity of the NAND flash chip, with the proposed hardware-tiling strategy that minimizes the data movement overhead between NPU and NAND flash chip. Specifically, the NAND flash chip, enhanced by our innovative in-flash computing and on-die ECC techniques, excels at performing precise lightweight on-die processing. Simultaneously, the NPU collaborates with the flash chip for matrix operations and handles special function computations beyond the flash's on-die processing capabilities. Overall, Cambricon-LLM enables the on-device inference of 70B LLMs at a speed of 3.44 token/s, and 7B LLMs at a speed of 36.34 token/s, which is over 22× to 45× faster than existing flash-offloading technologies, showing the potentiality of deploying powerful LLMs in edge devices.

Graph is widely utilized as a key data structure in many applications such as social network and recommendation systems. However, many real-world graphs are constructed with large-scale correlated data, which typically involves massive random memory accesses and distance calculation resulting in considerable processing time and energy consumption on CPUs and GPUs. In this work, we present GCiM, a specialized processing-in-memory architecture for efficient graph construction and updating. By directly deploying the computing units on the logic layer of the 3D stacked memory, GCiM benefits from memory-level parallelism and further improves the memory access efficiency with both optimized processing ordering and data layout. In addition, we notice that the computing engines for graph updating suffer from dramatic utilization imbalance and integrate a power gating module to cut down the power supply of the idle computing engines at runtime and further enhance the energy efficiency. According to our experiments, GCiM shows 634.64X and 56.27X speedup while consuming 1194.14X and 505.07X less energy compared to CPU and GPU respectively.

Automatic Test Pattern Generation (ATPG) is a crucial technology in the testing of digital circuits. The excessive backtracks during the ATPG process can consume considerable computational resources and deleteriously affect performance. In this study, we introduce an intelligent ATPG method based on reinforcement learning to reduce the number of backtracks and enhance performance. Specifically, the Q-learning algorithm is employed to learn an optimal backtracing strategy pattern from the ATPG data produced through path-oriented decision-making (PODEM). The learned model is then utilized to guide the backtracing decisions within the PODEM, thereby improving the performance of the ATPG process. Experimental results demonstrate that, compared with traditional heuristic strategies and the backtrace path selection strategy based on artificial neural network (ANN), the proposed method can reduce backtrack occurrences and enhance performance more effectively.

In Number Theoretic Transform (NTT) operation, more than half of the active energy consumption stems from memory accesses. Here, we propose a generalized design method to improve the energy efficiency of NTT operation by considering the effect of processing element (PE) geometry and memory organization on the data flow between PEs and memory. To decrease the number of data bits that are required to be accessed from the memory, a two-dimensional (2-D) PE array architecture is used. A pair of ping-pong buffers are proposed to transposed swap the coefficients to enable a single bank of memory to be used with the 2-D PE array to reduce the average memory bit access energy without compromising the throughput. Our experimental results show that this design method can produce NTT accelerators with up to 69.8% saving in average energy consumption compared with the existing designs based on multi-bank SRAM and one-bank SRAM with one-dimensional PE array with the same number of PEs and total memory size.

Graph convolutional neural network (GCN) extends deep learning to process graph data and demonstrates superior performance. However, due to the irregularity, graphs show inconsistent patterns across different regions, which leads to distinctions in data reusability and edge processing activity, and consequently poses impacts on hardware efficiency and resource utility. Prior accelerators seldom explore the distinct patterns across graph regions and adopt a fixed strategy for the whole graph without consideration for region-specific characteristics. In this paper, we identify the inconsistent patterns of graphs and characterize the distinctions between graph regions. Then, we propose an adaptive dataflow to adapt the region-specific patterns. Third, we implement PANG, a pattern-aware accelerator that can dynamically adjust the dataflow to exploit the reusability and alleviate the frequent destination switching. Evaluated on real-world datasets, PANG achieves significant performance improvement.

Graph-based vector search that finds best matches to user queries based on their semantic similarities using a graph data structure, becomes instrumental in data science and AI application. However, deploying graph-based vector search in production systems requires high accuracy and cost-efficiency with low latency and memory footprint, which existing work fails to offer. We present VStore, a graph-based vector search solution that collaboratively optimizes accuracy, latency, memory, and data movement on large-scale vector data based on in-storage computing. The evaluation shows that VStore exhibits significant search efficiency improvement and energy reduction while attaining accuracy over CPU, GPU, and ZipNN platforms.

Graph is widely utilized as a key data structure in many applications like social network and recommendation systems. However, real-world graph construction typically involves massive random memory accesses and distance calculation, resulting in considerable processing time and energy consumptions on CPUs and GPUs. In this work, we present GCiM, a specialized processing-in-memory architecture for efficient graph construction and update. By directly deploying the computing units on the logic layer of the 3D stacked memory, GCiM benefits from memory-level parallelism and further improves the memory access efficiency with both optimized processing ordering and data layout. According to our experiments, GCiM shows 634.64X and 53.29X speedup while consuming 1470.7X and 442.56X less energy compared to CPU and GPU respectively.

Graph neural networks (GNNs) emerge as a powerful approach to process non-euclidean data structures and have been proved powerful in various application domains such as social networks and e-commerce. While such graph data maintained in real-world systems can be extremely large and sparse, thus employing GNNs to deal with them requires substantial computational and memory overhead, which induces considerable energy and resource cost on CPUs and GPUs. In this article, we present a specialized accelerator architecture, EnGN, to enable high-throughput and energy-efficient processing of large-scale GNNs. The proposed EnGN is designed to accelerate the three key stages of GNN propagation, which is abstracted as common computing patterns shared by typical GNNs. To support the key stages simultaneously, we propose the ring-edge-reduce(RER) dataflow that tames the poor locality of sparsely-and-randomly connected vertices, and the RER PE-array to practice RER dataflow. In addition, we utilize a graph tiling strategy to fit large graphs into EnGN and make good use of the hierarchical on-chip buffers through adaptive computation reordering and tile scheduling. Overall, EnGN achieves performance speedup by 1802.9X, 19.75X, and 2.97X and energy efficiency by 1326.35X, 304.43X, and 6.2X on average compared to CPU, GPU, and a state-of-the-art GCN accelerator HyGCN, respectively.

Building FPGA-based graph learning accelerators is very time-consuming due to the low-level RTL programming and the complicated design flow of FPGA development. It also requires the architecture and hardware expertise from the Graph Neural Network (GNN) application developers to tailor efficient accelerator designs on FPGAs. This work proposes an automation framework, DeepBurning-GL, which is compatible with state-of-the-art graph learning frameworks such as Deep Graph Library so that the developers can easily generate application-specific GNN accelerators from the software-described models. First, DeepBurning-GL employs a GNN performance analyzer to locate the performance bottleneck of specific GNN applications and decide the major design architectures and parameters that meet the user-specified constraints. Second, DeepBurning-GL provides a series of pre-built design templates such as computing templates and memory templates, which can be parameterized and fused to generate the final accelerator design. It also includes an optimizer that conducts automatic optimization by adjusting the accelerator architectural parameters. In evaluation, we use DeepBurning-GL to generate customized accelerators on three different FPGA platforms for various GNN models and workloads. The experimental results show that the generated accelerators achieve 179.4X and 40.1X energy-efficiency boost over the CPU and GPU solutions on average and deliver a 6.28X speedup and 6.73X energy-efficiency improvement on average compared to the latest GNN accelerator HyGCN on Alveo U50.

Prior research on energy-efficient Convolutional Neural Network (CNN) inference accelerators mostly focus on exploiting the model sparsity, i.e., zero patterns in weight and activations, to reduce the on-chip storage and computation overhead. In this work, we found in addition to zero patterns, a larger group of repetitive patterns and values exists in the working-set of CNN inference task, which is defined as computation redundancy and induces unnecessary performance and storage overhead in CNN accelerators. Based on this observation, we proposed a redundancy-free architecture that detects and eliminates the repetitive computation and storage patterns in CNN for more efficient network inference. The architecture consists of two parts: the off-line parameter analyzer that extracts the repetitive patterns in the 3D tensor of parameters, and the dataflow accelerator. The proposed accelerator at first preprocesses the weight patterns and the dynamically generated activations, and then cache these intermediate results in special P2-cache banks for further usage in convolution or full-connection stage. It is evaluated in experiments that the proposed Cavoluche architecture removes up to 89% of the repetitive operations from the layer inference process and reduce 77% of on-chip storage space to store both redundancy-free weight and activations. It is seen in experiments that the implementation of Cavoluche outperforms the state-of-the-art mobile GPGPU in both performance and energy-efficiency. When compared to the latest sparsity base accelerators, Cavoluche also achieves better operation elimination effects.

Compute-centric architecture is suffering from the data moving overhead caused by memory wall, particularly for applications like large-scale data analysis based on deep learning technology. In this work, we designed an energy-efficient In-SSD Deep Learning Accelerator, InS-DLA, for Near-Data-Processing. InS-DLA directly operates on NAND Flash inside the Open Channel Solid-State-Drive where the target data are stored, eliminating the power and performance overhead caused by data movement. Experimental results reveal that the InS-DLA based FPGA prototype reduces energy consumption by 95.82% and 59.23%, compared to conventional CPU and GPU based deep learning systems.