Zhirong Chen, Kaiyan Chang, Zhuolin Li, Cangyuan Li, Xinyang He, Chujie Chen, Mengdi Wang, Haobo Xu, Yinhe Han, Huawei Li, Ying Wang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Large language model (LLM) inference demands vast memory capacity and hierarchical memory structures, but conventional architectures suffer from excessive energy and latency costs due to frequent data movement across deep memory tiers. To address this, we propose a wafer-scale SRAM-based Computing-in-Memory (CIM) architecture that performs all LLM operations in situ within the first-level SRAM, eliminating off-chip data migration and achieving unprecedented energy efficiency. However, wafer-scale SRAM CIM presents multiple challenges due to the limited first-level memory capacity, which requires efficient compute-memory resource allocation.

As GPUs and other accelerators become increasingly popular, optimizing I/O operations between on-chip and off-chip memory is increasingly critical. I/O analysis, however, is complex, requiring a deep understanding of application dataflow and memory hierarchy. Developing a practical I/O analysis methodology remains a timely challenge. Self-attention is employed extensively in transformer models, but its quadratic memory complexity poses significant challenges to modern memory systems. In this study, we explore how to use I/O analysis to develop optimal solutions for accelerating exact long-sequence self-attention. We first introduce a novel I/O analysis for tall-and-skinny matrix-matrix multiplication, which captures the dominant data movement behavior of long-sequence self-attention. Guided by systematic I/O analysis, we develop AttenIO, an I/O-driven accelerator for exact long-sequence self-attention with three key optimizations: (1) an analytically derived I/O-optimal tiling and scheduling to minimize I/O operations, (2) fine-grained three-level communication-computation overlapping to hide I/O stalls, and (3) parallel execution patterns for efficient softmax. Our evaluation shows that AttenIO achieves a 1.6×-8.8× speedup over the state-of-the-art solutions. Although AttenIO is designed for self-attention, it also highlights the broader potential of I/O analysis as a principled foundation for guiding high-performance I/O optimizations.

Fully Homomorphic Encryption (FHE) is a powerful privacy-preserving technology enabling secure computation on encrypted data, but it suffers from substantial performance overheads. Running FHE efficiently typically requires developing dedicated FHE accelerators, which can be costly and inflexible. Instead of pursuing entirely new accelerators, this paper explores an alternative paradigm: augmenting generic computing devices with a modular FHE-specific hardware extension unit (HEU) to create an efficient FHE engine. To make this paradigm viable, we propose three key innovations: (1) recognizing that some FHE operators are data-intensive and involve a massive volume of ciphertexts, we design the HEU with a 3D stacked memory-based architecture to handle data-intensive operators.We also provide software-level support to facilitate deploying FHE tasks on this extension-based architecture. (2) To capitalize on the hardware parallelism, we propose an adaptive offloading algorithm that intelligently distributes FHE operators between the computing device and the HEU. (3) To optimize the data layout and minimize the inter-tile data communications in the novel 3D stack memory, we propose a dedicated ciphertext mapping mechanism. Experimental results demonstrate that our work achieves substantial acceleration in FHE tasks.

Transformer-based models dominate modern AI workloads but exacerbate memory bottlenecks due to their quadratic attention complexity and ever-growing model sizes. Existing accelerators, such as Groq and Cerebras, mitigate off-chip traffic with large on-chip caches, while algorithmic innovations such as FlashAttention fuse operators to avoid materializing large attention matrices. However, as off-chip traffic decreases, our measurements show that on-chip SRAM accesses account for over 60% of energy in long-sequence workloads, making cache access the new bottleneck. We propose 3D-Flow, a hybrid-bonded, 3D-stacked spatial accelerator that enables register-to-register communication across vertically partitioned PE tiers. Unlike 2D multi-array architectures limited by NoC-based router-to-router transfers, 3D-Flow leverages sub-10 μm vertical TSVs to sustain cycle-level operator pipelining with minimal overhead. On top of this architecture, we design 3D-FlashAttention, a fine-grained scheduling method that balances latency across tiers, forming a bubble-free vertical dataflow without on-chip SRAM round-trips. Evaluations on Transformer workloads (OPT and QWEN models) show that our 3D spatial accelerator reduces 46–93% energy consumption and achieves 1.4×–7.6× speedups compared to state-of-the-art 2D and 3D designs.

Sparse Matrix-Dense Vector Multiplication (SpMV) is an important computational primitive which is bounded by memory bandwidth. Computing-in-memory (CIM) is regarded as an effective approach to reduce data movement. Due to the lack of flexibility in architectural design, current CIM-based SpMV accelerators struggle to simultaneously support high-parallelism computations and the storage of irregular sparse data. We propose a flexible CIM-based accelerator named MACAM for high-precision SpMV. Each array of MACAM can be configured into sparse or dense modes according to the local-sparsity of the sparse matrix. We propose a unified data layout approach that enables MACAM to meet the data storage requirements of different modes. We also propose a sparse storage format and a workload-balancing approach to further improve the performance of MACAM. Experiments show that MACAM achieves 167.26× speedup and 286.04× energy saving over the GPU baseline. MACAM also achieves 97.41× and 6.56× speedup and 213.65× and 10.06× energy saving compared with two state-of-the-art CIM-based SpMV accelerators.

Modern AI inference interleaves compute-dense matrix kernels with memory-sensitive element-wise and reduction operators. Symmetric RISC-V vector processors execute this mix with a uniform lane design, replicating heavy arithmetic units even when many instructions need only lightweight arithmetic. This paper presents AsymVPU, a hardware-software co-designed vector architecture that introduces fine-grained intra-lane asymmetry: one main processing element retains full FP64/FMA capability, while three auxiliary processing elements provide dense support for lightweight vector operations. AsymVPU couples this lane organization with a two-level hierarchical reduction network, a coordinated global load/store path, and compiler-inserted density hints that preserve the RVV programming abstraction. Implemented in SystemVerilog and evaluated in a same-flow 12 nm comparison against an Ara-derived symmetric baseline, AsymVPU achieves up to 2.4 × higher compute density for quantized workloads and 60% lower reduction latency, while retaining competitive performance on FMA-dominated kernels.

As ultra-large-scale integrated circuits continue to evolve, advanced packaging has become a promising approach to boost system performance, especially in high-performance computing applications. To support heterogeneous integration, multiple chiplets are integrated via large-area silicon interposers. However, due to reticle size limitations, interposer lithography requires multi-reticle stitching, which introduces high manufacturing costs and alignment-induced yield degradation. In this paper, we propose CIT-R3, a reticle-reuse-driven router that formulates routing and reticle reuse as a differentiable optimization problem. Additionally, a graph-patching algorithm is applied to enforce layout consistency across reused reticle regions. Experimental results on multi-chiplet benchmarks demonstrate significant reticle reuse improvements with minimal routing cost overhead.

Numerous quantum algorithms assume that classical data has already been converted into quantum states, a process known as Quantum State Preparation (QSP). However, achieving precise QSP requires a circuit depth that scales exponentially with the number of qubits, posing a significant challenge to realizing quantum advantage. Recent research explores Parameterized Quantum Circuits (PQCs) as an approximate alternative, offering improved scalability with reduced circuit depth. However, the iterative, state-by-state optimization required by this approach creates substantial runtime overhead, which severely limits its practicality.To improve the efficiency of approximate QSP, we introduce a novel two-stage framework that can potentially generate QSP circuits for arbitrary quantum states. In theoffline training stage, our model learns a direct mapping from target states to circuit parameters, thereby bypassing the need foronline, state-by-state optimizationduring theinference stage. Extensive evaluations show that our approach significantly reduces runtime overhead by up to 132×, making a steady step towards efficient neural approximate QSP.

Fully Homomorphic Encryption (FHE) is a promising privacy-preserving technique that has drawn increasing attention from academia and industry. It allows computation directly on encrypted data without decryption. However, FHE incurs intensive computations. Chiplet-based designs integrate multiple processors, delivering high performance and thereby are embraced by computation-intensive FHE tasks. Despite the chiplet-based system with various processors, it is designed for unencrypted applications, falling short in handling FHE with unique ciphertext manipulations. One common approach to make it capable of FHE is developing a new FHE accelerator. However, this approach overlooks existing abundant resources already in the system and introduces a large area overhead. In this paper, we propose Chiplever, a framework that empowers a non-FHE-tailored system to efficiently support FHE tasks via a hardware extension. Chiplever aims to leverage the existing resources already in the room for FHE tasks. To achieve this, (1) Chiplever introduces a hardware extension with an FHE unit providing efficient function support for FHE operators. (2) Chiplever proposes an FHE coordinator in the extension, which enables direct ciphertext transfer between the newly introduced extension and existing chiplets, achieving efficient integration of the extension. (3) Chiplever lowers the high-level homomorphic operations to primitive operators that can be matched by existing chiplets and constructs a fine-grained computation graph. Based on this, Chiplever employs a task scheduling algorithm, which partitions the FHE task across the extension and existing chiplets to exploit the parallelism between them and reduce the ciphertext communication overheads. With these hardware and software optimizations, Chiplever achieves efficient FHE acceleration. Compared with prior FHE ASICs, Chiplever achieves 9.6× 15.9× speedup and 6.2× 67.4× throughput improvement on TFHE, while consuming only 18.8% 35.6% of the area overhead of dedicated FHE ASICs.

The decomposition-based chiplet design paradigm, which breaks monolithic system-on-chips (SoCs) into smaller chiplets, has proven effective in reducing costs and accelerating development. The vast design space of multi-chiplet systems necessitates automated techniques to maximize their benefits while minimizing overheads. However, current chiplet decomposition flows lack support for exploring reusable chiplets, missing opportunities to efficiently amortize non-recurring engineering costs. To fully unleash the economic potential of chiplets, we propose CHIPLETIZER 2.0, a cost-driven framework that guides SoC decomposition and enables chiplet reuse across a line of products. This early-stage chiplet planning framework determines critical system parameters involving the partition, floorplan, and D2D interface, based on user-specified optimization goals. Experimental results demonstrate that, compared with the existing chiplet decomposition techniques, our proposed framework achieves significant cost improvements on realistic product lines with acceptable overheads.

This is a corrigendum for the article “A data-centric chip design agent framework for Verilog code generation” published in ACM Trans. Des. Autom. Electron Syst. 30, 6, Article 93 (October 2025), 27 pages.

Rigid-flexible robots, integrating soft materials with rigid structures, have garnered increasing research interest due to their enhanced capabilities, flexibility, and inherent safety. However, existing control algorithms for these robots often exhibit high computational complexity, hindering real-time implementation. This work proposes VelKoz , an accelerator generation framework tailored for rigid-flexible robot control. It enables users to program in MATLAB and generate synthesizable Verilog code for control algorithms. A key challenge addressed is the integration of robotics domain knowledge with the dataflow representations commonly used in hardware accelerator design. Experimental results demonstrate that the generated accelerators achieve orders-of-magnitude lower latency and energy consumption compared to general-purpose CPUs and outperform customized high-level synthesis (HLS) implementations by 5.3 ×.

With the slowdown of Moore’s Law, conventional monolithic chip architectures face challenges such as excessive die sizes and prohibitive manufacturing costs. Consequently, chiplets have emerged as a pivotal technology in the post-Moore era, attracting significant attention from both academia and industry. Multi-chiplet systems offer compelling advantages over monolithic ones, including enhanced integration density, reduced cost, and shortened time-to-market. However, realizing these benefits necessitates design flows capable of optimizing parameters across logical, physical, and circuit layers, which introduces substantial design complexity. Numerous design automation technologies have been proposed to address these challenges. This article provides a comprehensive overview of related advancements, categorizing chiplet design methodologies into two primary types: (i) top-down flows disintegrating existing hardware designs into chiplets and subsequently reintegrating them into multi-chiplet systems, and (ii) bottom-up flows combining existing chiplets into multi-chiplet systems based on user applications. This article begins by introducing foundational concepts, technical characteristics, and evaluation models relevant to multi-chiplet systems. We then systematically summarize the problem formulations, design spaces, and optimization techniques associated with top-down and bottom-up design flows. Finally, we discuss key challenges and potential future research directions in chiplet design automation, aimed at further harnessing the potential of chiplet-based integration.

Recent advances of large language models in the field of Verilog generation have raised several ethical and security concerns, such as code copyright protection and dissemination of malicious code. Researchers have employed watermarking techniques to identify codes generated by large language models. However, the existing watermarking works fail to protect RTL code copyright due to the significant syntactic and semantic differences between RTL code and software code in languages such as Python. This paper proposes a hardware watermarking framework RTLMarker that embeds watermarks into RTL code and deeper into the synthesized netlist. We propose a set of rule-based Verilog code transformations, ensuring the watermarked RTL code's syntactic and semantic correctness. In addition, we consider an inherent tradeoff between watermark transparency and watermark effectiveness and jointly optimize them. The results demonstrate RTLMarker's superiority over the baseline in RTL code watermarking.

Quantization is a widely-used compression technology to reduce the overhead of serving large language models (LLMs) on terminal devices and in cloud data centers. However, prevalent quantization methods, such as 8-bit weight-activation or 4-bit weight-only quantization, achieve limited performance improvements due to poor support for low-precision (e.g., 4-bit) activation. This work, for the first time, realizes practical W4A4KV4 serving for LLMs, fully utilizing the INT4 tensor cores on modern GPUs and reducing the memory bottleneck caused by the KV cache. Specifically, we propose a novel fine-grained mixed-precision quantization algorithm (FMPQ) that compresses most activations into 4-bit with negligible accuracy loss. To support mixed-precision matrix multiplication for W4A4 and W4A8, we develop a highly optimized W4Ax kernel. Our approach introduces a novel mixed-precision data layout to facilitate access and fast dequantization for activation and weight tensors, utilizing the GPU's software pipeline to hide the overhead of data loading and conversion. Additionally, we propose fine-grained streaming multiprocessor (SM) scheduling to achieve load balance across different SMs. We integrate the optimized W4Ax kernel into our inference framework, COMET, and provide efficient management to support popular LLMs such as LLaMA-3-70B. Extensive evaluations demonstrate that, when running LLaMA family models on a single A100-80G-SMX4, COMET achieves a kernel-level speedup of 2.88x over cuBLAS and a 2.02x throughput improvement compared to TensorRT-LLM from an end-to-end framework perspective.

Computing-in-memory (CIM) architectures demonstrate superior performance over traditional architectures. To unleash the potential of CIM accelerators, many compilation methods have been proposed, focusing on application scheduling optimization specific to CIM. However, existing compilation methods often overlook CIM's capability to switch dynamically between compute and memory modes, which is crucial for accommodating the diverse memory and computational needs of real-world deep neural network architectures, especially the emerging large language models. To fill this gap, we introduce CMSwitch, a novel compiler to optimize resource allocation for CIM accelerators with adaptive mode-switching capabilities, thereby enhancing the performance of DNN applications. Specifically, our approach integrates the compute-memory mode switch into the CIM compilation optimization space by introducing a new hardware abstraction attribute. Then, we propose a novel compilation optimization pass that identifies the optimal network segment and the corresponding mode resource allocations using dynamic programming and mixed-integer programming. CMSwitch uses the tailored meta-operator to express the compilation result in a generalized manner. Evaluation results demonstrate that CMSwitch achieves an average speedup of 1.31x compared to existing SOTA CIM compilation works, highlighting CMSwitch's effectiveness in fully exploiting the potential of CIM processors for a wide range of real-world DNN applications.

Basic Linear Algebra Subprograms (BLAS) is a foundational software library for linear algebra kernels, which is widely used in scientific and engineering computing. Existing BLAS accelerations mainly rely on CPUs and GPUs. Many operations in BLAS are data intensive, so they are constrained by the limited memory bandwidth of CPUs and GPUs. The computing-in-memory (CIM) technology can effectively alleviate the memory wall bottleneck and is particularly suitable for accelerating BLAS. In this paper, we propose the first CIM accelerator for BLAS, CIM-BLAS, based on non-volatile memory. CIM-BLAS includes a unified floating-point pipeline to support high-precision arithmetics. High efficiency of the accelerator is achieved by developing configurable data flows to support various BLAS functions. Compared with GPU implementations, CIMBLAS demonstrates several orders of magnitude performance and energy efficiency improvements for executing level-1 and level-2 BLAS functions, and can achieve an energy efficiency improvement of 2.6-24.1 $\times$ for executing level-3 BLAS functions. The improvement increases with the size of the matrix, indicating excellent scalability of CIM-BLAS. Application-level evaluations also demonstrate the potential of CIM for accelerating BLAS.

The billion-scale Large Language Models (LLMs) necessitate deployment on expensive server-grade GPUs with large-storage HBMs and abundant computation capability. As LLM-assisted services become popular, achieving cost-effective LLM inference on budget-friendly hardware becomes the current trend. This has sparked extensive research into relocating LLM parameters from expensive GPUs to external host memory. However, the restricted bandwidth between the host and GPU memory limits the inference performance of existing solutions. This work introduces Hermes, a budget-friendly system that leverages the near-data processing units (NDP) within commodity DRAM DIMMs to enhance the performance of a single consumer-grade GPU, achieving efficient LLM inference. We recognize that the inherent activation sparsity in LLMs naturally divides weight parameters into two categories, termed “hot” and “cold” neurons, respectively. Hot neurons, which consist of only approximately 20% of all weight parameters, account for 80% of the total computational load. In contrast, cold neurons make up the other 80% of parameters but are responsible for just 20% of the computational workload. Leveraging this observation, we propose a heterogeneous computing strategy: mapping hot neurons to a single computation-efficient GPU without large-capacity HBMs, while offloading cold neurons to NDP-DIMMs, which offer large memory size but limited computation capabilities. In addition, the dynamic nature of activation sparsity necessitates a real-time partition of hot and cold neurons and adaptive remapping of cold neurons across multiple NDP-DIMM modules. To tackle these issues, we introduce a lightweight predictor that ensures optimal real-time neuron partition and adjustment between GPU and NDP-DIMMs. Furthermore, we utilize a window-based online scheduling mechanism to maintain load balance among multiple NDP-DIMM modules. In summary, Hermes facilitates the deployment of LLaMA2-70B on consumer-grade hardware at a rate of 13.75 tokens/s and realizes an average 75.24 × speedup over the state-of-the-art offloading-based inference system on popular LLMs.

Large Language Models (LLMs) have emerged as the cornerstone of content generation applications due to their ability to capture relations between newly generated token and the full preceding context. However, this ability stems from the attention mechanism for decoding that retains the entire generation history as key value cache (KV cache). As the generated sequence lengthens, the KV cache expands, causing a substantial memory access bottleneck. In advanced LLM generation systems running on GPUs, the attention mechanism for decoding accounts for more than 50% of the total inference time when the KV cache length reaches 4096. To address this issue, this paper introduces LAD (Locality Aware Decoding), an LLM generation accelerator with algorithm-hardware enhancements that significantly decrease KV cache access, resulting in considerable speedups and energy savings. A key insight underlying LAD is that when the attention score for a specific position remains fixed over the next several decoding steps, it is unnecessary to repeatedly retrieve the associated key and value at each step to reproduce the computation. Our analysis reveals that numerous positions exhibit notable numerical locality in attention scores through multiple decoding steps. Leveraging these insights, we have designed an innovative attention decoding computation method that decreases the frequency of accessing the key and value for positions demonstrating good locality, all while maintaining decoding accuracy. Extensive experiments show that LAD generates sequences with an average ROUGE-1 similarity of 97% compared to those generated by the original model. When the length of KV cache exceeds 2048, the high configuration of LAD accelerator achieves on average (geomean) $10.7 \times$ speedup and $52.4 \times$ energy efficiency for the attention mechanism compared to the A100 GPU. For end-to-end model inference, it also achieves on average $2.3 \times$ speedup and $13.4 \times$ energy efficiency.

Layout Synthesis for Quantum Computing (LSQC) is a critical component of quantum design tools. Traditional LSQC studies primarily focus on optimizing for reduced circuit depth by adopting a device-centric design methodology. However, these approaches overlook the impact of classical processing and communication time, thereby being insufficient for Dynamic Quantum Circuits (DQC).To address this, we introduce CLASS, a controller-centric layout synthesizer designed to reduce inter-controller communication latency in a distributed control system. It consists of a two-stage framework featuring a hypergraph-based modeling and a heuristic-based graph partitioning algorithm. Evaluations demonstrate that CLASS effectively reduces communication latency by up to 100% with only a 2.10% average increase in the number of additional operations.

With the rise of fully homomorphic encryption-based private inference, data centers are anticipated to simultaneously handle two disparate computational demands: plaintext-based non-private inference (NPI) and ciphertext-based private inference (PI). Unfortunately, current solutions face challenges in addressing this trend. They either depend on costly, inflexible dedicated accelerators or utilize general-purpose hardware with inferior performance. This limitation underscores the urgent need for a unified architecture capable of serving both normal and privacy-sensitive users with high efficiency.However, the fundamental disparities in computation patterns and resource management between NPI and PI make their architectural fusion intricate. To bridge this gap, we explore their inherent similarities and apply fine-grained reconfiguration to maximize resource sharing. We propose RTPU, a reconfigurable multi-core architecture that can seamlessly switch between tensor-based plaintext and polynomial ring-based ciphertext computations. Building upon its reconfigurable computing fabric and parallelization mechanism, we introduce a kernel group-based scheduling strategy to optimize hardware utilization and QoS. Experimental results show that: i) The RTPU architecture achieves near-ASIC performance and beyond-ASIC flexibility with substantial silicon reuse between NPI and PI. ii) The RTPU scheduler sustains high resource utilization for multi-tenant workloads with varying privacy requirements.

The emergence of chiplet technology has elevated the IO chiplet to a critical component for scalable system I/O and memory access extension. There is a growing demand for more effective and flexible memory management strategies enabled by the IO chiplet, to support the increasing scale and complexity of chiplet-based systems. Currently, many clustering-based memory access architectures that rely on static or coarse-grained memory clustering struggle with imbalanced memory workloads and cross-chiplet communication, which degrade performance due to resource contention and increased latency. To address this challenge, we propose FlexIO, a scalable IO chiplet architecture with flexible memory controller mapping, optimized for scalable chiplet-based systems with non-uniform memory access. FlexIO introduces a hardware-based application-to-memory controller mapping layer, which enables reconfigurable mapping of memory requests for each application to specified memory controllers. This flexibility is further enhanced by a heuristic-based optimization framework that explores optimal memory mappings tailored to the target chiplet-based system. Experiments demonstrate that FlexIO with the optimization framework improves 24.6 % system throughput, reduces 30.1 % average memory access latency, and decreases$\mathbf{1 6. 5 \%}$worst-case memory access latency compared to prior designs.

One-shot post-training pruning enhances the deployment of billion-scale large language models (LLMs), with the pruning metric playing a pivotal role in determining which weights to remove. However, existing metrics underperform due to their reliance on a simple symbolic combination of weights and activations, overlooking imbalanced weight magnitudes and the disproportionate influence of activation outliers. To overcome these limitations, we introduce \textbf{BaWA}, a novel pruning metric that systematically \underline{Ba}lances \underline{W}eight and \underline{A}ctivation distributions for more effective pruning. BaWA introduces two key innovations: \textbf{magnitude normalization}, which mitigates weight imbalance across channels for fairer pruning decisions, and \textbf{outlier regularization}, which reduces the impact of activation outliers, ensuring more appropriate channel prioritization. To further enhance its effectiveness, BaWA incorporates an efficient and automatic framework for optimizing normalization and regularization hyperparameters. Extensive experiments validate BaWA as a state-of-the-art (SOTA) pruning metric. For instance, applying BaWA to induce 2:4 sparsity in Mistral-7B reduces perplexity in language comprehension by 2.49 and improves average downstream task accuracy by 3.08\%, outperforming the previous SOTA method Wanda.

Embodied AI agents responsible for executing interconnected, long-sequence household tasks often face difficulties with in-context memory, leading to inefficiencies and errors in task execution. To address this issue, we introduce KARMA, an innovative memory system that integrates longterm and short-term memory modules, enhancing large language models (LLMs) for planning in embodied agents through memory-augmented prompting. Karma distinguishes between long-term and short-term memory, with long-term memory capturing comprehensive 3D scene graphs as representations of the environment, while short-term memory dynamically records changes in objects' positions and states. This dualmemory structure allows agents to retrieve relevant past scene experiences, thereby improving the accuracy and efficiency of task planning. Short-term memory employs strategies for effective and adaptive memory replacement, ensuring the retention of critical information while discarding less pertinent data. Compared to state-of-the-art embodied agents enhanced with memory, our memory-augmented embodied AI agent improves success rates by$1.3 \times$and$2.3 \times$in Composite Tasks and Complex Tasks within the AI2-THOR simulator, respectively, and enhances task execution efficiency by$3.4 \times$and$62.7 \times$. Furthermore, we demonstrate that KARMA's plug-and-play capability allows for seamless deployment on real-world robotic systems, such as mobile manipulation platforms. Through this plug-and-play memory system, KARMA significantly enhances the ability of embodied agents to generate coherent and contextually appropriate plans, making the execution of complex household tasks more efficient. Our code is available at https://github.com/WZX0Swarm0Robotics/KARMA/tree/master.

Keypoint detection and description are fundamental tasks for a variety of computer vision applications. Due to the limited receptive field of convolutional neural networks, most existing methods based on deep learning mainly focus on the local features, instead of taking into account the global context from entire image. The purpose of this work is to enhance the detection and description process of keypoints by leveraging global information obtained from Transformer, and to boost the consistence between keypoints and descriptors through their interaction. Specifically, the above two improvements are respectively implemented through the Local & Global Context Aggregation (LGCA) Module and Point & Descriptor Cross Augmentation (PDCA) Module proposed in this article. The LGCA module, which can model the long-range context, is inserted a Feature Pyramid Network (FPN) to extract features which contain diverse scales and different receptive fields. Moreover, the PDCA module enhances descriptors by the geometry information of keypoints detected, while enhancing the keypoint detection process by the position coordinates of correctly matched descriptors. Finally, we design a lightweight model to improve the running efficiency. Extensive experiments on various tasks demonstrate that our method achieves a substantial performance improvement over the current feature extraction methods. Code is available at: https://github.com/meng152634/CA2Point.

Embodied AI robots have the potential to fundamentally improve the way human beings live and manufacture.Continued progress in the burgeoning field of using large language models to control robots depends critically on an efficient computing substrate, and this trend is strongly evident in manipulation tasks.In particular, today's computing systems for embodied AI robots for manipulation tasks are designed purely based on the interest of algorithm developers, where robot actions are divided into a discrete frame basis.Such an execution pipeline creates high latency and energy consumption.This paper proposes Corki, an algorithm-architecture co-design framework for real-time embodied AI-powered robotic manipulation applications.We aim to decouple LLM inference, robotic control, and data communication in the embodied AI robots' compute pipeline.Instead of predicting action for one single frame, * equal contribution.

As the number of chiplets in 2.5D IC continues to increase, existing chiplet placement method faces two main challenges: (1) the combinatorial explosion in the search space, and (2) the difficulty of achieving global optimization through iterativing chiplet and terminal placement. To tackle these challenges, we develop an efficient analytical RDL chiplet-terminal co-placement algorithm, to ensure simultaneous placement of chiplets and terminals. Our algorithm employs RDL chiplet-terminal co-placement in three stages: analytical global placement, legalization, and bump-terminal assignment, to achieve high-quality placement results that comply with design rules. Experimental results demonstrate that our algorithm reduces average wirelength by 31% compared to prior work for the common testcases, with a maximum speedup of up to 6500× in testcases with more than 10 chiplets.

The design of a scalable Quantum Control Architecture (QCA) faces two primary challenges.First, the continuous growth in qubit counts has rendered distributed QCA inevitable, yet the nondeterministic latencies inherent in feedback loops demand cycleaccurate synchronization across multiple controllers.Existing synchronization strategies -whether lock-step or demand-drivenintroduce significant performance penalties.Second, existing quantum instruction set architectures are polarized, being either too abstract or too granular.This lack of a unifying design necessitates recurrent hardware customization for each new control requirement, which limits the system's reconfigurability and impedes the path toward a scalable and unified digital microarchitecture.Addressing these challenges, we propose Distributed-HISQ, featuring: (i) HISQ, A universal instruction set that redefines quantum control with a hardware-agnostic design.By decoupling from quantum operation semantics, HISQ provides a unified language for control sequences, enabling a single microarchitecture to support various control methods and enhancing system reconfigurability.(ii) BISP, a booking-based synchronization protocol that can potentially achieve zero-cycle synchronization overhead.The feasibility and adaptability of Distributed-HISQ are validated through its implementation on a commercial quantum control system targeting superconducting qubits.We performed a comprehensive evaluation using a customized quantum software stack.Our results show that BISP effectively synchronizes multiple control boards, leading to a 22.8% reduction in average program execution time and a ∼ 5× reduction in infidelity when compared to an existing lock-step synchronization scheme.

Graph processing plays an important role in many practical applications. However, the inherent characteristics of graph processing, including random memory access and the low computation-to-communication ratio, make it difficult to efficiently execute on traditional computing architectures, such as CPUs and GPUs. Near-memory computing has the characteristics of low latency and high bandwidth. It is widely regarded as a promising direction for designing graph processing accelerators. However, the storage space of a single device cannot meet the demand of large-scale graph processing. Using multiple devices will bring lots of inter-device data transmission, which may counteract the benefits of near-memory computing. To fundamentally reduce the data transmission overhead, we propose a data-centric graph processing framework for systems with multiple near-memory computing devices. The framework uses a data-centric programming model as the software hardware interface. For software, we propose an optimized data flow and a heuristic multi-step weighted maximum matching algorithm to achieve efficient inter-device communication and ensure load balancing. For hardware, we design a data reuse driven task controller and a data type-aware on-chip memory, which can effectively improve the utilization of the on-chip memory. Compared with the two most recent near-memory graph accelerators, our framework significantly reduces energy consumption and inter-device communication.

Due to the demonstrated superiority, dynamic neural networks (NNs), which adapt their network structures to different inputs, have been recognized as an optimized alternative to conventional static NNs. However, researchers have not explored the implications of dynamic NN on neural processing unit (NPU) architecture design. Consequently, we analyze the characteristics and inefficient sources of executing dynamic NNs on existing hardware. From our analysis, existing NPUs, designed for static NNs, cannot effectively handle the execution of dynamic operator and agent-dependent data loading in dynamic NNs.To this end, we present DNA, an efficient accelerator optimized to deal with the challenges of running general dynamic NNs. Firstly, to improve the execution efficiency of dynamic operators, we propose a transverter-based online scheduling strategy to rapidly generate efficient scheduling for each dynamic operator. Secondly, to mitigate hardware idleness caused by the non-deterministic and agent-dependent data access patterns in dynamic NNs, we propose a novel predictor-based prefetching strategy that achieves effective data preloading with negligible cost. We implemented our accelerator, DNA, by integrating an additional online scheduler into a typical many-core baseline accelerator. According to our evaluation of various dynamic NNs, DNA achieves 3.48× speedup and 3.03× energy savings over the baseline accelerator.

The reconfigurable processing-in-memory (PIM) architecture has garnered significant attention in recent years due to its versatility and ability to overcome storage limitations. However, it faces challenges, such as overly complex mapping and routing caused by the fine granularity of basic logic units, and the inclusion of numerous redundant devices to achieve reconfigurability. To address these issues, we have designed a software-hardware co-design reconfigurable PIM architecture called Re-Meltrix. Its hardware architecture uses an resistive random-access memory array as the foundation, combined with well-designed peripheral circuits. Maintaining a controllable area, it integrates logic, storage, ternary content-address memory, and interconnection modes into a unified tile architecture and implements two-level independent interconnection within and between tiles. This approach achieves a single tile logic capacity multiple times that of the most advanced reconfigurable PIM architectures currently available, thereby resolving mapping and routing difficulties at the hardware level. Our proposed function synthesis, combined with the hardware architecture, specifically optimizes two-level interconnection separation and module segmentation, further reducing interconnection complexity and improving tile usage efficiency. Experiments have demonstrated that our architecture outperforms the state-of-the-art Liquid Silicon by 2.00–$4.31\times $in performance and reduces power consumption by 29%–68%. Compared with the previously published Meltrix, the area has decreased by 15%–35%, with the area and power consumption remaining almost unchanged.

In the past decade, various processing-in-memory (PIM) accelerators based on various devices, micro-architectures, and interfaces have been proposed to accelerate deep neural networks (DNNs). How to deploy DNNs onto PIM-based accelerators is the key to explore PIM’s high performance and energy efficiency. The scale of DNN models, the diversity of PIM accelerators, and the complexity of deployment are far beyond the human deployment capability. Hence, an automatic deployment methodology is indispensable. In this work, we propose PIMCOMP, an end-to-end DNN compiler tailored for PIM accelerators, achieving efficient deployment of DNN models on PIM hardware. PIMCOMP can adapt to various PIM architectures by using an abstract configurable PIM accelerator template with a set of pseudo instructions, which is a high-level abstraction of the hardware’s fundamental functionalities. Through a generic multilevel optimization framework, PIMCOMP realizes an end-to-end conversion from a high-level DNN description to pseudo instructions, which can be further converted to specific hardware intrinsics/primitives. The compilation addresses two critical issues in PIM-accelerated inference from a system perspective: 1) resource utilization and 2) dataflow scheduling. PIMCOMP adopts a flexible unfolding format to reshape and partition convolutional layers, adopts a weight-layout guided computation-storage-mapping approach to enhance resource utilization, and balances the system’s computation, memory access, and communication characteristics. For dataflow scheduling, we design two scheduling algorithms with different interlayer pipeline granularities to support varying application scenarios while ensuring high-computational parallelism. Experiments demonstrate that PIMCOMP improves throughput, latency, and energy efficiency across various architectures. PIMCOMP is open-sourced athttps://github.com/sunxt99/PIMCOMP-NN.

Graph mining, critical for discovering specific patterns within complex structures, is becoming increasingly important in our data-driven world. Due to their memory-bound nature, graph mining applications encounter significant limitations with conventional processor-centric systems, like central processing units (CPUs) and graphics processing units (GPUs), stemming from the costly data movement between memory and processing units. Memory-centric computing systems, such as processing-in-memory (PIM) where computation occurs directly within or near memory modules, have the potential to accelerate graph mining. However, accelerating graph mining applications with PIM presents three primary challenges: (1) the difficulty in utilizing locality, (2) the challenge of exploring parallelism, and (3) the complexity of workload offloading between PIM and CPU. Addressing these intricate challenges, we introduce ProMiner, a novel framework that integrates three key techniques through cohesive software and hardware co-design. First, we propose a partitioning method tailored for graph mining to enhance data locality. Second, we design a coarse-fine parallelism optimization scheme to explore parallelism across different levels of memory. Third, we introduce a concurrency-aware mechanism for performance estimation, aimed at identifying the optimal computing engine for workload offloading to maximize performance. Our experimental results demonstrate that ProMiner significantly advances the state-of-the-art in graph mining, achieving 48.8% and 29.9% execution time reduction over NDMiner and DIM- Mining, respectively.

Recent advances in large language models (LLMs) have demonstrated significant potential for automated hardware description language (HDL) code generation from high-level specifications. However, two critical challenges limit further progress in this domain: the scarcity of quality Verilog training data and the inability of current approaches to generate RTL code optimized for power, performance, and area (PPA) metrics. This article presents a comprehensive data-centric framework that addresses these limitations through innovations in both pre-fine-tuning data preparation and after-fine-tuning optimization strategies. In the pre-fine-tuning phase, we tackle the data scarcity problem with an automated design-data augmentation framework that generates high-volume, high-quality natural language specifications aligned with corresponding Verilog code and EDA scripts. Our approach creates a complete RTL-level feedback loop by augmenting EDA scripts, RTL code, and EDA tool feedback. In the after-fine-tuning phase, we focus on generating PPA-aware RTL code through a novel search and prompt framework. Our approach implements iterative filtering and selection of LLM-generated Verilog variants while providing high-quality predefined prompts, including composition and interface specifications. To evaluate the effectiveness of our data augmentation method, we fine-tune Llama 2-13B and Llama 2-7B models using the dataset generated by our augmentation framework. The results demonstrate a significant improvement in the Verilog generation tasks with LLMs. Moreover, the accuracy of Verilog generation surpasses that of the current state-of-the-art open-source Verilog generation model, increasing from 58.8% to 70.6% with the same benchmark. Our 13B model has a pass rate improvement compared with GPT-3.5 in Verilog generation and outperforms in EDA script (i.e., SiliconCompiler) generation with only 200 EDA script data. Additionally, to evaluate the effectiveness of the our agent framework, we compare the PPA on the GPT-3.5, where the results show that the agent refined RTL code can have a better quality than the generated RTL code only with GPT-3.5.

Hardware description language (HDL) code designing is a critical component of the chip design process, requiring substantial engineering and time resources. Recent advancements in large language models (LLMs), such as GPT series, have shown promise in automating HDL code generation. However, current LLM-based approaches face significant challenges in meeting real-world hardware design requirements, particularly in handling complex designs and ensuring code correctness. Our evaluations reveal that the functional correctness rate of LLM-generated HDL code significantly decreases as design complexity increases. In this article, we propose the AutoSilicon framework, which aims to scale up the hardware design capability of LLMs. AutoSilicon incorporates an agent system, which (1) allows for the decomposition of large-scale, complex code design tasks into smaller, simpler tasks; (2) provides a compilation and simulation environment that enables LLMs to compile and test each piece of code it generates; and (3) introduces a series of optimization strategies. Experimental results demonstrate that AutoSilicon can scale hardware designs to projects with code equivalent to over 10,000 tokens. In terms of design quality, it further improves the syntax correctness rate and functional correctness rate compared with approaches that do not employ any extensions. For example, compared to directly generating HDL code using GPT-4-turbo, AutoSilicon enhances the syntax correctness rate by an average of 35.8% and improves functional correctness by an average of 35.6%.

Chiplet technology has been a hot topic due to its potential for more efficient implementation of large-scale integrated circuits. In chiplet manufacturing, the general-purpose active interposer usually integrates chiplets from different vendors with a typical mesh network. This method of manufacturing is broadly recognized for its cost-efficiency. However, untrusted vendors make the chiplet system vulnerable to security threats such as timing side-channel attacks (TSA) based on network contention information. Even worse, the reliability of each chiplet is usually unknown beforehand to a general-purpose interposer’s manufacturer, so that TSAs can be on arbitrary chiplets at arbitrary time in the manufacturer’s view. To address this challenge, this work first quantitatively analyzes the attack patterns including reinforced styles, based on which, a resilient interposer architecture named Resister is proposed. A hardware defender is designed in every router to globally detect the malicious transaction patterns at runtime, and adaptively detour the transaction packets accordingly for security while maintaining the performance. According to the evaluation of GEM5 on SPEC 2017 and PARSEC benchmarks, Resister can effectively mitigate TSA with only a 1.7% performance overhead.

Computing-in-memory (CIM) architecture has become a promising solution to improve the parallelism of the multiply-and-accumulation (MAC) operation for artificial intelligence (AI) processors. Recently, revived CIM engine partly relieves the memory wall issue by integrating computation in/with the memory. However, current CIM solutions still require large data movements with the increase of the practical neural network model and massive input data. Previous CIM works only considered computation without concern for the memory attribute, leading to a low memory computing ratio. This article presents a static-random access-memory (SRAM)-based digital CIM macro supporting pipeline mode and computation-memory-aware technique to improve the memory computing ratio. We develop a novel weight driver with fine-grained ping-pong operation, avoiding the computation stall caused by weight update. Based on our evaluation, the peak energy efficiency is 19.78 TOPS/W at the 22-nm technology node, 8-bit width, and 50% sparsity of the input feature map.

The stagnation of Moore’s law stimulates the concept of breaking monolithic chips into smaller chiplets. However, tactic design partitioning remains an unaddressed issue despite its crucial role in chip product cost reduction. In this paper, we propose Chipletizer, a framework to guide the design partitioning for those who would benefit from chiplet reuse across a line of SoC products. The proposed generic framework supports the repartitioning of multiple SoCs into reusable chiplets economically and efficiently with user-specified parameters. Experimental results show that, compared with existing partitioning strategies, our proposed framework achieves notable cost improvement on realistic products with acceptable power and latency overheads.

Despite extensive efforts, existing approaches to design accelerators for optimization-based robotic applications have limitations. Some approaches focus on accelerating general matrix operations, but they fail to fully exploit the specific sparse structure commonly found in many robotic algorithms. On the other hand, certain methods require manual design of dedicated accelerators, resulting in inefficiencies and significant non-recurring engineering (NRE) costs.

Sparse matrix-matrix multiplication (SpMM) is a critical computational kernel in numerous scientific and machine learning applications. SpMM involves massive irregular memory accesses and poses great challenges to conventional cache-based computer architectures. Recently dedicated SpMM accelerators have been proposed to enhance SpMM performance. However, current SpMM accelerators still face challenges in adapting to varied sparse patterns, fully exploiting inherent parallelism, and optimizing cache performance. To address these issues, we introduce ACES, a novel SpMM accelerator in this study. First, ACES features an adaptive execution flow that dynamically adjusts to diverse sparse patterns. The adaptive execution flow balances parallel computing efficiency and data reuse. Second, ACES incorporates locality-concurrency co-optimizations within the global cache. ACES utilizes a concurrency-aware cache management policy, which considers data locality and concurrency for optimal replacement decisions. Additionally, the integration of a non-blocking buffer with the global cache enhances concurrency and reduces computational stalls. Third, the hardware architecture of ACES is designed to integrate all innovations. The architecture ensures efficient support across the adaptive execution flow, advanced cache optimizations, and fine-grained parallel processing. Our performance evaluation demonstrates that ACES significantly outperforms existing solutions, providing a 2.1× speedup and marking a substantial advancement in SpMM acceleration.

With the rapid up-scaling of transformer-based large language models (LLM), training these models is becoming increasingly demanding on novel parallel training techniques. Tensor partitioning is an extensively researched parallel technique, encompassing data and model parallelism, and has a significant influence on LLM training performance. However, existing state-of-the-art parallel training systems are based on incomplete tensor partitioning space, where the distribution of partitioned sub-operators is limited to the spatial dimension. We discover that introducing the temporal dimension into tensor partitioning of LLM training instance provides extra opportunities to avoid collective communication across devices, saving memory space and also overlapping device-to-device communication with computation. In this paper, we propose a new tensor partition primitive that distributes sub-operators along both the spatial and temporal dimensions to further explore communication and memory overhead reduction over current solutions. This new primitive creates a broader parallelization space and leads to parallel solutions that achieve better training throughput with lower peak memory occupancy compared to state-of-the-art techniques. To efficiently deploy optimized parallel transformer model training to multiple devices, we further present an optimization algorithm that can find optimal parallel solutions from our spatial-temporal tensor partition space with acceptable search time. Our evaluation shows that our optimized tensor partitioning achieves up to 1.68 × training throughput with 69% peak memory occupancy compared to state-of-the-art distributed training systems when training LLMs. Upon scaling to 32 GPUs, the geo-mean speedup across benchmarks is 1.30 ×. When applied in 3D parallelism, up to 1.46 × training throughput can be achieved.

Recent advances in large language models have demonstrated their potential for automated generation of hardware description language (HDL) code from high-level prompts. Researchers have utilized fine-tuning to enhance the ability of these large language models (LLMs) in the field of Chip Design. However, the lack of Verilog data hinders further improvement in the quality of Verilog generation by LLMs. Additionally, the absence of a Verilog and electronic design automation (EDA) script data augmentation framework significantly increases the time required to prepare the training dataset for LLM trainers. This paper proposes an automated design-data augmentation framework, which generates high-volume and high-quality natural language aligned with Verilog and EDA scripts. For Verilog generation, it translates Verilog files to an abstract syntax tree and then maps nodes to natural language with a predefined template. For Verilog repair, it uses predefined rules to generate the wrong verilog file and then pairs EDA Tool feedback with the right and wrong verilog file. For EDA Script generation, it uses existing LLM(GPT-3.5) to obtain the description of the Script. To evaluate the effectiveness of our data augmentation method, we finetune Llama2--13B and Llama2-7B models using the dataset generated by our augmentation framework. The results demonstrate a significant improvement in the Verilog generation tasks with LLMs. Moreover, the accuracy of Verilog generation surpasses that of the current state-of-the-art open-source Verilog generation model, increasing from 58.8% to 70.6% with the same benchmark. Our 13B model (ChipGPT-FT1) has a pass rate improvement compared with GPT-3.5 in Verilog generation and outperforms in EDA script (i.e., SiliconCompiler) generation with only 200 EDA script data.

Fully Homomorphic Encryption (FHE) is one of the most promising privacy-preserving techniques that has drawn increasing attention from academia and industry due to its ideal security. Chiplet-based designs integrate multiple dies into the package delivering high performance and thereby are embraced by the resources-hungry FHE. Despite the chiplet-based system with various specialized accelerators, it falls short in supporting FHE with the novel polynomial operations. For a chiplet-based system that is not tailored for FHE, one common approach to support FHE is designing a new dedicated accelerator, However, this full design-and-build approach overlooks the existing abundant resources of accelerators in the system and incurs repeated customization and resource waste.

Quantization is one of the most hardware-efficient ways to reduce inference costs for deep neural network (DNN) models. Nevertheless, with the continuous increase of DNN model sizes (240× in two years) and the emergence of large language models, existing static quantization methods fail to utilize the sparsity and redundancy of models sufficiently. Motivated by the pervasive dynamism in data tensors across DNN models, we propose a dynamic precision quantization algorithm to further reduce computational costs beyond statically quantized DNN models. Furthermore, we find that existing precision-flexible accelerators cannot support the DNN models with dynamic precision. To this end, we design a novel accelerator, Drift, and achieve online scheduling to efficiently support dynamic precision execution. We conduct experiments with various DNN models, including CNN-based and Transformer-based models. Evaluation results show that Drift achieves 2.85× speedup and 3.12× energy saving compared to existing precision-flexible accelerators with statically quantized models.

Graph convolutional network (GCN) has been proven powerful in various tasks for it combines both neural networks and graph processing operators. However, this characteristic makes GCN exhibit hybrid execution patterns, which is unfavorable for CPUs and GPUs. Therefore, designing specialized GCN accelerators is becoming a prevalent paradigm. Unfortunately, as graph scale continues to grow, existing GCN accelerators suffer from significant bandwidth consumption and memory footprint as they neglect the inherent semantic redundancy of vertex features. Although applying Product Quantization to GCN is a promising solution to reduce the sizeable graph data via distilling semantic redundancy, it introduces novel operations with unique patterns that existing GCN accelerators cannot support. In this paper, we propose GPACE, an energy-efficient GCN accelerator that can fully harness the potential of PQ to reduce bandwidth consumption and data movement. GPACE is designed with a lookup-efficient architecture and well-optimized dataflow to support the unique data access and computation pattern of PQ-GCN. In addition to leveraging PQ to distill semantic redundancy, we exploit the operation redundancy and propose a redundancy-aware architecture to detect and reduce types of redundant operations to achieve higher energy efficiency. Evaluations show GPACE achieves high speedup and energy saving compared with CPU, GPU, and specialized GCN accelerators.

Processing-in-memory architectures have been re-garded as a promising solution for CNN acceleration. Existing PIM accelerator designs rely heavily on the experience of experts and require significant manual design overhead. Manual design cannot effectively optimize and explore architecture implementations. In this work, we develop an automatic framework PIMSYN for synthesizing PIM-based CNN accelerators, which greatly facilitates architecture design and helps generate energy-efficient accelerators. PIMSYN can automatically transform CNN applications into execution workflows and hardware construction of PIM accelerators. To systematically optimize the architecture, we embed an architectural exploration flow into the synthesis framework, providing a more comprehensive design space. Experiments demonstrate that PIMSYN improves the power efficiency by several times compared with existing works.

Processing-in-memory (PIM) has shown extraordinary potential in accelerating neural networks. To evaluate the performance of PIM accelerators, we present an ISA-based simulation framework including a dedicated ISA targeting neural networks running on PIM architectures, a compiler, and a cycle-accurate configurable simulator. Compared with prior works, this work decouples software algorithms and hardware architectures through the proposed ISA, providing a more convenient way to evaluate the effectiveness of softwarelhardware optimizations. The simulator adopts an event-driven simulation approach and has better support for hardware parallelism. The framework is open-sourced at https://github.com/wangxy-2000/pimsim-nn.

Natural language interfaces have exhibited considerable potential in the automation of Verilog generation derived from high-level specifications through the utilization of large language models, garnering significant attention. Nevertheless, this paper elucidates that visual representations contribute essential contextual information critical to design intent for hardware architectures possessing spatial complexity, potentially surpassing the efficacy of natural-language-only inputs. Expanding upon this premise, our paper introduces an open-source benchmark1 for multi-modal generative models tailored for Verilog synthesis from visual-linguistic inputs, addressing both singular and complex modules. Additionally, we introduce an open-source visual and natural language Verilog query language framework to facilitate efficient and user-friendly multi-modal queries. To evaluate the performance of the proposed multi-modal hardware generative AI in Verilog generation tasks, we compare it with a popular method that relies solely on natural language. Our results demonstrate a significant accuracy improvement in the multi-modal generated Verilog compared to queries based solely on natural language. We hope to reveal a new approach to hardware design in the large-hardware-design-model era, thereby fostering a more diversified and productive approach to hardware design.

Sorting is one of the most fundamental operations in computer programming and used in countless algorithms. The performance of traditional von Neumann computers running sorting is limited by the bandwidth between memories and processors. Computing-in-memory (CiM) is a promising technology which has the potential to solve the “memory wall” bottleneck. CiM is suitable for data-intensive applications, and it is ideal for accelerating large-scale data sorting. In this paper, we propose a novel in-memory sorting accelerator, named MemSort, based on a proposed in-memory comparison array design based on emerging non-volatile devices. MemSort supports three sort operations including counting sort, merging sort, and the combination of counting sort and merging sort. We build a performance model for the combination sort which enables flexible allocation of resources under given constraints to meet the requirements of various applications for sorting. The evaluation results show that MemSort shows significant performance improvement and energy efficiency at both the system level and application level when processing large-scale data sorting. Compared with the CPU implementation, MemSort achieves energy savings of 19.69-72.75x and speedups of 24.48-38.58 x with the same power constraint. MemSort's throughput is at least 4.86 x higher than that of the recent FPGA-based sorting accelerator FANS. MemSort exhibits more than 11 x throughput and 4.03 x area efficiency, compared with the recent CiM - based sorting accelerator, RIME.

Graph pattern matching (GPM), a critical algorithm for discovering specific patterns within complex structures, is becoming increasingly important in the data-driven world. GPM applications are memory-bound and can be accelerated by memory-centric computing systems, such as processing-in-memory (PIM). However, there are three primary challenges when it comes to accelerating GPM applications with PIM: (1) difficulty in utilizing locality, (2) heavy data movement, and (3) heavy comparison overhead due to pruning. To address these challenges, we propose AceMiner, a framework to accelerate GPM applications with a software and hardware co-design per-spective using PIM. In AceMiner, we embed hybridCache, a novel in-DRAM cache system with lower access latency and optimized replacement policy, to leverage the potential locality and reduce data movement in PIM. Additionally, we introduce a comparison unit to address the huge pruning overhead. Experimental results show that AceMiner outperforms the state-of-the-art, achieving speedups of 40.2% and 13.3% over NDMiner and DIMMining respectively, with less energy consumption and design overhead.

Low-end field programmable gate arrays (FPGAs) are difficult to deploy typical convolutional neural networks (C- NNs) owing to the limited hardware resources and the increasing model computational complexity. Fast Fourier transform (FFT) is a promising solution for saving both computation and memory footprint by convolving in the frequency domain. However, few FPGA accelerators can take full advantage at the computation level, because of the distinct element-wise complex calculation in the frequency domain. In this work, we present an FPGA-based 8-bit inference accelerator (called FAF) that packs frequency-domain calculations into digital signal processing (DSP) blocks to fully utilize DSPs for performance boost. We then provide a mapping dataflow to maximize the reduction of redundant packing operations by frequency-domain data reuse. Evaluations based on representative CNN benchmarks show that our work can achieve 1.5-6.9× better power efficiency compared with representative FPGA baselines.

Graph pattern mining discovers important patterns in graphs. It is both computation-and memory-intensive, characterized by numerous set operations and irregular memory access. Graph pattern mining inherently involves a large number of independent tasks, helping to alleviate its computational bottleneck through parallel processing. However, after exploiting parallelism, memory access will become the primary bottleneck. Existing parallelism strategies severely result in redundant and inefficient memory access, making the performance memory bounded. This paper proposes TMiner, a graph pattern mining architecture with optimized memory performance through a systematically designed software-hardware stack. TMiner fundamentally reduces redundant memory access of parallel graph pattern mining in three aspects. (1) TMiner leverages a task partitioning approach based on disjoint neighbor vertex set access, reducing redundant memory access between PEs. (2) TMiner utilizes the global neighbor vertex information to coalesce the access from different neighbor vertex subsets at compilation time, which not only reduces redundant memory access within a task but also improves the data locality. (3) TMiner adopts a data reuse-oriented task scheduling mechanism, which dynamically migrates and merges tasks with similar memory access patterns, reducing redundant memory access within a PE at runtime. A DIMM-based near-memory architecture that exploits the DRAM's internal bandwidth is elaborated for high-performance graph pattern mining, which incorporates the proposed memory access optimization techniques and an extended ISA. Compared with the state-of-the-art software and hardware baselines, TMiner significantly improves the performance.

Sparse matrix multiplication is widely used in various practical applications. Different accelerators have been proposed to speed up sparse matrix-dense vector multiplication (SpMV), sparse matrix-sparse vector multiplication (SpMSpV), sparse matrix-dense matrix multiplication (SpMM), and sparse matrix-sparse matrix multiplication (SpMSpM). The performance of traditional sparse matrix multiplication accelerators is typically bounded by memory access due to the poor data locality and irregular memory access. In-memory computing (IMC) is a promising technique to alleviate the memory bottleneck. Previous IMC studies are mostly focused on accelerating a single sparse matrix multiplication function. In this paper, we propose GAS, a general-purpose IMC accelerator for sparse matrix multiplication. GAS integrates non-volatile memory based content-addressable memory (CAM) arrays and multiply-add computation (MAC) arrays to support sparse matrices represented in the double-precision floating-point format. Using a unified outer product based multiplication methodology, GAS supports the acceleration of SpMV, SpMSpv, SpMM, and SpMSpM. We further propose four optimization techniques to speed up the computation of GAS. GAS achieves significant speedups and energy savings over central processing unit (CPU) and graphics processing unit (GPU) implementations. Compared with state-of- the-art traditional and IMC-based accelerators, GAS not only supports more functions, but also achieves higher performance and energy efficiency.

Efficient deep learning models, especially optimized for edge devices, benefit from low inference latency to efficient energy consumption. Two classical techniques for efficient model inference are lightweight neural architecture search (NAS), which automatically designs compact network models, and quantization, which reduces the bit-precision of neural network models. As a consequence, joint design for both neural architecture and quantization precision settings is becoming increasingly popular. There are three main aspects that affect the performance of the joint optimization between neural architecture and quantization: quantization precision selection (QPS), quantization aware training (QAT), and neural architecture searching (NAS). However, existing works focus on at most twofold of these aspects, and result in secondary performance. To this end, we proposed a novel automatic optimization framework, DAQUDAQU is an ancient liquor fermentation process., that allows jointly searching for Pareto-optimal neural architecture and quantization precision combination among more than 1047 quantized subnet models. To overcome the instability of the conventional automatic optimization framework, DAQU incorporates a warm-up strategy to reduce the accuracy gap among different neural architectures, and a precision-transfer training approach to maintain flexibility among different quantization precision settings. Our experiments show that the quantized lightweight neural networks generated by DAQU consistently outperform state-of-the-art NAS and quantization joint optimization methods.

Silicon interposer re-usage is drawing attention for cost-effective multi-chiplet integrated systems. To address the communication awareness of inter/off-chiplet interconnect, the paper proposes a field-programmable interconnect fabric and develops its corresponding automatic physical integration tool. The tile-based fabric consists of turnout, cross-over boxes and parallel tracks. It features micro-bump-wise connecting flexibility and hardware efficiency. The automation flow performs chiplet location optimization and efficient bump-to-bump routing, supporting multi-lane bus interconnect and miscellaneous external ports. The methodology is validated by 9 different integration scenarios, where the routability is guaranteed when the local resource utilization ratio approaches 94.5%. The data’s maximum interconnect latency is 2.2 ns and the energy consumption is 1.18 pJ/bit at a bitrate of 1 Gbps. The latency consumes$16.5\times \sim ~53.4\times $fewer clock cycles than the state-of-the-art network-on-package-based reusable interposer architectures.

The resistive random-access memory (ReRAM) has widely been used to accelerate convolutional neural networks (CNNs) thanks to its analog in-memory computing capability. ReRAM crossbars not only store layers’ weights, but also perform in-situ matrix-vector multiplications which are core operations of CNNs. To boost the performance of ReRAM-based CNN accelerators, crossbars can be duplicated to explore more intra-layer parallelism. The crossbar allocation scheme can significantly influence both the computing throughput and bandwidth requirements of ReRAM-based CNN accelerators. Under the resource constraints (i.e., crossbars and memory bandwidths), how to find the optimal number of crossbars for each layer to maximize the inference performance for an entire CNN is an unsolved problem. In this work, we find the optimal crossbar allocation scheme by mathematically modeling the problem as a constrained optimization problem and solving it with a dynamic programming based solver. Experiments demonstrate that our model for CNN inference time is almost precise, and the proposed framework can obtain solutions with near-optimal inference time. We also emphasize that communication (i.e., data access) is an important factor and must also be considered when determining the optimal crossbar allocation scheme.

Convolutional neural networks (CNNs) are typically computationally heavy. Fast algorithms such as fast Fourier transforms (FFTs), are promising in significantly reducing computation complexity by replacing convolutions with frequency-domain element-wise multiplication. However, the increased high memory access overhead of complex weights counteracts the computing benefit, because frequency-domain convolutions not only pad weights to the same size as input maps, but also have no sharable complex kernel weights. In this work, we propose an FFT-based kernel-sharing technique called FS-Conv to reduce memory access. Based on FS-Conv, we derive the sharable complex weights in frequency-domain convolutions, which has never been solved. FS-Conv includes a hybrid padding approach, which utilizes the inherent periodic characteristic of FFT transformation to provide sharable complex weights for different blocks of complex input maps. We in addition build a frequency-domain inference accelerator (called Yixin) that can utilize the sharable complex weights for CNN accelerations. Evaluation results demonstrate the significant performance and energy efficiency benefits compared with the state-of-the-art baseline.

Application-specific instruction-set processors (ASIP) has been widely used to speedup specific applications based on general-purpose processor (CPU) ISA-extension and scalar/vector units customization. However, as deep neural processing unit (NPU) becomes a dominant IP in nowadays system-on-chip (SoC) designs, the rich computational and memory resource of the NPUs integrated into advanced CPUs should also be utilized to achieve an even better application performance boost than vector/scalar compute-unit customization only. In this paper, we propose APPEND, a novel framework that tries to enrich the ASIP design methodology by taking the co-designing of both NPU and RISC-V CPU into consideration. To fully utilize and customize the resources of CPU and NPU, APPEND automatically (1) identifies the NPU-compatible kernels from the target application and partition the applications in between the NPU and RISC-V CPU core, (2) based on the application performance specification, applies the necessary hardware parameterization and customization based on the RISC-V CPU and NPU templates, and also (3) generates the extended NPU instructions to accelerate the critical and compatible kernels of the target application.

Crossbar-based PIM DNN accelerators can provide massively parallel in-situ operations. A specifically designed compiler is important to achieve high performance for a wide variety of DNN workloads. However, some key compilation issues such as parallelism considerations, weight replication selection, and array mapping methods have not been solved. In this work, we propose PIMCOMP - a universal compilation framework for NVM crossbar-based PIM DNN accelerators. PIMCOMP is built on an abstract PIM accelerator architecture, which is compatible with the widely used Crossbar/IMA/Tile/Chip hierarchy. On this basis, we propose four general compilation stages for crossbar-based PIM accelerators: node partitioning, weight replicating, core mapping, and dataflow scheduling. We design two compilation modes with different inter-layer pipeline granularities to support high-throughput and low-latency application scenarios, respectively. Our experimental results show that PIMCMOP yields improvements of 1.6× and 2.4× in throughput and latency, respectively, relative to PUMA.

Sparse matrix-dense vector multiplication (SpMV) is widely used in various applications. The performance of traditional SpMV accelerators is bounded by memory. In-memory computing (IMC) is a promising technique to alleviate the memory bottleneck. The current IMC accelerator cannot support sparse storage format and in-situ floating-point multiplication at the same time. In this paper, we propose FSPA, an ferroelectric field-effect transistor (FeFET) based SpMV accelerator. FSPA integrates novel content-addressable memory (CAM) arrays and multiply-add computation (MAC) arrays to support sparse matrices represented in the floating-point format. FSPA achieves significant speedups and energy savings over CPU, GPU and two state-of-the-art IMC accelerators.

In this work, we propose Layer-Puzzle, a multi-task allocation and scheduling framework for multi-core NPUs. Based on the proposed latency-prediction model and dynamic parallelization scheme, Layer-Puzzle can generate near-optimal results for each layer under given hardware resources and traffic congestion levels. As an online scheduler, Layer-Puzzle performs a QoS-aware and dynamic scheduling method that picks the superior version from the previously compiled results and co-runs the selected tasks to improve system performance. Our experiments on MLPerf show that Layer-Puzzle can achieve up to 1.61X, 1.53X, and 1.95X improvement in ANTT, STP, and PE utilization, respectively.

This work proposes a ReRAM-based Computing-in-Memory (CIM) architecture for Neural Architecture Search (NAS) acceleration, ENASA, so that the compute-intensive NAS technology can be applied to various edge devices to customize the most suitable individual solution for their cases. In the popular one-shot NAS process, the system must repetitively evaluate the sampled sub-network within a large-scale supernet before converging to the best sub-network architecture. Thereby, how to map these iterative network inference tasks onto the CIM arrays makes a big difference in system performance. To realize efficient in-memory supernet sampling and evaluation, we design a novel mapping method that tactically executes a group of sub-nets in the CIM arrays, not only to boost the sub-net concurrency but also to eliminate the repetitive operations shared by these subnets. Meanwhile, to further enhance the subnet-level operation concurrency and sharing in the CIM arrays, we also tailor a novel CIM-friendly one-shot NAS algorithm that purposely samples those operation-sharing subnets in each iteration while still maintaining the convergence performance of NAS. According to the experimental results, our CIM NAS accelerator achieves an improvement of 196.6× and 1200× in performance speedup and energy saving respectively compared to the CPU+GPU baseline.

The attention mechanism is becoming an integral part of modern neural networks, bringing breakthroughs to Natural Language Processing (NLP) applications and even Computer Vision (CV) applications. Unfortunately, the superiority of attention mechanism comes from its ability to model relations between any two positions in long sequence, which incurs high inference overhead. For state-of-the-art AI workloads such as Bert or GPT-2, attention mechanism is reported to account up to 50% of the inference overhead. Previous works seek to alleviate this performance bottleneck by removing useless relations for each position and accelerate position-specific operations. However their attempts require selecting from a sequence of relations once for each position, which is essentially frequent on-the-fly pruning and breaks the inherent parallelism in attention mechanism. In this paper, we propose CTA, an algorithm-architecture co-designed solution that can substantially reduce theoretic complexity of attention mechanism, enabling significant speedup and energy saving. Inspired by the fact that the feature sequence encoded by attention mechanism contain a large number of semantic feature repetition, we propose a novel approximation scheme that can efficiently remove that repetition, only calculating attention among necessary features thus reducing computation complexity quadratically. To utilize this algorithmic bonus and empower high performance attention mechanism inference, we devise specialized architecture to efficiently support the proposed approximation scheme. Extensive experiments show that, on average, CTA achieves 27.7× speedup, 634.0× energy savings with no accuracy loss, and 44.2× speedup, 950.0× energy savings with around 1% accuracy loss over Nvidia V100-SXM2 GPU. Also, CTA achieves 22.8× speedup, 479.6× energy savings over ELSA accelerator+GPU system.

Field-programmable gate arrays (FPGAs) are popular for computational intensive applications and hardware accelerators recently. But they face limitations in memory capacity and its growth, resulting in excessive time spent on data access. The fixed capacity of embedded memory blocks also leads inflexibility and resource waste. Moreover, logic blocks in FPGAs which are insufficient for large-scale applications and fixed memory block positions both lead to high routing overhead. To address these issues, we propose a software-hardware co-designed polymorphic architecture called Meltrix. The hardware architecture, which uses RRAM arrays as the fundamental block, creates a unified fabric that can be reconfigured into logic, storage, and interconnection modes. We achieve multiple times of logic capacity compared with FPGAs' logic blocks and multi-level interconnections inside the tiles, which are used to solve the routing overhead problem in FPGAs. Moreover, the global routing complexity is further reduced by the proposed function synthesis framework, which isolates logic and memory components, synthesizes and maps them to configured tiles of Meltrix. Experiments show that, when comparing with commercial FPGAs and state-out-of-art Liquid-Silicon, Meltrix achieves 1.89-3.14× performance improvement and 2.08-4.17× power reduction in both logic-intensive and memory-intensive applications.

Memristor-based logic-in-memory (LIM) is an emerging technology that enables logic operations within memory, making it a promising solution for data-intensive applications. LIM architectures have different types according to where computations are executed, with each type being suitable for specific design objectives and application domains. However, mapping applications to a single LIM mode restricts the full utilization of different LIM modes. In this paper, we propose LIM-GEN, a data-guided framework for automated generation of heterogeneous LIM architectures. To take advantages of different LIM modes, three LIM modes are combined and used as building blocks to create heterogeneous architectures. Given the data-centric nature and large design space, there is an urgent need of developing new EDA tools for synthesizing such LIM architectures. LIM-GEN includes an automatic hardware synthesis flow, which takes behavior-level descriptions as input to generate application-specific architectures and dataflows. During synthesis, data distribution, task allocation and crossbar mapping are optimized through a design space exploration process. We evaluate LIM-GEN in several data-intensive applications and compare the generated heterogeneous architectures with synthesized architectures with a single LIM mode. The experimental results demonstrate significant improvements in latency, area and power consumption, brought by the heterogeneous architectures generated by LIM-GEN.

Quantum circuit simulation (QCS) is essential in the noisy intermediate scale quantum (NISQ) era when real quantum computers are scarce. However, fully tracking the states of a quantum system in QCS is highly challenging due to the exponential memory growth that significantly limits the computational reach of classical systems for QCS. Though it is straightforward to leverage secondary storage to extend the scale of QCS, excessive data movement between memory and storage dominates the simulation time, making this solution unrealistic. To tackle this challenge, we identify an intrinsic property of QCS and implement an open-source framework to effectively reduce data movement by >116x. We evaluate the framework on various benchmarks and demonstrate 4x memory reduction with only <20% overhead. On a memory constrained system, we show that it extends the scale of QCS to 32 qubits (64 GB memory requirement) while existing simulators are bounded to 28 qubits (4 GB memory requirement). Our implementation can be accessed via https://github.com/Zhaoyilunnn/qdao.

Graph convolutional neural network (GCN) extends deep learning to process graph data and demonstrates superior performance. However, due to the irregularity, graphs show inconsistent patterns across different regions, which leads to distinctions in data reusability and edge processing activity, and consequently poses impacts on hardware efficiency and resource utility. Prior accelerators seldom explore the distinct patterns across graph regions and adopt a fixed strategy for the whole graph without consideration for region-specific characteristics. In this paper, we identify the inconsistent patterns of graphs and characterize the distinctions between graph regions. Then, we propose an adaptive dataflow to adapt the region-specific patterns. Third, we implement PANG, a pattern-aware accelerator that can dynamically adjust the dataflow to exploit the reusability and alleviate the frequent destination switching. Evaluated on real-world datasets, PANG achieves significant performance improvement.

Attention-based pre-trained large models have demonstrated impressive performance in many domains such as natural language processing and computer vision. Unfortunately, due to the quadratic complexity incurred by calculating pairwise correlations across the entire input sequence, processing the attention mechanism becomes the arguably major bottleneck of the whole inference execution. To accelerate the attention mechanism with no loss of accuracy, we present a novel algorithm-architecture co-design that can substantially save runtime as well as energy spent on the attention mechanism. Inspired by the observation that only a small subset of content highly correlates with the others under attention, we devise a hardware-friendly content-based sparsity scheme to eliminate unnecessary relations, thus reducing computation complexity effectively. Furthermore, we develop a tailored hardware for this content-based sparse attention mechanism to best utilize this algorithm innovation. Experiments show that, compared with the implementation based on an Nvidia V100-SXM2 GPU, on average, our design achieves 63× speedup and 505× energy saving with no accuracy loss.

Rigid body dynamics is a core technology in the robotics field. In trajectory optimization and model predictive control algorithms, there are usually a large number of rigid body dynamics computing tasks. Using CPUs to process these tasks consumes a lot of time, which will affect the real-time performance of robots. To this end, we propose a multifunctional robot rigid body dynamics accelerator, named Dadu-RBD, to address the performance bottleneck. By analyzing different functions commonly used in robot dynamics calculations, we summarize their relationships and characteristics, then optimize them according to the hardware. Based on this, Dadu-RBD can fully reuse common hardware modules when processing different computing tasks. By dynamically switching the dataflow path, Dadu-RBD can accelerate various dynamics functions without reconfiguring the hardware. We design the Round-Trip Pipeline and Structure-Adaptive Pipelines for Dadu-RBD, which can greatly improve the throughput of the accelerator. Robots with different structures and parameters can be optimized specifically. Compared with the state-of-the-art CPU, GPU dynamics libraries and FPGA accelerator, Dadu-RBD can significantly improve the performance.

Recently, video processing tasks, such as video enhancement and analysis, have received increasing attention from both academics and industries. However, the current video processing procedure on edge decouples the decoding phase and the subsequent video processing tasks, missing the opportunity to accelerate the procedure by orchestrating video decoding and enhancement stages. Thus, we propose an intelligent video processing workflow and architecture(IVP) for cloud-edge video streaming. For edge devices that receive compressed videos, IVP can perform direct DNN-based video enhancement, e.g., super-resolution and frame-interpolation. By leveraging the metadata motion vectors and residuals extracted from the encoded video, our architecture will significantly eliminate unnecessary frame pixels being processed by the DNNs and improve execution efficiency. The proposed IVP and workflow are proved to reduce up to 90% of the processing latency while producing accurate and high-quality videos. Furthermore, we observe a significant portion of similar optical flow in time domain of continuous videos, which can be used to reduce the computation overhead. Thus, to utilize such temporal similarity of optical flow, the proposed IVP is upgraded to be capable of reusing previous computation results, which further improves energy efficiency of the whole system.

Recently, computing-in-memory (CiM) becomes a promising technology for alleviating the memory wall bottleneck. CiM is suitable for data-intensive applications, especially, cryptographic algorithms. Most current cryptographic accelerators are specific to a single function. It is expensive to accelerate different cryptographic algorithms with different accelerators. In this work, we first introduce a CiM architecture FeMIC that supports multioperand CiM operations, by exploring advantages of state-of-the-art ferroelectric field-effect transistors. Based on that, we propose a novel instruction set together with an accelerator architecture named FeCrypto which supports the acceleration of various cryptographic algorithms. Evaluation results show that FeCrypto has better performance and energy efficiency than software implementations. The energy-delay product (EDP) of FeCrypto is$118.4\times $and$1.93\times $lower than that of the dedicated AES accelerator AIM that is built based on phase-change memories (PCMs) and magnetic random-access memories (MRAMs), respectively. EDP is reduced by$44.7\times $compared with PCM-based EIM, a recent AES accelerator. Compared with MRAM-based EIM, the EDP overhead of FeCrypto for supporting multiple functions is 23.2%.

Convolutional neural networks (CNNs) (including 2D and 3D convolutions) are popular in video analysis tasks such as action recognition and activity understanding. Fast algorithms such as fast Fourier transforms (FFTs) are promising in significantly reducing computation complexity by transforming convolution into frequency domain. In frequency space, conventional spatial convolutions are replaced with simpler element-wise complex multiplications. Conventional application-specific-integrated-circuit (ASIC) based frequency-domain accelerators can achieve effective performance boost but come at the cost of significant energy consumption, owing to the hierarchical memory organization. We propose a frequency-domain resistive random access memory (ReRAM) based inference accelerator called FDA that can process element-wise complex multiplication in memory for both 2D and 3D CNNs. Each ReRAM-based frequency-domain process element (PE) with two ReRAM cells can perform an element-wise complex multiplication in two continuous execution cycles. We then provide a flexible dataflow to alleviate the redundant data movements by frequency-domain data reuse and inherent symmetrical characteristic for both 2D and 3D convolutions. Evaluation results based on representative both 2D and 3D CNN benchmarks demonstrate that FDA outperforms state-of-the-art baselines with better performance and energy efficiency.

Least-squares fitting (LSF) is a fundamental statistical method that is widely used in linear regression problems, such as modeling, data fitting, predictive analysis, etc. For large-scale data sets, LSF is computationally complex and poorly scaled due to the$O(N^{2})-O(N^{3})$computational complexity. The computing-in-memory technique has potential to improve the performance and scalability of LSF. In this paper, we propose a computing-in-memory accelerator based on resistive random-access memory (RRAM) devices. We not only utilize the conventional idea of accelerating matrix-vector multiplications by RRAM-based crossbar arrays, but also elaborate the hardware and the mapping strategy. Our approach has a unique feature that it can finish a complete LSF problem in$O$(1) time complexity. We also propose a scalable and configurable architecture such that the problem scale that can be solved is not restricted by the crossbar array size. Experimental results have demonstrated the superior performance and energy efficiency of our accelerator.

Graph processing involves lots of irregular memory accesses and increases demands on high memory bandwidth, making it difficult to execute efficiently on compute-centric architectures. Dedicated graph processing accelerators based on the processing-in-memory (PIM) technique have recently been proposed. Despite they achieved higher performance and energy efficiency than conventional architectures, the data allocation problem for communication minimization in PIM systems (e.g., hybrid memory cubes (HMCs)) has still not been well solved. In this paper, we demonstrate that the conventional “graph data allocation = graph partitioning” assumption is not true, and the memory access patterns of graph algorithms should also be taken into account when partitioning graph data for communication minimization. For this purpose, we classify graph algorithms into two representative classes from a memory access pattern point of view and propose different graph data partitioning strategies for them. We then propose two algorithms to optimize the partition-to-HMC mapping to minimize the inter-HMC communication. Evaluations have proved the superiority of our data allocation framework and the data movement energy efficiency is improved by 4.2-5 × on average than the state-of-the-art GraphP approach.

The “memory wall” bottleneck caused by the performance gap between processors and memories is getting worse. Computing-in-memory (CiM), a promising technology to alleviate the “memory wall” bottleneck, has recently attracted much attention. Conventional CiM architectures based on emerging nonvolatile devices have a major drawback that they need${N\,-\,1}$clock cycles to complete a CiM operation with${N}$operands, as they are natively designed for processing two operands. In this work, we propose FeMIC, a new CiM architecture based on ferroelectric field-effect transistors (FeFETs), which natively supports the computation of multiple operands. For a CiM operation with${N}$operands, FeMIC only needs$\left\lfloor {N/2} \right\rfloor$clock cycles. The simulation results based on a calibrated FeFET model reveal that FeMIC can significantly reduce the energy consumption when processing multi-operand CiM operations, compared with state-of-the-arts that use conventional CiM mechanisms.

Due to the irregular memory access and high bandwidth demanding, graph processing is usually inefficient on conventional computer architectures. The recent development of the processing-in-memory (PIM) technique such as hybrid memory cube (HMC) has provided a feasible design direction for graph processing accelerators. Although PIM provides high internal bandwidth, inter-node memory access is inevitable in large-scale graph processing, which greatly affects the performance. In this paper, we propose an HMC-based graph processing framework, GraphRing. GraphRing is a software-hardware codesign framework that optimizes inter-HMC communication. It contains a regularity- and locality-aware graph execution model and a ring-based multi-HMC architecture. The evaluation results based on 5 graph datasets and 4 graph algorithms show that GraphRing achieves on average 2.14× speedup and 3.07× inter-HMC communication energy saving, compared with GraphQ, a state-of-the-art graph processing architecture.

State-of-the-art convolutional neural network (CNN) accelerators are typically communication-dominate architectures. To reduce the energy consumption of data accesses and also to maintain the high performance, researches have adopted large amounts of on-chip register resources and proposed various methods to concentrate communication on on-chip register accesses. As a result, the on-chip register accesses become the energy bottleneck. To further reduce the energy consumption, in this work we propose an in-SRAM accumulation architecture to replace the conventional register files and digital accumulators in the processing elements of CNN accelerators. Compared with the existing in-SRAM computing approaches (which may not be targeted at CNN accelerators), the presented in-SRAM computing architecture not only realizes in-memory accumulation, but also solves the structure contention problem which occurs frequently when embedding in-memory architectures into CNN accelerators. HSPICE simulation results based on the 45nm technology demonstrate that with the proposed in-SRAM accumulator, the overall energy efficiency of a state-of-the-art communication-optimal CNN accelerator is increased by 29% on average.

Convolutional Neural Networks (CNNs) achieve state-of-the-art performance for perception tasks at the cost of billions of computational operations. In this paper, we propose a probabilistic prediction processing system, dubbed P3S, to eliminate redundant compute-heavy convolution operations by predicting whether output activations are zero-valued. By exploiting the probability characteristic of Gaussian-like distributed activations and weights in CNNs, P3S calculates the partial convolution across values greater than a standard deviation-related threshold, to predict the ineffectual output activations. P3S skips remaining convolutions and sets outputs to zero in advance if output activations are predicted to be zero. P3S reduces 67% computations within 0.2% accuracy loss and does not even require retraining or fine-tuning CNNs. We further implement a P3S-based CNN accelerator that achieves 2.02x speedup and 2.23x energy efficiency on average over the traditional accelerator. Compared with the state-of-the-art prediction-based accelerator with 3% accuracy degradation, our P$^3$S yields up to 1.49x speedup and 1.69x energy efficiency.

2.5D chiplet technology is gaining popularity for the efficiency of integrating multiple heterogeneous dies or chiplets on interposers, and it is also considered an ideal option for agile silicon system design by mitigating the huge design, verification, and manufacturing overhead of monolithic SoCs. Although it significantly reduces development costs by chiplet reuse, the design and fabrication of interposers also introduce additional high non-recurring engineering (NRE) costs and development cycles which might be prohibitive for application-specific designs having low volume.

Micro- and macro-architecture search have emerged as two popular NAS paradigms recently. Existing methods leverage different search strategies for searching micro- and macro- architectures. When using architecture parameters to search for micro-structure such as normal cell and reduction cell, the architecture parameters can not fully reflect the corresponding operation importance. When searching for the macro-structure chained by pre-defined blocks, many sub-networks need to be sampled for evaluation, which is very time-consuming. To address the two issues, we propose a new search paradigm, that is, leverage the attention mechanism to guide the micro- and macro-architecture search, namely AGNAS. Specifically, we introduce an attention module and plug it behind each candidate operation or each candidate block. We utilize the attention weights to represent the importance of the relevant operations for the micro search or the importance of the relevant blocks for the macro search. Experimental results show that AGNAS can achieve 2.46% test error on CIFAR-10 in the DARTS search space, and 23.4% test error when directly searching on ImageNet in the ProxylessNAS search space. AGNAS also achieves optimal performance on NAS-Bench-201, outperforming state-of-the-art approaches. The source code can be available at https://github.com/Sunzh1996/AGNAS.

With the success of Transformers in the computer vision field, the automated design of vision Transformers has attracted significant attention. Recently, MetaFormer found that simple average pooling can achieve impressive performance, which naturally raises the question of how to design a search space to search diverse and high-performance Transformer-like architectures. By revisiting typical search spaces, we design micro-meso-macro space to search for Transformer-like architectures, namely BurgerFormer. Micro, meso, and macro correspond to the granularity levels of operation, block and stage, respectively. At the microscopic level, we enrich the atomic operations to include various normalizations, activation functions, and basic operations (e.g., multi-head self attention, average pooling). At the mesoscopic level, a hamburger structure is searched out as the basic BurgerFormer block. At the macroscopic level, we search for the depth, width, and expansion ratio of the network based on the multi-stage architecture. Meanwhile, we propose a hybrid sampling method for effectively training the supernet. Experimental results demonstrate that the searched BurgerFormer architectures achieve comparable even superior performance compared with current state-of-the-art Transformers on the ImageNet and COCO datasets. The codes can be available at https://github.com/xingxing-123/BurgerFormer.

Autonomous racing has lately gained popularity because of its entertainment value and potential of advancing autonomous driving in high-speed situations. These high-speed racing efforts usually focus on a road domain with fixed dynamics. They cannot meet the challenge of policy adaptation between domains with large dynamics gaps. Meanwhile, existing policy adaptation methods either rely on experts to build new environments for policy training, or only handle a small dynamics gap for low-speed control tasks due to limited dynamics modeling and rigorous data collection assumptions. To overcome these drawbacks, we introduce DAARL, a novel policy adaptation algorithm that uses adversarial and reinforcement learning to bridge the large dynamics gap between different domains. It has two training stages. In the first training stage, a domain transfer function is learned by adversarial learning to better capture the dynamics gap. The single domain transfer function integrates with the source domain to implement the dynamics of different target domains virtually without the help of experts. We name these virtual domains the imaginary target domains. In the second training stage, the knowledge of the source-domain policy guides the reinforcement learning of a target-domain policy on an imaginary target domain. It improves the convergence of the target-domain policy. Five experiments have been conducted on a racing simulator with different road domains. All results show that DAARL outperforms baselines in terms of driving speed, stability, success rate, and domain scalability.

The growing complexity and diversity of deep neural network (DNN) applications have inspired intensive research on specialized DNN accelerators and also the design automation frameworks. Previous specialized NN acceleratos roughly fall into two categories of implementation, either the no-pipelined architecture that relies on a generic processing unit (PU) to sequentially execute the DNN layers in a layer-wise way, or the fully-pipelined architecture that dedicates interconnected customized PUs to the corresponding DNN layers in the model. Thus, such designs often suffer from either the resource under-utilization issue faced by no-pipelined accelerators or the resource scalability problem brought by the over-deep pipeline designs. In this work, we propose a novel class of design solution for DNN acceleration, segment-grained pipeline architecture (SPA). In the SPA accelerator, the targeted workload of DNN models will be divided into many segments and each segment will be sequentially executed on the shared interconnected PUs in a pipeline manner, so that they will benefit from both the efficiency of pipelined execution and also the flexibility of sharing PUs across different model layers. Particularly, we found that the efficiency of the implemented SPA accelerator significantly depends on the segmentation strategies of the models and the hardware resources assignment policy for PUs. Therefore, we introduce an automated design framework, AutoSeg, that includes a parameterized SPA accelerator template and a co-design engine that will generate the efficient model segmentation solution and hardware pipeline design parameters for the acceleration workload. Experimental results show that the SPA solutions generated by the AutoSeg framework achieve $1.2\times to 6.3\times$ speedup when compared to ASIC-based general DNN processors, and the FPGA designs implemented by AutoSeg also achieve as high as $3.4\times$ DSP efficiency and $3.6\times$ throughput improvement.

Processing-in-memory (PIM) architecture has demonstrated great potentials in accelerating numerous deep learning tasks. In particular, resistive random-access memory (RRAM) technology provides a promising hardware substrate for PIM accelerators, because it can support efficient in-situ vector-matrix multiplications (VMMs) with high-density RRAM crossbar arrays. However, such accelerators suffer from frequent and energy-intensive analog-to-digital (A/D) conversions, severely limiting their performance. This paper proposes a new PIM architecture to efficiently accelerate deep learning tasks by minimizing the required A/D conversions with neural approximated peripheral circuits. By characterizing the existing dataflows of state-of-the-art PIM architectures, we first propose a new dataflow by extending shift and add (S+A) operations into the analog domain before the final A/D conversion, which can remarkably reduce the required A/D conversions for a dot-product. We then elaborate on a neural approximation method to design both accumulation circuits (S+A) and quantization circuits (ADC) using RRAM crossbar arrays. Finally, we apply them to build a RRAM-based PIM accelerator--\textbf{Neural-PIM} based on the proposed analog dataflow and evaluate its system-level performances. Evaluations on different DNN benchmarks demonstrate that Neural-PIM can improve energy efficiency by 5.36x (1.73x) and speed up throughput by 3.43x (1.59x) without losing accuracy, compared to state-of-the-art RRAM-based PIM accelerators, i.e., ISAAC} (CASCADE)

Dynamic resources management in reconfigurable processors often manifests as a hard online decision-making task, which should yield premier solutions that must meet Quality-of-Service (QoS) requirements while maximizing the system’s efficiency. Most prior works rely on a hard-to-train predictor to model the complicated relationships between processor configurations and performance. To decide the proper resource allocation, the predictor needs to tentatively evaluate a group of possible configurations, and then decide the best configuration for the workload. This tedious process has an expensive runtime overhead for resource configuration in processors. Besides, prior works focus on improving the prediction accuracy, however, higher performance prediction cannot guarantee a good system outcome. Inspired by recent advances in adversarial learning, we present a generative adversarial network (GAN)-based framework, Amphis, which can directly generate the on-demand processor configuration for any scheduled-in application. By evaluating Amphis on a reconfigurable processor with 18 different workloads, our results demonstrate that the GAN-based method provides tremendous overhead reduction (up to 90%) compared to the SOTA prediction-based method WNNM while providing higher resource utilization.

Sparse convolution neural networks (CNNs) are promising in reducing both memory usage and computational complexity while still preserving high inference accuracy. State-of-the-art sparse CNN accelerators can deliver high throughput by skipping zero weights and/or activations. To operate on only nonzero weights and activations, sparse accelerators typically search pairs of nonzero weights and activations for multiplication-accumulation (MAC) operations. However, the conventional search operation results in a severe limitation in the processing element (PE) array scale because of the enormous demands of internal interconnection and memory bandwidth. In this article, we first provide a design principle to free the search process of sparse CNN accelerations. Specifically, the indexes of the static compressed weights access the dynamic activations directly to avoid the search process for MAC operations. We then develop two search-free inference accelerators, called Swan and Swan-flexible, for sparse CNN accelerations. Swan supports search-free sparse convolution accelerations for interconnection and bandwidth saving. Compared with Swan, Swan-flexible not only has the search-free capability but also comprises a configurable architecture for optimum throughput. We formulate a mathematical optimization problem by combining the configurable characterization with the compressive dataflow to optimize the overall throughput. Evaluations based on a place-and-route process show that the proposed designs, in a compact factor of 4096 PEs, achieve 1.5–$2.7\times $higher speedup and 6.0–$13.6\times $better energy efficiency than representative accelerator baselines with the same PE array scale.

Most of current processing-in-memory (PIM) architectures are application specific, that is, they can only accelerate particular functions, e.g., matrix-vector dot product for neural network acceleration. However, practical applications usually involve various functions. In order to accelerate different functions, various accelerators, and dedicated circuits have been proposed. In this work, by exploring the similarities among some commonly used dedicated circuits, we adopt ferroelectric field-effect transistors (FeFETs) to build a reconfigurable multifunctional memory architecture named Re-FeMAT. Re-FeMAT is composed of multiple processing elements (PEs). Each PE is not only a nonvolatile memory array, but also can perform logic operations (i.e., the PIM mode), convolutions (i.e., the binary convolutional neural network and the convolutional neural network (CNN) acceleration mode) and content search (i.e., the ternary content-addressable memory (TCAM) mode) without changing the circuit structure. Re-FeMAT can support applications that require multiple functions. As an example, by configuring different PEs to different working modes and using a simulated annealing algorithm or a tabu search algorithm to optimize the task-PE assignment, Re-FeMAT can completely accelerate few-shot learning applications. Our simulation results based on a calibrated FeFET model show that the proposed Re-FeMAT architecture achieves better performance and power efficiency than the previous FeMAT architecture. Compared with FeFET-based single-functional circuits, though the power dissipation of Re-FeMAT is higher in some modes, the power-delay product is still smaller. Compared with a state-of-the-art FeFET-based multifunctional accelerator named attention-in-memory, Re-FeMAT achieves lower power, latency, and energy when accelerating a complete few-shot learning task.

Load balance in vehicular ad hoc networks (VANETs) is a challenge in vehicle-to-vehicle computation offloading, due to stochastic requests of users, heterogeneous service capabilities and high mobility of vehicles, etc. This paper aims to fill this gap by formulating a problem for load balance in a VANET, with the objective of minimizing the maximum load under transmit power, storage capacity, per task completion time and energy consumption constraints. The formulated problem is proved to be NP-hard, then it is investigated by decomposing it into two subproblems, i.e., how to offload tasks for the case of fixed transmit power and how to adjust transmit power for the given offloading decision. For the first subproblem, an approximation algorithm is proposed by offloading the tasks in the vehicle with the maximum load to the vehicle with minimum load. Meanwhile, a deep reinforcement learning algorithm is proposed, in order to focus on the network dynamics. A coalition based algorithm, a distributed coalition based algorithm, as well as an incentive algorithm based on deep reinforcement learning, are proposed to maximize the total payoff for the selfishness of vehicles. For the second subproblem, an adjustment strategy for transmit power is customized to further reduce the computing load. The algorithms are evaluated on an integrated simulation platform with open street map, SUMO, NS-3 and dataset of Google cluster-usage traces. Simulation results show that, the proposed algorithms outperform three state-of-the-art works for most cases, in terms of the maximum load. The proposed distributed algorithm can significantly accelerate the proposed centralized algorithm with acceptable increase in maximum load. Besides, the load can be further reduced by the proposed adjustment strategy.

Neural Architecture Search (NAS) has made remarkable progress in automatically designing neural networks. However, existing differentiable NAS and stochastic NAS methods are either biased towards exploitation and thus may converge to a local minimum, or biased towards exploration and thus converge slowly. In this work, we propose a Dynamic and Differentiable Space-Architecture Search (DDSAS) method to address the exploration-exploitation dilemma. DDSAS dynamically samples space, searches architectures in the sampled subspace with gradient descent, and leverages the Upper Confidence Bound (UCB) to balance exploitation and exploration. The whole search space is elastic, offering flexibility to evolve and to consider resource constraints. Experiments on image classification datasets demonstrate that with only 4GB memory and 3 hours for searching, DDSAS achieves 2.39% test error on CIFAR10, 16.26% test error on CIFAR100, and 23.9% test error when transferring to ImageNet. When directly searching on ImageNet, DDSAS achieves comparable accuracy with more than 6.5 times speedup over state-of-the-art methods. The source codes are available at https://github.com/xingxing-123/DDSAS.

The memory wall bottleneck has caused a large portion of the energy to be consumed by data transfer between processors and memories when dealing with data-intensive workloads. By giving some processing abilities to memories, processing-in-memory (PIM) is a promising technique to alleviate the memory wall bottleneck. In this work, we proposed a novel PIM architecture by employing ferroelectric field-effect transistors (FeFETs). The proposed design, named FePIM, is able to perform in-memory bitwise logic and add operations between two selected rows or between one selected row and an immediate operand. By utilizing unique features of FeFET devices, we further propose novel solutions to eliminate simultaneous-read-and-write (SRAW) contentions such that stalls are eliminated. Experimental results show that FePIM reduces 15% of the memory access latency and 44% of the memory access energy, compared with an enhanced version of a state-of-the-art FeFET-based PIM design which cannot handle SRAW contentions.

3D convolutional neural networks (CNNs) are widely deployed in video analysis. Fast algorithms such as fast Fourier transforms (FFTs) are gaining popularity in reducing computation complexity for their superior capability of replacing convolutions with simpler element-wise multiplications. Conventional frequency-domain dedicated accelerators employ memory hierarchy organization for high throughput but at the expensive costs of a significant amount of data movements and energy consumptions. This paper presents F3D, a processingin-memory frequency-domain accelerator using resistive random access memory (ReRAM). F3D supports frequency-domain complex number multiplications directly in ReRAM-based crossbar architecture. We alleviate the overheads of redundant data movements in ReRAM-based complex number multiplications by data reuse and the inherent symmetry of inputs in the frequency space. Evaluation results demonstrate that F3D outperforms state-of-the-art accelerators with significant improvements in performance and energy efficiency.

Accelerating convolutional neural networks (CNNs) with resistive random-access memory (RRAM) based processing-in-memory systems has been recognized as a promising approach. However, conventional accelerators are usually mixed-signal circuits with digital-to-analog converters (DACs) and analog-to-digital converters (ADCs), which cause performance and energy efficiency degradation. In this work, we first analyze the problems in existing RRAM-based CNN accelerators and point out that there are redundant analog-to-digital (AD) conversions. To eliminate redundant AD conversions and also reduce AD conversion overhead, we propose the BRAHMS architecture, which is an RRAM-based CNN accelerator composed of reconfigurable RRAM crossbars and analog resistive content-addressable memory (ARCAM) arrays. We reorder the operations after a convolutional or fully-connected layer and form fused operators (FOPs), which are implemented as a whole by ARCAM arrays so that digital logic and ADCs are eliminated. BRAHMS realizes a mixed-signal pipeline which transmits data signals in the analog domain within an FOP and in the digital domain between FOPs to obtain high performance and energy efficiency. Detailed simulation results show that compared with an ISAAC-like architecture, BRAHMS improves the performance by several times and the energy efficiency by 10 + times on average.

The sparse linear solver is an important component in lots of scientific computing applications. For large-scale sparse linear systems, general-purpose processors such as CPUs and GPUs are facing challenges of high time complexity and massive data movements between processors and main memories. This work utilizes the ability of in-situ analog computing of RRAMs and builds an RRAMbased accelerator for iterative linear solvers.We first propose a basic principle of mapping iterative solvers onto RRAM-based crossbar arrays. The proposed principle eliminates not only the iterations but also the convergence condition. Based on the principle, we propose a scalable architecture that can solve large-scale sparse matrices in O(1) time complexity. Compared with a massively parallel iterative solver on GPU, our accelerator shows 100× higher performance and 1000× energy reduction. If the solution obtained by our accelerator is used as the seed for a further refinement on GPU, about 35% of the solving time and energy consumption can be saved compared with a pure GPU solving process.

Feature-based lidar odometry methods have attracted increasing attention due to their low computational cost. However, theoretically analysis of the effect of extracted features on pose estimation is still lacked. In this paper, we propose a method of key-feature selection for lightweight lidar inertial odometry, KFS-LIO, to further enhance the real-time performance by selecting the most effective subset of lidar feature constraints. Aiming at explaining the correlation between the feature distribution and state errors, a quantitative evaluation method of lidar constraints is introduced. In addition, to avoid recalculating the reprojection matrices in de-skewing step, we use the intermediate variables in IMU preintegration to compensate for lidar motion distortion. The experimental results demonstrate that KFS-LIO can reduce half of the LOAM features and provide comparable accuracy with the state-of-the-art odometry.

Processing-in-Memory (PIM) is considered a promising solution to improve the performance of graph-computing applications by minimizing the data movement between the host and memory. Which workload to offload and how to offload it to PIM logic determine whether the PIM architecture is well utilized. Offloading too much or too little workload from the host processor to the PIM side could hurt overall performance. On the other hand, the offloading granularity needs to be representative without losing generality. In this paper, we present CoPIM, a novel PIM workload offloading architecture that can dynamically determine which portion of the graph workload can benefit more from PIM-side computation. CoPIM focuses on the loop code blocks of graph applications and evaluates the necessity of offloading based on a concurrent memory access model. We also provide detailed architectural designs to support the offloading. In this way, CoPIM reduces the size of offloading instructions and also improves the overall performance with less energy consumption. The experimental results show that compared with other state-of-the-art PIM workload offloading frameworks, CoPIM achieves a speedup by the geometric mean of 19.5% and 11.4% than PEI and GraphPIM, respectively. On the other hand, CoPIM also reduces the un-core energy consumption by 6.8% and 6.5% on average over PEI and GraphPIM, respectively.

Memristor-based memory technology is one of the emerging memory technologies, which is a potential candidate to replace traditional memories. Efficient test solutions are required to enable the quality and reliability of such products. In previous works, fault models are caused by open, short and bridge defects and parametric variations during the fabrication. However, these fault models cannot describe the bridge defects that cause the state of the faulty cell to an undefined state. In this paper, we analyze the different effects of bridge defects and aggregate their faulty behavior into new fault models, undefined coupling fault and dynamic undefined coupling fault. In addition, an enhanced March algorithm is designed to detect all the modeled faults. In one resistor crossbar with$N$memristors, the enhanced March algorithm requires$8N$write and$7N$read operations with negligible hardware overhead. To reduce the test time, a March RC algorithm is proposed based on read operations with new reference currents, which requires$4N+2$write and$6N$read operations. Analytical results show that the proposed test algorithms can detect all the modeled faults outperforming all the previous methods. Subsequently, a Design-for-Testability scheme is proposed to implement March RC algorithm with a little area overhead.

Stereo vision is widely deployed on robots and drones to enable depth estimation at a low cost. The combination of lightweight deep neural network (DNN) and cost volumes algorithm is proved to possess the advantages of both high depth estimation accuracy and speed. However, currently there is no accelerator architecture compatible with both efficient DNN inference and cost generation algorithms such as stereo matching. This work proposes a stereo vision accelerator called Dadu-eye, dedicated to real-time processing of high-resolution image streams. The proposed architecture adopts a pipelined hardware design with the techniques of operation approximation and scheduling-level optimization. First, a cost estimation block is designed to generate cost volumes from both luminance and color information. Second, a super pipelined multiplication and accumulation array with a row scan-based fused-layer convolution scheduling is proposed to perform the encoding and decoding neural network efficiently. Finally, an optical flow block is designed and cooperates with the array to approximately predict half of the frames’ depth to achieve real-time (30fps) processing on 1080p view. Based on the SMIC 40 nm CMOS process, this stereo vision accelerator achieves 5.3 TOPS/W power efficiency and significantly reduces 81% off-chip memory access.

Approximate computing provides an emerging approach to design high performance and low power arithmetic circuits. The logarithmic multiplier (LM) converts multiplication into addition and has inherent approximate characteristics. In this article, dynamic range approximate LMs (DR-ALMs) for machine learning applications are proposed; they use Mitchell’s approximation and a dynamic range operand truncation scheme. The worst case (absolute and relative) errors for the proposed DR-ALMs are analyzed. The accuracy and the hardware overhead of these designs are provided to select the best approximate scheme according to different metrics. The proposed DR-ALMs are compared with the conventional LM with exact operands and previous approximate multipliers; the results show that the power-delay product (PDP) of the best proposed DR-ALM (DR-ALM-6) are decreased by up to 54.07 percent with the mean relative error distance (MRED) decreasing by 21.30 percent compared with 16-bit conventional design. Case studies for three machine learning applications show the viability of the proposed DR-ALMs. Compared with the exact multiplier and its conventional counterpart, the back-propagation classifier with DR-ALMs with a truncation length larger than 4 has a similar classification result for the three datasets; the K-means clustering application with all DR-ALMs has a similar clustering result for four datasets; and the handwritten digit recognition application with DR-ALM-5 or DR-ALM-6 for LeNet-5 achieves similar or even slightly higher recognition rate.

Sparsification is an efficient solution to reduce the demand of on-chip memory space for deep convolutional neural networks (CNNs). Most of state-of-the-art CNN accelerators can deliver high throughput for sparse CNNs by searching pairs of nonzero weights and activations, and then sending them to processing elements (PEs) for multiplication-accumulation (MAC) operations. However, their PE scales are difficult to be increased for superior and efficient computing because of the significant internal interconnect and memory bandwidth consumption. To deal with this dilemma, we propose a sparsity-aware architecture, called Swan, which frees the search process for sparse CNNs under limited interconnect and bandwidth resources. The architecture comprises two parts: a MAC unit that can free the search operation for the sparsity-aware MAC calculation, and a systolic compressive dataflow that well suits the MAC architecture and greatly reuses inputs for interconnect and bandwidth saving. With the proposed architecture, only one column of the PEs needs to load/store data while all PEs can operate in full scale. Evaluation results based on a place-and-route process show that the proposed design, in a compact factor of 4096 PEs, 4.9TOP/s peak performance, and 2.97W power running at 600MHz, achieves 1.5-2.1× speedup and 6.0-9.1× higher energy efficiency than state-of-the-art CNN accelerators with the same PE scale.

Video prediction is a pixel-wise dense prediction task to infer future frames based on past frames. Missing appearance details and motion blur are still two major problems for current models, leading to image distortion and temporal inconsistency. We point out the necessity of exploring multi-frequency analysis to deal with the two problems. Inspired by the frequency band decomposition characteristic of Human Vision System (HVS), we propose a video prediction network based on multi-level wavelet analysis to uniformly deal with spatial and temporal information. Specifically, multi-level spatial discrete wavelet transform decomposes each video frame into anisotropic sub-bands with multiple frequencies, helping to enrich structural information and reserve fine details. On the other hand, multilevel temporal discrete wavelet transform which operates on time axis decomposes the frame sequence into sub-band groups of different frequencies to accurately capture multifrequency motions under a fixed frame rate. Extensive experiments on diverse datasets demonstrate that our model shows significant improvements on fidelity and temporal consistency over the state-of-the-art works. Source code and videos are available at https://github.com/Bei-Jin/STMFANet.

Deep Convolution Neural network (DCNN) pruning is an efficient way to reduce the resource and power consumption in a DCNN accelerator. Exploiting the sparsity in the weight matrices of DCNNs, however, is nontrivial if we deploy these DC-NNs in a crossbar-based Process-In-Memory (PIM) architecture, because of the crossbar structure. Structural pruning-exploiting a coarse-grained sparsity, such as filter/channel-level pruning-can result in a compressed weight matrix that fits the crossbar structure. However, this pruning method inevitably degrades the model accuracy. To solve this problem, in this paper, we propose PIM-PRUNE to exploit the finer-grained sparsity in PIM-architecture, and the resulting compressed weight matrices can significantly reduce the demand of crossbars with negligible accuracy loss. Further, we explore the design space of the crossbar, such as the crossbar size and aspect-ratio, from a new point-of-view of resource-oriented pruning. We find a trade-off existing between the pruning algorithm and the hardware overhead: a PIM with smaller crossbars is more friendly for pruning methods; however, the resulting peripheral circuit cause higher power consumption. Given a specific DCNN, we can suggest a sweet-spot of crossbar design to the optimal overall energy efficiency. Experimental results show that the proposed pruning method applied on Resnet18 can achieve up to 24.85× and 3.56× higher compression rate of occupied crossbars on CifarlO and Imagenet, respectively; while the accuracy loss is negligible, which is 4.56× and 1.99× better than the state-of-art methods.

Previous neural network accelerators tailored to video analysis only accept data of RGB/YUV domain, requiring decompressing the video that are often compressed before transmitted from the edge sensors. A compressed video processing accelerator can remove the decoding overhead, and gain performance speedup by operating on more compact input data. This work proposes a novel deep learning accelerator architecture, Alchemist, which predicts results directly from the compressed video bitstream instead of reconstructing the full RGB images. By utilizing the metadata of motion vector and critical blocks extracted from bitstream, Alchemist contributes to remarkable performance speedup of 5x with negligible accuracy loss.

Collision detection is a fundamental task in motion planning of robotics. Typically, the performance of collision detection is the bottleneck of an entire motion planning, and so does the energy consumption. Several hardware accelerators have been proposed for collision detection, which achieves higher performance and energy efficiency than general-purpose CPUs and GPUs. However, existing accelerators are still facing the limited memory bandwidth bottleneck, due to the large data volume required by the parallel processing cores and the limited DRAM bandwidth. In this work, we propose a novel collision detection accelerator by employing the processing-in-memory technique. We elaborate the in-memory processing architecture to fully utilize the internal bandwidth of DRAM banks. To make the algorithm and hardware suitable for in-memory processing to be highly efficient, a set of innovative software and hardware techniques are also proposed. Compared with a state-of-the-art ASIC-based collision detection accelerator, both performance and energy efficiency of our accelerator are significantly improved.

Recently, processing-in-memory (PIM) is gaining much attention because it could minimize data movement by conducting computation in memory. Existing PIM solutions require a number of additional procedures during the setup-time, including code re-writing and re-compiling, code annotations, and detailed program profiling, etc. These requirements, however, potentially prevent existing executable binaries benefiting from PIM architectures. For old binary legacies without any source code, it is impossible to run them on existing PIM architectures. To solve these challenges, we propose a transparent and universal PIM (TUPIM), a novel PIM architecture that can execute unmodified binaries and at the same time take advantages of PIM. TUPIM is a significant advance over the state-of-the-art because it transparently expends the scope of PIM to deploy all applications without any source code, programming models, or compiler modifications. Experiments show that TUPIM can get 2.2x speedup on average (up to 3.67x) and 15.7% energy reduction, compared with conventional CPU-only executions.

Real-time motion planning is important for robot movement. In motion planning, path search and collision detection are two performance bottlenecks. In this paper, we adopt a range-based matching scheme with ternary content-addressable memories (TCAMs) to accelerate the processes of both nearest neighbor search and collision detection. In our approach, the nearest node search and collision detection can be both processed in a few TCAM lookup cycles. The evaluation shows that the TCAM-based accelerator is 236× faster than CPU for motion planning tasks. It is 5.4× faster and at least 8.8× more energy-efficient than a state-of-the-art dedicated ASIC-based accelerator.

In current convolutional neural network (CNN) accelerators, communication (i.e., memory access) dominates the energy consumption. This work provides comprehensive analysis and methodologies to minimize the communication for CNN accelerators. For the off-chip communication, we derive the theoretical lower bound for any convolutional layer and propose a dataflow to reach the lower bound. This fundamental problem has never been solved by prior studies. The on-chip communication is minimized based on an elaborate workload and storage mapping scheme. We in addition design a communication-optimal CNN accelerator architecture. Evaluations based on the 65nm technology demonstrate that the proposed architecture nearly reaches the theoretical minimum communication in a three-level memory hierarchy and it is computation dominant. The gap between the energy efficiency of our accelerator and the theoretical best value is only 37-87%.

Research on accelerators for robotics is increasing. This article introduces the kinematics, motion planning and collision detection algorithms and our accelerators in robotics, and then analyzes their advantages, disadvantages and bottlenecks. In view of the shortcomings of the existing accelerators, this paper will show a series accelerators named "DaDu" that we have proposed. For kinematics, we have proposed Dadu [1] to accelerate the inverse kinematics algorithm, which achieves 1700x speedup than the CPU implementation, 30x speedup than the GPU implementation, and 776x higher energy efficiency than the GPU implementation. For motion planning, we have proposed Dadu-P [2] to accelerate the PRM algorithm. It can get 26.5x speedup than an existing CPU-based approach for collision detection. Furthermore, with an incremental approach, the performance of motion planning can further be improved by 10x while the solution quality is degraded by 10% only. For the collision detection algorithm in motion planning, the proposed accelerator Dadu-CD [3] elaborates the in-memory processing architecture, achieving at least 5x speedup than Dadu-P in the total planning time and 9.55x lower energy consumption than Dadu-P.

Decision making for autonomous driving is a safety-critical control problem. Prior works of safe reinforcement learning either tackle the problem with reward shaping or with modifying the reinforcement learning exploration process. However, the former cannot guarantee the safety during the learning process, while the latter relies heavily on expertise to design exquisite exploration policy. Currently, only short-term decision makings for low-speed driving were achieved in road scenes with basic geometries. In this paper, we propose a two-stage safe reinforcement learning algorithm to automatically learn a long-term policy for high-speed driving that guarantees safety during the entire training. In the first learning stage, model-free reinforcement learning is followed by a rule-based safeguard module to avoid danger at low speed without expert ne-tuning. In the second learning stage, the rule-based module is replaced with a data-driven counterpart to develop a closed-form analytical safety solution for high-speed driving. Moreover, an adaptive reward function is designed to match the different objectives of the two learning stages for faster convergence to an optimal policy. Experiments are conducted on a racing simulator TORCS which has complex racing tracks (e.g. sharp turns, hills). Compared with the state-of-the-art baselines, the results show that our method achieves zero safety violation and quickly converges to a more efficient and stable policy with an average speed of 127 km/h (3.3% higher than the best result of baselines) and an average swing of 3.96 degrees.

Sparse neural networks (SNNs) are emerging as a promising technique for resource-limited intelligent embedded systems because of the compact model size and the un compromised accuracy. Recently, most of the dedicated neural network accelerators are beginning to exploit the sparsity of neural network models for performance boost and energy saving. However, existing sparsity-aware accelerators fail to support both sparse weights and activations in neural networks or support them at the same time for both convolutional (Conv) layers and fully connected (FC) layers, which dominate the computational time of neural networks. In this article, we propose a novel sparsity-aware accelerator architecture, called Swallow, to sufficiently improve the inference performance by eliminating ineffectual weights and activations of neural networks. Swallow comprises: 1) a 2-D systolic architecture that fully utilizes the sparsity of both weights and activations in both Conv and FC layers and 2) a sparsity-aware dataflow which is optimized to reuse both weights and activations and to achieve high processing element (PE) utilization by sparse matrix multiplication tiling. Comprehensive evaluations based on a place-and-route process show that Swallow, with 614 GOP/s peak performance and 1.26-W power, outperforms a state-of-the-art sparsity-aware accelerator Cambricon-X by 1.32× in term of energy efficiency.

Inference efficiency is the predominant design consideration for modern machine learning accelerators. The ability of executing multiply-and-accumulate (MAC) significantly impacts the throughput and energy consumption during inference. However, MAC operation suffers from significant ineffectual computations that severely undermines the inference efficiency and must be appropriately handled by the accelerator. The ineffectual computations are manifested in two ways: first, zero values as the input operands of the multiplier, waste time and energy but contribute nothing to the model inference; second, zero bits in nonzero values occupy a large portion of multiplication time but are useless to the final result. In this article, we propose an ineffectual-free yet cost-effective computing architecture, called split-and-accumulate (SAC) with two essential bit detection mechanisms to address these intractable problems in tandem. It replaces the conventional MAC operation in the accelerator by only manipulating the essential bits in the parameters (weights) to accomplish the partial sum computation. Besides, it also eliminates multiplications without any accuracy loss, and supports a wide range of precision configurations. Based on SAC, we propose an accelerator family called Tetris and demonstrate its application in accelerating state-of-the-art deep learning models. Tetris includes two implementations designed for either high performance (i.e., cloud applications) or low power consumption (i.e., edge devices), respectively, contingent to its built-in essential bit detection mechanism. We evaluate our design with Vivado HLS platform and achieve up to 6.96× performance enhancement, and up to 55.1× energy efficiency improvement over conventional accelerator designs.

This work investigates how to effectively train binarized neural networks (BNNs) for the specialized low-precision neural accelerators. When mapping BNNs onto the specialized neural accelerators that adopt fixed-point feature data representation and binary parameters, due to the operation overflow caused by short fixed-point coding, the BNN inference results from the deep learning frameworks on CPU/GPU will be inconsistent with those from the accelerators. This issue leads to a large deviation between the training environment and the inference implementation, and causes potential model accuracy losses when deployed on the accelerators. Therefore, we present a series of methods to contain the overflow phenomenon, and enable typical deep learning frameworks like Tensorflow to effectively train BNNs that could work with high accuracy and convergence speed on the specialized neural accelerators.

As an energy-efficient hardware solution for deep neural network (DNN) inference, systolic accelerators are particularly popular in both embedded and datacenter computing scenarios. Despite their excellent performance and energy efficiency, however, systolic DNN accelerators are naturally facing a resource under-utilization problem - not all DNN models can well match the fixed processing elements (PEs) in a systolic array implementation, because typical DNN models vary significantly from applications to applications. Consequently, state-of-the-art hardware solutions are not expected to deliver the nominal (peak) performance and energy efficiency as claimed because of resource under-utilization. To deal with this dilemma, this study proposes a novel systolic DNN accelerator with a flexible computation mapping and dataflow scheme. By providing three types of parallelism and dynamically switching among them: channel-direction mapping, planar mapping, and hybrid, our accelerator offers the adaptability to match various DNN models to the fixed hardware resources, and thus, enables flexibly exploiting PE provision and data reuse for a wide range of DNN models to achieve optimal performance and energy efficiency.

The ever-growing processing ability of in-memory processing logic makes the data sharing and coherence between processors and in-memory logic play an increasingly important role in Processing-in-Memory (PIM) systems. Unfortunately, the existing state-of-the-art coarse-grained PIM coherence solutions suffer from unnecessary data movements and stalls caused by a data ping-pong issue. This work proposes CuckooPIM, a criticality-aware and less-blocking coherence mechanism, which can effectively avoid unnecessary data movements and stalls. Experiments reveal that CuckooPIM achieves 1.68x speedup on average comparing with coarse-grained PIM coherence.

No abstract available.

Disk and memory faults are the leading causes of server breakdown. A proactive solution is to predict such hardware failure at the runtime and then isolate the hardware at risk and backup the data. However, the current model-based predictors are incapable of using the discrete time-series data, such as the values of device attributes, which conveys high-level information of the device behavior. In this paper, we propose a novel deep-learning based prediction scheme for system-level hardware failure prediction. We normalize the distribution of samples' attributes from different vendors to make use of diverse training sets. We propose a temporal Convolution Neural Network based model that is insensitive to the noise in the time dimension. Finally, we design a loss function to train the model with extremely imbalanced samples effectively. Experimental results from an open S.M.A.R.T data set and an industrial data set show the effectiveness of the proposed scheme.

The technological breakthrough in Generative Adversarial Networks (GAN) has propelled the advancement of content generative applications such as AI-based paintings, style transfer, and music composition. However, in contrast to previous deep learning models for prediction and categorization, generative networks generally rely on instance normalization (IN) layer for better feature distribution, which performs significantly better than batch normalization(BN) in image style-transfer, image to image translation, etc. Unlike batch or group normalization that can be fused into convolutional layers and ignored during the network inference stage, an instance normalization layer induces intensive computation and memory access. However, prior deep learning accelerator designs for traditional Neural Network and Generative Adversarial Networks mostly focus on the acceleration of convolution and deconvolution layer but lack of support for IN operations, which could become a performance bottleneck on edge devices with insufficient computational power. To address this problem, we propose an inference accelerator for content generation (ACG-Engine) aimed to support the fundamental operations of generative networks, including convolution layers, deconvolution layers, specifically instance normalization layer. We performed a hardware-aware mathematical transformation of the IN operation for less computation complexity and memory-friendliness, so that it can be efficiently mapped to the classic 2D processing element array. Owing to the proposed optimization techniques, ACG-Engine achieves 4.56X speedup and improve power efficiency up to 29X compared to prior baseline acceleration scheme in generative network acceleration. In addition, ACG-Engine can achieve performance comparable to the classic CNN-specific accelerators with negligible power consumption and area overhead.

The performance gap between the processors and the main memory is continuously widening, known as the memory wall bottleneck. Emerging nonvolatile devices have the ability of in-memory processing, and thus, have the potential to partially alleviate the memory wall bottleneck. People have adopted nonvolatile devices to build various accelerators that are targeted at different problems and applications. In this work, we adopt one of the emerging nonvolatile devices, the ferroelectric field-effect transistor (FeFET), to build a multifunctional in-memory processing unit, which is named FeMAT. From a structural point of view, FeMAT is an FeFET-based memory array composed of 3T-based cells. From a functional point of view, FeMAT not only is a nonvolatile memory, but also can perform some logic operations (i.e., the processing-in-memory (PIM) mode), binary convolutions (i.e., the binary convolutional neural network (BCNN) acceleration mode) and content searching (i.e., the ternary content-addressable memory (TCAM) mode) in the memory. These functions are seamlessly fused into the FeFET-based memory array and can be configured online without changing the circuit structure. Superior energy efficiency is demonstrated by our experiments and comparisons with a resistive random-access memory (ReRAM) based equivalence, as well as a TCAM and a BCNN accelerator based on complementary metal-oxide-semiconductor (CMOS) devices.

The reconfigurability of FPGA makes it flexible for different applications. However, an FPGA may be delivered, designed, and deployed by different persons during its lifecycle, so anyone who can access the FPGA may bring in security issues. This paper proposes an implementation method of a parametric hardware Trojan in the FPGA. This hardware Trojan does not add any extra circuits, so many existing detection methods based on analyzing the design files are invalid.

The year of 2019 marks the 50thIEEE International Test Conference (ITC) - the world's first and premier conference dedicated to electronic test technology. As one of the participating conferences of the ITC global test forum (GTF), China Test Conference (CTC) was invited to join the celebration of the ITC's 50thanniversary. CTC was initiated in Beijing by the test community in China in year of 2000, in the background that China began to highly encourage the developing of its integrated circuit (IC) industry, while test plays a major role in guaranteeing IC's quality and reliability. CTC has been held biennially for ten times and been included into the technical meetings of the IEEE Computer Society Test Technology Technical Council (TTTC). It has been a connection of the test community in China with the international test communities and conferences. The 10thCTC was very successful and was selected as the TTTC Affiliated Most Populous Technical Meeting in 2018.

Graphics processing units (GPUs) have been widely adopted to accelerate the training of deep neural networks (DNNs). Although the computational performance of GPUs has been improving steadily, the memory size of modern GPUs is still quite limited, which restricts the sizes of DNNs that can be trained on GPUs, and hence raises serious challenges. This paper introduces a framework, referred to as moDNN (memory optimal DNN training on GPUs), to optimize the memory usage in DNN training. moDNN supports automatic tuning of DNN training code to match any given memory budget (not smaller than the theoretical lower bound). By taking full advantage of overlapping computations and data transfers, we develop new heuristics to judiciously schedule data offloading and prefetching transfers, together with convolution algorithm selection, to optimize memory usage. We further devise a new sub-batch size selection method which also greatly reduces memory usage. moDNN can save memory usage up to 59×, compared with an ideal case which assumes that the GPU memory is sufficient to hold all data. When executing moDNN on a GPU with 12 GB memory, the training time is increased by only 3 percent, which is much shorter than that incurred by the best known approach, vDNN. Furthermore, we propose an optimization strategy for moDNN on multiple GPUs again by utilizing the idea of overlapping data transfers and GPU computations. The results show that 3.7× speedup is attained on four GPUs.

Conventional high-performance processors employ hardware data prefetcher to reduce the cache miss rate by exploiting data locality in memory access. In the scenario of PIM-enabled system, current prefetching methods are likely to induce unnecessary data moving between memory and CPU, resulting in significant performance slow-down if ignoring the data access pattern of the In-Memory cores of PIM architecture. In this paper, we propose a novel PIM-oriented Prefetching (PIMCH), to fix this problem for the heterogeneous system with both conventional CPU cores and PIM architectures. PIMCH consider the memory behavior of both CPU cores and PIM cores, and refers to their data mapping to dynamically adjust prefetching methods, which enables cooperative memory prefetching in Processing-In-Memory architecture.

As a critical operation in robotics, motion planning consumes lots of time and energy, especially in a dynamic environment. Through approaches based on general-purpose processors, it is hard to get a valid planning in real time. We present an accelerator to speed up collision detection, which costs over 90% of the computation time in motion planning. Via the octree-based roadmap representation, the accelerator can be reconfigured online and support large roadmaps. We in addition propose an effective algorithm to update the roadmap in a dynamic environment, together with a batched incremental processing approach to reduce the complexity of collision detection. Experimental results show that our accelerator achieves 26.5X speedup than an existing CPU-based approach. With the incremental approach, the performance further improves by 10X while the solution quality is degraded by 10% only.

The field of image and video recognition has been propelled by the rapid development of deep learning in recent years. With its fascinating accuracy and generalization ability, deep CNNs have shown remarkable performance in large-scale and real-life image dataset. However, accommodating computation-intensive CNN-based image detection frameworks on power-constrained devices is considered more challenging than desktop or warehouse computing systems. Instead of emphasizing purely on detection accuracy, Low Power Image Recognition Challenge (LPIRC) is initiated to highlight the energy-efficiency of different image recognition solutions, and it witnesses the advancement of cost-effective image recognition technology in aspects of both algorithmic and architecture innovation. This paper introduces the cost-effective CNN-based object detection solutions that reached an improved tradeoff between energy and accuracy for mobile CPU+GPU SoCs, which is the winner of LPIRC2016, and it also analyzes the implications of both recent hardware and algorithm advancement on such a technique. It is demonstrated in our evaluation that the performance growth of embedded SoCs and CNN models have clearly contributed to a sheer growth of mAP/WH in current CNN-based object detection solutions, and also shifted the balance between accuracy and energy-cost in the contest solution design when we seek to maximize the efficiency score defined by LPIRC through design parameter exploration.

Semantic scene completion predicts volumetric occupancy and object category of a 3D scene, which helps intelligent agents to understand and interact with the surroundings. In this work, we propose a disentangled framework, sequentially carrying out 2D semantic segmentation, 2D-3D reprojection and 3D semantic scene completion. This three-stage framework has three advantages: (1) explicit semantic segmentation significantly boosts performance; (2) flexible fusion ways of sensor data bring good extensibility; (3) progress in any subtask will promote the holistic performance. Experimental results show that regardless of inputing a single depth or RGB-D, our framework can generate high-quality semantic scene completion, and outperforms state-of-the-art approaches on both synthetic and real datasets.

Data compression has been intensively studied to increase the utility of cache, network-on-chip (NoC), and main memory in energy-efficient processors. However, prior solutions to data compression often add remarkable compression and decompression delay to the critical path of memory access, which is thought as the major factor limiting its application to commodity processors. Unlike prior work that deals with memory compression or network compression separately, this paper proposes a unified on-chip distributed data compressor (DISCO), to enable near-zero-latency cache and memory block compression for chip multiprocessors adopting nonuniform cache access. DISCO integrates a multimode cache compressor into the NoC routers and overlaps the de/compression latency with the queuing delay in the network. In addition, cache block evicted to or fetched from the main memory can also be compressed or decompressed during the network queuing time in this unified DISCO compressor. With the support of congestion-awareness, it is shown in the evaluation that DISCO, which unifies the compression solution of the memory hierarchy, dramatically decreases the compression overhead of isolated techniques, and significantly boosts the efficiency of data moving and store.

Aiming at real-life problems, microrobotic systems have gained more and more attention. However, limited achievable performance of microrobotic system prevents it from carrying out complex tasks. Current research work propose customize designs for different applications and incorporate dedicated accelerator for high energy efficiency. However, not only such techniques require significant manual effort and expertise for specified applications, but also the accelerator itself dictates unnegligible amount of chip resources. So in this paper we propose ApproxEye, a partial approximate computation reuse framework to accelerate microrobotic computer vision. Leveraging computation locality, ApproxEye reuses previous “similar” computations to reduce redundant computations. To squeeze every piece of computation reuse opportunity, ApproxEye proposes to 1) heuristically define optimal reuse granularity and 2) apply adaptive reuse requirements for different computations. Moreover, to reduce latency of computation reuse, ApproxEye tailors a parallel implemented search scheme for approximate computation reuse. Experimental results show ApproxEye could effectively exploit the potential of computation reuse and achieve 57.05% speedup on average.

Multi-granularity memory system provides multiple access granularities for the applications with various spatial localities. In the multi-granularity access pattern, the one-size-bandwidth NoC design cannot utilize the bandwidth efficiently. We propose a novel NoC design, called BoDNoC, which can merge multiple narrow subnets to provide various bandwidths for access data. The new design also adopts an optimization algorithm to take full advantage of bandwidth provision. Experimental results show that BoDNoC can improve the throughput by 23.5% and reduce the energy consumption by 37.2% in comparison with one-size-bandwidth NoC design.

This paper surveys how to use Convolutional Neural Networks (CNN) to hypothesize object location and categorization from images or videos in mobile heterogeneous SoCs. Recently a variety of CNN-based object detection frameworks have demonstrated both increasing accuracy and speed. Though they are making fast progress in high quality image recognition, state-of-the-art CNN-based detection frameworks seldom discuss their hardware-depended aspects and the cost-effectiveness of real-time image analysis in off-the-shelf low-power devices. As the focus of deep learning and convolutional neural nets is shifting to the embedded or mobile applications with limited power and computational resources, scaling down object detection framework and CNNs is becoming a new and important direction. In this work we conduct a comprehensive comparative study of state-of-the-art real-time object detection frameworks about their performance, cost-effectiveness/energy-efficiency (in the metric of mAP/Wh) in off-the-shelf mobile GPU devices. Based on the analysis results and observation in investigation, we propose to adjust the design parameters of such frameworks and employ a design space exploration procedure to maximize the energy-efficiency (mAP/Wh) of real-time object detection solution in mobile GPUs. As shown in the benchmarking result, we successfully boost the energy-efficiency of multiple popular CNN-based detection solutions by maximizing the utility of computation resources of SoC and trading-off between prediction accuracy and energy cost. In the second Low-Power Image Recognition Challenge (LPIRC), our system achieved the best result measured in mAP/Energy on the embedded Jetson TX1 CPU+GPU SoC.

Kinematics is the basis of robotic control, which manages the robots' movement, walking and balancing. As a critical part of Kinematics, the Inverse Kinematics (IK) will consume more time and energy to figure out the solution with the degrees of freedom increase. It goes beyond the ability of general-purpose processor based methods to provide real-time IK solver for manipulators with high degree of freedom. In this paper, we present a novel parallel algorithm, Quick-IK, based on the Jacobian transpose method. Via speculative searching in parallel, Quick-IK can reduce the number of iterations by 97% for the baseline Jacobian transpose method. In addition, we propose a novel specialized architecture, IKAcc, to boost the energy efficiency of Quick-IK through hardware acceleration. The evaluation shows that IKAcc can solve IK problem in 12 milliseconds for a 100 degrees of freedom manipulator. In addition, IKAcc can achieve 1700x performance speed-up over the CPU implementation of the original Jacobian transpose method and 30x speedup over the GPU implementation of Quick-IK. At same time, IKAcc achieves about 776x higher energy efficiency than the GPU implementation of Quick-IK.

Convolutional Neural Networks (CNN) are very computation-intensive. Recently, a lot of CNN accelerators based on the CNN intrinsic parallelism are proposed. However, we observed that there is a big mismatch between the parallel types supported by computing engine and the dominant parallel types of CNN workloads. This mismatch seriously degrades resource utilization of existing accelerators. In this paper, we propose a flexible dataflow architecture (FlexFlow) that can leverage the complementary effects among feature map, neuron, and synapse parallelism to mitigate the mismatch. We evaluated our design with six typical practical workloads, it acquires 2-10x performance speedup and 2.5-10x power efficiency improvement compared with three state-of-the-art accelerator architectures. Meanwhile, FlexFlow is highly scalable with growing computing engine scale.

Refresh operations consume substantial energy and bandwidth in high-density dynamic random-access memory (DRAM) memory. The trend of increasing refresh overhead limits the scalability of DRAM memory that refreshes all cells at the same rate, because the refresh rate setting depends on the worst-case weak cell manufactured in unstable process technology. To cope with this issue, fine-grained refresh (FGR) is proposed to eliminate the unnecessary refresh operations caused by minor weak cells. Even JEDEC's DDR4 DRAM specification announces the support of FGR, which is likely to evolve and become a standard in future DRAM. Unfortunately, according to our key observation, the effectiveness of FGR is greatly confined by the procedure of refresh-oblivious device integration because all memory devices within a module have to be controlled and refreshed in a lockstep way after the step of assembly. In this paper, we are the first to propose a holistic FGR-oriented DRAM optimization framework, retention-aware DRAM assembly and repair (RADAR), to enhance the effectiveness of FGR in DRAM modules. RADAR includes two novel techniques applicable at the stage of DRAM assembly. The first one is retention-aware device clustering that integrates the “compatible” devices to achieve low refresh rate through a preassembly testing and retention-aware matching method. The second technique, Microfix, exploits the hierarchical DRAM array structure and its redundancy to fix critical weak DRAM rows through fine-grained row and subarray remapping. With this optimization architecture, RADAR, the refresh overhead of DRAM dual in-line memory modules can be dramatically reduced as implied in the experiments.

Multilevel spin toque transfer RAM (STT-RAM) is a suitable storage device for energy-efficient neural network accelerators (NNAs), which relies on large-capacity on-chip memory to support brain-inspired large-scale learning models from conventional artificial neural networks to current popular deep convolutional neural networks. In this paper, we investigate the application of multilevel STT-RAM to general-purpose NNAs. First, the error-resilience feature of neural networks is leveraged to tolerate the read/write reliability issue in multilevel cell STT-RAM using approximate computing. The induced read/write failures at the expense of higher storage density can be effectively masked by a wide spectrum of NN applications with intrinsic forgiveness. Second, we present a precision-tunable STT-RAM buffer for the popular general-purpose NNA. The targeted STT-RAM memory design is able to transform between multiple working modes and adaptable to meet the varying quality constraint of approximate applications. Lastly, the reconfigurable STT-RAM buffer not only enables precision scaling in NNA but also provides adaptiveness to the demand for different learning models with distinct working-set sizes. Particularly, we demonstrate the concept of capacity/precision-tunable STT-RAM memory with the emerging reconfigurable deep NNA and elaborate on the data mapping and storage mode switching policy in STT-RAM memory to achieve the best energy efficiency of approximate computing.

Approximate computing, which trades off computation quality (e.g, accuracy) and computation efforts, has becoming a promising technique to improve performance for many mission-non-critical and error-tolerant applications. The computations in such applications usually exhibit superior value locality, i.e, computations performed by a function or code region are very likely to reproduce “similar” results. Reusing the similar results can bypass redundant computations, as long as “exact” results are not mandatory. However, conventional computation reuse techniques are less effective in approximate computing paradigm. The input values of two computation instances have to be identical to reuse one for another, hence “exact” in nature.We propose ACR, an approximate computation reuse framework, to enable computation reuse for approximate computing. ACR relaxes the exact matching requirement in inputs to some extent regulated by “similarity” quantification, thereby shifting the exact computation reuse paradigm to its approximate counterpart. We furthermore propose an input significance-aware similarity quantification scheme through statistical approaches. Experimental result shows ACR could effectively exploit the potential of computation reuse for approximate computing and reduce 47.6% computations on average for a set of approximate applications.

Convolutional neural networks (CNN) accelerators have been proposed as an efficient hardware solution for deep learning based applications, which are known to be both compute-and-memory intensive. Although the most advanced CNN accelerators can deliver high computational throughput, the performance is highly unstable. Once changed to accommodate a new network with different parameters like layers and kernel size, the fixed hardware structure, may no longer well match the data flows. Consequently, the accelerator will fail to deliver high performance due to the underutilization of either logic resource or memory bandwidth. To overcome this problem, we proposed a novel deep learning accelerator, which offers multiple types of data-level parallelism: inter-kernel, intra-kernel and hybrid. Our design can adaptively switch among the three types of parallelism and the corresponding data tiling schemes to dynamically match different networks or even different layers of a single network. No matter how we change the hardware configurations or network types, the proposed network mapping strategy ensures the optimal performance and energy-efficiency. Compared with previous state-of-the-art NN accelerators, it is possible to achieve a speedup of 4.0x-8.3x for some layers of the well-known large scale CNNs. For the whole phase of network forward-propagation, our design achieves 28.04% PE energy saving, 90.3% on-chip memory energy saving on average.

Data compression has been proposed to increase the utility of on-chip memory space or Network-on-Chip (NoC) bandwidth in energy-efficient processors. However, such techniques usually add additional compression and decompression latency to the critical path of memory access, which is one of the major factors limiting their application to processors. In contrast to prior work that deals with either cache compression or network compression separately, this study proposes a unified on-chip DIStributed data COmpressor, DISCO, to enable near-zero latency cache/NoC compression for chip multi-processors (CMPs) adopting Non-Uniform Cache Access (NUCA). DISCO integrates data compressors into NoC routers and seeks opportunity to overlap the de/compression latency with the NoC queuing delay through a coordinated NoC scheduling and cache compression mechanism With the support of DISCO that unifies the solutions of on-chip data compression, it is shown in evaluation that DISCO significantly boosts the efficiency of on-chip data caching and data moving.

Recent advances in Neural Networks (NN) are enabling more and more innovative applications. As an energy-efficient hardware solution, machine learning accelerators for CNNs or traditional ANNs are also gaining popularity in the area of embedded vision, robotics and cyberphysics. However, the design parameters of NN models vary significantly from application to application. Hence, it's hard to provide one general and highly-efficient hardware solution to accommodate all of them, and it is also impractical for the domain-specific developers to customize their flown hardware targeting on a specific NN model. To deal with this dilemma, this study proposes a design automation tool, DeepBurning, allowing the application developers to build from scratch learning accelerators that targets their specific NN models with custom configurations and optimized performance. DeepBurning includes a RTL-level accelerator generator and a coordinated compiler that generates the control flow and data layout under the user-specified constraints. The results can be used to implement FPGA-based NN accelerator or help generate chip design for early design stage. In general, DeepBurning supports a large family of NN models, and greatly simplifies the design flow of NN accelerators for the machine learning or AI application developers. The evaluation shows that the generated learning accelerators burnt to our FPGA board exhibit great power efficiency compared to state-of-the-art FPGA-based solutions.

Heterogeneous manycore architectures have shown to be highly promising to boost power efficiency through two independent ways: (1) enabling massive thread-level parallelism, called “scale-out” approach, and (2) enabling thread migration between heterogeneous cores, called “scale-up” approach. How to accurately model the profitability of power efficiency of the two ways, particularly in an analytical and computational-effective manner, is essential to reap the power efficiency of such architectures. We propose a comprehensive analytical model to predict the power efficiency from the two independent ways. Given power efficiency is measured by performance per watt, this model is composed of a performance and a power model. The performance model is built by two orthogonal functions a and β. Function a describes the scale-out speedup from multithreading; function β presents the scale-up speedup from core heterogeneity. Thus, the performance model can clearly capture the overall speedup of any multithreading and thread-to-core mapping strategies. The power model predicts the power of corresponding scale-out and scale-up configurations. It simultaneously captures the power variations caused by thread synchronization and thread migration between heterogeneous cores. We build both performance and power model in an analytical way and keep the computational complexity in mind. This merit leads to a suit of comprehensive and low-complexity models for runtime management. These models are validated on large-scale heterogeneous manycore architecture with full-system simulations. For performance prediction, the average error is below 12 percent, lower than that of the state-of-the-art methods. For power prediction, the average error is 7.74 percent. On top of the models, we introduce two heuristic scheduling algorithms, performance-oriented MAX-P and power efficiency-oriented MAX-E, to demonstrate the usage of these models. The results show that MAX-P outperforms the state-of-the-art methods by 18 percent in performance averagely; MAX-E outperforms the baseline by 70 percent in power efficiency on average.

Voltage-frequency island (VFI)-based design has been widely exploited for optimizing system energy of embedded multicore chip in recent years. The existing work either constructed a single static VFI partition for all kinds of applications or required per-core voltage domain configuration. However, the former solution is hard to find one optimal VFI partition for diverse applications while the latter one suffers from high hardware cost. In this article, we propose a cost effective energy optimization framework based on dynamically reconfigurable VFI (D-VFI). Our framework treats a small number of cores as dynamic cores (D-cores) and configures each of them with an independent voltage domain. At runtime, the D-cores can be pieced together with neighboring static VFIs by scaling their operating voltages. This can dynamically construct the optimal VFI partitions for different kinds of applications, thus achieving more aggressive energy optimization under low cost. To identify the D-cores, we propose a rules constrained task scheduling and VFI partitioning algorithm. Moreover, we analyze the task schedules to determine the optimal scaling intervals which can accommodate voltage scaling induced latency. Experimental results demonstrate that the effectiveness of the proposed scheme.

The rapid growth of cloud services dictates increasingly powerful datacenters to maintain the high quality of service (QoS). It's a common practice in virtually all tiers of datacenters to continuously upgrade the datacenters, i.e. replacing outdated and failed servers with more advanced and efficient ones. However, how to upgrade a datacenter in the most cost-efficient strategy remains unclear, and however this problem goes increasingly challenging given the great diversity of applications. In practice, the datacenters' operators usually resort to expending the scale of servers. The preferred servers are either expensive but high-performance, or, by contrast, cheap but low-power. Whatever sever preferences, how to justify the cost-efficiency is still an open problem. We claim that a cost-efficient upgrading strategy should be fully aware of not only the capacity and cost of various servers, but also the resource demands of target applications. We model this strategy as a recommendation problem: recommending the “best” servers to a datacenter. We propose “EcoUp”, a model-based framework that faithfully rates the cost efficiency of server candidates, relying on which an optimal server portfolio can be derived. The performance prediction on candidate servers is realized by employing a sophisticated latent factor model (LFM). The cost mainly involves the server purchasing cost and energy bill. Given the application distribution, EcoUp can give an optimal server portfolio under a certain capital budget. We use Google trace, a big profiling dataset opened by Google, to validate the performance prediction. Experimental results show that the error rate is below 8 percent on average. Meanwhile, we build a comprehensive upgrading procedure on a local cluster to evaluate the potential of EcoUp. The results show that our approach significantly outperforms two conventional upgrading strategies by 12.3 and 33.6 percent in terms of system throughput, respectively.

The limited write endurance is one of the major obstacles for phase-change random access memory (PRAM)-based main memory. Traditionally, wear-leveling (WL) techniques were proposed to enhance its lifetime by balancing write traffic. However, these techniques do not concern the endurance variation in PRAM chips. When different PRAM cells have distinct endurance, balanced writes results in lifetime degradation due to the weakest cells. In this paper, we first define a new metric-wear rate (i.e., writes/endurance) considering both the write traffic and endurance distribution from application and hardware, respectively. After investigating the writing behavior of applications and endurance variation, we propose an architecture-level leveling mechanism to balance wear rate of cells across the PRAM chip. Hardware and algorithm to support the proposed leveling mechanism are presented. Moreover, there is an important tradeoff between endurance improvement and swapping data volume. To co-optimize endurance and swapping, this situation is formulated as a maximum weight perfect matching problem in bipartite graph. Thereafter, a novel algorithm that minimizes wear-rate and swapping by employing Kuhn-Munkras algorithm is proposed to maximize PRAM lifetime and minimize performance degradation. The experimental results show ~17× lifetime improvement over prior WL.

In this paper, we investigate the feasibility of voltage adjustment in a large capacity cache, and propose the architecture of voltage-adaptable nonuniform cache access (VANUCA) that exploits near-threshold computing and multivoltage domain to approach the limit of Vddin a low-power cache. However, the adoption of near-threshold voltage (NTV) leads to a rocketing error probability in SRAM arrays, which has to be addressed by effective fault-tolerant techniques. Instead of using error correction code or data duplication, the VANUCA exploits the natural data redundancy across the whole memory hierarchy to enable fast fault recovery in the NTV cache. Based on the discovered data resilience and the multi-Vddarchitecture, the VANUCA is able to match vulnerable/invulnerable data clusters to available high-/low-voltage domains by utilizing the data migration mechanism in dynamic NUCA. The proposed VANUCA includes two important architectural techniques: 1) static assignment that assumes a fixed voltage domain partitioning and 2) DataMotion that dynamically fits the working set into heterogeneous cache banks through Vddswitching. Experimental results show that the VANUCA achieves considerable improvements in energy efficiency over the conventional single-voltage domain NUCA cache.

In 3-D-stacked memory chips, the problem of power supply integrity (PSI) is aggravating due to the additional through-silicon-via resistance and the higher current density in 3-D power delivery network. In particular, for the 3-D phase-change memory (PCM) well known for its high-amplitude programming current, IR-drop violation poses a serious threat that enforces a strict guard band of requesting concurrence, and consequently reduces the write throughput. This paper presents the implication of an IR-drop phenomenon in a 3-D PCM cube, and investigates IR-drop's impacts on write management in the PCM. From the obtained SPICE simulation results, we find that the issued writes have to meet the IR-drop constraint to be reliably processed, and then propose a PSI conscious write scheduler to improve the write performance within the constraint of the IR-drops and the power budget in the 3-D PCM cube. First, a Bloom-filter-based method is proposed to avoid the invalid write decisions for the PCM. Second, to support fine-grained write management in the cutting-edge PCM, we develop an inexpensive approach, weighted token assignment (WTA), to filter out PSI-unsafe write decisions by employing a support vector machine-based learning model. Last, a write reordering policy is proposed to cooperate with WTA and optimize the total write throughput for better memory performance. In the simulated hybrid main memory composed of both dynamic random access memory and 3-D PCM, the proposed scheduler significantly improves the write throughput.

Networks-on-Chip (NoC) gradually becomes a main contributor of chip-level power consumption. Due to the temporal and spatial heterogeneity of on-chip traffic, existing power management approaches cannot adapt the NoC power consumption to its traffic intensity, and hence lead to a suboptimal power efficiency. They either resort to over-provisioned NoC design that only suits for traffic spatial distribution, or coarse-grained power gating that only serves traffic temporal variation. In this paper, we propose a novel NoC architecture called Shuttle Networks-on-Chip (ShuttleNoC). By permitting packets shuttling between multiple subnetworks, localized power adaptation can be achieved. Experimental results show that ShuttleNoC could achieve optimal power efficiency with up to 23.5% power savings and 22.3% performance boost in comparison with traditional heterogeneity-agnostic NoC designs.

Emerging highly-parallel and big data applications have renewed the research interest in Processing-in-Memory (PIM) architectures. However, moving powerful processing unit into the CMOS-incompatible DRAM chips is not cost-effective for large capacity memory. In this work, we observe that Non-Volatile Memory is often naturally incorporated with basic logics like Data Comparison Write or Flip-n-Write modules that are essential for cell SET/REST operation. In contrast to other conventional PIM or Near Data Computing (NDC) architectures, ProPRAM, as a typical Active NVM, abandons the design approach of moving accelerators or customized processors into memory devices, but begins with exploiting the existing resources inside the memory chips to accelerate the key non-compute-intensive functions for emerging big data applications. With slight hardware and architectural modification, we succeed to expose the transparent peripheral logics to the application layer through instruction set extension and exploit them for in-field bulk data processing with limited hardware cost. Compared to conventional CPU-centric systems, ProPRAM achieves an excellent optimization on energy-efficiency (15x) for important data-intensive micro-benchmarks and kernels.

Refresh operations consume substantial energy and bandwidth in high-density DRAM memory. To cope with this issue, Fine-Grained Refresh (FGR) is recently proposed to eliminate unnecessary refresh operations caused by minor weak cells. Even JEDEC's DDR4 DRAM specification announces the support of FGR to make DRAM refresh more scalable. Unfortunately, we observe that the effectiveness of FGR is greatly confined by the procedure of refresh-oblivious device integration because all memory devices within a module have to be controlled and refreshed in a lockstep way after the step of assembly. In this work, we firstly propose to intelligently integrate the "compatible" devices through a pre-assembly testing and retention-aware matching method. Second, we reuse the reconfiguration structure from yield-oriented remapping mechanism in memory chips and propose Microfix to create a balanced distribution of retention time in memory banks through fine-grained row-address tuning. With this optimization architecture, RADAR, we can eliminate the refresh overhead of produced memory modules by 28% on average.

Multi-level STT-RAM cell is able to boost the memory density at the expense of read/write reliability. However, the induced data integrity issue in STT-RAM memory can be effectively masked by a wide spectrum of applications with intrinsic forgiveness, which belong to the specific domain such as multimedia, synthesis and mining. In this work, we leverage the reconfigurable capability of MLC STT-RAM to provide variable-precision data storage for popular machine learning architectures. The targeted STT-RAM memory design is able to transform between multiple work modes and adaptable to meet the varying quality constraint of approximate applications. Particularly, we demonstrate the concept of precision-tunable STT-RAM memory with the emerging Convolution Neural Network accelerators and elaborate on the data mapping policy in STT-RAM memory to achieve the best energy-efficiency.

Workload consolidation is widely used in modern cloud processors to reduce total cost of ownership. Performance isolation has to be enforced between consolidated workloads to achieve controllable quality of service. Networks-on-chip (NoCs), as a major shared resource, often incur traffic interference and violate performance isolation criteria. Previous work resorts to strict isolation strategy that partitions NoC into independent regions to isolate core-to-core communication traffic. However, strict isolation either results in low consolidation density or degrades network performance, and more importantly, cannot be applied to memory access traffic. To address these weaknesses, we propose a novel performance isolation strategy in NoC, called relaxed isolation (RISO). It permits underutilized routers and links to be shared by multiple applications, and, at the same time, it keeps the aggregated traffic in check to enforce performance isolation. Experimental results show that RISO could effectively improve consolidation density and network performance in synergy.

The confluence of 3-D integration and network-on-chip (NoC) provides an effective solution to the scalability problem of on-chip interconnects. In 3-D integration, through-silicon via (TSV) is considered to be the most promising bonding technology. However, TSVs are also precious link resources because they consume significant chip area and possibly lead to routing congestion in the physical design stage. In addition, TSVs suffer from serious yield losses that shrink the effective TSV density. Thus, it is necessary to implement a TSV-economical 3-D NoC architecture in cost-effective design. For symmetric 3-D mesh NoCs, we observe that the TSVs bandwidth utilization is low and they rarely become the contention spots in networks as planar links. Based on this observation, we propose the TSV sharing (TS) scheme to save TSVs in 3-D NoC by enabling neighboring routers to share the vertical channels in a time division multiplexing way. We also investigate different TS implementation alternatives and show how TS improves TSV-effectiveness (TE) in multicore processors through a design space exploration. In experiments, we comprehensively evaluate TSs influence on all layers of system. It is shown that the proposed method significantly promotes TE with negligible performance overhead.

In chip multiprocessors (CMPs), nonuniform cache architecture (NUCA) is often employed to organize last-level cache (LLC) banks through network-on-chip (NoC). Because of the shrinking feature size and unstable operating environment, severe reliability problems unavoidably emerge and cause frequent on-chip component (e.g., cores, cache banks, routers) failures. Typical fault-tolerant CMPs should possess the feature of graceful degradation and function normally with deactivated tiles. However, for CMPs adopting static NUCA, certain physical address areas will become inaccessible when cache banks in a CMP node are isolated from the system. To protect cache from such threats induced by either online or offline faults, we survey several potential solutions and propose the utility-driven node remapping technique by reusing the resources in NoC. In our NoC-assisted remapping scheme, cache accesses to isolated banks are so redirected that cache space contention are successfully balanced and relieved in shared-LLC, thus ensuring the least performance penalty caused by fault isolation. Our experimental results show significant performance improvement over conventional resizing approaches such as set reduction.

Energy efficiency is a primary design concern for embedded multiprocessor system-on-chips (MPSoCs). Recently, Voltage-Frequency Island (VFI) - based design paradigm was introduced for fine-grained power management, which can seamlessly combine with the task scheduling algorithm to optimize system energy. However, the ever-increasing variabilities cause large uncertainty on delay and power. Such statistical nature in performance parameters easily makes deterministic energy optimization hard to achieve desirable performance yield, defined as the probability of the design meeting timing constraints of the system. In this paper, we propose a variation-aware statistical energy optimization framework, which takes account of performance yield constraints in energy-aware task scheduling, voltage assignment and VFI partitioning process. Energy optimization sensitivity, defined as energy variations of the task under voltage scaling, combines with the statistical slack of the task to guide the overall optimization flow. Experimental results demonstrate the effectiveness of the proposed scheme.

Heterogeneous many-cores can deliver high performance or energy efficiency. There are two orthogonal ways to improve performance: 1) scale-out by exploiting thread-level parallelism, and 2) scale-up by enabling core heterogeneity. Predicting the performance of such architecture is increasingly challenging. We propose a comprehensive performance model Amphisbaena, or Φ, built from two orthogonal functions α and β. Function α describes the scale-out speedup and function β handles the scale-up speedup. The Φ model can clearly tell not only the overall speedup of a given multithreading and core mapping strategy, but also how to improve the multithreading and core mapping, hence should be a promising performance predictor for future heterogenous many-cores. The results show that Φ model's error rate is within 12%, which is lower than state-of-the-art methods. We demonstrate the application of Φ model by introducing a heuristic scheduling algorithm, which outperforms the baselines by 13% on average.

The load power range of modern processors is greatly enlarged because many advanced power management techniques like dynamic voltage frequency scaling, Turbo boosting, and Near Threshold Voltage technologies are incorporated. However, the power saving may be offset by power loss in power delivery; moreover, as the efficiency of power delivery varies greatly with different load conditions, conventional power delivery designs cannot maintain high efficiency over the entire voltage range. We propose SuperRange, a wide operational range power delivery scheme. SuperRange complements the power delivery capability of on-chip voltage regulator and off-chip voltage regulator. Experimental results show SuperRange has an average 70% power conversion efficiency over wide operational range which outperforms conventional power delivery schemes. And it also exhibits superior resilience to power-constrained systems.

With the increase of storage density, DRAM refresh leads to higher overhead of power and bandwidth, particularly in emerging 3D stacked memory design like Hybrid Memory Cube (HMC). To exploit the hardware resources for a smarter solution, we propose a data-aware refresh control scheme, Trial and Error (Trial-n-Error), which leverages the data-pattern dependence characteristics of the cells' retention time to reduce refresh operations. Trial-n-Error is a systematic approach that employs our proposed Synergy Testing to capture the refresh bottleneck of DRAM memory: “weak” cells that have a relatively shorter retention time. By locating the dominant weak cells sensitized by applications, Trial-n-Error can avoid the worst-case refresh setting, and adjust the refresh rate under the control of our self-tuning algorithm. Thus, Trial-n-Error can gradually approach to the possible lower-bound of refresh rate for less energy and memory bandwidth consumption. In experiments of 3D-stacked DRAMs, we successfully eliminate an average of 28% refresh operations and save 21% refresh energy for a set of pre-profiled synthetic data patterns and real benchmarks.

Power efficiency is increasingly critical to battery-powered smartphones. Given that the using experience is most valued by the user, we propose that the power optimization should directly respect the user experience. We conduct a statistical sample survey and study the correlation among the user experience, system runtime activities, and computational performance of an application processor. We find that there exists a minimal frequency requirement, called “saturated frequency”. Above this frequency, the device consumes more power but provides little improvements in user experience. This study motivates an intelligent self-adaptive scheme, SmartCap, that automatically identifies the most power-efficient state of the application processor. Compared to prior Linux power adaptation schemes, SmartCap can help save power from 11% to 84%, depending on applications, with little decline in user experience.

Fault-tolerant routing is usually used to provide reliable on-chip communication for many-core processors. This paper focuses on a special class of algorithms that do not use virtual channels. One of the major challenges is to keep the network deadlock free in the presence of faults, especially those locating on network edges. State-of-the-art solutions address this problem by either disabling all nodes of the faulty network edges or including all faults into one faulty block. Therefore, a large number of fault-free nodes will be sacrificed. To address this problem, the proposed ZoneDefense routing not only includes faults into convex faulty blocks but also spreads the faulty blocks' position information in corresponding columns. The nodes, which know the position of faulty blocks, form the defense zones. Therefore, packets can find the faulty blocks and route around them in advance. Exploiting the defense zones, the proposed ZoneDefense routing could tolerate many more faults with significantly reduced sacrificed fault-free nodes compared with the state-of-the-art algorithms. Furthermore, the ZoneDefense routing does not degrade the network performance in the absence of faults, and could get similar performance as its counterparts in the presence of faults.

Aggressive technology scaling in modern chips resulted in complicated faulty timing behaviors, which necessitate undesirable long development cycle and high test volumes to ensure product quality. To reduce the test time, cost-effective and timing-efficient test selection algorithms are used to choose optimal test inputs from a large-volume test set. In this paper, we define an approximate longest sensitized path (ALSP) metric to derive the longest sensitized path for all transition faults (TFs) from the detectability of TFs with very low computational complexity. With the ALSP metric, a general public utilities-based parallel test selection method is proposed to choose a small test set with high delay test quality from the timing-unaware n-detection test set. Our results demonstrate the comparison with a commercial automatic test pattern generation tool and a previous timing-aware test selection method targeting small delay defects, and confirm that our test selection algorithm can achieve better delay test coverage and higher n -detection fault coverage with steeper fault coverage curves of ordered patterns, for the same pattern count.

Cloud service providers use workload consolidation technique in many-core cloud processors to optimize system utilization and augment performance for ever extending scale-out workloads. Performance isolation usually has to be enforced for the consolidated workloads sharing the same many-core resources. Networks-on-chip (NoC) serves as a major shared resource, also needs to be isolated to avoid violating performance isolation. Prior work uses strict network isolation to fulfill performance isolation. However, strict network isolation either results in low consolidation density, or complex routing mechanisms which indicates prohibitive high hardware cost and large latency. In view of this limitation, we propose a novel NoC isolation strategy for many-core cloud processors, called relaxed isolation (RISO). It permits underutilized links to be shared by multiple applications, at the same time keeps the aggregated traffic in check to enforce performance isolation. The experimental results show that the consolidation density is improved more than 12% in comparison with previous strict isolation scheme, meanwhile reducing network latency by 38.4% on average.

Power efficiency is increasingly critical to battery-powered smartphones. Given the using experience is most valued by the user, we propose that the power optimization should directly respect the user experience. We conduct a statistical sample survey and study the correlation among the user experience, the system runtime activities, and the minimal required frequency of an application processor. This study motivates an intelligent self-adaptive scheme, SmartCap, which automatically identifies the most power-efficient state of the application processor according to system activities. Compared to prior Linux power adaptation schemes, SmartCap can help save power from 11% to 84%, depending on applications, with little decline in user experience.

Power constraint for modern processors becomes a very serious problem with the increasing core counts and cache capacity in multi/many core processors. Compared with processing cores with mature techniques like DVFS to alleviate the situation, last level cache which consume largest portion of processor chip needs effective power management strategy. In this paper, we explore the feasibility of Near-Threshold Voltage(NTV) SRAM and Multi-Voltage Domain (Multi-VDD) for power reduction in large capacity cache. To prevent data corruption in cache, we propose redundancy-based data salvaging technique for fault recovery. To solve the dilemma of power reduction and reliability guarantee, we try to match vulnerable/invulnerable data sets to high/low voltage domains. Different from previous work, we take into consideration multi-bit errors and redundancy masking effects in Multi-VDD cache. Experimental results show that our Multi-VDD cache achieves considerable improvements in energy efficiency.

3-D technology that stacks silicon dies with through silicon vias (TSVs) is a promising solution to overcome the interconnect scaling problem in giga-scale integrated circuits (ICs). Thermal dissipation is a major challenge for 3-D integration and prior thermal-balanced task scheduling methods for 3-D multiprocessor system-on-chips (MPSoCs) typically balance power gradient across vertical stacks based on the assumption of strong thermal correlation among processing cores within a stack. On the other hand, 3-D MPSoCs typically employ network-on-chip (NoC) as the communication infrastructure which consumes a large portion of the energy budget. As TSVs consume much less energy than horizontal links in 3-D MPSoCs when transmitting the same amount data due to the reduced interconnect distance between vertical adjacent cores, it motivates to allocate heavily communicating tasks within the same vertical stack as much as possible, and thus traffic is restricted in the third dimension to reduce interconnect energy. However, aggregating active tasks within the same stack probably exacerbates the power density and result in hot spots. In this paper, we explore the tradeoff between thermal and interconnect energy when allocating tasks in 3-D Homogeneous MPSoCs, and propose an efficient heuristic. Experimental results show that the proposed technique can reduce interconnect energy by more than 25% on average with almost the same peak temperature when compared with prior thermal-balanced solutions.

Small delay defect (SDD) and aging-induced circuit failure are both prominent reliability concerns for nanoscale integrated circuits. Faster-than-at-speed testing is effective on SDD detection in manufacturing testing, which is always implemented by designing a suite of test signal generation circuits on the chip. Meanwhile, the integration of online aging sensors is becoming attractive in monitoring aging-induced delay degradation in the runtime. These design requirements, if implemented in separate ways, will increase the complexity of a reliable design and consume more die area. In this paper, a unified capture scheme is proposed to generate programmable clock signals for the detection of both SDDs and circuit aging. Our motivation arises from the observations that SDD detection and online aging prediction both need to capture circuit response ahead of the functional clock. The proposed aging-resistant design method enables the offline test circuit to be reused in online operations. Reversed short channel effect is also exploited to make the underlying circuit resilient to process variations. The proposed scheme is validated by intensive HSPICE simulations. Experimental results demonstrate the effectiveness in terms of low area, power, and performance overheads.

Concurrent trace is an emerging challenge when debugging multicore systems. In concurrent trace, trace buffer becomes a bottleneck since all trace sources try to access it simultaneously. In addition, the on-chip interconnection fabric is extremely high hardware cost for the distributed trace signals. In this paper, we propose a clustering-based scheme which implements concurrent trace for debugging Network-on-Chip (NoC) based multicore systems. In the proposed scheme, a unified communication framework eliminates the requirement for interconnection fabric which is only used during debugging. With clustering scheme, multiple concurrent trace sources can access distributed trace buffer via NoC under bandwidth constraint. We evaluate the proposed scheme using Booksim and the results show the effectiveness of the proposed scheme.

The widening gap between the fast-increasing transistor budget but slow-growing power delivery and system cooling capability calls for novel architectural solutions to boost energy efficiency. Leveraging the fact of surging “dark silicon” area, we propose a hybrid scheme to use both on-chip and off-chip voltage regulators, called “AgileRegulator”, for a multicore system to explore both coarse-grain and fine-grain power phases. We present two complementary algorithms: Sensitivity-Aware Application Scheduling (SAAS) and Responsiveness-Aware Application Scheduling (RAAS) to maximally achieve the energy saving potential of the hybrid regulator scheme. Experimental results show that the hybrid scheme achieves performance-energy efficiency close to per-core DVFS, without imposing much design cost. Meanwhile, the silicon overhead of this scheme is well contained into the “dark silicon”. Unlike other application specific schemes based on accelerators, the proposed scheme itself is a simple and universal solution for chip area and energy trade-offs.

Three-dimensional (3D) integration and Network-on-Chip (NoC) are both proposed to tackle the on-chip interconnect scaling problems, and extensive research efforts have been devoted to the design challenges of combining both. Through-silicon via (TSV) is considered to be the most promising technology for 3D integration, however, TSV pads distributed across planar layers occupy significant chip area and result in routing congestions. In addition, the yield of 3D integrated circuits decreased dramatically as the number of TSVs increases. For symmetric 3D mesh NoC, we observe that the TSVs' utilization is pretty low and adjacent routers rarely transmit packets via their vertical channels (i.e. TSVs) at the same time. Based on this observation, we propose a novel TSV squeezing scheme to share TSVs among neighboring router in a time division multiplex mode, which greatly improves the utilization of TSVs. Experimental results show that the proposed method can save significant TSV footprint with negligible performance overhead.

Very large scale integrated circuits typically employ Network-on-Chip (NoC) as the backbone for on-chip communication. As technology advances into the nanometer regime, NoCs become more and more susceptible to permanent faults such as manufacturing defects, device wear-out, which hinder the correct operations of the entire system. Therefore, effective fault-tolerant techniques are essential to improve the reliability of NoCs. Prior work mainly focuses on introducing redundancies, which can't achieve satisfactory reliability and also involve large hardware overhead, especially for data path components. In this paper, we propose fine-grained data path salvaging techniques by splitting data path components, i.e., links, input buffers and crossbar into slices, instead of introducing redundancies. As long as there is one fault-free slice for each component, the router can be functional. Experimental results show that the proposed solution achieves quite high reliability with graceful performance degradation even under high fault rate.

Three dimensional (3D) System-on-Chips (SoCs) that typically employ through-silicon vias (TSVs) as vertical interconnects, emerge as a promising solution to continue Moore's law. Whereas, it also brings challenging problems, one of which is the test wrapper chain design and optimization, especially for circuit-partitioned 3D SoCs in which scan chains can cross among layers. Test time is the primary goal for wrapper chain design, both for 2D and 3D SoCs. The 3D SoC wrapper chain design problem can be converted into the well-studied2D one by projecting wrapper chain components of all layers to one virtual layer. Thereafter, we can leverage 2D optimization algorithms to determine the composition of wrapper chains and thus guarantee minimal testing time for 3D SoCs. One specific thing for circuit-partitioned 3D SoCs is that TSVs are needed to connect cross-layer wrapper structures to form the wrapper chains. As TSVs occupy planar chip area and will aggravate the routing congestion problem, it is necessary to reduce TSVs for test purpose as much as possible. In this work, we observe that by varying the connection orders of wrapper chain components, e.g., scan chains and I/O cells, the TSVs consumed vary significantly. Based on the above, we formulate this problem and propose novel heuristic to tackle it. Experimental results show that the proposed solution can save on average 33.2% amount of TSVs when compared to a prior intuitive method.

The limited write endurance of phase change random access memory (PRAM) is one of the major obstacles for PRAM-based main memory. Wear leveling techniques were proposed to extend its lifetime by balancing writes traffic. Another important concern that need to be considered is endurance variation in PRAM chips. When different PRAM cells have distinct endurance, balanced writes will result in lifetime degradation due to the weakest cells. Instead of balancing writes traffic, in this paper we propose wear rate leveling (WRL), a variant of wear leveling, to balance wear rates (i.e., writes traffic/edudrance) of cells across the PRAM chip. After investigating writing behavior of applications and endurance variation, we propose an architecture-level WRL mechanism. Moreover, there is an important tradeoff between endurance improvement and swapping data volume. To co-optimize endurance and swapping, a novel algorithm, Max Hyper-weight Rematching, is proposed to maximize PRAM lifetime and minimize performance degradation. Experimental results show 19x endurance improvement to prior Wear Leveling.

A critical concern for post-silicon debug is the need to control the chip at clock cycle level. In a single clock chip, run-stop control can be implemented by gating the clock signal using a stop signal. However, data invalidation might occur when it comes to multiple-clock chips. In this paper, we analyze the possible data invalidation, including data repetition and data loss, when stopping and resuming a multiple-clock chip. Furthermore, we propose an efficient solution to eliminate data repetition and data loss. Theoretical analysis and simulation experiments are both conducted for the proposed solution. We implement the proposed Design-for-Debug (DfD) circuit with SMIC 0.18μm technology and simulate the data transfer across clock domains using SPICE tool. The results show that both data repetition and data loss can be avoided with the proposed solution, even if metastability occurs.

The emerging nanophotonic technology can avoid the limitation of I/O pin count, and provide abundant memory bandwidth. However, current DRAM organization has mainly been optimized for a higher storage capacity and package pin utilization. The resulted data fetching mechanism is quite inefficient in performance and energy saving, and cannot effectively utilize the abundant optical bandwidth in off-chip communication. This paper inspects the opportunity brought by optical communication, and revisits the DRAM memory architecture considering the technology trend towards multiprocessors. In our FlexMemory design, super-line prefetching is proposed to boost system performance and promote energy efficiency, which leverages the abundant photonic bandwidth to enlarge the effective data fetch size per memory cycle. To further preserve locality and maintain service parallelism for different workloads, page folding technique is employed to achieve adaptive data mapping in photonics-connected DRAM chips via optical wavelengths allocation. By combining both techniques, surplus off-chip bandwidth can be utilized and effectively managed adapting to the workloads intensity. Experimental results show that our FlexMemory achieves considerable improvements in performance and energy efficiency.

Applications' traffic tends to be bursty and the location of hot-spot nodes moves as time goes by. This will significantly aggregate the blocking problem of wormhole-routed Network-on-Chip (NoC). Most of state-of-the-art traffic balancing solutions are based on fully adaptive routing algorithms which may introduce large time/space overhead to routers. Partially adaptive routing algorithms, on the other hand, are time/space efficient, but lack of even or sufficient routing adaptiveness. Reconfigurable routing algorithms could provide on-demand routing adaptiveness for reducing blocking, but most of them are off-line solutions due to the lack of a practical model to dynamically generate deadlock-free routing algorithms.

The aggressive technology scaling poses serious challenges to lifetime reliability. A parament challenge comes from a variety of aging mechanisms that can cause gradual performance degradation of circuits. Prior work shows that such progressive degradation can be reliably detected by dedicated aging sensors, which provides a good foundation for proposing a new scheme to improve lifetime reliability. In this paper, we propose ReviveNet, a hardware-implemented aging-aware and self-adaptive architecture. Aging awareness is realized by deploying dedicated aging sensors, and self-adaptation is achieved by employing a group of synergistic agents. Each agent implements a localized timing adaptation mechanism to tolerate aging-induced delay on critical paths. On the evaluation, a reliability model based on widely used weibull distribution is presented. Experimental results show that, without compromising with any nominal architectural performance, ReviveNet can improve the Mean-Time-To-Failure by up to 48.7 percent, at the expense of 9.5 percent area overhead and small power increase.

Traditional DVFS schemes are oblivious to fine-grained adaptability resulting from path-grained timing imbalance. With the awareness of such fine-grained adaptability, better power-performance efficiency can be obtained. We propose a new scheme, MicroFix, to exploit such fine-grained adaptability. We first show the potential resulted from the path-grained timing imbalance and then present a new technique, Timing Interpolation, to reap the fine-grained adaptability for power reduction. Moreover, to eliminate the conservative margins of traditional DVFS, unlike the previous approaches such as Razor that reactively handle the delay errors (induced by aggressively scaled voltage/frequcncy) by enabling error detection and recovery, we propose a proactive approach by error prediction, thereby obviate the high-cost recovery routines. MicroFix was evaluated based on ISCAS89 benchmarks and the floating-point unit adopted by OpenSPARC T1 processor. Compared to ideal traditional DVFS schemes, the experimental results show that for most of the evaluated circuits, MicroFix can help saving up to 20% power consumption without compromising with frequency, at the expense of less than 5% area overhead. Compared to nonideal DVFS schemes (with 10% voltage margin), the power reduction can even reach up to 38% on average.

In ultra-deep submicrometer technology, soft errors and device aging are two of the paramount reliability concerns. Although many studies have been done to tackle the two challenges, most take them separately so far, thereby failing to reach better performance-cost tradeoffs. To support a more efficient design tradeoff, we propose a unified fault detection scheme—stability violation-based fault detection (SVFD), by which the soft errors (both single event upset and single event transient), aging delay, and delay faults can be uniformly dealt with. SVFD grounds on a new fault model, stability violation, derived from analysis of signal behavior. SVFD has been validated by conducting a set of intensive Hspice simulations targeting the next-generation 32-nm CMOS technology. An application of SVFD to a floating-point unit (FPU) is also evaluated. Experimental results show that SVFD has more versatile fault detection capability for fault detection than several schemes recently proposed at comparable overhead in terms of area, power, and performance.

Statistical static timing analysis (SSTA) considering process variation and aging effects is usually used to analyze circuit lifetime reliability at design phase. A key challenge for statistical lifetime reliability analysis is that an accurate statistical timing model is needed to carefully model practical variation distribution as well as delay correlation. In this work, P2CLRAF, a circuit lifetime reliability analysis framework is proposed. It calibrates pre-silicon SSTA result by learning the collected data from path delay testing at post-silicon timing validation phase. A neural network inside P2CLRAF is trained to learn variation distribution and delay correlation based on the statistic of path delay testing. The learned information is then fed back to SSTA to further improve the accuracy of circuit lifetime reliability analysis. Experimental results demonstrate the effectiveness of the proposed analysis framework.

In this paper, we propose a binary-tree waveguide connected Optical-Network-on-Chip (ONoC) to accelerate the establishment of the lightpath. By broadcasting the control data in the proposed power-efficient binary-tree waveguide, the maximal hops for establishing lightpath is reduced to two. With extensive simulations and analysis, we demonstrate that the proposed ONoC significantly reduces the setup time, and then the packet latency.

Topology virtualization techniques are proposed for NoC-based many-core processors with core-level redundancy to isolate hardware changes caused by on-chip defective cores. Prior work focuses on homogeneous cores with symmetric performance and optimizes on-chip communication only. However, core-to-core performance asymmetry due to manufacturing process variations poses new challenges for constructing virtual topologies. Lower performance cores may scatter over a virtual topology, while operating systems typically allocate tasks to continuous cores. As a result, parallel applications are probably assigned to a region containing many slower cores that become bottlenecks. To tackle the above problem, in this paper we present a novel performance-asymmetry-aware reconfiguration algorithm Bubble-Up based on a new metric called core fragmentation factor (CFF). Bubble-Up can arrange cores with similar performance closer, yet maintaining reasonable hop distances between virtual neighbors, thus accelerating applications with higher degree of parallelism, without changing existing allocation strategies for OS. Experimental results show its effectiveness.

Process, Voltage, and Temperature (PVT) variations can significantly degrade the performance benefits expected from next nanoscale technology. The primary circuit implication of the PVT variations is the resultant timing emergencies. In a multi-core processor running multiple programs, variations create spatial and temporal unbalance across the processing cores. Most prior schemes are dedicated to tolerating PVT variations individually for a single core, but ignore the opportunity of leveraging the complementary effects between variations and the intrinsic variation unbalance among individual cores. We find that the notorious delay impacts from different variations are not necessary aggregated. Cores with mild variations can share the violent workload from cores suffering large variations. If operated correctly, variations on different cores can help mitigating each other and result in a variation-mild environment. In this paper, we propose Timing Emergency Aware Thread Migration (TEA-TM), a delay sensor-based scheme to reduce system timing emergencies under PVT variations. Fourier transform and frequency domain analysis are conducted to provide the insights and the potential of the PVT co-optimization scheme. Experimental results show on average TEA-TM can help save up to 24% throughput loss, at the same time improve the system fairness by 85%.

We present nGFSIM, a GPU-based fault simulator for stuck-at faults which can report the fault coverage of one-to n-detection for any specified integer n using only a single run of fault simulation. nGFSIM, which explores the massive parallelism in the GPU architecture and optimizes the memory access and usage, enables accelerated fault simulation without the need of fault dropping. We show that nGFSIM offers a 25X speedup in comparison with a commercial tool and enables new applications in test selection.

Large scale Chip-Multiprocessors (CMPs) generally employ Network-on-Chip (NoC) to connect the last level cache (LLC), which is generally organized as distributed NUCA (non-uniform cache access) arrays for scalability and efficiency. On the other hand, aggressive technology scaling induces severe reliability problems, causing on-chip components (e.g., cores, cache banks, routers) failure due to manufacture defects or on-line hardware faults. Typical degradable CMPs should possess the ability to work around defects by disabling faulty components. For static NUCA architecture, when cache banks attached to a computing node are disabled, however, certain physical address sections will no longer be accessible. Prior approaches such as sets reduction introduced in Intel Xeon processor 7100 series enable turning off cache banks by masking certain sets bits in physical address1, which greatly wastes cache capacity. In this paper, we propose to tackle the above problem in a finer granularity to restrict the capacity loss in NUCA cache. Cache accesses to isolated nodes are redirected based on the utility-driven address remapping scheme that reduces data blocks conflicts in fault-tolerant shared-LLC. We evaluate our technique using GEMS simulator. Experimental results show that address remapping achieves significant improvement over the conventional cache sizing scheme.

Negative bias temperature instability (NBTI) has been a significant reliability concern in current digital circuit design due to its effect of increasing the path delay with time and in turn degrading the circuit performance. NBTI degradation has strong dependence on input pattern and duty cycles. Based on this observation, we propose to apply multiple input vectors to the combination circuit in a non-uniform way during standby mode. Multiple input vectors can enhance the capability to control the circuit nodes, achieve smaller duty cycles to reduce the stress time of gates and thus mitigate static NBTI. A constrained multi-object optimization model is formalized to find the optimal combination of duty cycles for timing-critical paths, which in turn minimizes the increase of path delay. An ATPG-like procedure is then presented to generate the corresponding input vectors. Experimental results demonstrate that the delay increase of timing-critical paths can be mitigated significantly under long time NBTI effect (10-year) by only applying a small number of vectors.

Recently, selective encoding of scan slices is proposed to compress test data. This encoding technique, unlike many other compression techniques encoding all the bits, only encodes the target-symbol by specifying single bit index and copying group data. In this paper, we propose an extended selective encoding which presents two new techniques: flexible grouping strategy, X bits exploitation and filling strategy. Flexible grouping strategy is able to decrease the number of encoded groups to improve compression ratio. X bits exploitation and filling strategy can exploit a large number of don't care bits to reduce testing power with no compression ratio loss. Experimental results show that the proposed technique needs less test data storage volume and reduces average weighted switching activity by 24.7%, peak weighted switching activity by 11.6% during scan shift compared to selective encoding.

This paper describes the scan test challenges and techniques used in the Godson-3 microprocessor, which is a scalable multicore processor based on the SMOC (scalable mesh of crossbar) on-chip network and targets high-end applications. Advanced techniques are adopted to achieve the scalable, low-power and low-cost scan architecture at the challenge of limited I/O resources and large scale of transistors. To achieve a scalable and flexible test access, a highly elaborate TAM (test access mechanism) is implemented with supporting multiple test instructions and test modes. Taking advantage of multiple cores embedding in the processor, scan partitions are employed to reduce test power and test time, and test compression with more than 10X compression ratio are utilized to decrease the scan chain length. To further decrease test time, a data-synchronous-comparator (DSC) is proposed for comparing the scan responses of the identical cores.

In ultra-deep submicro technology, two of the paramount reliability concerns are soft errors and device aging. Although intensive studies have been done to face the two challenges, most take them separately so far, thereby failing to reach better performance-cost tradeoffs. To support a more efficient design tradeoff, we present a new fault model, stability violation, derived from analysis of signal behavior. Furthermore, we propose a unified fault detection scheme-stability violation based fault detection (SVFD), by which the soft errors (both single event upset and single event transient), aging delay, and delay faults can be uniformly handled. SVFD can greatly facilitate soft error-resistant and aging-aware designs. SVFD is validated by conducting a set of intensive Hspice simulations targeting 65 nm CMOS technology. Experimental results show that SVFD has more robust capability for fault detection than previous schemes at comparable overhead in terms of area, power, and performance.

Traditional DVFS schemes are oblivious to fine-grained adaptability resulting from path-grained timing imbalance. With the awareness of such fine-grained adaptability, better power-performance efficiency can be obtained. We propose a new approach, MicroFix, to exploit such fine-grained adaptability. We first reveal the potential of the path-grained timing imbalance and then present a novel implementation of MicroFix. Moreover, to eliminate the conservative margins of traditional DVFS, unlike the previous approaches that reactively handle the delay errors (induced by aggressively scaled voltage/frequcncy) by error detection and recovery strategies, we propose a proactive approach by error prediction. MicroFix was evaluated based on the floating-point unit adopted by OpenSPARC T1 processor. Compared against traditional DVFS schemes, the experimental results shows that MicroFix improves the EDP (Energy-Delay Product) up to 35% for high-performance circuits and PDP (Power-Delay Product) to 28% for low-power circuits, while at the expense of only 7% area overhead.

Thread-level redundancy in Chip Multiprocessors(TLR-CMP) is efficient for soft error tolerance. Process variation causes core-to-core (C2C) performance asymmetry across a chip, which should be taken into consideration for application scheduling. In this paper, two types of variations beyond C2C are introduced, i.e., inter-pair and intra-pair variation in TLR-CMP. Intra-pair performance asymmetry can affect the performance of applications differently. Based on the above observation, we firstly formalize the variation aware scheduling in TLR-CMP as a 0-1 programming problem,to maximize the system weighted throughput. An efficient scheduling algorithm, named IntraVarF&AppSen, is then proposed to tackle this problem, which can be proved to be optimal when the number of applications to be scheduled is equal to the number of core pairs. Simulation on a 64-core CMP shows 2.8%-4% improvement in weighted throughput when compared to prior VarF&AppIPC algorithm.

The Network-on-Chip (NoC) meshes are limited by the reliability constraint, which impels us to exploit the fault tolerant routing. Particularly, one of the main design issues is minimizing the loss of non-faulty routers at the presence of faults. To address that problem, we propose a new fault tolerant routing, which has the following two distinct advantages: First, it keeps a network deadlock-free by utilizing restricted intermediate nodes rather than adding virtual channels (VC). This characteristic leads to an area-efficient router. Second, in the proposed routing algorithm, the rounds of DOR are not limited by the number of VC's anymore. As a consequence, the number of sacrificed non-faulty routers is significantly reduced. We demonstrate above advantages through extensive simulations. The experimental results show that under the limitation of VC's, the proposed routing algorithm always sacrifices the minimal number of non-faulty routers compared to previous solutions.

Bugs are tending to be unavoidable in the design of complex integrated circuits. It is imperative to identify the bugs as soon as possible by post-silicon debug. The main challenge for post-silicon debug is the observability of the internal signals. This paper exploits the fact that it is not necessary to observe the error free states. Then we introduce "suspect window" and present a method for determining its boundary. Based on suspect window, we propose a debug approach to achieve high observability by reusing scan chain. Since scan dumps take place only in suspect window, debug time is greatly reduced. Experiment results demonstrate the effectiveness of the proposed approach.

Homogeneous manycore systems are emerging for tera-scale computation and typically utilize Network-on-Chip (NoC) as the communication scheme between embedded cores. Effective defect tolerance techniques are essential to improve the yield of such complex integrated circuits. We propose to achieve fault tolerance by employing redundancy at the core-level instead of at the microarchitecture level. When faulty cores exist on-chip in this architecture, however, the physical topologies of various manufactured chips can be significantly different. How to reconfigure the system with the most effective NoC topology is a relevant research problem. In this paper, we first show that this problem is an instance of a well known NP-complete problem. We then present novel solutions for the above problem, which not only maximize the performance of the on-chip communication scheme, but also provide a unified topology to Operating System and application software running on the processor. Experimental results show the effectiveness of the proposed techniques.

Homogeneous manycore processors are emerging for tera-scale computation. Effective defect tolerance techniques are essential to improve the yield of such complex integrated circuits. In this paper, we propose to achieve fault tolerance by employing redundancy at the core-level instead of at the microarchitecture-level. When faulty cores existing on-chip in this architecture, how to reconfigure the processor with the most effective topology is a relevant research problem. We present novel solutions for this problem, which not only maximize the performance of the manycore processor, but also provide a unified topology to operating system and application software running on the processor. Experimental results show the effectiveness of the proposed techniques.

The continuous development of VLSI technology is shrinking the minimal sizes to nanometer region, making circuits more susceptible to transient error. In this paper, we present a frequency analysis method to accurately estimate the possible propagation of transient fault-due glitches through a CMOS combinational circuit. We use the frequency feature of signal and frequency response of electrical system to analyze the propagation of transient error. Experiments show that on average, our approach provides approximately 95% accuracy and several orders of magnitude faster with respect to HSTICE simulation.

An embedded test stimulus decompressor is presented for the test patterns decompression, which can reduce the required channels and vector memory of automatic test equipment (ATE) for complex processor circuit. The proposed decompressor mainly consists of a periodically alterable MUX network which has multiple configurations to decode the input information flexibly and efficiently. In order to reduce the number of test patterns and configurations, a test patterns compaction algorithm, using CI-Graph merging, is proposed. With the proposed periodically alterable MUX network and the patterns compaction algorithm, smaller test data volume and required external pins can be achieved as compared to previous techniques

We proposed clustered random access scan (CRAS) architecture to reduce test data volume. CRAS makes use of the compatibility of the test stimuli to cluster the scan cells, and assigns every cluster a unique address. The compression ratio upper bound of CRAS is analyzed based on the random graph theory. Experimental results on ISCAS'89 benchmarks and two industry designs show that the proposed CRAS architecture can yield on average 67.3% reduction in test data volume, with reasonable area and routing overhead than scan design

Packet classification is important in fulfilling the requirements of new services such as policy-based routing in next generation networks. In this paper, we propose a novel bit vector based two-dimensional packet classification algorithm called Group Bit Vector. The key feature of the proposed algorithm is its ability to set the length of the bit vector to perform AND operation (the operation bit vector) regardless of the size of rule databases by effectively aggregating the rules and constructing particular groups of a limited number of members. Moreover, the proposed algorithm avoids the problem of false matches, which may be brought by rules aggregation. Thus, the length of the operation bit vector can be adjusted to be suitable to the width of memories to improve the performance. Experimental results demonstrate the flexibility and effectiveness of the proposed algorithm.

Utilizing an on-chip decompressor is an efficient method to reduce test data volume in multiple-scan-chain designs. This paper investigates a new technique to implement the decompressor by combinational circuits. The proposed architecture drives a large number of internal scan chains with far fewer external input pins, thus delivering significant reductions in test data volume. Based on the analysis of compatible relationships among scan slices, the number of external scan inputs can be minimized. The effectiveness and applicability of the proposed scheme are demonstrated by experimental results.

This paper addresses the problem of test response compaction. In order to maximize compaction ratio, a single-output encoder based on check matrix of a (n, n-1, m, 3) convolutional code is proposed. Theoretic analysis for this encoder is presented to avoid two and any odd erroneous bit cancellations, handle one unknown bit(X bit) and diagnose one erroneous bit. The X-bits tolerance capacity can be enhanced by choosing a proper memory size and weight of check matrix, which can also be obtained by an optimized input assignment algorithm. The theoretic analysis and experimental results on aliasing shows the efficiency of the proposed encoder.

This paper presents a decompression architecture using a periodically alterable MUXs decompressor for scan data volume reduction. Compared to static XOR network, the periodically alterable MUXs decompressor has multiple configurations to decode the input information more efficiently. Three different DFT techniques are proposed to handle hard, firm and soft cores, respectively. With the proposed pattern decompression algorithms and scan decompression architecture, smaller test data volume and test application time can be achieved as compared to previous techniques.

Conventional serial connection of internal scan chains brings the power and time penalty. A parallel core wrapper design (pCWD) approach is presented in this paper for reducing test power and test application time. The pCWD utilizes overlapping scan slices to reduce the number of scan slices loading. Experimental results on d695 of ITC2002 benchmark demonstrated that, about 2/spl times/ shift time and 20/spl times/ test power reduction can be achieved.

Along with more pre-designed and pre-verified cores are integrated into a single chip to construct an entire system, the test application time increases significantly. This paper presents a novel test scheduling solution, unlike previous techniques that take advantage of balanced scan chains of every single core, utilizing the balance of pairwise combined cores. Experimental results for two ITC '02 SOC benchmarks show that the pair balance-based test scheduling technique achieves less test time compared to the previous approaches.

This paper presents a test resource partitioning technique based on an efficient single-output response compaction design called quotient compactor (q-Compactor). Some design theorems of quotient compactor are presented to achieve full diagnostics ability, minimize error cancellation and handle the X bits in the outputs of the CUT The quotient compactor can also be moved to the load-board to reduce the number of ATE channels required. Our experimental results on the ISCA S89 benchmark circuits and an MPEG 2 decoder SOC show that the proposed compaction scheme is very efficient.