Diffusion Policy (DP) has demonstrated strong potential in robotic visuomotor control, offering robust generalization and seamless integration of multi-modal data. However, its complex model structure and increasing multi-modal inputs have brought latency and power challenges for edge resource-constrained robotic platforms. To address the above challenges, we identify the potential intra-model and inter-model redundancies in DP. We observe that DP relies on frequent multi-modal inputs such as images and text during execution. However, the demands of fine-grained robotic manipulation result in substantial intra-modal similarity across consecutive image frames, which, combined with inter-modal semantic redundancy between images and language, indicates that much of the input information is repetitive and potentially compressible. Yet prior works have not exploited these characteristics for targeted optimization. We therefore propose a hardware–software co-design accelerator. On the algorithmic side, we introduce self- and cross-guided modal compression, leveraging intra- and inter-modality similarity to reduce redundant computation within the key DP modules. On the hardware side, we design a tailored architecture that supports multiple operators with optimized sparse memory access, lightweight computation engines, and reconfigurable on-chip dataflow, substantially reducing energy cost. Experimental results demonstrate a 26× speedup over a high-performance GPU while consuming only 1.5 W, enabling low-power and real-time robotic control on edge robotic devices.

Optical computing is becoming a promising solution to meet the growing computational demands of increasingly large-scale deep neural networks (DNNs). However, high power consumption from analog-to-digital (ADC) and digital-to-analog (DAC) conversions poses significant challenges for optical computing. Spiking Neural Networks (SNNs), with their binary spike-based input and output, show the potential to address this issue by reducing the need for high-precision DAC/ADC.In order to exploit the complementary nature of optical computing and spike-based processing, this paper proposes the Partially Incoherent Configurable Optical Computing Architecture for SNN Acceleration (PICoSNN). We address three critical challenges: phase errors in coherent optical computing, limited configurability in weight-stationary architectures, and inefficient mapping of general SNNs to optical computing hardware. We integrate partially incoherent tensor cores with optical leaky integrate-and-fire neurons, minimizing ADC/DAC overhead while supporting dynamic weight mapping. Further, we propose KV Spiking Self-Attention to enable efficient attention with 1-bit multiplications. Experimental results show that PICoSNN achieves up to 70.54× higher throughput and 8.13× lower energy consumption compared to ASIC implementations, while delivering 15.46× better throughput per area and 17.67× better energy efficiency per area than state-of-the-art photonic accelerators.

Compute-in-memory (CiM) for edge AI inference operates under strict memory and energy constraints. While unstructured pruning reduces model size and computation, efficiently deploying the sparse weights on CiM’s dense, regular arrays remains challenging. Existing studies either incur high indexing overhead by storing per-element indexing metadata, or achieve limited compression by relying on scarce structural patterns within unstructured weights. The column packing method, which avoids the high overhead of per-element indexing and offers rich compression potential, shows promise to reconcile unstructured sparsity with CiM’s regular compute pattern, but its direct application to CiM is hindered by heuristic grouping algorithms that either yield suboptimal compression or sacrifice model accuracy.To bridge this gap and unlock the potential of column packing for CiM, this study presents EGO, an algorithm-hardware co-designed framework. EGO overcomes the inefficiency of heuristic grouping by introducing a combinatorially optimized grouping algorithm, which formulates column packing as minimum-cost graph matching. A digital CiM architecture is co-designed with the EGO column grouping formulation, which features a custom Sparsity Processing Unit (SPU) to enable efficient activation routing while preserving CiM’s dense and regular dataflow. Circuit-level simulations show that EGO achieves 1.4–3.7x average improvement in energy efficiency and 1.2–1.8x average improvement in area efficiency compared to previous state-of-the-art methods.

Compute-in-Memory (CiM) has emerged as a promising paradigm to overcome the memory bottleneck of von Neumann architectures in data-intensive applications. While SRAM-based CiM benefits from mature fabrication support and high design flexibility, it suffers from significant access energy due to limited memory density. Recent advances in ROM-based CiM provide a high-density, energy-efficient alternative for deploying entire deep neural network (DNN) models on-chip, often assisted by small SRAM CiM modules to enhance task-level flexibility. However, existing ROM CiM architectures still face critical challenges in further scaling memory density and achieving finer-grained flexibility improvement.This paper presents FSDB, a digital hybrid ROM/SRAM CiM architecture to address these limitations. FSDB incorporates a folded-store compressed ROM CiM macro implemented using a sparsity-aware quantization methodology, achieving a record-high memory density of 40.2 Mb/mm2in a 28nm CMOS technology. Furthermore, the proposed dynamic-broaden computing architecture enables updates to parameters stored in ROM, providing kernel-level reconfigurability and cross-model scalability. Experimental results on an extended ResNet-50 demonstrate that FSDB improves inference accuracy by >5% on ImageNet compared to prior state-of-the-art (SOTA) flexible ROM CiM architectures.

Recent years have witnessed the rapid progress and broad application of diffusion probabilistic models (DPMs). Sampling from DPMs can be viewed as solving an ordinary differential equation (ODE). Despite the promising performance, the generation of DPMs usually consumes much time due to the large number of function evaluations (NFE). Though recent works have accelerated the sampling to around 20 steps with high-order solvers, the sample quality with less than 10 NFE can still be improved. In this paper, we propose a unified sampling framework (USF++) to study the optional strategies for solver. Under this framework, we further reveal that taking different solving strategies at different timesteps may help further decrease the truncation error, and a carefully designed solver schedule has the potential to improve the sample quality by a large margin. Therefore, we propose a new sampling framework based on the exponential integral formulation that allows free choices of solver strategy at each step and design specific decisions for the framework. Moreover, we apply evolutionary search to find outstanding solver schedules which outperform the state-of-the-art sampling methods on CIFAR-10, ImageNet, and LSUN-Bedroom datasets. Specifically, we achieve 3.89 FID with 5 NFE on CIFAR-10 dataset and 8.62 FID with 3 NFE on LSUN-Bedroom dataset, outperforming the SOTA method significantly. We further apply searching to Stable-Diffusion model and get an acceleration ratio of 2×, showing the feasibility of sampling in very few steps without retraining the neural network.

Neural networks (NNs) have exhibited excellent performance in various fields of artificial intelligence. However, the primary operations in these mainstream models, including matrix-vector multiplication (MVM), element-wise multiplication (EWM), and depth-wise convolution (DWConv), require massive data movements during computation, which greatly impacts NNs’ inference performance. The emerging Processing-In-Memory (PIM) architectures have shown great potential to overcome the memory wall problem. However, constrained by the supported data format and operator type, directly adopting PIM architectures for neural network acceleration faces three challenges: (1) Floating-point (FP) format has been widely adopted for ensuring high algorithm accuracy. However, Resistive Random-Access Memory (RRAM)-based analog PIM architectures perform integer (INT) MVMs in the analog domain, limiting their application to the more accurate FP format; (2) Static Random-Access Memory (SRAM)-based digital PIM architectures require additional circuits to support the FP format, and the SRAM capacity cannot satisfy the storage requirement of latest large language models (LLMs); (3) When performing the operators with few accumulation steps, such as EWMs and DWConvs, only few memory units in PIM architecture are activated, resulting in severe device under-utilization. To tackle the above challenges, this article proposes an RRAM and 3D-SRAM-based hybrid PIM architecture, achieving FP-based algorithm accuracy, high device utilization, and high energy efficiency. At the software level , we first analyze the impact of quantization errors on NN’s inference accuracy. For the quantization error-insensitive MVM operations, we propose the PIM-oriented exponent-free non-uniform (PN) data format. The proposed PN format can be flexibly adjusted to fit the non-uniform distribution and approach FP-based algorithm accuracy using bit-slicing-based full INT operations. For the quantization error-sensitive EWM/DWConv operations, we introduce the multiplication-free approximated FP multiplications to reduce the additional hardware overhead. At the hardware level , we propose a hybrid PIM architecture, including an RRAM analog PIM using shift-and-add for PN-based MVMs, and a 3D-SRAM digital PIM with high utilization for DWConv/EWM operations. Extensive experiments on CNNs and attention-free LLMs validate that the proposed PIM architecture achieves up to 99.4× and 33.9× speedup with 5697.7× and 8.2× energy efficiency improvement compared to GPU and PIM-baseline, respectively. With the proposed PN format and approximated FP multiplications, the algorithm accuracy of CNNs and attention-free LLMs can be improved by up to 3.01% and 10.18%, respectively.

While large Transformer models have exhibited outstanding performance on multimodal tasks, the underlying massive parameters land up with memory-wall issues. To address this bottleneck, SRAM-based compute-in-memory (CiM) is a promising technique. However, frequent off-chip weight loading due to limited on-chip capacity could severely limit the systemlevel energy efficiency. Recently, a high-density CiM structure at 16.4Mb/mm2, YOLoC, has shown the potential of complete on-chip deployment of a large detection model using transistor-based read-only-memory (ROM). However, it is still challenging to deploy even larger Transformer models. With opportunities provided by LoRA for finetuning large pretrained models on ROM-CiM with very light SRAM-CiMs, this work achieves ultra-high density up to 165.6Mb/mm2 by eliminating the use of transistors for ROM-CiM with a proposed 3D-METRO and a 3D stacking array on the mature CMOS process. Unlike the usual belief that parasitics have negative impacts, this work observes that parasitics can be utilized for data storage. Furthermore, a local recovering unit (LRU) is proposed for addressing the interference due to the transistor-less structure. 3D-METRO achieves ultra-high density improvement over the previous YOLoC, which is hundreds of times higher than that of SRAM-CiM, enabling the opportunity for large language model (LLM) deployment on a single chip with 28x energy efficiency improvement.

Owing to mature fabrication support and high flexibility, static random-access memory (SRAM) has become a very promising candidate for compute-in-memory (CiM) in accelerating deep neural networks (DNNs). However, SRAM-based CiM has low memory density and thus very limited total on-chip capacity, resulting in frequent weights reloading and additional power consumption during end-to-end inference tasks. Analog ROM CiM increases memory density but suffers from low computing density caused by A/D converter (ADC) limitation. To address these challenges, for the first time, a fully digital compute-in-read-only-memory (DCiROM) design approach is proposed in this paper. DCiROM introduces a novel ROM-logic fusion CiM that successfully reduces CiM area by 51% while maintaining high memory density and computing performance. By reusing multiply-and-accumulation (MAC) resources, DCiROM further achieves flexibility with a minimal area cost. We have implemented a DCiROM chip loaded 3024Kb ResNet-56 parameters using 65nm CMOS technology. This macro achieves 10.2x-55.7x higher normalized FoM (memory density x computing density) than the state-of-the-art CiM works. It also reduces 2.9x-9.9x energy consumption per image inference than SRAM CiM works when considering off-chip access.

City-scale traffic signal control (TSC) involves thousands of heterogeneous intersections with varying topologies, making cooperative decision-making across intersections particularly challenging. Given the prohibitive computational cost of learning individual policies for each intersection, some researchers explore learning a universal policy to control each intersection in a decentralized manner, where the key challenge is to construct a universal representation method for heterogeneous intersections. However, existing methods are limited to universally representing information of heterogeneous ego intersections, neglecting the essential representation of influence from their heterogeneous neighbors. Universally incorporating neighborhood information is nontrivial due to the intrinsic complexity of traffic flow interactions, as well as the challenge of modeling collective influences from neighbor intersections. To address these challenges, we propose CityLight, which learns a universal policy based on representations obtained with two major modules: a Neighbor Influence Encoder to explicitly model neighbor's influence with specified traffic flow relation and connectivity to the ego intersection; a Neighbor Influence Aggregator to attentively aggregate the influence of neighbors based on their mutual competitive relations. Extensive experiments on five city-scale datasets, ranging from 97 to 13,952 intersections, confirm the efficacy of CityLight, with an average throughput improvement of 11.68% and a lift of 22.59% for generalization. Our codes and datasets are released: https://github.com/tsinghua-fib-lab/CityLight.

Vision-Language Models (VLMs) have enabled a variety of real-world applications. The large parameter size of VLMs brings large memory and computation overhead which poses significant challenges for deployment. Post-Training Quantization (PTQ) is an effective technique to reduce the memory and computation overhead. Existing PTQ methods mainly focus on large language models (LLMs), without considering the differences across other modalities. In this paper, we discover that there is a significant difference in sensitivity between language and vision tokens in large VLMs. Therefore, treating tokens from different modalities equally, as in existing PTQ methods, may over-emphasize the insensitive modalities, leading to significant accuracy loss. To deal with the above issue, we propose a simple yet effective method, Modality-Balanced Quantization (MBQ), for large VLMs. Specifically, MBQ incorporates the different sensitivities across modalities during the calibration process to minimize the reconstruction loss for better quantization parameters. Extensive experiments show that MBQ can significantly improve task accuracy by up to 4.4% and 11.6% under W3A16 and W4A8 quantization for 7B to 70B VLMs, compared to SOTA baselines. Additionally, we implement a W3A16 GPU kernel that fuses the dequantization and GEMV operators, achieving a 1.4× speedup on LLaVA-onevision-7B on the RTX 4090. The code is available at https://github.com/thu-nics/MBQ.

Long-context inference has become a central focus in recent self-regressive Transformer research. However, challenges still remain in performing decode stage due to the memory bandwidth bottleneck of attention mechanisms and the substantial memory overhead associated with KV cache. Although attention sparsity has been proposed as a potential solution, conventional sparsity methods that rely on heuristic algorithms often suffer from accuracy degradation when applied to ultra-long sequences. To break through the dilemma between accuracy-performance and bandwidth-capacity, this work proposes DIAS, a distancebased irregular attention sparsity approach with processing-inmemory (PIM) architecture. DIAS employs approximate topK attention (AKAttention) scores through graph-based search to improve inference efficiency while maintaining accuracy. Furthermore, a scalable tree-like PIM (TreePIM) architecture is introduced to achieve both memory capacity and bandwidth improvement by isolating enormous memory access for KV cache into the PIM units. Evaluations on various configurations of DIAS for Longbench with Llama3-405B models with 1 M sequence length show up to 75 times speedup compared with the state-of-the-art LLM accelerator, with accuracy drop of less than $1 \%$. Index Terms-AI and Machine Learning, Architecture & System Design

Compute-in-memory (CiM) has become a promising candidate for edge AI by reducing data movements through insitu operations. However, this emerging computational paradigm also poses the vulnerability of model leakage as the weights are stored in plaintext for computing. While prior works have explored lightweight encryption methods, CiM is usually considered a separate module instead of a system component, leaving the origin of keys unclear and unprotected. Physical unclonable functions (PUFs) offer a potential origin of keys, but a comprehensive framework for securing key generation and delivery remains lacking. Besides, the complementary ciphertext storage incurs substantial costs and degrades the performance. This work proposes PUFiM, a robust and efficient security solution for edge computing based on ferroelectric FETs (FeFETs). For the first time, a strong PUF is synergized with CiM to enable authentication, key generation, and encrypted computations within a unified array for comprehensive protection. To achieve this synergization, a high-density hybrid storage and computation approach combining PUF and weight bits via multi-level cell (MLC) FeFETs is proposed. Besides, two PUF enhancement techniques and a novel mapping scheme are developed to improve security and efficiency further. Results show that PUFiM withstands PUF modeling attacks with up to $\mathbf{1 0 M}$ samples. Moreover, PUFiM reduces the inference accuracy by $\gt 60 \%$ under 95% key leakage and achieves $\gt 9.7 \times$ compute density and $\gt 1.2 \times$ energy efficiency improvement compared with the state-of-the-art SRAM/NVM secure CiMs.

Transformer-based video generation models have demonstrated significant potential in content creation. However, the current state-of-the-art model employing “ 3 D full attention” encounters substantial computation and storage challenges. For instance, the attention map size for $\operatorname{Cog}$ VideoX-5B requires 56.50 GB, and generating a video of 49 frames takes approximately 1 minute on an NVIDIA A100 GPU under FP16. Although model quantization has proven effective in reducing both memory and computational costs, applying it to video generation models still faces challenges in preserving algorithm performance while ensuring efficient hardware processing. To address these issues, we introduce PARO, a video generation accelerator with patternaware reorder-based attention quantization. PARO investigates the diverse attention patterns of 3D full attention and proposes a novel reorder technique to unify these patterns into a unified “block diagonal” structure. Block-wise mixed precision quantization is further applied to achieve lossless compression under an average bitwidth of 4.80 bits. In terms of hardware, to overcome the limitation of existing mixed-precision computing units could not fully utilize the attention map bitwidth to accelerate $Q K$ multiplication, PARO designs an output-bitwidth aware mixedprecision processing element (PE) array through hardwaresoftware co-design. This approach ensures that the mixedprecision characteristics are fully utilized to enhance hardware efficiency in the bottleneck attention computation. Experiments demonstrate that PARO delivers up to $2.71 \times$ improvement in end-to-end performance compared to an NVIDIA A100 GPU and achieves up to $6.38 \sim 7.05 \times$ speedup over state-of-the-art ASICbased accelerators on the CogVideoX-2B and 5B models.

Compute-in-Memory (CiM) is a promising technique for energy-efficient deep neural network (DNN) inference to miti-gate the memory bottleneck. Unfortunately, conventional SRAM-based CiM has a low density and limited on-chip capacity, resulting in undesired weight reloading from off-chip DRAM. The emerging high-density ROM-based CiM architecture has recently revealed the opportunity of deploying large-scale DNNs on-chip, with optional assisting SRAM to ensure moderate flexibility. However, prior analog-domain ROM CiM still suffers from limited memory density improvement and low computing area efficiency due to stringent array structure and large A/D converter (ADC) overhead. This paper presents DSC-ROM, a fully digital sparsity-compressed compute-in-ROM architecture to address these challenges. DSC-ROM introduces a fully synthesizable macro-level design methodology that achieves a record-high memory density of 27.9 Mb/mm2in a 28nm CMOS technology. Experimental results show that the macro area efficiency of DSC-ROM improves by 5.6-6.6x compared with prior analog-based ROM CiM. Furthermore, a novel weight fine-tuning technique is proposed to ensure task transfer flexibility and reduce required assisting SRAM cells by 94.4%. Experimental results show that DSC-ROM designed for ResNet-18 pre-trained on ImageNet dataset achieves <0.5% accuracy loss in CIFAR-10 and FER2013, compared with the fully SRAM-based CiM.

Near DRAM Processing (NDP) architectures have emerged to be a promising solution for commercializing in-memory computing and addressing the “memory wall” problem, especially for the memory-intensive machine learning (ML) workloads. In NDP architectures, the Processing Units (PUs) are distributed next to different memory units to exploit the high internal bandwidth. Therefore, in order to fully utilize the bandwidth advantage of NDP architectures for ML applications, meticulous evaluations and optimizations of data placement in DRAM and workload scheduling among different PUs are required. However, existing simulation and compilation tools face two insuperable obstacles to achieving these targets. On the one hand, tools for traditional von Neumann architectures only focus on the data access behaviors between the host and DRAM and treat DRAM as a whole part, which cannot support NDP architectures with multiple independent processing and memory units working simultaneously. On the other hand, existing NDP simulators and compilers are designed for specific DRAM technology and NDP architecture, lacking compatibility for various NDP architectures. In order to overcome these challenges and optimize data mapping and workload scheduling for different NDP architectures, we propose UniNDP, a unified NDP compilation and simulation tool for ML applications. Firstly, we propose a unified tree-based NDP hardware abstraction and the corresponding instruction set, enabling the support for various NDP architectures based on different DRAM technologies. Secondly, we design a cycle-accurate and instruction-driven NDP simulator to evaluate hardware performance by accurately tracking the working status of memory elements and PUs. The accurate simulation can provide effective guidance for compilation. Thirdly, we design an NDP compiler that optimizes data partition, mapping, and workload scheduling in different DRAM hierarchies. Furthermore, to enhance the compilation efficiency, we propose a hardware status-guided search space pruning strategy and a fast performance predictor using DRAM timing parameters. Extensive experimental results show that, compared to existing mapping and compilation methods, UniNDP can achieve 1.05-3.43 $\times$ speedup across multiple NDP architectures and different ML workloads. Furthermore, based on the results of UniNDP, we provide insights for the future NDP architecture design and deployment in ML applications.

Compound AI systems showcase impressive performance and versatility compared to single AI models by combining large language models (LLMs) with various smaller expert models. The major bottleneck of compound AI lies in frequent data movement due to the massive parameters and dynamic routing mechanisms. Compute-In-Memory (CiM) has demonstrated great potential to mitigate the memory wall. However, constrained by the rigid array structure, existing CiM accelerators struggle to meet more general and diverse model compression demands of compound AI, such as fine-grained pruning for expert models and outlier-aware quantization for LLM-based router models. The lack of support for agile model compression hinders the deployment of compound AI systems on CiM accelerators.To fully unlock the potential of CiM in accelerating compound AI, we present DANCE, a dual-side N:M sparse compressed digital CiM architecture with cross-layer co-optimizations: (i) At the circuit level, DANCE introduces a customized set-associative selection circuit to extract N:M sparse patterns for both weights and activations, maintaining high parallelism; (ii) At the architecture level, DANCE explores a novel design paradigm that integrates fine-grained pruning and outlier-aware quantization into a unified N:M sparsity compression framework. Experimental results show that DANCE achieves up to 4.36× energy efficiency improvement with <1% accuracy loss for ResNet-18 on CIFAR-100, and up to 2.59× energy efficiency improvement with <0.5 perplexity increase for Llama-7B on WikiText-2, compared to the conventional digital CiM baseline.

The emerging analog Processing-in-Memory (PIM) architectures have shown great potential to overcome the memory wall problem and accelerate neural network (NN) inference. However, different from digital architectures, the computation accuracy of analog PIM architectures is directly impacted by various errors, which are related to both software and hardware parameters. Existing PIM simulators mainly adopt the bit-and-crossbar slicing paradigm to evaluate the accuracy under various errors. Each MVM operation is performed bit by bit and crossbar by crossbar, which is extremely time-consuming, especially for models with a larger number of parameters, such as large language models (LLMs).In this work, we propose an error-injected robustness metric, unifying various errors into the weight dimension and facilitating joint error analysis. Based on the error-injected robustness metric, we propose a Non-Ideal PIM Accuracy (NIPA) evaluation model for relative accuracy evaluation, considering the coupling effect (i.e., various errors can be affected by the same factor) among various errors using NN’s prior information. We further propose a non-slicing absolute accuracy evaluation method, eliminating the need for the time-consuming bit-and-crossbar slicing process. Extensive experiments on CNNs and LLMs validate that the proposed NIPA evaluation model achieves high correlations of up to 0.91 with the absolute accuracy evaluated by DNN+NeuroSim. At the same time, compared to existing bit-and-crossbar slicing evaluation methods, the proposed non-slicing absolute accuracy evaluation method achieves up to 105.8× speedup with average evaluation errors as low as 0.29%.

Diffusion Models (DM) and Consistency Models (CM) are two types of popular generative models with good generation quality on various tasks. When training DM and CM, intermediate weight checkpoints are not fully utilized and only the last converged checkpoint is used. In this work, we find proper checkpoint merging can significantly improve the training convergence and final performance. Specifically, we propose LCSC, a simple but effective and efficient method to enhance the performance of DM and CM, by combining checkpoints along the training trajectory with coefficients deduced from evolutionary search. We demonstrate the value of LCSC through two use cases: (a) Reducing training cost. With LCSC, we only need to train DM/CM with fewer number of iterations and/or lower batch sizes to obtain comparable sample quality with the fully trained model. For example, LCSC achieves considerable training speedups for CM (23$\times$ on CIFAR-10 and 15$\times$ on ImageNet-64). (b) Enhancing pre-trained models. When full training is already done, LCSC can further improve the generation quality or efficiency of the final converged models. For example, LCSC achieves better FID using 1 number of function evaluation (NFE) than the base model with 2 NFE on consistency distillation, and decreases the NFE of DM from 15 to 9 while maintaining the generation quality. Applying LCSC to large text-to-image models, we also observe clearly enhanced generation quality.

Diffusion transformers have demonstrated remarkable performance in visual generation tasks, such as generating realistic images or videos based on textual instructions. However, larger model sizes and multi-frame processing for video generation lead to increased computational and memory costs, posing challenges for practical deployment on edge devices. Post-Training Quantization (PTQ) is an effective method for reducing memory costs and computational complexity. When quantizing diffusion transformers, we find that existing quantization methods face challenges when applied to text-to-image and video tasks. To address these challenges, we begin by systematically analyzing the source of quantization error and conclude with the unique challenges posed by DiT quantization. Accordingly, we design an improved quantization scheme: ViDiT-Q (**V**ideo \& **I**mage **Di**ffusion **T**ransformer **Q**uantization), tailored specifically for DiT models. We validate the effectiveness of ViDiT-Q across a variety of text-to-image and video models, achieving W8A8 and W4A8 with negligible degradation in visual quality and metrics. Additionally, we implement efficient GPU kernels to achieve practical 2-2.5x memory optimization and a 1.4-1.7x end-to-end latency speedup.

With the development of the Internet of Things (IoT), in-sensor computing techniques, such as the frame differencing and edge detection, is an effective solution for reducing large-scale sensing data transmission activities and related costs by eliminating the spatiotemporal redundancy. In these in-sensor computing tasks, thin-film transistor (TFT) technologies have gained particular interest because of their inherent support for large-area sensing-computing integration. However, existing TFT-based in-sensor computing works face challenges of high device variations, high-cost analog readouts and insufficient optimization of sequential redundancy. To address these challenges, we propose a robust spatiotemporal optimization-based in-sensor computing architecture, enabling low-power, low-latency, and efficient data compression. By combining frame differencing and edge detection, the data sparsity is greatly improved while the data movement is reduced. We develop a 4μm low-temperature polysilicon (LTPS) TFT digital standard cell library and implement a Rice compression encoder based on it. The system-level evaluations exhibit 21.5× compression ratio, 6.9× energy efficiency and 5.2× speedup improvement compared with traditional compression schemes in large-area scenarios.

Compute-In-Memory (CiM) is an emerging architecture designed to address the memory wall issue in deep neural network (DNN) inference. However, both the ADC in analog CiM (ACiM) and the adder trees in digital CiM (DCiM) contribute to significant energy and area overhead. In response to these challenges, binary neural networks (BNNs) have been proposed recently. Nevertheless, accuracy degradation poses a serious challenge to the application of BNNs in CiM due to errors in partial-sum accumulations. Furthermore, post-processing steps involving binary activation, such as ReLU, scaling, and bias addition, introduce redundant computing that cannot be effectively optimized by BNN-CiM.This work proposes a novel software-hardware co-optimization approach aimed at enabling an ADC-free analog CiM design while maintaining accuracy. Multi-Level binary computing fusion techniques comprising redundant load isolation based row fusion, in-array parallelism adaption based block fusion, and high-precision post-process elimination based layer fusion address the serious accuracy issues associated with conventional BNN algorithms. In contrast with past over 10% accuracy lost BNN-CiM on practical dataset CIFAR-10 and ImageNet, this work achieves more than 2.2x energy efficiency and 7.4x memory density than state-of-the-art with only 2% accuracy loss.

Organic thin-film transistor (OTFT) is a promising device technology for flexible large-area high-channel-count active-matrix neurostimulation system due to its flexibility and biocompatibility. However, circuits made by OTFT might be sensitive to device variation. As a result, it is difficult to achieve precise neurostimulation without any compensation structure in the pixel circuits. This work proposes a 6T2C threshold voltage compensation circuit for neurostimulation, which has low output current variation of 10.53%, reduced from the variation of 17.85% without compensation. We also improve the OTFT fabrication process with encapsulation to allow the circuits to operate under an electrolyte environment. Using the pixel circuits, we implement a 256-channel active-matrix neurostimulation system. The system can output stimulation with any pattern and allow each channel to output independently and simultaneously.

The rapid growth in the parameter count of large language models (LLMs) in recent years has placed higher demands on the density of compute-in-memory (CiM) solutions. Read-Only memory (ROM), due to its high-density advantages, has emerged as a promising CiM cell type, offering substantial task-level energy efficiency improvements over SRAM CiM. However, traditional 2D ROM CiM approaches are limited by 2D fabrication constraints, restricting scalability for LLM deployment. To address this limitation, this work explores a novel 3D back-end-of-line (BEOL)-compatible device, the channel-all-around (CAA)-IGZO TFT. Here, we propose a 3D ROM CiM with an ultra-dense cell structure and a high-throughput computing scheme. Additionally, we introduce a hybrid 3D CiM accelerator architecture that integrates both ROM and eDRAM for unprecedented density and flexibility. Evaluation results show that the proposed 3D ROM CiM, with 16 CAA-IGZO stacked layers, achieves an ultra-high memory density of 31.19 Mb/mm2/layer, a computation density of 167.6 TOPS/mm2, and high computing accuracy with a compute SNR (CSNR) of 22.6 dB, underscoring its potential for edge large-scale model acceleration. Based on this, when deployed with a LoRA-tuned GPT-2 model, the proposed hybrid 3D eDRAM-ROM architecture shows 1.7× improvement in area efficiency compared to the eDRAM-only counterpart.

3D Gaussian Splatting (3DGS) has emerged as a promising approach for high-fidelity scene reconstruction and has been widely adopted in Simultaneous Localization and Mapping (SLAM) systems.3DGS SLAM requires incremental training and rendering of Gaussians in real-time from continuous camera viewpoints.To match the streaming nature of SLAM, 3DGS-based mapping must sustain over 30 frames per second (FPS), which is a widely recognized threshold for maintaining accurate tracking and mapping quality.Existing GPU-based solutions and prior accelerators fall short of this target, primarily due to redundant training computation, unnecessary loss computing, and irregular memory access patterns.To address these challenges, we propose REACT3D, a real-time edge accelerator designed for incremental training in 3DGS SLAM systems.At the algorithmic level, we introduce spatial consistency and convergence aware sparsification, which eliminates redundant computation in both forward and backward rendering by predicting under-optimized regions based on spatial coherence and convergence dynamics.At the architectural level, we design a pixel blockwise fine-grained dataflow to eliminate explicit loss computing, establish a tightly coupled pipeline, and improve hardware utilization.Furthermore, we develop a Content Addressable Memory (CAM)-based Dual-index Gaussian Buffer to resolve discontinuous * Equal contribution.

Large Language Models (LLMs) achieve impressive reasoning capabilities at the cost of substantial inference overhead, posing substantial deployment challenges. Although distilled Small Language Models (SLMs) significantly enhance efficiency, their performance suffers as they fail to follow LLMs' reasoning paths. Luckily, we reveal that only a small fraction of tokens genuinely diverge reasoning paths between LLMs and SLMs. Most generated tokens are either identical or exhibit neutral differences, such as minor variations in abbreviations or expressions. Leveraging this insight, we introduce **Roads to Rome (R2R)**, a neural token router that selectively utilizes LLMs only for these critical, path-divergent tokens, while leaving the majority of token generation to the SLM. We also develop an automatic data generation pipeline that identifies divergent tokens and generates token-level routing labels to train the lightweight router. We apply R2R to combine R1-1.5B and R1-32B models from the DeepSeek family, and evaluate on challenging math, coding, and QA benchmarks. With an average activated parameter size of 5.6B, R2R surpasses the average accuracy of R1-7B by 1.6×, outperforming even the R1-14B model. Compared to R1-32B, it delivers a 2.8× wall-clock speedup with comparable performance, advancing the Pareto frontier of test-time scaling efficiency.

Learning Nash equilibrium (NE) in complex zero-sum games with multi-agent reinforcement learning (MARL) can be extremely computationally expensive. Curriculum learning is an effective way to accelerate learning, but an under-explored dimension for generating a curriculum is the difficulty-to-learn of the subgames –games induced by starting from a specific state. In this work, we present a novel subgame curriculum learning framework for zero-sum games. It adopts an adaptive initial state distribution by resetting agents to some previously visited states where they can quickly learn to improve performance. Building upon this framework, we derive a subgame selection metric that approximates the squared distance to NE values and further adopt a particle-based state sampler for subgame generation. Integrating these techniques leads to our new algorithm, Subgame Automatic Curriculum Learning (SACL), which is a realization of the subgame curriculum learning framework. SACL can be combined with any MARL algorithm such as MAPPO. Experiments in the particle-world environment and Google Research Football environment show SACL produces much stronger policies than baselines. In the challenging hide-and-seek quadrant environment, SACL produces all four emergent stages and uses only half the samples of MAPPO with self-play. The project website is at https://sites.google.com/view/sacl-neurips.

Deploying a lightweight quantized model in compute-in-memory (CIM) might result in significant accuracy degradation due to reduced signal-noise rate (SNR). To address this issue, this paper presents ZEBRA, a zero-bit robust-accumulation CIM approach, which utilizes bitwise zero patterns to compress computation with ultra-high resilience against noise due to circuit non-idealities, etc. First, ZEBRA provides a cross-level design that successfully exploits value-adaptive zero-bit patterns to improve the performance in robust 8-bit quantization dramatically. Second, ZEBRA presents a multi-level local computing unit circuit design to implement the bitwise sparsity pattern, which boosts the area/energy efficiency by 2x-4x compared with existing CIM works. Experiments demonstrate that ZEBRA can achieve10% accuracy loss. Such robustness leads to much more stable accuracy for high-parallelism inference on large models in practice.

Recently, sparse tensor algebra (SpTA) plays an increasingly important role in machine learning. However, due to the unstructured sparsity of SpTA, the general-purpose processors (e.g., GPU and CPU) are inefficient because of the underutilized hardware resources. Sparse kernel accelerators are optimized for specific tasks. However, their dedicated processing units and data paths cannot effectively support other SpTA tasks with different dataflow and various sparsity, resulting in performance degradation. This paper proposes FEASTA, a Flexible and Efficient Accelerator for Sparse Tensor Algebra. To process general SpTA tasks with various sparsity efficiently, we design FEASTA meticulously from three levels. At the dataflow abstraction level, we apply the Einstein Summation on the sparse fiber tree data structure to model the unified execution flow of general SpTA as joining and merging the fiber tree. At the instruction set architecture (ISA) level, a general SpTA ISA is proposed based on the execution flow. It includes different types of instructions for dense and sparse data, achieving flexibility and efficiency at the instruction level. At the architecture level, an instruction-driven architecture consisting of configurable and high-performance function units is designed, supporting the flexible and efficient ISA. Evaluations show that FEASTA has 5.40× geomean energy efficiency improvements compared to GPU among various workloads. FEASTA delivers 1.47× and 3.19× higher performance on sparse matrix multiplication kernels compared to state-of-the-art sparse matrix accelerator and CPU extension. Across diverse kernels, FEASTA achieves 1.69-12.70× energy efficiency over existing architectures.

Compute-in-memory (CiM) architecture is an emerging energy-efficient processing paradigm that has attracted widespread attention in AI and Internet of Things (IoT) applications. To protect statically stored sensitive data in CiM, designers have implemented various hardware obfuscation techniques in CiM architectures. However, we observe that existing CiM obfuscation defense strategies are based on straightforward static-key deployment strategies, which pose vulnerabilities from the perspective of key-pruning algorithms for de-obfuscation.

In recent years, there has been significant progress in the development of text-to-image generative models. Evaluating the quality of the generative models is one essential step in the development process. Unfortunately, the evaluation process could consume a significant amount of computational resources, making the required periodic evaluation of model performance (e.g., monitoring training progress) impractical. Therefore, we seek to improve the evaluation efficiency by selecting the representative subset of the text-image dataset. We systematically investigate the design choices, including the selection criteria (textural features or image-based metrics) and the selection granularity (prompt-level or set-level). We find that the insights from prior work on subset selection for training data do not generalize to this problem, and we propose FlashEval, an iterative search algorithm tailored to evaluation data selection. We demonstrate the effectiveness of FlashEval on ranking diffusion models with various configurations, including architectures, quantization levels, and sampler schedules on COCO and DiffusionDB datasets. Our searched 50-item subset could achieve compa-rable evaluation quality to the randomly sampled 500-item subset for COCO annotations on unseen models, achieving a 10x evaluation speedup. We release the condensed subset of these commonly used datasets to help facilitate diffusion algorithm design and evaluation, and open-source FlashE-val as a tool for condensing future datasets, accessible at https://github.com/thu-nics/FlashEval.

In-sensor computing has emerged as a promising approach to mitigating huge data transmission costs between sensors and processing units. Recently, the emerging application scenarios have raised more demands of sensory technology for large-area and flexible integration. However, with thin-film technologies that are capable of providing flexible and large-area integration support, the implementation of in-sensor computing can be strongly restricted due to the low device performance, large-area integration variation, and costly interface between sensors and CMOS processors. To address this challenge, we propose an in-sensor computing architecture to facilitate high-parallelism NN pre-processing and effective data compression. The boundaries of computing parallelism are expanded by adopting compact ROM-based compute-in-memory scheme next to sensing array. Differential-frame computing provides not only excellent robustness, but also high data sparsity. A bio-inspired data compression method with residual recovery caches and zero-skip circuits further enhances output sparsity without accumulated error. Based on the proposed cross-layer design optimization, an LTPS TFT-based ROM CiM chip has been fabricated and experimentally measured. The system-level evaluation demonstrates 3.85× speedup and 5.10× energy efficiency improvement compared with traditional architecture with separated sensors and processors, outperforming existing in-sensor computing works in large-area thin-film technology scenarios.

Sparse Matrix-Matrix Multiplication (SpMM) is one of the key operators in many fields, showing dynamic features in terms of sparsity, element distribution, and data dependency. Previous studies have proposed FPGA-based SpMM accelerators with fixed configurations of on-chip dataflow, leaving three major challenges unsolved: 1) Partitioning matrices with the fixed sub-matrix size to fit limited on-chip buffer on FPGA leads to performance loss because the optimal sub-matrix size to minimize memory access varies with dynamic sparsity. 2) The fixed row-wise allocation scheme of sparse elements in streaming architecture leads to unbalanced workloads because of dynamic element distribution across sparse matrix rows. 3) Read-after-write (RAW) hazard caused by floating-point adder makes the elements in one row cannot be processed consecutively. Architectures with fixed execution order rely on time-consuming pre-processing to deal with dynamic data dependency. Motivated by the observation that fixed configurations lead to performance loss, we propose DySpMM by introducing the dynamic design methodology to SpMM architectures. The configurable data distributor is introduced to enable dynamic sub-matrix size, achieving up to 3.79× less memory access amount. The element-wise allocator is designed for dynamic workload balance, improving utilization up to 3.74×. The interleaved reorder unit is proposed to reorder the elements and dynamically avoid RAW hazards at runtime, avoiding time-consuming pre-processing. We implement DySpMM on U280 FPGA, and the evaluation shows that it achieves 1.42× geomean throughput compared with the state-of-the-art accelerator Sextans and 1.78× energy efficiency compared with V100S GPU.

Hyperdimensional computing (HDC) is an emerging paradigm that employs hypervectors (HV s) to emulate cognitive tasks. In HDC, the most time-consuming and power-hungry process is encoding, the first step that maps raw data into HV s. There have been non-volatile memory (NVM) based computing-in-memory (CiM) HDC encoding designs, which exploit the intrinsic HDC characteristics of high parallelism, massive data, and robustness. These NVM-based CiMs have shown great potential in reducing encoding time and power consumption. Among them, the ferroelectric field-effect transistor (FeFET) based designs show ultra-high energy efficiency. However, existing FeFET-based HDC encoding designs face the challenges of energy -consuming current-mode addition, inefficient HV storage, limited endurance, and single encoding method support. These challenges limit the energy efficiency, lifetime, and versatility of the designs. This work proposes an energy-efficient charge-domain FeFET-based in-memory HDC encoder, i.e., CafeHD, with extended lifetime, good versatility, and comparable accuracy. Area-efficient charge-domain computing is proposed in HDC encoding for the first time, which enables CafeHD with ultra-low power and high scalability. An HV merging technique is explored to improve the performance. A low-cost partial MAJ interface is also proposed to reduce writes. Besides, CafeHD also supports two widely used encoding methods. Results show that CafeHD on average achieves 10.9×/12.7×/3.5× speedup and 103.3×/21.9×/6.3× energy effi-ciency with ~84 % write times reduction and similar accuracy compared with the state-of-the-art ReRAM/PCMlFeFET-based CiM design for HDC encoding, respectively.

Keys grant access to devices and are the core secrets in logic obfuscation. Typically, keys are stored in tamper-proof memory and are subsequently delivered to logic locking modules through scan chains. However, recent physical attacks have successfully extracted keys directly from registers, challenging the security of the prior scan obfuscation/blocking efforts. This paper mitigates the threat of direct value extraction by proposing TroScan, an architecture that leverages the internal frequency of register chains to activate trigger circuits. We propose three key generation methods for typical defense scenarios and gate-aware obfuscation optimization. To the authors' best knowledge, this work presents the first on-chip key delivery obfuscation architecture against Electro-Optical Frequency Mapping (EOFM) attacks. Evaluation shows ~100% key obfuscation effectiveness under two EOFM attack targets. For overheads, we demonstrate the worst-case fault coverage rate of 97.6%, average area/power overheads of 7.5%/11.8%, and an average key generation success rate of 98% across 80 process voltage temperature (PVT) conditions.

Dynamic neural network is an emerging research topic in deep learning. Dynamic networks selectively skip redundant computations conditioned on the input during inference (i.e., dynamic inference). And they have demonstrated superior trade-offs between accuracy and inference efficiency. However, memory I/O turns irregular and dominant because of the fine-grained computation skip in dynamic networks. Processing-In-Memory (PIM) can perform Matrix-Vector Multiplications inside the memory, eliminating the data movement of network parameters. So, it is promising to address the memory I/O challenge. However, deploying dynamic networks on PIM architectures faces severe performance degradation caused by (1) Pipeline stall when deciding on computation to be skipped. (2) Mismatch between fine-grained algorithm computation skip and coarse-grained hardware computing granularity. (3) Improper proxy of hardware performance during training. To tackle these problems, we propose DyPIM, the dynamic inference-enabled PIM accelerator with software-hardware co-optimizations. At the algorithm level, a PIM-friendly dynamic network with a standalone mask generation network and a throughput-optimal training technique is proposed. At the hardware level, a PIM architecture supporting dynamic networks is proposed, with a pipeline controller to process the dynamic dataflow. Peripheral circuits are also designed in processing units to enable non-contiguous activating of non-zero wordlines to better utilize the computation skip. Experiments show that DyPIM can achieve 1.52x to 2.74x speedup and 2.05x to 3.95x throughput improvement over the existing PIM architectures for Res Net networks.

Attention-free large language models (LLMs), such as Mamba and RWKV, have emerged as promising architectures to address the quadratic attention complexity of Transformer models. The inference bottleneck of these models lies in memory-bound matrix-vector multiplications (MVMs) and element-wise multiplications (EWMs). The emerging RRAM/SRAM-based Processing-In-Memory (PIM) architectures have shown great potential to overcome the memory wall problem. However, constrained by the supported data format and operator type, directly adopting PIM architectures for attention-free models faces three challenges: (1) RRAM-based analog PIM architectures perform integer (INT) MVMs using voltage, current, and conductance in the analog domain, limiting their application to the more accurate floating point (FP) data format; (2) SRAM-based digital PIM architectures require additional decoder circuits to support FP format, and the SRAM capacity cannot satisfy the storage requirement of LLMs; (3) When performing EWMs using PIM architectures, only one row/column or the diagonal memory cells are activated, resulting in severe device under-utilization.

Nonvolatile memory (NVM)-based computing-in-memory (CiM) has shown promising prospects in deep neural network (DNN) inference at the edge thanks to its nonvolatility and high density. Moreover, most NVMs support multi-level cell (MLC) storage, which can further boost energy efficiency and storage density. However, MLC NVM-based CiMs suffer from degraded accuracy due to device nonidealities, including large variations, nonlinear current distribution, and state drifts. Although prior works have explored various mitigation measures, such as hybrid SLC/MLC, write-and-verify, and local recovery units, the substantial costs from software support, energy, latency, and area still limit the performance. Therefore, the tradeoff between inference accuracy, storage density and compute density has become a vital challenge in NVM-based CiMs.

Tabular data are a widely used format in data science, and tree-based Machine Learning (ML) models are powerful tools and outperform Deep Neural Network (DNN) with higher accuracy for tasks on tabular data. However, computing multiple trees on massive tabular data via conventional von Neumann architectures suffers from irregular memory accesses. Prior work utilizes Analog Content Addressable Memories (ACAMs) to gain great speedup, but the analog matching method is vulnerable to device and voltage variations, and the limited density of 2D memory makes frequent data movement still inevitable for large-scale tabular data.

Recent years have witnessed the rapid progress and broad application of diffusion probabilistic models (DPMs). Sampling from DPMs can be viewed as solving an ordinary differential equation (ODE). Despite the promising performance, the generation of DPMs usually consumes much time due to the large number of function evaluations (NFE). Though recent works have accelerated the sampling to around 20 steps with high-order solvers, the sample quality with less than 10 NFE can still be improved. In this paper, we propose a unified sampling framework (USF) to study the optional strategies for solver. Under this framework, we further reveal that taking different solving strategies at different timesteps may help further decrease the truncation error, and a carefully designed \emph{solver schedule} has the potential to improve the sample quality by a large margin. Therefore, we propose a new sampling framework based on the exponential integral formulation that allows free choices of solver strategy at each step and design specific decisions for the framework. Moreover, we propose $S^3$, a predictor-based search method that automatically optimizes the solver schedule to get a better time-quality trade-off of sampling. We demonstrate that $S^3$ can find outstanding solver schedules which outperform the state-of-the-art sampling methods on CIFAR-10, CelebA, ImageNet-64, and LSUN-Bedroom datasets. Specifically, we achieve 2.69 FID with 9 NFE and 6.86 FID with 5 NFE on CIFAR-10 dataset, outperforming the SOTA method significantly. We further apply $S^3$ to Stable-Diffusion model and get an acceleration ratio of 2$\times$, showing the feasibility of sampling in very few steps without retraining of the neural network.

This work aims at decreasing the end-to-end generation latency of large language models (LLMs). One of the major causes of the high generation latency is the sequential decoding approach adopted by almost all state-of-the-art LLMs. In this work, motivated by the thinking and writing process of humans, we propose Skeleton-of-Thought (SoT), which first guides LLMs to generate the skeleton of the answer, and then conducts parallel API calls or batched decoding to complete the contents of each skeleton point in parallel. Not only does SoT provide considerable speed-ups across 12 LLMs, but it can also potentially improve the answer quality on several question categories. SoT is an initial attempt at data-centric optimization for inference efficiency, and showcases the potential of eliciting high-quality answers by explicitly planning the answer structure in language.

Post-training quantization (PTQ) has emerged as a promising technique to reduce the cost of large language models (LLMs). Specifically, PTQ can effectively mitigate memory consumption and reduce computational overhead in LLMs. To meet the requirements of both high efficiency and performance across diverse scenarios, a comprehensive evaluation of quantized LLMs is essential to guide the selection of quantization methods. This paper presents a thorough evaluation of these factors by evaluating the effect of PTQ on Weight, Activation, and KV Cache on 11 model families, including OPT, LLaMA2, Falcon, Bloomz, Mistral, ChatGLM, Vicuna, LongChat, StableLM, Gemma, and Mamba, with parameters ranging from 125M to 180B. The evaluation encompasses five types of tasks: basic NLP, emergent ability, trustworthiness, dialogue, and long-context tasks. Moreover, we also evaluate the state-of-the-art (SOTA) quantization methods to demonstrate their applicability. Based on the extensive experiments, we systematically summarize the effect of quantization, provide recommendations to apply quantization techniques, and point out future directions. The code can be found in https://github.com/thu-nics/qllm-eval.

Implantable neuroprostheses require stimulators with high channel counts and mechanical flexibility. Organic thin-film transistor (OTFT), an essential building block for flexible circuits and system, is a promising candidate. However, the development of photolithographic OTFTs for complete bioelectronic system integration remains a challenge, due to their limited yield and uniformity. This paper reports a 4-mask photolithographic OTFT circuit integration technology, which shows a high device yield of 100% (50/50) and small device variation in threshold voltage of 0.64 V and in mobility of 4.9%. Using a device-circuit-system co-design approach, we demonstrate an active-matrix neurostimulation array comprised of 1024 pixels of a 4T1C stimulation circuits, in which independent stimulation intensity levels can be programmed and current stimulus at all channels can output simultaneously. The electrical function of the complete neurostimulation system is verified, showing a small variation of 15.59% for the output stimulation currents among pixels. This OTFT-based neurostimulation system provides a potential solution for the next-generation neurostimulators with high channel counts and mechanical flexibility.

Teaching to improve student models (e.g., knowledge distillation) is an extensively studied methodology in LLMs. However, in human education, teaching enhances not only the students but also the teachers by fostering more rigorous and clearer reasoning, as well as deeper knowledge building. We ask: Can LLMs also learn by teaching (LbT) for better reasoning? If the answer is yes, we can potentially unlock the possibility of continuously advancing the models without solely relying on human-produced data or stronger models. In this paper, we provide a preliminary exploration of this question. We show that LbT ideas can be incorporated into existing LLM training/prompting pipelines and bring improvements. Specifically, we design three methods, each mimicking one of the three levels of LbT: observing students' feedback, learning from the feedback, and learning iteratively, with the goal of improving answer accuracy without training or improving models' inherent capability with fine-tuning. We reveal some findings: (1) Teaching materials that make it easier for students to learn (via in-context learning) have clearer and more accurate logic; (2) Weak-to-strong generalization: LbT might help improve strong models by teaching weak models; (3) Diversity in students might help: teaching multiple students could be better than teaching a single student or the teacher alone. We hope that our exploration can inspire future research on LbT and, more broadly, the adoption of advanced education techniques to improve LLMs. The code and website are at https://github.com/imagination-research/lbt and https://sites.google.com/view/llm-learning-by-teaching.

In class-incremental learning (CIL), when deep neural networks learn new classes, their recognition performance in old classes will drop significantly. This phenomenon is widely known as catastrophic forgetting. To alleviate catastrophic forgetting, existing methods store a small portion of old class data with a memory buffer and replay it while learning new classes. These methods suffer from a severe imbalance problem between old and new classes. In this paper, we discover that the imbalance problem in CIL makes it difficult to preserve the feature relation of old classes and hard to learn the feature relation between old and new classes. To mitigate the above two issues, we design a triplet contrastive preserving (TCP) loss to preserve old knowledge, and propose an asymmetric augmented contrastive learning (A2CL) method to learn new classes. Comprehensive experiments demonstrate the effectiveness of our method, which increases the average accuracies by 1.26% and 0.95% on CIFAR-100 and ImageNet. Especially under smaller memory buffer settings where the imbalance problem is more severe, our method can surpass the baselines by a large margin (up to 3.2%). We also show that TCP can be easily plugged into other methods and further improve their performance.

Transformer networks have been increasingly successful in various fields. The input sequence lengths have become much larger as the algorithm and task complexity develops, which is challenging due to high computational and storage cost. Softmax and LayerNorm are bottleneck nonlinear operators in ultra-long sequence Transformer networks. To improve the efficiency of Softmax, assumption-based and quantization-based Softmax approaches are introduced. However, the sparsity potential to accelerate Softmax itself is not fully discovered. To improve the efficiency of LayerNorm, some works reduce the input size, and some works explore the pipeline. However, the sparsity potential is also not yet explored. To address these challenges, this article presents the ULSeq-TA software–hardware co-design framework. The software includes 1) the grouped sparse Softmax method to leverage the data magnifying characteristic to explore the middle and post-Softmax sparse processing and 2) the dual-path sparse LayerNorm method which explores the dimensional significance for sparse calculation. The hardware includes 1) an attention fusion architecture which reduces the on-chip memory with fused operators; 2) the grouped sparse Softmax core; and 3) the dual-path sparse LayerNorm core. Experiments show that the software achieves$4.45\times $and$7.59\times $computation reduction with little output difference for Softmax and LayerNorm, respectively. The hardware architecture supports at most 32768 sequence length with only 186-kB on-chip memory and achieves$1.75\times -1.98\times $and$3.22\times -4.32\times $speedups for sparse Softmax core and sparse LayerNorm core with little accuracy loss, respectively.

As domain-specific training data is recognized as valuable intellectual property, acquiring well-trained weights in Convolutional Neural Networks (CNN) has emerged as a new threat to the neural network design community. To design a CNN accelerator that is resilient to side-channel threats, it is crucial to have an accurate and efficient security-driven framework at the early design stage. However, there is no standard way to perform root-cause analysis on the power side channel that exists in FPGA-based CNN accelerators. Therefore, we build RE-Specter, a framework that facilitates security-driven design space exploration (DSE) across various building components, combination patterns, and parallelism configurations in CNNs. The goal is to fully understand the power side-channel effects resulting from architectural modifications or optimization decisions. We further compare the benchmarks considering precision, resource utilization, and power side-channel leakage. Finally, we experimentally explore the design space of various architectural features. The experimental results show that low-bit precision delivers more secure architectures (68.9× among DSPs, 2439× among LUTs) in Measurement-To-Disclosure (MTD), but mixed-precision strategies are necessary to maintain the model accuracy. For loop optimization, in 16-parallel scenario, accumulator-based architecture outperforms the architecture featuring an adder tree with the improvements of 8.28× in MTD and 1.38× in PST.

In the era of intelligent IoT, huge amount of sensor data is collected and then transmitted to processor elements in edge devices or cloud servers. The latency and energy consumption in this process have been a bottleneck and are becoming more severe. To mitigate this problem, the idea of combining sensors, memory and processors for collectively handling the data, has been proposed and explored actively in recent efforts. In this work, thin-film transistor (TFT), which has been widely adopted in display devices and flexible sensors, is exploited. It is shown that, while TFT is promising for large-area sensing, it also shows a great potential for computing and storing data for large-area and low-cost edge sensors. More specifically, we have fabricated and measured two large-area TFT-based near-sensor computing-in-memory (CiM) chips adopting embedded DRAM (eDRAM) and ROM structure respectively. We further give a detailed analysis of the integration of CiM arrays and sensor arrays to realize a sensing and data pre-process system. Measurement and simulation results show that such TFT-based solutions can accomplish real-time sensing and multiply-accumulate (MAC) processing in the analog field, which simplifies the system design with lowered energy and latency in our neural network evaluations.

Few-shot 3D point cloud segmentation segments novel categories in point cloud scenes with only limited annotations. However, most current methods do not consider query content when exploring support prototypes, and thus suffer from intra-class variations between objects and incomplete representation of category information from annotated support samples. In this paper, we propose a novel Query-Guided support Prototype exploration Network (QGPNet) to tackle this challenge. Firstly, we present a point feature alignment module, which leverages geometry relationship between prototypes and query points, to tackle data misalignment caused by intra-class variations, and thus prevents incorrect label propagation from prototypes to query points. Secondly, we design a prototype feature mining strategy, which progressively harvests diverse support prototypes in the interaction with query features, to fully utilize the category information provided by annotated samples. Additionally, we introduce a semantic-aware data augmentation strategy for query samples in the training process, potentially improving the generalization ability of support prototypes on query samples. Extensive experiments on two indoor 3D datasets S3DIS and ScanNet demonstrate that QGPNet outperforms previous state-of-the-art methods by a large margin.

Object detection via deep neural networks has undergone considerable advancements in recent years. Yet, the detection of smaller objects, specifically those with a few pixels (i.e.,2pixels), is still challenging compared with large objects (i.e., > 962pixels). Existing methods commonly apply high-resolution features or complex super-resolution strategies based on the two-stage Faster Region Convolutional Neural Network (RCNN). They sequentially apply localization and classification stages after a shared feature map extracted by one single backbone network. However, these methods cause low detection accuracy of small objects, high computational overhead, and waste of hardware resources. In this paper, we develop a high-accuracy and real-time small object detection system with negligible computational overhead and low hardware idleness. At the software level, we propose a two-stage Coarse-to-Fine Decoupling RCNN (CFD RCNN) with three techniques: (1) The shared backbone decoupling for localization and classification to achieve high accuracy for both tasks; (2) The training method using backbone feature upsampling for localization with low computational overhead; (3) The object cropping strategy from the original high-resolution image for high-accuracy classification. At the hardware level, we propose a virtualized FPGA accelerator with the Dynamic Resource Allocation (DRA) strategy. The DRA strategy reallocates the hardware resources, considering the workload and resource preference of each stage in CFD RCNN to reduce hardware idleness. Extensive experiments on the TT100K and GTSDB datasets using Xilinx ZCU102 FPGA show that the proposed small object detection system can achieve 2.9% improvement in mean average precision (mAP) compared with state-of-the-art (SOTA) algorithms and raised the throughput from 18.9 FPS to > 26.0 FPS (~1.37×) compared with existing accelerators.

Ferroelectric Field Effect Transistors (FeFETs) have spurred increasing interest in both memories and computing applications, thanks to their CMOS compatibility, low-power operation, and high scalability. However, new security threats to the FeFET-based memories also arise. A major threat is the power analysis side-channel attack (P-SCA), which exploits the power traces of the memory access to obtain data information. There have been several effective efforts on resistive nonvolatile memories (NVMs), but they fail to meet the requirements for secure FeFET-based memories due to the different capacitive FeFETs load. Directly applying these existing countermeasures to the P-SCA protection for FeFETs induces huge challenges, especially for the balance between power side-channel resistance and corresponding overheads. To address this issue, we leverage the unique features of FeFETs and propose ProtFe , namely the protection methods for FeFET-based memories, including the pipelined multi-step write strategy ( PiMWrite ) and the split array design ( SpA ). PiMWrite is proposed for general FeFET-based memories, and inserts specially designed intermediate states to mitigate information leakage with pipelined steps to reduce overheads. SpA is proposed for custom FeFET-based memories, and simultaneously writes two split portions of the array with shared minimized peripherals to go beyond the balance between security and overheads. Simulation results show that PiMWrite expands the search space of a single power trace to 21× and involves nearly zero hardware penalties. SpA presents 33× search space improvement with negligible latency, 0.6% area, and only 7.1% energy overhead. ProtFe achieves improved balance between security and overheads, compared with the state-of-the-art works.

Logic camouflage is a widely adopted technique that mitigates the threat of intellectual property (IP) piracy and overproduction in the integrated circuit (IC) supply chain. Camouflaged logic achieves functional obfuscation through physical-level ambiguity and post-manufacturing programmability. However, discussions on programmability are confined to the level of logic cells/gates, limiting the broader-scale application of logic camouflage. In this work, we propose a novel module-level configuration methodology for programmable camouflaged logic that can be implemented without additional hardware ports and with negligible resources. We prove theoretically that the configuration of the programmable camouflaged logic cells can be achieved through the inputs and netlist of the original module. Further, we propose a novel lightweight ferroelectric FET (FeFET)-based reconfigurable logic gate (rGate) family and apply it to the proposed methodology. With the flexible replacement and the proposed configuration-aware conversion algorithm, this work is characterized by the input-only programming scheme as well as the combination of high output error rate and point-function-like defense. Evaluations show an average of >95% of the alternative rGate location for camouflage, which is sufficient for the security-aware design. We illustrate the exponential complexity in function state traversal and the enhanced defense capability of locked blackbox against Boolean Satisfiability (SAT) attacks compared with key-based methods. We also preserve an evident output Hamming distance and introduce negligible hardware overheads in both gate-level and module-level evaluations under typical benchmarks.

Adversarial attacks have threatened modern deep learning systems by crafting adversarial examples with small perturbations to fool the convolutional neural networks (CNNs). To alleviate that, ensemble training methods are proposed to facilitate better adversarial robustness by diversifying the vulnerabilities among the sub-models, simultaneously maintaining comparable natural accuracy as standard training. Previous practices also demonstrate that enlarging the ensemble can improve the robustness. However, conventional ensemble methods are with poor scalability, owing to the rapidly increasing complexity when containing more sub-models in the ensemble. Moreover, it is usually infeasible to train or deploy an ensemble with substantial sub-models, owing to the tight hardware resource budget and latency requirement. In this work, we propose Ensemble-in-One (EIO), a simple but effective method to efficiently enlarge the ensemble with a random gated network (RGN). EIO augments a candidate model by replacing the parametrized layers with multi-path random gated blocks (RGBs) to construct an RGN. The scalability is significantly boosted because the number of paths exponentially increases with the RGN depth. Then by learning from the vulnerabilities of numerous other paths within the RGN, every path obtains better adversarial robustness. Our experiments demonstrate that EIO consistently outperforms previous ensemble training methods with smaller computational overheads, simultaneously achieving better accuracy-robustness trade-offs than adversarial training methods under black-box transfer attacks. Code is available at https://github.com/cai-y13/Ensemble-in-One.git

Deep learning (DL) based methods have significantly pushed forward the state-of-the-art for image restoration (IR) task. Nevertheless, DL-based IR models are highly computation- and memory-intensive. The surging demands for processing higher-resolution images and multi-task paralleling in practical mobile usage further add to their computation and memory burdens. In this paper, we reveal the overlooked memory redundancy of the IR models and propose a Memory-Oriented Structural Pruning (MOSP) method. To properly compress the long-range skip connections (a major source of the memory burden), we introduce a compactor module onto each skip connection to decouple the pruning of the skip connections and the main branch. MOSP progressively prunes the original model layers and the compactors to cut down the peak memory while maintaining high IR quality. Experiments on real image denoising, image super-resolution and low-light image enhancement show that MOSP can yield models with higher memory efficiency while better preserving performance compared with baseline pruning methods.

Graph Attention Network (GAT) has demonstrated better performance in many graph tasks than previous Graph Neural Networks (GNN). However, it involves graph attention operations with extra computing complexity. While a large amount of existing literature has researched GNN acceleration, few have focused on the attention mechanism in GAT. The graph attention mechanism makes the computation flow different. Therefore, previous GNN accelerators can not support GAT well. Besides, GAT distinguishes the importance of neighbors and makes it possible to reduce the workload through runtime tailoring. We present NTGAT, a software-hardware co-design approach to accelerate GAT with runtime node tailoring. Our work comprises both a runtime node tailoring algorithm and an accelerator design. We propose a pipeline sorting method and a hardware unit to support node tailoring during inference. The experiments show that our algorithm can reduce up to 86% of aggregation workload while incurring slight accuracy loss (<0.4%). And the FPGA based accelerator can achieve up to 3.8× speedup and 4.98× energy efficiency comparing to the GPU baseline.

Data quantization is an effective method to accelerate neural network training and reduce power consumption. However, it is challenging to perform low-bit quantized training: the conventional equal-precision quantization will lead to either high accuracy loss or limited bit-width reduction, while existing mixed-precision methods offer high compression potential but failed to perform accurate and efficient bit-width assignment. In this work, we propose DYNASTY, a block-wise dynamic-precision neural network training framework. DYNASTY provides accurate data sensitivity information through fast online analytics, and maintains stable training convergence with an adaptive bit-width map generator. Network training experiments on CIFAR-100 and ImageNet dataset are carried out, and compared to 8-bit quantization baseline, DYNASTY brings up to 5.1× speedup and 4.7× energy consumption reduction with no accuracy drop and negligible hardware overhead.

Multi-level cell (MLC) NVM-based CiM has become a promising candidate in computing-in-memory (CiM) designs because of its non-volatility, high cell density, and improving compatibility with the CMOS process. However, most MLC CiM faces the challenges of non-ideal device limitations, including the low on/off ratio, large device-to-device variations, and read disturbances, which limit the computing accuracy, reliability, and throughput performance. This work proposes Victor, a variation-resilient approach using cell-clustered charge-domain computing for high-density and high-throughput MLC CiM. A cell-clustered-computing with local recovery unit (LRU) design methodology is proposed to improve matrix-vector-multiplication (MVM) reliability and throughput. To showcase the capability of Victor, 2b-4b MLC Resistive RAM (RRAM) is taken as an example for design and evaluation. Results show that Victor reaches 3.56x energy efficiency, 4x variation tolerance compared with the prior ratio-based MLC CiM. In addition, the throughput is improved by 3.1x with less than 1% DNN accuracy loss. Moreover, a dynamic boundary adaption approach is proposed to restore the accuracy loss of state drifting, which in return reduces the energy and latency overhead by 100x and 1.25x, respectively, compared with the conventional write-and-verify approach.

Single Photon Avalanche Diode (SPAD)-based direct time-of-flight (dToF) depth sensors are widely used in Internet of Things (IoT) devices due to their high accuracy. Existing SPAD-based dToF sensors measure depth by continually accumulating the depth-measured value in a histogram. However, histogram-based methods typically have low convergence speed (~10 frames per second (FPS)) and large memory overhead (MB-level), hindering their use in real-time embedded IoT devices. To overcome these two challenges, we propose SSC, a histogram-free Spatial and Statistical Correlation based depth measurement method. On the one hand, SSC applies the spatial correlation of the adjacent pixels to accelerate the convergence speed. On the other hand, SSC explores the statistical correlation of depth measurements to reduce the memory overhead. In order to implement SSC with small hardware area and low power, we design mert-dToF, a memory-efficient and real-time dToF sensor for efficient execution. mert-dToF abstracts mainly operations in SSC into four basic operators and designs corresponding hardware with a fine-grained pipeline to maximize resource reuse and computational parallelism. Extensive experiments show that compared with state-of-the-art (SOTA) histogram-based dToF sensors, mert-dToF achieves ~8% accuracy improvement and 7.80× speedup (from 6.24 FPS to 48.70 FPS). The memory overhead is reduced by up to 60.91% (from 48 KB to 18.75 KB).

Genome sequence analysis is a powerful tool in medical and scientific research. Considering the inevitable sequencing errors and genetic variations, approximate string matching (ASM) has been adopted in practice for genome sequencing. However, with exponentially increasing bio-data, ASM hardware acceleration is facing severe challenges in improving the throughput and energy efficiency with the accuracy constraint.This paper presents ASMCap, an ASM acceleration approach for genome sequence analysis with hardware-algorithm co-optimization. At the circuit level, ASMCap adopts charge-domain computing based on the capacitive multi-level content addressable memories (ML-CAMs), and outperforms the state-of-the-art ML-CAM-based ASM accelerators EDAM with higher accuracy and energy efficiency. ASMCap also has misjudgment correction capability with two proposed hardware-friendly strategies, namely the Hamming-Distance Aid Correction (HDAC) for the substitution-dominant edits and the Threshold-Aware Sequence Rotation (TASR) for the consecutive indels. Evaluation results show that ASMCap can achieve an average of 1.2x (from 74.7% to 87.6%) and up to 1.8x (from 46.3% to 81.2%) higher F1score (the key metric of accuracy), 1.4x speedup, and 10.8x energy efficiency improvement compared with EDAM. Compared with the other ASM accelerators, including ResMA based on the comparison matrix, and SaVI based on the seeding strategy, ASMCap achieves an average improvement of 174x and 61x speedup, and 8.7e3x and 943x higher energy efficiency, respectively.

Graph-based approximate nearest neighbor search (ANNS) algorithms achieve the best accuracy for fast high-recall searches on billion-scale datasets. Because of the irregular and large-volume data access, existing CPU-based systems suffer from heavy data movements when dealing with graph-based ANNS algorithms. Near-memory-computing (NMC) architectures have demonstrated great potential in boosting the performance of big-data processing. However, existing NMC architectures face two serious problems when processing graph-based ANNS algorithms: (1) the memory capacity of main memory level NMC (e.g., 64GB) cannot meet the storage requirement of ANNS on billion-scale datasets (e.g., 800GB), resulting in heavy data transfers between main memory and storage; (2) the contradiction between the irregular and fine-grained graph access and the page-level read granularity hinder the throughput of storage level NMC.This paper proposes Pyramid, the processing-in-hierarchical-memory architecture for graph-based ANNS on billion-scale datasets. Pyramid combines the internal bandwidth benefits of main memory level NMC with the capacity benefits of storage level NMC. A hierarchical graph-cluster-based ANNS is also proposed for Pyramid. It transforms the irregular data access on large-scale graphs into the irregular access on small-scale graphs at the main memory level and regular sequential in-cluster access at the storage level. Experimental results show that with the same recall of 0.9, Pyramid improves the throughput by 21.1~72.8× and 26.0~50.7× compared with existing CPU/GPU-based ANNS systems on million-scale and billion-scale datasets, respectively.

Processing-in-memory (PIM) architectures have shown great abilities for neural network (NN) acceleration on edge devices that demand low latency under severe area constraints. Heterogeneous PIM architectures with different PIM implementation approaches such as RRAM-based PIM and SRAM-based PIM can further improve the performance. However, the automatic generation of heterogeneous PIM architectures faces the following two unresolved problems. First, existing work has not considered the design for heterogeneous PIM-based NN accelerators with multiple memory technologies. Second, for PIM with insufficient memory on edge devices, it is challenging to find the optimal runtime weight scheduling strategy in an O(L!) optimization space for the NN with L layers.In this paper, we propose PIM-HLS, an automatic hardware generation tool for heterogeneous PIM-based NN accelerators. Aiming at the problems above, we first point out that heterogeneous PIM can improve the performance under severe area constraints. Then we optimize the architectures for each NN layer by taking the advantage of different memory technologies. We also define the optimization problem of runtime weight scheduling and mapping for the first time, and propose a dynamic-programming-based weight scheduling algorithm to reduce the optimization space to O(L2). We implement PIM-HLS to automatically generate the hardware code and the instructions. Results show that we achieve an averagely 5.9× speedup with 72.8% less area compared with state-of-the-art PIM designs.

Triangle counting (TC) is one of the most fundamental graph analysis tools with a wide range of applications. Modern triangle counting algorithms traverse the graph and perform set intersections of neighbor sets to find triangles. However, existing triangle counting approaches suffer from the heavy off-chip memory access and set intersection overhead. Thus, we propose CLAP, the first content addressable memory (CAM) based triangle counting architecture with the software and hardware co-optimizations. To reduce off-chip memory access and the number of set intersections, we propose the first force-based node index reorder method. It simultaneously optimizes both data locality and the computation amount. Compared with random node indices, the reorder method reduces the off-chip memory access and the set intersections by 61% and 64%, respectively, while providing$\mathbf{2.19}\times$end-to-end speedup. To improve the set intersection parallelism, we propose the first CAM-based triangle counting architecture under chip area constraints. We enable the high parallel set intersection by translating it into content search on CAM with full parallelism. Thus, the time complexity of the set intersection reduces from$O(m+n)$or$O(n\log m)$to$O(n)$. Extensive experiments on real-world graphs show that CLAP achieves$\mathbf{39}\times, \mathbf{27}\times$, and$\mathbf{78}\times$speedup over state-of-the-art CPU, GPU, and processing-in-memory baselines, respectively. The software code is available at: https://github.com/thu-nics/CLAP-triangle-counting

Deep Neural Networks (DNNs) have made significant breakthroughs in various fields. However, their enormous computations and parameters seriously hinder their applications. Emerging Processing-In-Memory (PIM) architectures provide extremely high energy efficiency to accelerate DNN computing. Moreover, Network-on-Chip (NoC) based PIM architectures significantly improve the scalability of PIM architectures. However, the contradiction between high communication and limited NoC bandwidth introduces severe communication conflicts. Existing work neglects the impact of communication conflicts. On the one hand, neglecting communication conflicts leads to the lack of precise performance estimations in the mapping process, making it hard to find optimal results. On the other hand, communication conflicts cause low NoC bandwidth utilization in the schedule process. And there is over 70% latency gap in existing work caused by communication conflicts. This paper proposes communication conflict optimized mapping and schedule strategies for NoC-based PIM architectures. The proposed mapping strategy constructs communication conflict graphs to model communication conflicts. Based on this constructed graph, we adopt a Graph Neural Network (GNN) as a precise performance estimator. Our schedule strategy predefines the communication priority and NoC communication behavior tables for target DNN workloads. In this way, it can improve the NoC bandwidth utilization effectively. Compared with existing work, for typical classification DNNs on the CIFAR and ImageNet datasets, the proposed strategies reduce 78% latency and improve the throughput by 3.33× on average with negligible deployment and hardware overhead. Experimental results also show that our strategies decrease the average gap to ideal cases without communication conflicts from 80.7% and 70% to 12.3% and 1.26% for latency and throughput, respectively.

Phase-change memory (PCM) and resistive memory (RRAM) are promising alternatives to traditional memory technologies. However, both PCM and RRAM suffer from limited write endurance. Wear-leveling (WL) techniques are essential to extend the lifetime of these memories before experiencing endurance faults. Beyond the additional usage afforded by WL, row-sparing and focused error correction can extend the lifetime further after wear faults appear. Unfortunately, the need for extended WL techniques continues to become more pressing as scaling exacerbates process variation. Similarly, scaling causes challenges such as more severe noise and crosstalk to traditional DRAM.In this paper, we propose novel fault-aware WL schemes to allocate write frequencies according to the strength of the rows and handle the imbalance of writes in columns. We use runtime detection schemes to identify weak rows and protect them prior to wear out. In particular, row-level WL, aka RETROFIT, leverages the spare rows provided for redundancy to be used strategically to guard against early cell wear out. RETROFIT is compatible with error correction schemes that guarantee to mitigate hard faults and error-correcting codes (ECC). Rather than discard retired rows, when any spare row completely replaces a retired row, we retarget the retired row to assist with column sparing. It becomes a group of Page Protecting Pointers (PPPs), which utilizes otherwise discarded error correction potential to further enhance the leveling ability of RETROFIT. To relieve column-level imbalance, we apply idle error correction bits before they are used to reduce average bit flips. The evaluation demonstrates that RETROFIT and enhanced RETROFIT with the PPPs improve lifetime by as much as 0.64× and 5.4× in the average case, respectively, over state-of-the-art row-level method while also reducing area overhead. In the worst-case scenario, these improvements further increase to 2.6× and 16.0×. Combined with the proposed column-level WL, enhanced RETROFIT realizes an overall 1.5× memory lifetime improvement over the perfectly uniform wear-leveling with equal storage overhead.

The tensor cores in modern GPUs lead to significant performance improvement in matrix multiplication, which is the primary operation in deep learning. However, existing hardware architectures face unstructured sparsity in deep learning, resulting in algorithm inflexibility and hardware inefficiency. The previous tensor core architecture requires matrices to be pruned into 2:4 sparse patterns, leading to algorithm inflexibility. Customized accelerators introduce extra architectures (e.g., interconnection networks for dynamic data routing or buffers for avoiding data conflicts) for unstructured sparse matrices, leading to hardware inefficiency. To tackle the contradiction between algorithm inflexibility and hardware inefficiency, we propose Two-level Sparsity Tensor Core (TSTC) in this paper. TSTC points out that the unstructured sparsity which enables algorithm flexibility can be maintained at the coarse-grained level, while hardware efficiency which requires structured sparsity can be ensured at the fine-grained level. For algorithm flexibility, we propose Flexible Sparse Block (FSB) pattern. FSB enables unstructured sparse matrices can be divided into fine-grained blocks with different structured sparsity. As a result, using FSB leads to up to 7.29x speed up compared with other formats. For hardware efficiency, we propose Dynamic Extendible Reduction Network (DERN). DERN enables different structured sparse reductions by only extending the data width on the standard reduction network without introducing interconnections or buffers. DERN enables TSTC to achieve 7.19x more energy savings under a similar speed. We also propose the whole flow, which can automatically deploy different sparse deep learning algorithms to TSTC. According to extensive experiments, TSTC achieves 1.24 x ~7.69 x speedup and 3.68 x~4.17 x energy savings than the tensor core and the SOTA customized accelerator.

The concept of multi-level cell (MLC) enabled by emerging memory device technologies has introduced new opportunities for memory density improvement, including in the cache scenarios with some high-endurance technologies. However, the access latency of different bits within an MLC memory cell is inherently nonuniform, which raises challenges in utilizing the MLC technology for low-latency cache. To exploit the access performance of the MLC cache, the key is identifying the hot data blocks and mapping them to fast MLC bits. Prior works perform the hot/cold data management based on block-wise access patterns with considerable hardware overheads. Inspired by the memory hierarchy, this work proposes a new concept of in-cell hierarchical victim cache as embedded memory and systematically presents the cache architecture, operating mechanism, design space exploration, optimizations, and evaluations. By utilizing the slow bits as the victim buffer, lower hit latency with low implementation overheads is achieved. Based on the in-cell victim cache, two optimization techniques, namely selective victim retrieval, and victim-bypassing write (VBW) are proposed, to further improve performance and prolong cache endurance, respectively. Evaluation results show that the MLC victim cache significantly improves the average system performance by 20.2% over conventional MLC cache and achieves 98% performance of the upper bound implemented with 2x memory cells SLC. The proposed VBW also reduces energy consumption by 21% and improves lifetime by over 80%, showing a new promising dimension for future MLC cache design.

Voxel-based methods have achieved state-of-the-art performance for 3D object detection in autonomous driving. However, their significant computational and memory costs pose a challenge for their application to resource-constrained vehicles. One reason for this high resource consumption is the presence of a large number of redundant background points in Lidar point clouds, resulting in spatial redundancy in both 3D voxel and BEV map representations. To address this issue, we propose an adaptive inference framework called Ada3D, which focuses on reducing the spatial redundancy to compress the model’s computational and memory cost. Ada3D adaptively filters the redundant input, guided by a lightweight importance predictor and the unique properties of the Lidar point cloud. Additionally, we maintain the BEV features’ intrinsic sparsity by introducing the Sparsity Preserving Batch Normalization. With Ada3D, we achieve 40% reduction for 3D voxels and decrease the density of 2D BEV feature maps from 100% to 20% without sacrificing accuracy. Ada3D reduces the model computational and memory cost by 5×, and achieves 1.52× / 1.45× end-to-end GPU latency and 1.5× / 4.5× GPU peak memory optimization for the 3D and 2D backbone respectively.

Diffusion probabilistic models (DPMs) are a new class of generative models that have achieved state-of-the-art generation quality in various domains. Despite the promise, one major drawback of DPMs is the slow generation speed due to the large number of neural network evaluations required in the generation process. In this paper, we reveal an overlooked dimension---model schedule---for optimizing the trade-off between generation quality and speed. More specifically, we observe that small models, though having worse generation quality when used alone, could outperform large models in certain generation steps. Therefore, unlike the traditional way of using a single model, using different models in different generation steps in a carefully designed model schedule could potentially improve generation quality and speed simultaneously. We design OMS-DPM, a predictor-based search algorithm, to determine the optimal model schedule given an arbitrary generation time budget and a set of pre-trained models. We demonstrate that OMS-DPM can find model schedules that improve generation quality and speed than prior state-of-the-art methods across CIFAR-10, CelebA, ImageNet, and LSUN datasets. When applied to the public checkpoints of the Stable Diffusion model, we are able to accelerate the sampling by 2x while maintaining the generation quality.

Embedding retrieval is a crucial task for recommendation systems. Graph-based approximate nearest neighbor search (GANNS) is the most commonly used method for retrieval, and achieves the best performance on billion-scale datasets. Unfortunately, the existing CPU- and GPU-based GANNS systems are difficult to optimize the throughput under the latency constraints on billion-scale datasets, due to the underutilized local memory bandwidth (5-45%) and the expensive remote data access overhead (∼ 85% of the total latency). In this paper, we first introduce a practically ideal GANNS architecture for billion-scale datasets, which facilitates a detailed analysis of the challenges and characteristics of distributed GANNS systems. Then, at the architecture level, we propose DF-GAS, a Distributed FPGA-as-a-Service (FPaaS) architecture for accelerating billion-scale Graph-based Approximate nearest neighbor Search. DF-GAS uses a feature-packing memory access engine and a data prefetching and delayed processing scheme to increase local memory bandwidth by 36-42% and reduce remote data access overhead by 76.2%, respectively. At the system level, we exploit the “full-graph + sub-graph” hybrid parallel search scheme on distributed FPaaS system. It achieves million-level query-per-second with sub-millisecond latency on billion-scale GANNS for the first time. Extensive evaluations on million-scale and billion-scale datasets show that DF-GAS achieves an average of 55.4 ×, 32.2 ×, 5.4 ×, and 4.4 × better latency-bounded throughput than CPUs, GPUs, and two state-of-the-art ANNS architectures, i.e., ANNA [23] and Vstore [27], respectively.

Neural architecture search (NAS) can automatically discover well-performing architectures in a large search space and has been shown to bring improvements to various applications. However, the computational burden of NAS is huge, since exploring a large search space can need evaluating more than thousands of architecture samples. To improve the sample efficiency of search space exploration, predictor-based NAS methods learn a performance predictor of architectures, and utilize the predictor to sample worth-evaluating architectures. The encoding scheme of NN architectures is crucial to the predictor's generalization ability, and thus crucial to the efficacy of the NAS process. To this end, we have designed a generic Graph-based neural ArchiTecture Encoding Scheme (GATES), a more reasonable modeling of NN architectures that mimics their data processing. Nevertheless, GATES is unaware of the concrete computing semantic of NN operations or architectures. Thus, the learning of operation embeddings and weights in GATES can only exploit the information in architectures-performance pairs. We propose GATES++, which incorporates multifaceted information about NN's operation-level and architecture-level computing semantics into its construction and training, respectively. Experiments on benchmark search spaces show that both the operation-level and architecture-level information can bring improvements alone, and GATES++ can discover better architectures after evaluating the same number of architectures.

Deep Neural Network (DNN) INFerence-as-a-Service (INFaaS) is the dominating workload in current data centers, for which FPGAs become promising hardware platforms because of their high flexibility and energy efficiency. The dynamic and multi-tenancy nature of INFaaS requires careful design in three aspects: multi-tenant architecture, multi-DNN scheduling, and multi-core mapping. These three factors are critical to the system latency and energy efficiency but are also challenging to optimize since they are tightly coupled and correlated. This paper proposesH3M, an automatic Design Space Exploration (DSE) framework to jointly optimize thearchitecture,scheduling, andmappingfor serving INFaaS on cloud FPGAs. H3M explores: (1) the architecture design space withHeterogeneousspatialMulti-tenantsub-accelerators, (2) layer-wise scheduling forHeterogeneousMulti-DNNworkloads, and (3) single-layer mapping to theHomogeneousMulti-corearchitecture. H3M beats state-of-the-art multi-tenant DNN accelerators, Planaria and Herald, by up to 7.5× and 3.6× in Energy-Delay-Product (EDP) reduction on the ASIC platform. On the Xilinx U200 and U280 FPGA platforms, H3M offers 2.1-5.7× and 1.8-9.0× EDP reduction over Herald.

Fault simulation is a critical component of the automatic test pattern generation (ATPG) tool, which is widely used in chip development. The CPU–GPU heterogeneous system can accelerate fault simulation. However, existing work faces the following challenges: 1) Path Divergence: The simulation path of different faults is not uniform, which leads to low parallel efficiency of different GPU threads; 2) Unbalanced Workload: The load of different computing units is not balanced, leading to serious differences in the execution time of each part; and 3) Poor Scalability: When the circuit scale increases, the GPU memory is limited and the simulation has strong structural dependence, which makes the simulation difficult. In this work, we propose an adaptive multidimensional parallel fault simulation framework based on the CPU–GPU heterogeneous system. We adaptively select different simulation approaches according to different circuit scales. In detail, we use the fanout-free region (FFR) grouping method to solve the problem of path divergence. We also use a combination of static and dynamic load balancing to tradeoff data handling and the execution time of each computing unit. We limit the queue length used in the GPU to improve the scalability of the simulation. To further accelerate, we propose the 4-D parallel architecture on multiple GPUs. Extensive experimental results show that our fault simulator based on 8 GPU is$105.7\times $faster than the commercial tool on average. For tens of millions of gate-level circuits, our fault simulator based on one GPU is up to$25.9\times $faster than the CPU single-threaded simulator.

The memristor-based Processing-In-Memory (PIM) architectures have been proven to be a potential architecture to store enormous parameters and execute the complicated computations of Deep Neural Networks (DNNs) efficiently. Existing PIM studies focus on designing high energy-efficient hardware architecture and algorithm-hardware co-optimization for better performance. However, the impacts of the algorithms and hardware architectures on the performance intersect with each other. Only optimizing the algorithms or the hardware architectures can not realize the optimal design. Therefore, the co-exploration of NN models and PIM architecture is necessary. However, for one thing, the co-exploration space size of NN models and PIM architectures is extremely huge, and is challenging to search. For another, during the co-exploration process, time-consuming PIM simulators are needed to evaluate various design candidates and pose a heavy time burden. To tackle these problems, we propose an efficient co-exploration framework of NN models and PIM architectures, named . In, the co-exploration space is carefully designed to adapt both NN models and PIM architectures. Besides, in order to improve search efficiency, we propose an evolutionary search algorithm with adaptive parameter priority (ESAPP). In addition, introduces a multi-level joint simulator to alleviate the problem of time-consuming evaluation. The experimental results show that the proposed co-exploration framework can find better NN models and PIM architectures than existing studies in only six GPU hours (9.8 48.2× speedup). At the same time, can improve the accuracy of co-design results by 15.3% and reduce the energy-delay-product (EDP) by 5.96× compared with existing work.

As a new algorithm of graph embedding, graph neural networks (GNNs) have been widely used in many fields. However, GNN computing has the characteristics of both sparse graph processing and dense neural network, which make it difficult to be deployed efficiently on the existing graph processing accelerators or neural network accelerators. Recently, some GNN accelerators have been proposed, but the following challenges have not been fully solved: 1) the minibatch GNN inference scenario has the potential of software and hardware co-design, which can bring 30% computation amount reduction, and this is not well utilized. Besides, the cost of message flow graph construction is large and may account for more than 50% of the total delay; 2) the feature aggregation has a large amount of data access and relatively small amount of computation, which leads to low on-chip data reuse, only 10% of dense computing; and 3) without the optimization of sparse computing units, simple memory bank and cross bar architecture can easily lead to bank access conflict and load imbalance, reducing the utilization of computing units to less than 60%. In order to solve the above problems, we propose a algorithm-hardware co-design scheme to accelerate GNN inference, which includes three technologies: 1) a reuse-aware sampling method is proposed for minibatch inference scenarios, which reduces 30% of the calculation and improves the on-chip reusability of local data; 2) through the nodewise parallelism-aware quantization, the features and weights are quantized to integers with eight or four bits, which reduces the amount of memory access by at least four times; and 3) an accelerator supporting the above technologies is designed and evaluated, and different operations are supported by the sampling-inference integration architecture. The multibank on-chip memory pool is designed to support data reuse, and edge stream reordering is used to reduce data access conflicts, improving the utilization of computing units by$1.5\times $. Combined with the above technologies, the experiments show that our design achieves$9.2\times $speedup and$29\times $energy efficiency improvement compared with the Deep Graph Library framework running on servers equipped with CPU and GPU.

In the age of Artificial Intelligence (AI), the huge data movements between memory and computing units become the bottleneck of von Neumann architectures, i.e., the “memory wall” problem. In order to tackle this challenge, Processing-In-Memory (PIM) architectures are proposed, which perform in-situ computations in memory and give alternative solutions to boost the computing energy efficiency and performance. Because of the large-scale Neural Network (NN) algorithm models and the huge hardware design space, various factors affect computing accuracy and performance, bringing the need for efficient PIM modeling and evaluation tools. In this work, we propose a behavior-level modeling tool, MNSIM 2.0, to model the performance of PIM architectures efficiently. At the hardware level, MNSIM 2.0 provides a hierarchical PIM modeling structure with flexible architecture configurability and components extensibility. Moreover, the first unified PIM memory array model is proposed for describing both digital and analog PIM. At the algorithm level, MNSIM 2.0 supports the PIM-based NN computing accuracy simulation considering various architecture and device parameters. A PIM-oriented NN model training and quantization flow is also integrated to improve the performance gain brought by PIM. At the scheduling level, MNSIM 2.0 adopts a universal scheduling description compatible with different scheduling strategies. Validation using fabricated PIM macros shows the relative modeling error rate of MNSIM 2.0 is 3:8 5:5%. Case studies show that MNSIM 2.0 enables PIM design space explorations, influences analysis of device parameters, and architecture design insight discoveries.

SRAM-based computing-in-memory (CIM) has been widely explored to accelerate neural networks (NNs). However, it is challenging to store all weights of many modern NNs due to limited on-chip SRAM capacity. This bottleneck induces a large amount of off-chip DRAM accesses and impedes the improvement of performance and energy efficiency. This paper proposes a new approach of computing in resistive random-access memory (ReRAM)-assisted energy- and area-efficient SRAM (CREAM) for accelerating large-scale NNs while eliminating the DRAM access. The NN weights are all stored in high-density on-chip ReRAMs and restored to the proposed non-volatile SRAM (nvSRAM) CIM cells with array-level parallelism. Furthermore, to deal with the influence of ReRAM and CMOS variations, a novel layer-wise and bit-wise weight-configuration search algorithm is proposed by leveraging different sensitivity of each layer in NN models. A data-aware weight-mapping method is also presented to efficiently map NN models to ReRAMs in CREAM for high computation parallelism. The experiment results show$10.3\times $weight storage density over the standard 6T SRAM array. Evaluations of ResNet-18 and VGG-9 on CIFAR-10/CIFAR-100 datasets show up to$3.47\times $and$1.70\times $energy efficiency over two baseline designs of SRAM-CIM and ReRAM-CIM, respectively, in addition to 15.6% higher accuracy than ReRAM-CIM under device variations.

Performing data-intensive tasks in the von Neumann architecture is challenging to achieve both high performance and energy efficiency due to the memory wall bottleneck. Compute-in-memory (CiM) is a promising mitigation approach by enabling parallel and in-situ multiply-accumulate (MAC) operations within the memory array. Thanks to the good matching of capacitors, SRAM-based charge-domain CiM (Q-CiM) has shown its potential for higher row-wise parallelism. However, the peripheral circuits of Q-CiM, such as the input drivers and analog-digital converters (ADCs), limit further improvement of throughput and area efficiency. This paper proposes a single-ADC multi-bit accumulation CiM macro architecture SAMBA, which can perform multi-bit MAC operation with ReLU of two vectors in one CiM cycle by only a single A/D conversion to mitigate the ADC overhead. In addition, post-correction methods are proposed to compensate the non-linearity of sensitive circuit modules in SAMBA to recover the accuracy drop due to the capacitor mismatch. A proof-of-concept macro is fabricated in a 65nm process and achieves 51.2GOPS throughput and 10.3TOPS/W energy efficiency, while showing 88.6% accuracy on CIFAR-10 and 64.8% accuracy on the CIFAR-100 with VGG-8 model.

This paper presents an approach to the mitigation of harmonic distortions in wideband current-steering digital-to-analog converters (DACs). This approach enables code-independent constant-switching-activity data-weighted-averaging (CSA-DWA) with the extra area and power overhead by exploiting redundant current sources. With CSA-DWA, a 16-bit 4.0-GS/s calibration-free DAC is designed in 65 nm CMOS. To achieve high-speed low-complexity CSA-DWA decoding, the most-significant-bit (MSB) segment is set to 5 bits. The MSB switching activities are regulated to be constant with 1-bit randomized switching activity to minimize the non-linearity due to the MSB switching activity truncation errors in the CSA-DWA decoder. Furthermore, a power delivery scheme is adopted to reduce the IR-drop mismatch between the switching elements. Experimental results show that this DAC achieves$>$70 dBc spurious-free dynamic range (SFDR) and$< -80$dBc third-order intermodulation distortion (IM3) up to 1 GHz. With the proposed CSA-DWA, SFDR and IM3 are improved by 4–15 dB and 5–14 dB, respectively, across the Nyquist band.

The rapid development of edge artificial intelligence (AI) raises high requirements for data-intensive neural network (NN) computing and storage of edge devices, under a limited chip footprint and energy supply source. As a promising approach for energy-efficient processing, computing-in-memory (CiM) has been widely explored in recent efforts to mitigate the data transmission bottleneck. However, CiM with small on-chip memory capacity results in expensive data reloads, limiting its deployment in large-scale NN applications. Moreover, the increased leakage under advanced CMOS scaling lowers the energy efficiency. In this work, device-circuit synergy based on the indium-gallium-zinc-oxide (IGZO) thin-film transistor (TFT) is adopted to address these challenges. First, 4-transistor-1-capacitor (4T1C) IGZO eDRAM CiM is proposed with higher density than SRAM-based CiM and enhanced data retention by both lower device leakage and a differential cell structure. Second, exploiting the back-end-of-line (BEOL) compatibility and vertical integration of emerging channel-all-around (CAA) IGZO devices, 3D eDRAM CiM is proposed, which paves the way for IGZO-based CiM with ultra-high density. Circuit techniques including time-interleaved computing and differential refresh are proposed to guarantee accuracy under large-capacity 3D CiM. As a proof of concept, a$128 \times 32$CiM array is fabricated under a foundry low-temperature poly-crystalline and oxide (LTPO) technology, demonstrating high computing linearity and long data retention. Benchmarks on scaled 45nm IGZO technology show energy efficiency of 686 TOPS/W for array only, and 138 TOPS/W while considering peripheral overheads.

Bitwise logic-in-memory (BLiM) is a promising approach to efficient computing in data-intensive applications by reducing data movement between memory and processing units. However, existing BLiM techniques have challenges towards higher energy efficiency and speed: (i) DC power in computing and result sensing is significant in most existing RRAM and MRAM based BLiM solutions; (ii) before the computation result could be stored back to the same memory array, existing BLiM has to sense the result first, at the cost of extra power and latency due to the sense amplifiers (SAs). Targeting at higher energy efficiency and speed, this work proposes a new BLiM approach in 2-transistor/ cell (2T/C) and 3T/C topologies based on ferroelectric field-effect transistors (FeFETs), supporting a variety of computing functions. For the first time, this new approach supports SA-free direct write-back, and consumes no static power for computing and sensing with proposed fully dynamic computing and sensing schemes. Another highlight is that this work further minimizes the dynamic power by (i) reducing the chance of bitline charging activities and (ii) recycling the bitline charge in sensing multi-operand operations. Compared with prior BLiM methods based on nonvolatile memories, evaluation shows 3.0x–100x latency and 1.3x–200x energy improvement for typical in- memory XOR operation, which further leads to 3.0x–58x and 3.2x–78x savings of latency and energy, respectively, for the application of advanced-encryption standard (AES).

Intermittent systems powered by ambient energy harvesting are becoming popular for the benefits of an infinite lifetime and minimum maintenance requirements. Nonvolatile processors (NVPs) enable continual task executions under an unstable power supply with an efficient reactive checkpointing strategy. However, recovering concurrent peripherals in an intermittent system may incur significant overhead once power failures take place, and the recovery of interrupts also lacks discussion in existing works. Noticing the different optimization directions between responsive checkpointing within NVPs and proactive checkpointing required to recover concurrent peripherals, this paper proposes REMARK, an NVP architecture enabling hybrid checkpointing and efficient peripheral recovery. REMARK expands the current NVP structure with a hybrid backup/restore module, a peripheral handler and an interrupt handler, which addresses the recovery problem of both peripherals and interrupts efficiently. A REMARK chip is fabricated to verify the proposed architecture. Results show that the execution efficiency is improved by$13\times $compared with the state-of-the-art. With programming optimization, another 36.5% performance improvement can be achieved.

Time interleaving (TI) is an effective approach to higher speed conversion of current-steering digital-to-analog converters (DACs). However, achieving high linearity performance for these TI DACs is challenging during interleaving synchronization, output current summation, and parasitic capacitance control. This article exploits the design of a 6.0-GS/s 14-bit two-channel time-interleaved DAC in a 65-nm CMOS process for communication systems. A novel asymmetric current-tree summation network is proposed to reduce the current summation nonlinearity in the DAC. A differential clock phase and duty-cycle calibration scheme is also adopted while achieving low complexity. Furthermore, a current source layout optimization scheme is proposed that significantly reduces the parasitic capacitance of interleaving switches and improves the linearity. Measurement results of the fabricated DAC show 6–20-dB spurious-free dynamic range (SFDR) improvement with the proposed techniques, achieving >60-dB SFDR up to 1355 MHz and >50-dB SFDR up to the Nyquist.

Non-volatile computing-in-memory (nvCIM) is a novel architecture used for deep neural networks (DNNs) because it can reduce the movement of data between computing units and memory units. As sparsity has made great progress in DNNs, the existing nvCIM architecture is only optimized for structured sparsity but little for unstructured sparsity. To solve this problem, the sparsity-aware nvCIM macro is proposed to improve the computing performance and network classification accuracy, and to support both structured and unstructured sparsity. First, the analog switch array is used to take advantage of the structured sparsity and to improve the computing parallelism. Second, the low-resolution current-mode analog-to-digital converter (CMADC) is designed to optimize the unstructured sparsity. Experimental results show that the peak equivalent energy efficiency of the proposed nvCIM macro is 9.1 TOPS/W (A8W8, 8-bit activations and 8-bit weights) with only 0.51% accuracy loss, and 584.9 TOPS/W (A1W1), which is 4.8 -$7.5\times$compared to the state-of-the-art nvCIM macros.

Low-bit quantization is a big challenge for neural network training. Conventional training hardware adopts FP32 to accumulate the partial-sum result, which seriously degrades energy efficiency. In this paper, a technology called dynamic group accumulation (DGA) is proposed to reduce the accumulation error. First, we model the proposed group accumulation method and give the optimal DGA algorithm. Second, we design a training architecture and implement a hardware-efficient DGA unit. Third, we make a comprehensive analysis of the DGA algorithm and training architecture. The proposed method is evaluated on CIFAR and ImageNet datasets, and results show that DGA can reduce accumulation bit-width by 6 bits while achieving the same precision as the static group method. With the FP12 DGA, the CNN algorithm only loses 0.11% accuracy in ImageNet training, and our architecture saves 32% of power consumption compared to the FP32 baseline.

Computing-in-memory (CiM) is a promising technique to achieve high energy efficiency in data-intensive matrix-vector multiplication (MVM) by relieving the memory bottleneck. Unfortunately, due to the limited SRAM capacity, existing SRAM-based CiM needs to reload the weights from DRAM in large-scale networks. This undesired fact weakens the energy efficiency significantly. This work, for the first time, proposes the concept, design, and optimization of computing-in-ROM to achieve much higher on-chip memory capacity, and thus less DRAM access and lower energy consumption. Furthermore, to support different computing scenarios with varying weights, a weight fine-tune technique, namely Residual Branch (ReBranch), is also proposed. ReBranch combines ROM-CiM and assisting SRAM-CiM to achieve high versatility. YOLoC, a ReBranch-assisted ROM-CiM framework for object detection is presented and evaluated. With the same area in 28nm CMOS, YOLoC for several datasets has shown significant energy efficiency improvement by 14.8x for YOLO (DarkNet-19) and 4.8x for ResNet-18, with <8% latency overhead and almost no mean average precision (mAP) loss (−0.5% ~ +0.2%), compared with the fully SRAM-based CiM.

Sparse Matrix-Matrix Multiplication (SpMM) has served as fundamental components in various domains. Many previous studies exploit GPUs for SpMM acceleration because GPUs provide high bandwidth and parallelism. We point out that a static design does not always improve the performance of SpMM on different input data (e.g., >85% performance loss with a single algorithm). In this paper, we consider the challenge of input dynamics from a novel auto-tuning perspective, while following issues remain to be solved: (1) Orthogonal design principles considering sparsity. Orthogonal design principles for such a sparse problem should be extracted to form different algorithms, and further used for performance tuning. (2) Nontrivial implementations in the algorithm space. Combining orthogonal design principles to create new algorithms needs to tackle with new challenges like thread race handling. (3) Heuristic adaptability to input dynamics. The heuristic adaptability is required to dynamically optimize code for input dynamics.

The memristor-based Processing-In-Memory (PIM) architectures have shown great potential to boost the computing energy efficiency of Neural Networks (NNs). Existing work concentrates on hardware architecture design and algorithm-hardware co-optimization, but neglects the non-negligible impact of the correlation between NN models and PIM architectures. To ensure high accuracy and energy efficiency, it is important to co-design the NN model and PIM architecture. However, on the one hand, the co-exploration space of NN model and PIM architecture is extremely tremendous, making searching for the optimal results difficult. On the other hand, during the co-exploration process, PIM simulators pose a heavy computational burden and runtime overhead for evaluation. To address these problems, in this paper, we propose an efficient co-exploration framework for the NN model and PIM architecture, named Gibbon. In Gibbon, we propose an evolutionary search algorithm with adaptive parameter priority, which focuses on subspace of high priority parameters and alleviates the problem of vast co-design space. Besides, we design a Recurrent Neural Network (RNN) based predictor for accuracy and hardware performances. It substitutes for a large part of the PIM simulator workload and reduces the long simulation time. Experimental results show that the proposed co-exploration framework can find better NN models and PIM architectures than existing studies in only seven GPU hours (8.4~41.3× speedup). At the same time, Gibbon can improve the accuracy of co-design results by 10.7% and reduce the energy-delay-product by 6.48× compared with existing work.

Recently, Graph Convolutional Networks (GCNs) have shown powerful learning capabilities in graph processing tasks. Computing GCNs with conventional von Neumann architectures usually suffers from limited memory bandwidth due to the irregular memory access. Recent work has proposed Processing-In-Memory (PIM) architectures to overcome the bandwidth bottleneck in Convolutional Neural Networks (CNNs) by performing in-situ matrix-vector multiplication. However, the performance improvement and computation parallelism of existing CNN-oriented PIM architectures is hindered when performing GCNs because of the large scale and sparsity of graphs. To tackle these problems, this paper presents a parallelism enhancement framework for PIM-based GCN architectures. At the software level, we propose a fixed-point quantization method for GCNs, which reduces the PIM computation overhead with little accuracy loss. We also introduce the vertex clustering algorithm to the graph, minimizing the inter-cluster links and realizing cluster-level parallel computing on multi-core systems. At the hardware level, we design a Resistive Random Access Memory (RRAM) based multi-core PIM architecture for GCN, which supports the cluster-level parallelism. Besides, we propose a coarse-grained pipeline dataflow to cover the RRAM write costs and improve the GCN computation throughput. At the software/hardware interface level, we propose a PIM-aware GCN mapping strategy to achieve the optimal tradeoff between resource utilization and computation performance. We also propose edge dropping methods to reduce the inter-core communications with little accuracy loss. We evaluate our framework on typical datasets with multiple widely-used GCN models. Experimental results show that the proposed framework achieves$698\times, 89\times$, and$41\times$speedup with$7108\times,255\times$, and$31\times$energy efficiency enhancement compared with CPUs, GPUs, and ASICs, respectively.

Motivated by reducing the data transfer activities in data-intensive neural network computing, SRAM-based compute-in-memory (CiM) has made significant progress. Unfortunately, SRAM has low density and limited on-chip capacity. This makes the deployment of large models inefficient due to the frequent DRAM access to update the weight in SRAM. Recently, a ROM-based CiM design, YOLoC, reveals the unique opportunity of deploying a large-scale neural network in CMOS by exploring the intriguing high density of ROM. However, even though assisting SRAM has been adopted in YOLoC for task transfer within the same domain, it is still a big challenge to overcome the read-only limitation in ROM and enable more flexibility. Therefore, it is of paramount significance to develop new ROM-based CiM architectures and provide broader task space and model expansion capability for more complex tasks.

Visual indoor exploration requires agents to explore a room in a limited time. Currently, planning-based solutions have a time-consuming inference stage and require many handcrafted parameters in different scenes. Reinforcement Learning (RL) schemes on the other hand solve these problems by automatically updating flexible policies and affording faster inference time. Spurred by the advantages of RL, we introduce Spatial Attention Visual Exploration (SAVE), which is based on Active Neural SLAM (ANS) [1]. Specifically, we propose a novel RL-based global planner named Spatial Global Policy (SGP) that utilizes spatial information to promote efficient exploration through global goal guidance. SGP has two major components: a transformer-based spatial-attention module encoding spatial interrelation between the agent and different regions to perform spatial reasoning, and a hierarchical spatial action selector to infer global goals for faster training. The map representations are aligned through our spatial adjustor. Experiments on the Habitat photo-realistic simulator [2] demonstrate that SAVE outperforms current planning-based methods and RL variants, reducing at least 10% of the processing steps, 15% of the repeat ratio, and affording an x2 to x4 faster execution time than planning-based methods.

Autonomous exploration and mapping of unknown terrains employing single or multiple robots is an essential task in mobile robotics and has therefore been widely investigated. Nevertheless, given the lack of unified data sets, metrics, and platforms to evaluate the exploration approaches, we develop an autonomous robot exploration benchmark en-titled Explore-Bench. The benchmark involves various explo-ration scenarios and presents two types of quantitative metrics to evaluate exploration efficiency and multi-robot cooperation. Explore-Bench is extremely useful as, recently, deep rein-forcement learning (DRL) has been widely used for robot exploration tasks and achieved promising results. However, training DRL-based approaches requires large data sets, and additionally, current benchmarks rely on realistic simulators with a slow simulation speed, which is not appropriate for training exploration strategies. Hence, to support efficient DRL training and comprehensive evaluation, the suggested Explore-Bench designs a 3-level platform with a unified data flow and 12 × speed-up that includes a grid-based simulator for fast evaluation and efficient training, a realistic Gazebo simulator, and a remotely accessible robot testbed for high-accuracy tests in physical environments. The practicality of the proposed benchmark is highlighted with the application of one DRL-based and three frontier-based exploration approaches. Fur-thermore, we analyze the performance differences and provide some insights about the selection and design of exploration methods. Our benchmark is available at https://github.com/efc-robot/Explore-Bench.

Graph mining, which finds specific patterns in the graph, is becoming increasingly important in various domains. We point out that accelerating graph mining suffers from the following challenges: (1) Heavy comparison for pruning: Pruning technique is widely used to reduce search space in graph mining. It applies constraints on vertex indices and involves massive index comparisons. (2) Low parallelism of set operations: The typical graph mining algorithms can be expressed as a series of set operations between neighbors of vertices, which suffer from low parallelism if vertices are streaming to the computation units. (3) Heavy data transfer: Graph mining needs to transfer intermediate data with two orders of magnitude larger than the original data volume between CPU and memory.

In normally-off instant-on applications, power-gating of the embedded memory is an effective way for higher power efficiency by preventing long-standby-time leakage energy. Recent efforts of nonvolatile SRAM (nvSRAM) design with in-cell NVM element backup provide an efficient way for both normal-mode computing and off-mode backup and restore (B&R) operations. For these efforts, circuit innovations are required to achieve optimal balance between B&R energy and area overheads. In this paper, we report a novel 8T/cell FeFET-based nvSRAM design that outperforms prior FeFET-based designs with higher density, while still maintaining the advantage of only sub-fJ energy for each B&R operation, 363x lower than the existing RRAM-based nvSRAM design. Compared with prior FeFET-based designs, this design reduces the B&R transistor count per cell from 4 to only 2, which leads to a significant total area overhead reduction of 11%.

Approximate Nearest Neighbor Search (ANNS) is widely used in many fields (e.g., recommender systems). In recent years, the graph-based ANNS methods have attracted the attention of many researchers due to their superiority compared to non-graph-based methods. Compared with traditional recommender systems, mobile recommender systems have higher latency requirements. The graph-based ANNS method faces the following challenges that make it difficult to meet the requirements. (1) Poor connectivity. Due to the limitation of the construction algorithm, the connectivity of the graph is poor, which in turn affects the search performance. (2) Redundant search. The existing search algorithm uses sufficiently long search steps for all queries to achieve high search accuracy. However, the query search steps follow the long-tailed distribution that brings the redundant search, e.g., for more than 40 % of the queries, 87.4 % of the search overhead is redundant. We propose two optimization strategies to tackle the above challenges. (1) Reverse connection enhancement strategy. In the graph construction process, we increase the in-degree of the point to be inserted to enhance the graph connectivity, while keeping the out-degree low to maintain the high search efficiency. (2) Query aware early termination strategy. We identify regional features to predict the number of remaining search steps to achieve dynamic search termination and reduce the redundant search overhead. Finally, we verify the proposed solutions on multiple representative datasets. Compared with the state-of-the-art graph-based algorithm, our solutions can improve the search speed up to 1.21x when the recall rate equals 0.95.

Neural architecture search tries to shift the manual design of neural network (NN) architectures to algorithmic design. In these cases, the NN architecture itself can be viewed as data and needs to be modeled. A better modeling could help explore novel architectures automatically and open the black box of automated architecture design. To this end, this work proposes a new encoding scheme for neural architectures, the Training-Analogous Graph-based ArchiTecture Encoding Scheme (TA-GATES). TA-GATES encodes an NN architecture in a way that is analogous to its training. Extensive experiments demonstrate that the flexibility and discriminative power of TA-GATES lead to better modeling of NN architectures. We expect our methodology of explicitly modeling the NN training process to benefit broader automated deep learning systems. The code is available at https://github.com/walkerning/aw_nas.

Various low-bit quantized methods have been widely exploited and shown decent performance on 2D vision tasks in recent years. Complemented with 2D images, 3D point clouds provide an opportunity to understand the surrounding environ-ment better. However, low-bit quantization methods designed for 2D vision tasks are not readily transferable to 3D point clouds due to the higher dimension of 3D data and the increased proportion of activations. In this work, we propose a novel quantization framework, DASCQ, for 3D point cloud processing. First, a new distribution-aware strategy (DA) is presented to decrease the deviation caused by extremely low-bit quantization through activation and weight distribution analysis. Second, a soft constraint manner (SC) is designed to smooth the training of quantized networks which suffer from backward propagation errors. We evaluate our approach on two 3D point cloud datasets, ModelNet40 and S3DIS. Results indicate that the performance of the proposed approach is superior to other state-of-the-art quantization methods on both shape classification and scene semantic segmentation tasks.

In recent years, convolutional neural networks (CNNs) have achieved significant advancements in various fields. However, the computation and storage overheads of CNNs are overwhelming for Internet-of-Things devices. Both network pruning algorithms and hardware accelerators have been introduced to empower CNN inference at the edge. Network pruning algorithms reduce the size and computational cost of CNNs by regularizing unimportant weights to zeros. However, existing works lack intrakernel structured types to tradeoff between sparsity and hardware efficiency, and the index storage for irregularly pruned networks is significant. Hardware accelerators leverage the sparsity of pruned CNNs to improve energy efficiency. However, their process element (PE) utilization rate is low because of uneven sparsity among input convolutional kernels. To overcome these problems, we propose PACA: a Pattern pruning Algorithm and Channel-fused high PE utilization Accelerator for CNNs. It includes three parts: a pattern pruning algorithm to explore the intrakernel sparsity type and reduce the index storage, a channel-fused hardware architecture to reduce the PEs’ idle rate and improve the performance, and a heuristic and taboo search-based smart fusion scheduler to analyze the idle PE problem and schedule the channel fusion in hardware. To demonstrate the effectiveness of PACA, we have implemented the software parts by Python and the hardware architecture by RTL codes. Experimental results on various datasets show that compared with an existing work, PACA can reduce the index storage overhead by$3.47\times $–$5.63\times $with 3.85–9.12 average patterns, and it can improve the hardware performance by$2.01\times $–$5.53\times $because of PEs’ idle rate reduction.

Multirobot exploration (MR-Exploration) is a primary task providing the location and map for many multirobot applications. To improve system performance, convolutional neural network (CNN) is introduced by recent researches into critical components in MR-Exploration, such as feature-point extraction (FE) and place recognition (PR). This CNN-based MR-Exploration needs to simultaneously run multiple CNN models and complex postprocessing algorithms. This significantly challenges the hardware platforms of embedded systems. Previous researches reveal that an FPGA is ideal for CNN processing on embedded platforms. Such accelerators usually process different models in sequence, while they cannot schedule multiple tasks at runtime. Furthermore, the postprocessing of CNNs is computationally intensive and becomes the bottleneck of the whole system. To handle such problems, we propose an interruptible CNN accelerator for multirobot exploration (INCAME) framework to rapidly deploy the robot applications on FPGAs. In INCAME, we propose an interrupt method based on virtual instructions to support multitasking on CNN accelerators. INCAME also includes hardware modules for accelerating the postprocessing of the CNN-based components. Organically, it integrates the postprocessing and CNN backbone by sharing memory. Experimental results reveal that INCAME enables multitask scheduling on the CNN accelerator with negligible performance degradation (0.3%). INCAME enables embedded FPGAs to perform MR-Exploration in real time (20 fps) via the multitask support and postprocessing acceleration.

Convolutional neural networks (CNNs) have been widely used in many tasks, but training CNNs is time consuming and energy hungry. Using the low-bit integer format has been proved promising for speeding up and improving the energy efficiency of CNN inference, while CNN training can hardly benefit from such a technique because of the following challenges: 1) the integer data format cannot meet the requirements of the data dynamic range in training, resulting in the accuracy drop; 2) the floating-point data format keeps sizeable dynamic range with much more exponent bits, thus using it results in higher accumulation power than using the integer data format; and 3) there are some specially designed data formats (e.g., with group-wise scaling) that have the potential to deal with the former two problems but common hardware platforms cannot support them efficiently. To tackle all these challenges and make the training phase of CNNs benefit from the low-bit format, we propose a low-bit training framework for CNNs to pursue a better tradeoff between accuracy and energy efficiency: 1) we adopt element-wise scaling to increase the dynamic range of data representation, which significantly reduces the quantization error; 2) group-wise scaling with hardware friendly factor format is designed to reduce the element-wise exponent bits without degrading the accuracy; and 3) we design the customized hardware unit that implements the low-bit tensor convolution arithmetic with our multilevel scaling data format. Experiments show that our framework achieves a superior tradeoff between the accuracy and the bit-width than previous low-bit training studies. For training various models on CIFAR-10, using 1-bit mantissa and 2-bit exponent is adequate to keep the accuracy loss within 1%. On larger datasets like ImageNet, using 4-bit mantissa and 2-bit exponent is adequate. Through the energy consumption simulation of the whole network, we can see that training a variety of models with our framework could achieve$4.9\times $–$10.2\times $higher energy efficiency than full-precision arithmetic.

Computing-in-memory (CIM) is a new architecture which is more energy-efficient than the Von Neumann architecture due to the fact that it performs calculation in the memory units which can reduce a large amount of data movement. Nowadays, CIM with non-volatile memory (nvCIM), such as resistive random access memory (RRAM), has become a research frontier to further improve computing performance. Recent works mainly explored how to improve the computing performance of nvCIM, but seldom paid attention to the problem of accuracy loss. In this paper, we propose the nvCIM framework, which can systematically analyze the relationship between the classification accuracy of network models and main analog factors. Based on the nvCIM framework, we further provide detailed optimization methods, including RRAM features, array properties, and ADC parameters. The adaptive voltage-controlled SET and pulse-controlled RESET (VSPR) program-verify scheme is proposed to achieve high-resolution RRAM. And the margin enhancement based current-mode sense amplifier (MECSA) and offset reduction based analog-to-digital converter (ORADC) are proposed to improve the accuracy of analog part computing. Experimental results show that the macro-level and system-level energy efficiency is 112.1 TOPS/W and 9.86 TOPS/W respectively with less than 3.51% loss in contrast to the ideal accuracy, which is$2.9\times $-$25.9\times $compared to the energy efficiency of existing RRAM based nvCIM accelerators.

Always-on intelligent visual perception applications are widely deployed in edges in the AIoT era. In order to eliminate power costs of data conversion and transmission, this paper proposes Senputing, an ultra-low-power processing-in-sensor chip that completely fuses sensing and computing together for a BNN-based hierarchical processing system. This chip could operate in two modes. In computation mode, photocurrents are directly utilized for computing without being converted into voltages, and the computation results of 1-st BNN layer are directly sent out to subsequent BNN processors for an always-on coarse classification, eliminating conversion power and storage cost of raw images. Once an interested objected is detected, this chip switches to sensor mode and sends raw images to potential full-precision processors or cloud servers for fine-grained recognition or segmentation. A$32\times 32$prototype is fabricated with 180nm CMOS process. It accomplishes MNIST dataset classification task with the accuracy of 93.76% and the power consumption of 147nW at 156fps, achieving$13.1\times $energy efficiency compared with state-of-the-art work.

INFerence-as-a-Service (INFaaS) has become a primary workload in the cloud. However, existing FPGA-based Deep Neural Network (DNN) accelerators are mainly optimized for the fastest speed of a single task, while the multi-tenancy of INFaaS has not been explored yet. As the demand for INFaaS keeps growing, simply increasing the number of FPGA-based DNN accelerators is not cost-effective, while merely sharing these single-task optimized DNN accelerators in a time-division multiplexing way could lead to poor isolation and high-performance loss for INFaaS. On the other hand, current cloud-based DNN accelerators have excessive compilation overhead, especially when scaling out to multi-FPGA systems for multi-tenant sharing, leading to unacceptable compilation costs for both offline deployment and online reconfiguration. Therefore, it is far from providing efficient and flexible FPGA virtualization for public and private cloud scenarios. Aiming to solve these problems, we propose a unified virtualization framework for general-purpose deep neural networks in the cloud, enabling multi-tenant sharing for both the Convolution Neural Network (CNN), and the Recurrent Neural Network (RNN) accelerators on a single FPGA. The isolation is enabled by introducing a two-level instruction dispatch module and a multi-core based hardware resources pool. Such designs provide isolated and runtime-programmable hardware resources, which further leads to performance isolation for multi-tenant sharing. On the other hand, to overcome the heavy re-compilation overheads, a tiling-based instruction frame package design and a two-stage static-dynamic compilation, are proposed. Only the lightweight runtime information is re-compiled with ∼1 ms overhead, thus guaranteeing the private cloud’s performance. Finally, the extensive experimental results show that the proposed virtualized solutions achieve up to 3.12× and 6.18× higher throughput in the private cloud compared with the static CNN and RNN baseline designs, respectively.

As one type of associative memory, content-addressable memory (CAM) has become a critical component in several applications, including caches, routers, and pattern matching. Compared with the conventional CAM that could only deliver a “matched or not-matched” result, emerging multilevel CAM (ML-CAM) is capable of delivering “the degree of match” with multilevel distance calculation. This feature has been desired in applications that need beyond-Boolean matching results. However, existing ML-CAM designs are limited by the bit-cell device discharging current mismatch and vulnerability to the timing of sensing operations for distance calculation. This inherent constraint makes it difficult to further improve the accuracy and scalability toward higher accuracy and higher dimension matching. In this work, we propose CapCAM, a multilevel Capacitive Content Addressable Memory. It could be implemented based on either static random-access memory (SRAM) or emerging technologies, e.g., the ferroelectric field-effect transistor (FeFET). CapCAM could provide linear and stable voltage drop scaled by the match degree and need no strict timing for result sensing, which embraces the high-accuracy and high-scalability search. The inherent enabler of CapCAM is the charge-domain computing mechanism. This article will present the basic concept, operating mechanisms, detailed circuit designs, and circuit-level simulations of CapCAM. Besides, we apply CapCAM to few-shot learning applications and compare CapCAM with the current-domain TCAM designs. Results show 99.2% accuracy for a five-way five-shot classification task with our proposed CapCAM design while considering 1-fF capacitors, 20-domain FeFETs, and 256 columns. In contrast, the prior work based on discharging dynamics requires strict timing controls and suffers from accuracy degradation under the same configuration, which demonstrates CapCAM’s capability of low-power, accurate, and scalable multilevel CAM (ML-CAM) computing.

Block-circulant based compression is a popular technique to accelerate neural network inference. Though storage and computing costs can be reduced by transforming weights into block-circulant matrices, this method incurs uneven data distribution in the frequency domain and imbalanced workload. In this paper, we propose RAB: a Reconfigurable Architecture Block-Circulant Neural Network Accelerator to solve the problems via two techniques. First, a fine-grained frequency-domain quantization is proposed to accelerate MAC operations. Second, a reconfigurable architecture is designed to transform FFT/IFFT modules into MAC modules, which alleviates the imbalanced workload and further improves efficiency. Experimental results show that RAB can achieve 1.9x/1.8x area/energy efficiency improvement compared with the state-of-the-art block-circulant compression based accelerator.

Nowadays, deep neural network (DNN) has played an important role in machine learning. Non-volatile computingin-memory (nvCIM) for DNN has become a new architecture to optimize hardware performance and energy efficiency. However, the existing nvCIM accelerators focus on system-level performance but ignore analog factors. In this paper, the sense margin and offset are considered in the proposed nvCIM framework. The margin enhancement based current-mode sense amplifier (MECSA) and the offset reduction based analog-to-digital converter (ORADC) are proposed to improve the accuracy of the ADC. Based on the above methods, the nvCIM framework is displayed and the experiment results show that the proposed framework has an improvement on area, power, and latency with the high accuracy of network models, and the energy efficiency is 2.3 - 20.4x compared to the existing RRAM based nvCIM accelerators.

The automatic speech recognition (ASR) system is becoming increasingly irreplaceable in smart speech interaction applications. Nonetheless, these applications confront the memory wall when embedded in the energy and memory constrained Internet of Things devices. Therefore, it is extremely challenging but imperative to design a memory-saving and energy-saving ASR system. This paper proposes a joint-optimized scheme of network compression with approximate memory for the economical ASR system. At the algorithm level, this work presents block-based pruning and quantization with error model (BPQE), an optimized compression framework including a novel pruning technique coordinated with low-precision quantization and the approximate memory scheme. The BPQE compressed recurrent neural network (RNN) model comes with an ultra-high compression rate and finegrained structured pattern that reduce the amount of memory access immensely. At the hardware level, this work presents an ASR-adapted incremental retraining method to further obtain optimal power saving. This retraining method stimulates the utility of the approximate memory scheme, while maintaining considerable accuracy. According to the experiment results, the proposed joint-optimized scheme achieves 58.6% power saving and 40x memory saving with a phone error rate of 20%.

Autonomous driving is becoming a hot topic in both academic and industrial communities. Traditional algorithms can hardly achieve the complex tasks and meet the high safety criteria. Recent research on deep learning shows significant performance improvement over traditional algorithms and is believed to be a strong candidate in autonomous driving system. Despite the attractive performance, deep learning does not solve the problem totally. The application scenario requires that an autonomous driving system must work in real-time to keep safety. But the high computation complexity of neural network model, together with complicated pre-process and post-process, brings great challenges. System designers need to do dedicated optimizations to make a practical computing platform for autonomous driving. In this paper, we introduce our work on efficient computing platform design for autonomous driving systems. In the software level, we introduce neural network compression and hardware-aware architecture search to reduce the workload. In the hardware level, we propose customized hardware accelerators for pre- and post-process of deep learning algorithms. Finally, we introduce the hardware platform design, NOVA-30, and our on-vehicle evaluation project.

Memristor based Processing-In-Memory (PIM) systems give alternative solutions to boost the computing energy efficiency of Convolutional Neural Network (CNN) based algorithms. However, Analog-to-Digital Converters' (ADCs) high interface costs and the limited size of the memristor crossbars make it challenging to map CNN models onto PIM systems with both high accuracy and high energy efficiency. Besides, it takes a long time to simulate the performance of large-scale PIM systems, resulting in unacceptable development time for the PIM system. To address these problems, we propose a reliability-aware training framework and a behavior-level modeling tool (MNSIM 2.0) for PIM accelerators. The proposed reliability-aware training framework, containing network splitting/merging analysis and a PIM-based non-uniform activation quantization scheme, can improve the energy efficiency by reducing the ADC resolution requirements in memristor crossbars. Moreover, MNSIM 2.0 provides a general modeling method for PIM architecture design and computation data flow; it can evaluate both accuracy and hardware performance within a short time. Experiments based on MNSIM 2.0 show that the reliability-aware training framework can improve 3.4x energy efficiency of PIM accelerators with little accuracy loss. The equivalent energy efficiency is 9.02 TOPS/W, nearly 2.6~4.2x compared with the existing work. We also evaluate more case studies of MNSIM 2.0, which help us balance the trade-off between accuracy and hardware performance.

Neural networks running on cloud servers have been widely used in smart surveillance, but they require high bandwidth to upload videos. Edge-cloud collaborative encoding based on ROI (Region-Of-Interest) can reduce bandwidth requirement, but it suffers from inaccurate ROI detection due to feedback latency and undetected new targets. To address the above challenges, we propose an object detection system named PETRI. It adopts a latency-hiding pipeline workflow with adaptive keyframe interval selection for different input videos, and utilizes a retro-tracking method to find undetected targets. While achieving negligible impact on model accuracy, the proposed PETRI can save up to 66.44% and 30.25% bandwidth compared with the cloud only method and the previous state-of-art work respectively.

Ternary content addressable memory (TCAM) has been a critical component in caches, routers, etc., in which density, speed, power efficiency, and reliability are the major design targets. There have been the conventional low-write-power but bulky SRAM-based TCAM design, and also denser but less reliable or higher-write-power TCAM designs using nonvolatile memory (NVM) devices. Meanwhile, some TCAM designs using dynamic memories have been also proposed. Although dynamic design TCAM is denser than CMOS SRAM TCAM and more reliable than NVM TCAM, the conventional row-by-row refresh operations land up with a bottleneck of interference with normal TCAM activities. Therefore, this paper proposes a custom low-power dynamic TCAM using nanoelectromechanical (NEM) relay devices utilizing one-shot refresh to solve the memory refresh problem. By harnessing the unique NEM relay characteristics with a proposed novel cell structure, the proposed TCAM occupies a small footprint of only 3 transistors (with two NEM relays integrated on the top through the back-end-of-line process), which significantly outperforms the density of 16-transistor SRAM-based TCAM. In addition, evaluations show that the proposed TCAM improves the write energy efficiency by 2.31x, 131x, and 13.5x over SRAM, RRAM, and FeFET TCAMs, respectively; The search energy-delay-product is improved by 12.7x, 1.30x, and 2.83x over SRAM, RRAM, and FeFET TCAMs, respectively.

3D mapping is a fundamental task in robot applications. The traditional mapping methods mainly rely on spatial discretization, in which the amount of data that needs to be stored is large, and the representation ability is limited. As a continuous probability model, the Gaussian Mixture Model (GMM) has a small memory footprint and high-fidelity representation ability. Thus the GMM map is superior to discrete map representations in basic robot tasks such as navigation and localization. The general method of building GMM maps is the iterative Expectation-Maximization (EM) algorithm with K-means initialization. The EM and K-means algorithms are computation-intensive, making it challenging to meet real-time 30 fps mapping requirements on the embedded robot systems. This paper proposes a Gaussian mixture model mapping and navigation engine (GAME) on embedded FPGA to accelerate the mapping process. To achieve fully pipelined with minimal hardware resource cost, we design a unified dataflow and hardware architecture for both K-means and EM for GMM. We analyze different quantization strategies for higher parallelism and find a low-bit quantization method with mixed 8/16-bit data representation, bringing negligible loss in accuracy. Combining the unified dataflow and the mixed-bit data quantization, GAME enables real-time GMM mapping and navigation on embedded robots. The experimental results on ZCU102 show that our proposed hardware-software co-optimization framework on FPGA can run over 60× faster than on a GeForce 1080Ti GPU and over 490× faster than on an Nvidia Jetson TX2, and achieves 59 fps.

With the ever-growing demands for online Artificial Intelligence (AI), the hardware virtualization support for deep learning accelerators is vital for providing AI capability in the cloud. Three basic features, multi-task, dynamic workload, and remote access, are fundamental for hardware virtualization. However, most of the deep learning accelerators do not support concurrent execution of multiple tasks. Besides, the SOTA multi-DNN scheduling algorithm for NN accelerators neither consider the multi-task concurrent execution and resources allocation for the multi-core DNN accelerators. Moreover, existing GPU virtualized solutions could introduce a huge remote access latency overhead, resulting in a severe system performance drop.

Content-addressable memory (CAM) has been a critical component in pattern matching and also machine-learning applications. Recently emerged CAM that is capable of delivering multi-level distance calculation is promising for applications that need matching results beyond Boolean results of ?matched" and ?not matched". However, existing multi-level CAM designs are constrained by the bit-cell device discharging current mismatch and the strict timing of sensing operations for distance calculation. This fact results in the challenge of further improving the accuracy and scalability towards higher-resolution and higher-dimension matching. This work presents a multi-level CAM design that is capable of delivering high-accuracy and high-scalability search, which is immune to the discharging device mismatch and needs no strict timing for result sensing. The inherent enabler is the charge-domain computing mechanism. This work will present the operating mechanisms, the circuit simulation, and content-matching evaluation results, showing the promise towards high reliability, high energy efficiency, and high scalability.

Sampled Dense-Dense Matrix Multiplication (SDDMM) is a core component of many machine learning systems. SDDMM exposes a substantial amount of parallelism that favors throughput-oriented architectures like the GPU. However, accelerating it on GPUs is challenging in two aspects: the poor memory access locality caused by the sparse sampling matrix with the poor parallelism caused by the dot-product reduction of vectors in two dense matrices. To address both challenges, we present PRedS to boost SDDMM efficiency with a suite of Parallel Reduction Scheduling optimizations. PRedS uses Vectorized Coarsen 1-Dimensional Tiling (VCT) to benefit the online locality of loading the dense matrix. PRedS uses Integrated Interleaving Reduction (IIR) to increase thread occupancy in the parallel reduction. PRedS also leverages Warp-Merged Tiling (WMT) to preserve occupancy and parallelism when reducing very long arrays. Enhanced with GPU-intrinsic vectorized memory loading, PRedS achieves a geometric speedup of 29.20× compared to the vendor library. PRedS achieves up to 8.31× speedup over state-of-the-art implementations on the SuiteSparse benchmark.

This paper presents dynamic switching sequence (DSS) for current-steering digital-to-analog converters (DACs). Unlike conventional static switching sequence (SSS), the proposed DSS dynamically selects from pre-defined switching sequences, and achieves a minimized integral nonlinearity (INL) that is even lower than the lower bound of traditional SSS. Moreover, it works effectively with the digital pre-distortion (DPD) technique to further reduce the residual nonlinearity. Simulation results of a 16-bit segmented DAC show an average INL reduction from 37.9 LSBs (SSS, with 17.7 LSBs as the lower bound) and 10.3 LSBs (SSS+DPD, normalized), to 8.2 LSBs (with DSS) and 0.2 LSBs (DSS+DPD).

In the era of Intelligent IoT, huge amount of sensor data is collected and then transmitted to processor elements in edge devices or cloud servers. The latency and energy consumption in this process have been a bottleneck and are becoming more severe. To mitigate this problem, the idea of combining sensors, memory and processors for collectively handling the data, has been proposed and explored actively in recent efforts. In this work, thin-film transistor (TFT), which has been widely adopted in display devices and flexible sensors, is exploited. It is shown that, while TFT is promising for near-sensor processing architecture, it also shows a great potential for computing and storage for large-area and low-cost edge sensors. More specifically, we propose an almost-nonvolatile near-sensor computing-in-memory (CiM) array based on indium-gallium- zinc-oxide (IGZO) TFT, and further, integrate the CiM array with a sensor array to be a sensing and data pre-process system. We show that such a TFT-based solution can accomplish realtime sensing and multiply-and-accumulate (MAC) processing in the analog field, which simplifies the system design with lowered energy and latency in our neural network evaluations.

This paper presents a correlation-based 3rd-order inter-stage gain error background calibration technique for high-performance pipeline ADCs, with the reduced signal swing overhead to only 8% during the pseudo noise (PN) sequence injection. This is achieved by exploiting the use of paired comparators along with a dedicated calibration algorithm, namely the adaptive dither injection calibration (ADIC) technique, which further alleviates the trade-off between the signal-to-noise ratio and the non-linearity. Monte Carlo simulation results of a 16-bit 100MS/s ADC show that with an amplifier of 20dB loop gain, the spurious-free dynamic range (SFDR) is improved from 64.8dB to a mean value of 106.6dB and a standard deviation of 1.1dB.

We introduce an automatic curriculum algorithm, Variational Automatic Curriculum Learning (VACL), for solving challenging goal-conditioned cooperative multi-agent reinforcement learning problems. We motivate our curriculum learning paradigm through a variational perspective, where the learning objective can be decomposed into two terms: task learning on the current curriculum, and curriculum update to a new task distribution. Local optimization over the second term suggests that the curriculum should gradually expand the training tasks from easy to hard. Our VACL algorithm implements this variational paradigm with two practical components, task expansion and entity curriculum, which produces a series of training tasks over both the task configurations as well as the number of entities in the task. Experiment results show that VACL solves a collection of sparse-reward problems with a large number of agents. Particularly, using a single desktop machine, VACL achieves 98% coverage rate with 100 agents in the simple-spread benchmark and reproduces the ramp-use behavior originally shown in OpenAI’s hide-and-seek project.

Conducting efficient performance estimations of neural architectures is a major challenge in neural architecture search (NAS). To reduce the architecture training costs in NAS, one-shot estimators (OSEs) amortize the architecture training costs by sharing the parameters of one supernet between all architectures. Recently, zero-shot estimators (ZSEs) that involve no training are proposed to further reduce the architecture evaluation cost. Despite the high efficiency of these estimators, the quality of such estimations has not been thoroughly studied. In this paper, we conduct an extensive and organized assessment of OSEs and ZSEs on five NAS benchmarks: NAS-Bench-101/201/301, and NDS ResNet/ResNeXt-A. Specifically, we employ a set of NAS-oriented criteria to study the behavior of OSEs and ZSEs, and reveal their biases and variances. After analyzing how and why the OSE estimations are unsatisfying, we explore how to mitigate the correlation gap of OSEs from three perspectives. Through our analysis, we give out suggestions for future application and development of efficient architecture performance estimators. Furthermore, the analysis framework proposed in our work could be utilized in future research to give a more comprehensive understanding of newly designed architecture performance estimators. The code is available at https://github.com/walkerning/aw_nas.

Deep learning-based intelligent electrocardiogram (ECG) diagnosis algorithms heavily rely on large annotated datasets. Unfortunately, in the context of ECG diagnosis, privacy issues and the high cost of data annotations lead to a shortage of ECG datasets which severely limits the performance of the state-of-the-art ECG diagnosis algorithms. In this paper, we propose a novel instance-level contrastive learning scheme for ECG signals, namely CLECG, to mine effective information from unlabeled data. During the pre-training, CLECG encourages the representations of different augmented views of the same signal (positive samples) to be similar and increases the distance between representations of augmented views from the different signals (negative samples). The whole pre-training process does not require any form of labeling. Experimental results show that the proposed CLECG strategy outperforms other self-supervised methods and supervised transfer learning strategies.

Due to the significant variability in waveforms and characteristics of ECG signals, developing fully automatic (i.e., requires no expert assistance) heartbeat classification algorithms with satisfactory performance on domain-shifted data remains challenging. In this letter, we propose a novel Mixup Asymmetric Tri-training (MIAT) method to improve the generalization ability of heartbeat classifiers in domain shift scenarios. First, we develop an ECG-based tri-branch CNN model, including one shared feature encoder followed by three branch networks. Next, to obtain target-discriminative features progressively, the tri-branch CNN is trained asymmetrically in each domain adaptation cycle, where two branches are used to assign pseudo-labels to the target domain samples and the third branch is trained on these pseudo-labeled target samples. Moreover, three kinds of mixup regularizations are incorporated into the training process. Experimental results on MITDB and SVDB show that the proposed MIAT outperforms the state-of-the-art methods in terms of F1-macro score and demonstrate the effectiveness of each mixup regularization.

Structured light (SL) based three-dimensional (3D) imaging technology has been widely employed in many fields of computer vision. However, currently available SL based depth sensors are sensitive to imaging noises, which severely limits the performance of subsequent advanced vision tasks. In this letter, we propose a robust and practical SL illumination pattern coding method based on epipolar geometry. The proposed pattern can effectively alleviate coding redundancy in traditional global random speckle SL patterns and make the stereo matching more robust to noise. Meanwhile, this coding strategy supplies sufficient non-local similar blocks, which inspires us to propose a uni-direction block-stacking 3D filtering algorithm to further improve the 3D imaging quality. To verify the proposed algorithms, we developed a prototype using the off-the-shelf projector and camera. Both simulation and real scene experimental results show that the proposed methods can achieve high-quality 3D imaging performance under different noisy conditions.

Convolutional neural network (CNN) has been widely deployed in various processors for intelligent visual signal processing. However, the large amount of activations and weights in CNN causes huge power consumption on SRAM access. Data-adaptive SRAM design is a widely studied method to reduce SRAM reading power based on the utilization of data patterns, while current designs only exploit data patterns in a coarse granularity, and have no advantages when faced with randomly distributed weight data. In this article, we propose a hardware–software co-design scheme to reduce SRAM reading power for CNN processing. First, we propose a reconfigurable data-adaptive SRAM architecture with column data segmentation (CDS-RSRAM) to utilize data patterns. Data in one column is partitioned into several segments, and finer-grained data patterns are exploited within each segment for further reading power reduction. Then, a novel training method—minimum segmented neighbor difference (miniSND)—is proposed for enhancing the correlation of weights. MiniSND improves the similarity of weights without classification accuracy degradation, thus weights could benefit from CDS-RSRAM and be read out with less power consumption. Simulation results demonstrate that the co-design scheme saves up to 66%(8b)/89%(2b) power consumption compared with 8T SRAM.

Always-on keyword spotting (KWS) that detects wake-up words has been the indispensable module in the voice interaction system. However, the ultra-low-power embedded devices put forward strict requirements on energy consumption, latency, and recognition accuracy of KWS. In this work, we propose a near-sensor processing architecture of feature-configurable distributed network (NS-FDN) for always-on KWS applications. The proposed distributed network adapts to the flexible keywords demands in the actual scene by splitting the conventional single network into distributed sub-networks. We design a channel-independent training framework to improve the recognition accuracy of distributed networks. The speech features are evaluated and the redundancy is reduced in NS-FDN, which can also configure the speech features to further reduce the computing complexity and improve processing speed. For deeper optimization, we implement a 65nm-process prototype chip with near-sensor mixed-signal processing architecture avoiding energy-consuming analog-to-digital converter. By improving the system, algorithm, and hardware designs of the KWS, our co-optimized architecture eliminates the energy consumption bottleneck long-standing in conventional KWS systems and achieves state-of-the-art system performance. The experiment results show that NS-FDN achieves 31.6% energy consumption savings, 1.6 times memory savings, 57 times speedup, and 3.4% higher recognition accuracy compared with the state of the art.

With the down-scaling of CMOS technology, the design complexity of very large-scale integrated is increasing. Although the application of machine learning (ML) techniques in electronic design automation (EDA) can trace its history back to the 1990s, the recent breakthrough of ML and the increasing complexity of EDA tasks have aroused more interest in incorporating ML to solve EDA tasks. In this article, we present a comprehensive review of existing ML for EDA studies, organized following the EDA hierarchy.

With the fast evolvement of embedded deep-learning computing systems, applications powered by deep learning are moving from the cloud to the edge. When deploying neural networks (NNs) onto the devices under complex environments, there are various types of possible faults: soft errors caused by cosmic radiation and radioactive impurities, voltage instability, aging, temperature variations, malicious attackers, and so on. Thus, the safety risk of deploying NNs is now drawing much attention. In this article, after the analysis of the possible faults in various types of NN accelerators, we formalize and implement various fault models from the algorithmic perspective. We propose Fault-Tolerant Neural Architecture Search (FT-NAS) to automatically discover convolutional neural network (CNN) architectures that are reliable to various faults in nowadays devices. Then, we incorporate fault-tolerant training (FTT) in the search process to achieve better results, which is referred to as FTT-NAS. Experiments on CIFAR-10 show that the discovered architectures outperform other manually designed baseline architectures significantly, with comparable or fewer floating-point operations (FLOPs) and parameters. Specifically, with the same fault settings, F-FTT-Net discovered under the feature fault model achieves an accuracy of 86.2% (VS. 68.1% achieved by MobileNet-V2), and W-FTT-Net discovered under the weight fault model achieves an accuracy of 69.6% (VS. 60.8% achieved by ResNet-18). By inspecting the discovered architectures, we find that the operation primitives, the weight quantization range, the capacity of the model, and the connection pattern have influences on the fault resilience capability of NN models.

Ternary content addressable memory (TCAM) is one type of associative memory and has been widely used in caches, routers, and many other mapping-aware applications. While the conventional SRAM-based TCAM is high speed and bulky, there have been denser but slower and less reliable nonvolatile TCAMs using nonvolatile memory (NVM) devices. Meanwhile, some CMOS TCAMs using dynamic memories have been also proposed. Although dynamic TCAM could be denser than the 16T SRAM TCAM and more reliable than the nonvolatile TCAMs, CMOS dynamic TCAMs still suffer from the row-by-row refresh energy and time overheads. In this article, we propose dynamic TCAM using nanoelectromechanical (NEM) relays (DyTAN), and utilize one-shot refresh (OSR) to solve the memory refresh problem. By exploiting the unique NEM relay characteristics, DyTAN outperforms the existing works in the balance between density, speed, and power efficiency. Compared with the 16T SRAM-based TCAM, the 5T CMOS dynamic TCAM, the 2T2R TCAM, and the 2FeFET TCAM, evaluations show that the proposed DyTAN reduces the write energy by up to$2.3\times $,$1.3\times $,$131\times $, and$13.5\times $, and improves the search energy-delay-product (EDP) by up to$12.7\times $,$1.7\times $,$1.3\times $, and$2.8\times $, respectively.

With the fast evolvement of deep-learning specific embedded computing systems, applications powered by deep learning are moving from the cloud to the edge. When deploying NNs onto the edge devices under complex environments, there are various types of possible faults: soft errors caused by atmospheric neutrons and radioactive impurities, voltage instability, aging, temperature variations, and malicious attackers. Thus the safety risk of deploying neural networks at edge computing devices in safety-critic applications is now drawing much attention. In this paper, we implement the random bit-flip, Gaussian, and Salt-and-Pepper fault models and establish a multi-objective fault-tolerant neural architecture search framework. On top of the NAS framework, we propose Fault-Tolerant Neural Architecture Search (FT-NAS) to automatically discover convolutional neural network (CNN) architectures that are reliable to various faults in nowadays edge devices. Then we incorporate fault-tolerant training (FTT) in the search process to achieve better results, which we called FTT-NAS. Experiments show that the discovered architecture FT-NAS-Net and FTT-NAS-Net outperform other hand-designed baseline architectures (58.1%/86.6% VS. 10.0%/52.2%), with comparable FLOPs and less parameters. What is more, the architectures trained under a single fault model can also defend against other faults. By inspecting the discovered architecture, we find that there are redundant connections learned to protect the sensitive paths. This insight can guide future fault-tolerant neural architecture design, and we verify it by a modification on ResNet-20-ResNet-M.

Convolutional Neural Networks (CNNs) have made breakthroughs in various fields, while the energy consumption becomes enormous. Processing-In-Memory (PIM) architectures based on emerging non-volatile memory (e.g., Resistive Random Access Memory, RRAM) have demonstrated great potential in improving the energy efficiency of CNN computing. However, there is still much room for improvement in the energy efficiency of existing PIM architectures. On the one hand, current work shows that high resolution Analog-to-Digital Converters (ADCs) are required for maintaining computing accuracy, but they dominate more than 60% energy consumption of the entire system, damaging the energy efficiency benefits of PIM. On the other hand, the characteristic of computing in the analog domain in PIM accelerators leads to the computing energy consumption is influenced by the specific input and weight values. However, as far as we know, there is no energy efficiency optimization method based on this characteristic in existing work. To solve these problems, in this paper, we propose an energy-efficient quantized and regularized training framework for PIM accelerators, which consists of a PIM-based non-uniform activation quantization scheme and an energy-aware weight regularization method. The proposed framework can improve the energy efficiency of PIM architectures by reducing the ADC resolution requirements and training low energy consumption CNN models for PIM, with little accuracy loss. The experimental results show that the proposed training framework can reduce the resolution of ADCs by 2 bits and the computing energy consumption in the analog domain by 35%. The energy efficiency, therefore, can be enhanced by $3.4 \times$ in our proposed training framework.

Ferroelectric FETs (FeFETs) have emerged as a promising multi-level/cell (MLC) nonvolatile memory (NVM) candidate for low-power applications. This originates from the advantages of both efficient memory access and intrinsic device-level in-memory computing flexibilities. However, there still exist challenges for FeFET MLC NVM: (i) high power consumption in read operations due to high-gain requirement for sense amplifiers during sensing, and (ii) high latency and energy consumption in write operations with conventional recursive program-and-verify. Targeting at lower power, less latency, and higher density, this work investigates and optimizes the read and write approaches to MLC FeFET NVM design: (i) Adaptive FeFET memory State Mapping (ASM) between the FeFET drain-source current and the digital states to increase the sensing margin; (ii) Adaptive FeFET Gate Biasing (AGB) read methods that adopt the optimized FeFET gate voltage to boost the sensible dynamic range and to store more levels of states per cell; (iii) Adaptive Prediction-based Direct (APD) write methods that minimize the program-andverify activities. Evaluations show significant latency and energy improvement. Furthermore, the number of sensible levels of states per cell is also increased with an enhanced dynamic sensing range and an enhanced sensing margin.

Neural Architecture Search (NAS) is a promising approach to discover good neural network architectures for given applications. Among the three basic components in a NAS system (search space, search strategy, and evaluation), prior work mainly focused on the development of different search strategies and evaluation methods. As most of the previous hardware-aware search space designs aimed at CPUs and GPUs, it still remains a challenge to design a suitable search space for Deep Neural Network (DNN) accelerators. Besides, the architectures and compilers of DNN accelerators vary greatly, so it is quite difficult to get a unified and accurate evaluation of the latency of DNN across different platforms. To address these issues, we propose a black box profiling-based search space tuning method and further improve the latency evaluation by introducing a layer adaptive latency correction method. Used as the first stage in our general accelerator-aware NAS pipeline, our proposed methods could provide a smaller and dynamic search space with a controllable trade-off between accuracy and latency for DNN accelerators. Experimental results on CIFAR-10 and ImageNet demonstrate our search space is effective with up to 12.7% improvement in accuracy and 2.2x reduction of latency, and also efficient by reducing the search time and GPU memory up to 4.35x and 6.25x, respectively.

Recently CNN-based methods have made remarkable progress in broad fields. Both network pruning algorithms and hardware accelerators have been introduced to accelerate CNN. However, existing pruning algorithms have not fully studied the pattern pruning method, and current index storage scheme of sparse CNN is not efficient. Furthermore, the performance of existing accelerators suffers from no-load PEs on sparse networks. This work proposes a software-hardware co-design to address these problems. The software includes an ADMM-based method which compresses the patterns of convolution kernels with acceptable accuracy loss, and a Huffman encoding method which reduces index storage overhead. The hardware is a fusion-enabled systolic architecture, which can reduce PEs' no-load rate and improve performance by supporting the channel fusion. On CIFAR-10, this work achieves 5.63× index storage reduction with 2-7 patterns among different layers with 0.87% top-1 accuracy loss. Compared with the state-of-art accelerator, this work achieves 1.54×-1.79× performance and 25%-34% reduction of no-load rate with reasonable area and power overheads.

Deploying intelligent visual algorithms in terminal devices for always-on sensing is an attractive trend in the IoT era. In-sensor-processing architecture is proposed to reduce power consumption on A/D conversion and data transmission, which performs pre-processing and only converting low-throughput features. However, current designs still require high energy consumption on photoelectric conversion and analog data movement. In this paper, two methods are proposed to improve the energy efficiency of in-sensor-processing architecture, including direct photocurrent computation and 2D kernel scheduling. Photocurrents are directly involved in computation to avoid data conversion; thus the indispensable imaging power is also utilized for computing. Since the location of the pixel data is fixed, data scheduling is conducted on digital weights to eliminate analog data storage and movement. We implement a prototype chip with an array of 32 × 32 units to calculate the first layer of binarized LeNet-5. The post-simulation shows that the proposed architecture reaches the energy efficiency of 11.49TOPs/W, about 14.8x higher than previous works.

In recent years, Convolutional Neural Network (CNN) has been widely used in robotics, which has dramatically improved the perception and decision-making ability of robots. A series of CNN accelerators have been designed to implement energy-efficient CNN on embedded systems. However, despite the high energy efficiency on CNN accelerators, it is difficult for robotics developers to use it. Since the various functions on the robot are usually implemented independently by different developers, simultaneous access to the CNN accelerator by these multiple independent processes will result in hardware resources conflicts.To handle the above problem, we propose an INterruptible CNN Accelerator (INCA) to enable multi-tasking on CNN accelerators. In INCA, we propose a Virtual-Instruction-based interrupt method (VI method) to support multi-task on CNN accelerators. Based on INCA, we deploy the Distributed Simultaneously Localization and Mapping (DSLAM) on an embedded FPGA platform. We use CNN to implement two key components in DSLAM, Feature-point Extraction (FE) and Place Recognition (PR), so that they can both be accelerated on the same CNN accelerator. Experimental results show that, compared to the layer-by-layer interrupt method, our VI method reduces the interrupt respond latency to 1%.

This paper proposes low-power design of embedded dynamic random-access memory (eDRAM) using emerging nanoelectromechanical (NEM) relay devices. The motivation of this work is to reduce the standby refresh power consumption through the improvement of retention time of eDRAM cells. In this paper, it is revealed that the tunable beyond-CMOS characteristics of emerging NEM relay devices, especially the ultra-high OFF-state drain-source resistance, open up new opportunities with device-circuit co-design. In addition, the pull- in and pull-out threshold voltages are tilled to fit the operating mechanisms of eDRAM, so as to support low-voltage operations along with long retention time. Excitingly, when low-gate-leakage thick-gate transistors are used together, the proposed NEM-relay- based eDRAM exhibits so significant retention time improvement that it behaves almost "nonvolatile". Even if using thin-gate transistors in a 130nm CMOS, the evaluation of the proposed eDRAM shows up to 63x and 127x retention time improvement at 1.0V and 1.4V supply, respectively. Detailed performance benchmarking analysis, along with the practical CMOS- compatible NEM relay model, the eDRAM design and optimization considerations, is included in this paper.

Feature-point extraction is a fundamental step in many applications, such as image matching and Simultaneous Localization and Mapping (SLAM). The CNN-based feature-point extraction methods have made significant signs of progress in both feature-point detection and descriptor generation compared with handcrafted processes. However, the computational and storage complexity makes it difficult for CNN to run on real-time embedded systems. In this paper, we aim to deploy the advanced CNN-based feature-point extraction methods onto real-time embedded FPGA systems. We optimize the softmax data flow so that the computation of softmax and NMS can be reduced by 64×. We generate the normalized descriptors after picking the feature-points with the highest confidence so that the computation cost of normalization is reduced by 1500×. We use fixed-point in both of the CNN backbone and the postprocessing operations, and implement them on the ZCU102 FPGA platform. The experimental results show that our proposed hardware-software co-design CNN-based feature-point extraction method outperforms the handcrafted techniques. Our feature-point extraction on the embedded platform runs at the speed of 20 fps, meeting the real-time requirement.

FPGAs have shown great potential in providing low-latency and energy-efficient solutions for deep neural network (DNN) inference applications. Currently, the majority of FPGA-based DNN accelerators in the cloud run in a time-division multiplexing way for multiple users sharing a single FPGA, and require re-compilation with $\sim$100s overhead. Such designs lead to poor isolation and heavy performance loss for multiple users, which are far away from providing efficient and flexible FPGA virtualization for neither public nor private cloud scenarios. To solve these problems, we introduce a novel virtualization framework for instruction architecture set (ISA) based on DNN accelerators by sharing a single FPGA. We enable the isolation by introducing a two-level instruction dispatch module and a multi-core based hardware resources pool. Such designs provide isolated and runtime-programmable hardware resources, further leading to performance isolation for multiple users. On the other hand, to overcome the heavy re-compilation overheads, we propose a tiling-based instruction frame package design and two-stage static-dynamic compilation. Only the light-weight runtime information is re-compiled with $\sim$1 ms overhead, thus the performance is guaranteed for the private cloud. Our extensive experimental results show that the proposed virtualization design achieves 1.07-1.69x and 1.88-3.12x throughput improvement over previous static designs using the single-core and the multi-core architectures, respectively.

Multi-Robot Exploration (MR-Exploration) that provides the location and map is a basic task for many multi-robot applications. Recent researches introduce Convolutional Neural Network (CNN) to critical components in MR-Exploration, like Feature-point Extraction (FE) and Place Recognition (PR), to improve the system performance. Such CNN-based MR-Exploration requires running multiple CNN models simultaneously, together with complex post-processing algorithms, greatly challenges the hardware platforms, which are usually embedded systems. Previous researches have shown that FPGA is a good candidate for CNN processing on embedded platforms. But such accelerators usually process different models sequentially, lacking the ability to schedule multiple tasks at runtime. Furthermore, post-processing of CNNs in FE is also computation consuming and becomes the system bottleneck after accelerating the CNN models. To handle such problems, we propose an INterruptible CNN Accelerator for Multi-Robot Exploration (INCAME) framework for rapid deployment of robot applications on FPGA. In INCAME, we propose a virtual-instruction-based interrupt method to support multi-task on CNN accelerators. INCAME also includes hardware modules to accelerate the post-processing of the CNN-based components. Experimental results show that INCAME enables multi-task scheduling on the CNN accelerator with negligible performance degradation (0.3%). With the help of multi-task supporting and post-processing acceleration, INCAME enables embedded FPGA to execute MR-Exploration in real time (20 fps).

FPGAs have shown great potential in providing low-latency and energy-efficient solutions for deep learning applications, especially for the deep neural network (DNN). Currently, the majority of FPGA based DNN accelerators are designed for single-task and static-workload applications, making it difficult to adapt to the multi-task and dynamic-workload applications in the cloud. To meet these requirements, DNN accelerators need to support multi-task concurrent execution and low-overhead runtime resources reconfiguration. However, neither instruction set architecture (ISA) based nor template-based FPGA accelerators can support both functions at the same time. In this paper, we introduce a novel FPGA virtualization framework for ISA-based DNN accelerators in the cloud. As for the design goals of supporting multi-task and runtime reconfiguration, we propose a two-level instruction dispatch module and deep learning hardware resources pooling technique at the hardware level. As for the software level, we propose a tiling-based instruction frame package design and two-stage static-dynamic compilation. Furthermore, we propose a history information aware scheduling algorithm for the proposed ISA-based deep learning accelerators in the cloud scenario. According to our evaluation on Xilinx VU9P FPGA, the proposed virtualization method achieves 1.88x to 2.20x higher throughput and 1.36x to 1.77x lower latency against the static baseline design.

Memristor based neuromorphic computing systems give alternative solutions to boost the computing energy efficiency of Neural Network (NN) algorithms. Because of the large-scale applications and the large architecture design space, many factors will affect the computing accuracy and system's performance. In this work, we propose a behavior-level modeling tool for memristor-based neuromorphic computing systems, MNSIM 2.0, to model the performance and help researchers to realize an early-stage design space exploration. Compared with the former version and other benchmarks, MNSIM 2.0 has the following new features: 1. In the algorithm level, MNSIM 2.0 supports the inference accuracy simulation for mixed-precision NNs considering non-ideal factors. 2. In the architecture level, a hierarchical modeling structure for PIM systems is proposed. Users can customize their designs from the aspects of devices, interfaces, processing units, buffer designs, and interconnections. 3. Two hardware-aware algorithm optimization methods are integrated in MNSIM 2.0 to realize software-hardware co-optimization.

Although compression techniques like pruning or quantization are beneficial for accelerators' energy efficiency, the large search space makes finding the appropriate compression scheme difficult. Besides, most existing works ignore the combination of both pruning and quantization. In this paper, we propose a reinforcement learning (RL) based joint compression framework to find the appropriate pruning ratio and quantization bit-width for accelerators. By interacting with the energy model of the target accelerator, the RL agent can learn the effect of compression scheme on both accuracy and energy efficiency. Through a long trial-and-error process, the agent can finally reach an optimal trade-off between accuracy and energy efficiency. Compared with control groups whose compression hyper-parameters are not jointly optimized, the proposed framework can achieve at least 25% energy reduction with higher accuracy or much higher accuracy with small disadvantages on energy. Compared with 8-bit quantized baseline, the framework can achieve 90% and 85% energy reduction on Cifar10 and Cifar100 respectively.

Flexible capacitive pressure sensing system (FCPSS) is promising in the area of healthcare, robotics, and Internet of Things (IoT). As the size of the sensing array increases, designing energy-efficient FCPSS is getting challenging. This work provides a comprehensive solution for low-power FCPSS design, where major contributions are as follows. 1) Crosstalk-induced measurement error in a crossbar structure FCPSS is first studied and an accurate and low-power linear iterative algorithm is proposed for on-chip sensing array calibration (SAC). 2) Binary Neural Network (BNN)-based spatial-temporal adaptive sensing scheme for FCPSS is first proposed to utilize the sparsity of sampling and to further improve energy efficiency. Combined with the clock-gating-friendly low-power sensor interface, the system consumes 31.39 μJ energy and gains 95.04% capacitor measurement accuracy for each sensing operation on a 10×10 array, achieving 116× energy reduction compared with the state-of-the-art technology.

Compute-in-memory (CiM) is a promising method for mitigating the memory wall problem in data-intensive applications. The proposed bitwise logic-in-memory (BLiM) is targeted at data intensive applications, such as database, data encryption. This work proposes a low-power BLiM approach using the emerging nonvolatile ferroelectric FETs with direct write-back and data-adaptive dynamic sensing interface. Apart from general-purpose random-access memory, it also supports BLiM operations such as copy, not, nand, xor, and full adder (FA). The novel features of the proposed architecture include: (i) direct result-write-back based on the remnant bitline BLiM charge that avoids bitline sensing and charging operations; (ii) a fully dynamic sensing interface that needs no static reference current, but adopts data-adaptive voltage references for certain multi-operand operations, and (iii) selective bitline charging from wordline (instead of pre-charging all bitlines) to save power and also enable direct write-back. Detailed BLiM operations and benchmarking against conventional approaches show the promise of low-power computing with the FeFET-based circuit techniques.

Keyword spotting (KWS) is a crucial front-end module in the whole speech interaction system. The always-on KWS module detects input words, then activates the energy-consuming complex backend system when keywords are detected. The performance of the KWS determines the standby performance of the whole system and the conventional KWS module encounters the power consumption bottleneck problem of the data conversion near the microphone sensor. In this paper, we propose an energy-efficient near-sensor processing architecture for always-on KWS, which could enhance continuous perception of the whole speech interaction system. By implementing the keyword detection in the analog domain after the microphone sensor, this architecture avoids energy-consuming data converter and achieves faster speed than conventional realizations. In addition, we propose a lightweight gated recurrent unit (GRU) with negligible accuracy loss to ensure the recognition performance. We also implement and fabricate the proposed KWS system with the CMOS 0.18μm process. In the system-view evaluation results, the hardware-software co-design architecture achieves 65.6% energy consumption saving and 71 times speed up than state of the art.

Activation I/O traffic is a critical bottleneck of video neural network processor. Recent works adopted an inter-frame difference method to reduce activation size. However, current methods can't fully adapt to the various precision and sparsity in differential data. In this paper, we propose the multi-channel precision-sparsity-adapted codec, which will separate the differential activation and encode activation in multiple channels. We analyze the most adapted encoding of each channel, and select the optimal channel number with the best performance. A two-channel codec hardware has been implemented in the ASIC accelerator, which can encode/decode activations in parallel. Experiment results show that our coding achieves 2.2x-18.2x compression rate in three scenarios with no accuracy loss, and the hardware has 42x/174x improvement on speed and energy-efficiency compared with the software codec.

The acceleration of Graph Neural Networks (GNNs) requires efficient and framework-compatible Sparse-Dense Matrix-Matrix Multiplication (SpMM). From the compatibility perspective, the sophisticated sparse matrix representations in state-of-the-art SpMM designs cause heavy preprocessing overhead for the framework. From the efficiency perspective, optimizations for SpMV (Sparse Matrix-Vector) do not apply well to SpMM, leading to redundant and uncoalesced global memory access. We propose GE-SpMM1, which takes the CSR format consistent with GNN frameworks to enable integration without the format transformation overhead. We use Coalesced Row Caching to ensure coalesced access to both sparse and dense data in the global memory. We use Coarse-grained Warp Merging to reduce redundant data loading among GPU warps. Experiments on a real-world graph dataset demonstrate up to 1.41× speedup over Nvidia cuSPARSE [1] and up to 1.81× over GraphBLAST [2]. We embed GE-SpMM in GNN frameworks and get up to 3.67× speedup on popular GNN models like GCN [3] and GraphSAGE [4].

Neural network (NN) computing is energy-consuming on traditional computing systems, owing to the inherent memory wall bottleneck of the von Neumann architecture and the Moore's Law being approaching the end. Non-volatile memories (NVMs) have been demonstrated as promising alternatives for constructing computing-in-memory (CIM) systems to accelerate NN computing. However, NVM-based NN computing systems are vulnerable to the confidentiality attacks because the weight parameters persist in memory when the system is powered off, enabling an adversary with physical access to extract the well-trained NN models. The goal of this article is to find a solution for thwarting the confidentiality attacks. We define and model the weight encryption problem. Then we propose an effective framework, containing a sparse fast gradient encryption (SFGE) method and a runtime encryption scheduling (RES) scheme, to guarantee the confidentiality security of NN models with a negligible performance overhead. Moreover, we improve the SFGE method by incrementally generating the encryption keys. Additionally, we provide variants of the encryption method to better fit quantized models and various mapping strategies. The experiments demonstrate that only encrypting an extremely small proportion of the weights (e.g., 20 weights per layer in ResNet-101), the NN models can be strictly protected.

Nonvolatile memory (NVM)-based training-in-memory (TIME) systems have emerged that can process the neural network (NN) training in an energy-efficient manner. However, the endurance of NVM cells is disappointing, rendering concerns about the lifetime of TIME systems, because the weights of NN models always need to be updated for thousands to millions of times during training. Gradient sparsification (GS) can alleviate this problem by preserving only a small portion of the gradients to update the weights. However, conventional GS will introduce nonuniform writes on different cells across the whole NVM crossbars, which significantly reduces the excepted available lifetime. Moreover, an adversary can easily launch malicious training tasks to exactly wear-out the target cells and fast break down the system. In this article, we propose an efficient and effective framework, referred as SGS-ARS, to improve the lifetime and security of TIME systems. The framework mainly contains a structured GS (SGS) scheme for reducing the write frequency, and an aging-aware row swapping (ARS) scheme to make the writes uniform. Meanwhile, we show that the back-propagation mechanism allows the attacker to localize and update fixed memory locations and wear them out. Therefore, we introduce Random-ARS and Refresh techniques to thwart adversarial training attacks, preventing the systems from being fast broken in an extremely short time. Our experiments show that when TIME is programmed to train ResNet-50 on ImageNet dataset, $356\times $ lifetime extension can be achieved without sacrificing the accuracy much or incurring much hardware overhead. Under the adversarial environment, the available lifetime of TIME systems can still be improved by $84\times $ .

The emerging resistive random-access memory (RRAM) has been widely applied in accelerating the computing of deep neural networks. However, it is challenging to achieve highprecision computations based on RRAM due to the limits of the resistance level and the interfaces. Low bit-width convolutional neural networks (CNNs) provide promising solutions to introduce low bit-width RRAM devices and low bit-width interfaces in RRAM-based computing system (RCS). While open questions still remain regarding: 1) how to make matrix splitting when a single crossbar is not large enough to hold all parameters of one weight matrix; 2) how to design a pipeline to accelerate the inference based on line buffer structure; and 3) how to reduce the accuracy drop due to the parameter splitting and data quantization. In this paper, we propose an RRAM crossbar-based low bit-width CNN (LB-CNN) accelerator. We make detailed discussion on the system design, including the matrix splitting strategies to enhance the scalability, and the pipelined implementation based on line buffers to accelerate the inference. In addition, we propose a splitting and quantizing while training method to incorporate the actual hardware constraints with the training. In our experiments, low bit-width LeNet-5 on RRAM show much better robustness than multibit models with device variation. The pipeline strategy achieves approximately 6.0× speedup to process each image on ResNet-18. For low-bit VGG-8 on CIFAR-10, the proposed accelerator saves 54.9% of the energy consumption and 48.3% of the area compared with the multibit VGG-8 structure.

Convolutional neural networks (CNNs) have become powerful algorithms in various tasks. Application-specific integrated circuit (ASIC) has been widely used to accelerate CNN on mobile platforms because of its tremendous energy efficiency and performance. Meanwhile, CNNs have become much sparser with the development of network pruning algorithms. Recent works have employed different methods to improve the energy efficiency and performance of ASIC accelerators by utilizing the sparsity character of CNN. However, some of these methods suffer from tremendous output memory overhead and performance degradation induced by hash collisions. To overcome the aforementioned problem, we propose GAAS: an efficient group associated architecture and scheduler module for sparse CNN accelerators. It achieves smaller output memory overhead and higher performance compared with the state-of-the-art accelerator. Our proposed method GAAS mainly consists of two parts: 1) an$n$-way group associated architecture to reduce the output memory overhead and 2) a scheduler module to improve the performance. Besides, a load-balancing algorithm is proposed and implemented in the scheduler module to improve the performance by reducing the hash collision rate. To demonstrate the efficiency of GAAS, we implement a 4-way image-principal associated architecture with a$16\times 16$PE array and the scheduler module on our proposed method. The experimental results on AlexNet, VGG16, ResNet18, and MobileNet show that GAAS can reduce the output memory overhead by 50%, and it can surely improve the performance of them by$1.53\times $,$1.62\times $,$1.46\times $, and$1.55\times $, respectively.

The scale-invariant feature transform (SIFT) algorithm is still one of the most reliable image feature extraction methods. Despite its excellent robustness on various image transformations, SIFT's intensive computational burden has been severely preventing it from being used in real-time and energy-efficient embedded machine vision systems. To reduce processing time and energy cost while executing SIFT, an analog signal processing architecture, analog signal processing (ASP)SIFT, is proposed in this article. In ASP-SIFT, the Gaussian pyramid construction, difference-of-Gaussian (DoG) pyramid construction and keypoint locating, which are the primary steps of the keypoint detection part of the SIFT algorithm, are done directly with analog circuit networks. Thus, by completing keypoint detection in the analog domain, the total processing time is approximately equal to the settling time of the circuit network. Besides, by adopting a current-mode circuit network operating in the subthreshold region, the power dissipation would be very low. Simulation results show that the total processing speed for a typical video graphics array (VGA)-format (640 × 480) image is up to 2.3 kframes per second, which is at least 3.26× faster than the state-of-the-art digital hardware accelerators, while the system power is 94.5 mW and the energy consumption is only 40 μJ per frame.

Large-scale graph processing has drawn great attention in recent years. The emerging metal-oxide resistive random access memory (ReRAM) and ReRAM crossbars have shown huge potential in accelerating graph processing. However, the sparse feature of natural graphs hinders the performance of graph processing on ReRAMs. Previous work of graph processing on ReRAMs stored and computed edges separately, leading to high energy consumption and long latency of transferring data. In this paper, we present GraphSAR, a sparsity-aware processing-in-memory large-scale graph processing accelerator on ReRAMs. Computations over edges are performed in the memory, eliminating overheads of transferring edges. Moreover, graphs are divided considering the sparsity. Subgraphs with low densities are further divided into smaller ones to minimize the waste of memory space. According to our extensive experimental results, GraphSAR achieves 4.43x energy reduction and 1.85x speedup (8.19x lower energy-delay product, EDP) against previous graph processing architecture on ReRAMs (GraphR [1]).

In recent years, ASIC CNN Accelerators have attracted great attention among researchers for the high performance and energy efficiency. Some former works utilize the sparsity of CNN networks to improve the performance and the energy efficiency. However, these methods bring tremendous overhead to the output memory, and the performance suffers from the hash collision. This paper presents: 1) an N-Way Group Association Architecture to reduce the memory overhead for Sparse CNN Accelerators; 2) a Sparse Data Group Association Load Balancing Algorithm which is implemented by the Scheduler module in the architecture to reduce the collision rate and improve the performance. Compared with the state-of-art accelerator, this work achieves either 1) 1.74x performance with 50% memory overhead reduction in the 4-way associated design or 2) 1.91x performance without memory overhead reduction the 2-way associated design, which is close to the theoretical performance limit (without collision).

ReRAM-based processing-in-memory (PIM) architecture is a promising solution for deep neural networks (NN), due to its high energy efficiency and small footprint. However, traditional PIM architecture has to use a separate crossbar array to store either positive or negative (P/N) weights, which limits both energy efficiency and area efficiency. Even worse, imbalance running time of different layers and idle ADCs/DACs even lower down the whole system efficiency. This paper proposes AERIS, an Area/Energy-efficient 1T2R ReRAM based processing-In-memory NN System-on-a-chip to enhance both energy and area efficiency. We propose an area-efficient 1T2R ReRAM structure to represent both P/N weights in a single array, and a reference current cancelling scheme (RCS) is also presented for better accuracy. Moreover, a layer-balance scheduling strategy, as well as the power gating technique for interface circuits, such as ADCs/DACs, is adopted for higher energy efficiency. Experiment results show that compared with state-of-the-art ReRAM-based architectures, AERIS achieves 8.5x/1.3x peak energy/area efficiency improvements in total, due to layer-balance scheduling for different layers, power gating of interface circuits, and 1T2R ReRAM circuits. Furthermore, we demonstrate that the proposed RCS compensates the non-ideal factors of ReRAM and improves NN accuracy by 5.2% in the XNOR net on CIFAR-10 dataset.

Making embedded memory symmetric provides the capability of memory access in both rows and columns, which brings new opportunities of significant energy and time savings if only a portion of data in the words need to be accessed. This work investigates the use of ferroelectric field-effect transistors (FeFETs), an emerging nonvolatile, low-power, deeply-scalable, CMOS-compatible transistor technology, and proposes a new 3-transistor/cell symmetric nonvolatile memory (SymNVM). With ~1.67x higher density as compared with the prior FeFET design, significant benefits of energy and latency improvement have been achieved, as evaluated and discussed in depth in this paper.

Convolutional Neural Networks (CNNs) play a vital role in machine learning. Emerging resistive random-access memories (RRAMs) and RRAM-based Processing-In-Memory architectures have demonstrated great potentials in boosting both the performance and energy efficiency of CNNs. However, restricted by the immature process technology, it is hard to implement and fabricate a CNN accelerator chip based on multi-bit RRAM devices. In addition, existing single bit RRAM based CNN accelerators only focus on binary or ternary CNNs which have more than 10% accuracy loss compared with full precision CNNs. This paper proposes a configurable multi-precision CNN computing framework based on single bit RRAM, which consists of an RRAM computing overhead aware network quantization algorithm and a configurable multi-precision CNN computing architecture based on single bit RRAM. The proposed method can achieve equivalent accuracy as full precision CNN but also with lower storage consumption and latency via multiple precision quantization. The designed architecture supports for accelerating the multi-precision CNNs even with various precision among different layers. Experiment results show that the proposed framework can reduce 70% computing area and 75% computing energy on average, with nearly no accuracy loss. And the equivalent energy efficiency is 1.6 ~ 8.6× compared with existing RRAM based architectures with only 1.07% area overhead.

Training convolutional neural network (CNN) usually requires large amount of computation resource, time and power. Researchers and cloud service providers in this region needs fast and efficient training system. GPU is currently the best candidate for CNN training. But FPGAs have already shown good performance and energy efficiency as CNN inference accelerators. In this work, we design a compressed training process together with an FPGA-based accelerator for energy efficient CNN training. We adopt two of the widely used model compression methods, quantization and pruning, to accelerate CNN training process. The difference between inference and training brought challenges to apply the two methods in training. First, training requires higher data precision. We use the gradient accumulation buffer to achieve low operation complexity while keeping gradient descent precision. Second, sparse network results in different types of functions in forward and back-propagation phases. We design a novel architecture to utilize both inference and back-propagation sparsity. Experimental results show that the proposed training process achieves similar accuracy compared with traditional training process with floating point data. The proposed accelerator achieves 641GOP/s equivalent performance and 2.86x better energy efficiency compared with GPU.

Neural Networks (NNs) have made a significant breakthrough in many fields, while they also pose a great challenge to hardware platforms since the state-of-the-art neural networks are both communicational- and computational-intensive. Researchers proposed model compression algorithms using sparsification and quantization, along with specific hardware architecture designs, to accelerate various applications. However, the irregularity of memory access caused by the sparsity severely damages the regularity of intensive computation loops. Therefore, the architecture design for sparse neural networks is crucial to better software and hardware co-design for neural network applications. To face these challenges, this paper first analyzes the computation patterns of different NN structures and unify them into the form of sparse matrix-vector multiplication, sparse matrix-matrix multiplication, and element-wise multiplication. On the basis of the EIE which supports only the fully-connected network and recurrent neural network (RNN), we expand it to support the convolution neural network (CNN) using the input vector transform unit. This paper designs a multi-precision multiplier with supporting datapath, which makes the proposed architecture have a better acceleration effect in the low-bit quantization with the same hardware architecture. The proposed accelerator architecture can achieve the equivalent performance and energy efficiency up to 574.2 GOPS, 42.8 GOPS/W for CNN and 110.4 GOPS, 8.24 GOPS/W for RNN under 4-bit quantization on Xilinx XCKU115 FPGA running at 200MHz. And it is the state-of-the-art accelerator supporting CNN-RNN-based models like the long-term recurrent convolutional network with 571.1 GOPS performance and 42.6 GOPS/W energy efficiency under 4-bit data format.

High-precision digital-to-analog converter (DAC) is a critical component in process control, data acquisition, and testing instruments. In order to achieve high resolution and a wide-temperature range, conventional designs have been adopting high-cost thin-film resistors with laser-trimming to improve the matching property and thus the DAC resolution. In this work, targeting at lowering the cost, we propose an analog resistor redundancy, full code, and piecewise-linear calibration scheme to enable the use of low-cost poly resistors in a standard CMOS process. In order to overcome the drift over a wide temperature range, a feedback circuit is proposed to guarantee that the resistance of the switch can track the resistor with temperature changes. Therefore, the DAC can be calibrated under a specific temperature and tested under an arbitrary temperature from -40℃ to 85℃ using the same calibration codes. The 16b DAC was implemented in a 0.25μm 5V CMOS process with 5V CMOS devices and poly resistors rather than thin film resistor. The test result shows that it can achieve the INL≤0.5LSB, INL≤4LSB, and INL≤4LSB at 25℃, -40℃, and 85℃, respectively, using the same calibration code. It settles in 1μs and it has below 5nV ⋅ s glitch. The current consumption is 1.7mA from 5V voltage supplies.

Approximate computing has drawn considerable attention to both academia and industry in the area of DNN hardware. Despite substantial efforts to design approximate circuits and building blocks, the resilience of DNN layers and structures remains an untapped field to explore. This paper presents an efficient framework to evaluate DNN resilience with fine-grained approximate operations, such as multipliers, adders and low-bit operators. The framework can execute large-scale approximate DNNs with relatively less time overhead. Massive experiments are conducted with the proposed framework to reveal the relationship between network structures and error tolerance. Additionally, a case study of fine-tuning the approximate DNN is presented.

Neural network (NN) computing is energy-consuming on traditional computing systems, owing to the inherent memory wall bottleneck of the von Neumann architecture and the Moore's Law being approaching the end. Non-volatile memories (NVMs) have been demonstrated as promising alternatives for constructing computing-in-memory (CiM) systems to accelerate NN computing. However, NVM-based NN computing systems are vulnerable to the confidentiality attacks because the weight parameters persist in memory when the system is powered off, enabling an attacker with physical access to extract the well-trained NN models. The goal of this work is to find a solution for thwarting the confidentiality attacks. We define and model the weight encryption problem. Then we propose an effective framework, containing a sparse fast gradient encryption (SFGE) method and a runtime encryption scheduling (RES) scheme, to guarantee the confidentiality security of NN models with a negligible performance overhead. The experiments demonstrate that only encrypting an extremely small proportion of the weights (e.g., 20 weights per layer in ResNet-101), the NN models can be strictly protected.

Approximate computing is a promising paradigm to deal with large computing workloads in fault-tolerant applications, providing opportunities to improve hardware efficiency of Deep Neural Networks (DNNs). However, it is still difficult to apply highly approximate arithmetics (e.g., multipliers) to DNNs due to the effect of error accumulation and the convergence problem in re-training phase. To tackle this limitation, we propose a hardware-software co-design algorithm, namely Incremental Network Approximation (INA). By addressing the convergence problem, INA promotes fault tolerance of DNNs, and yields more tradeoffs between accuracy and implementation cost. Experiments show that the approximate inference models re-trained by INA could achieve up to 80% hardware reduction in various hardware design level, while the classification accuracy degradation is less than 2%. Moreover, the experiments also exhibit the generality of INA algorithm for applying to various approximate multiplier design.

Memristor-based logic design gives an alternative solution to improve the energy efficiency of computing systems, benefiting from combining the memory with computing units. Inspired by this thought, previous work has demonstrated various memristor-based logic families with different attributes and computation patterns. Besides, some logic synthesis tools are designed for specific memristive logic implementations. However, the poor universality and the neglect of realistic constraints in memory largely restrict the utility of these logic synthesis tools. In this paper, we propose a general logic synthesis framework for memristor-based logic design, containing a universal abstract description method for memristive logic, a mapping rules generator, and a synthesis and mapping flow. The proposed logic synthesis framework is suitable for various types of existing memristor-based logic families and takes the memory status into consideration. It is also possible to handle future memristive devices and logic families by providing the universal abstraction interface. Furthermore, we also design a circuit-partitioning-based synthesis acceleration strategy to tackle with the long synthesis time problem. Experimental results show that, our framework can generate mapping results under the restriction of limited resource, while the existing synthesis tools may fail under the same restriction, and achieve comparable synthesis results with the same resource as the existing synthesis tools, which is enough for computation and storage. And the proposed acceleration scheme can achieve ~ 1000× speedup compared with the initial one.

High energy consumption of conventional memory modules (e.g., DRAMs) hinders the further improvement of large-scale graph processing's energy efficiency. The emerging resistive random-access memory (ReRAM) has shown great potential in providing an energy-efficient memory module. However, the performance of ReRAMs suffers from data access patterns with poor locality and large amounts of written data, which are common in graph processing. In this paper, we propose HyVE, a Hybrid Vertex-Edge memory hierarchy for energy-efficient graph processing. In HyVE, we avoid random access and data written to ReRAM modules. HyVE can reduce memory energy consumption by 86.17 percent compared with conventional memory systems. We have also proposed data sharing and bank-level power-gating schemes, which improve the energy efficiency by 1.60x and 1.53x. By analyzing the graph processing model on ReRAMs, we show that ReRAMs are good for read-intensive operations in graph processing (e.g., reading edges), while ReRAM crossbars are not suitable for processing edges because of heavy writing overheads. Our evaluations show that the optimized design achieves two orders of magnitude and 5.90x energy efficiency improvement compared with the CPU-based and conventional memory hierarchy based designs, respectively. Moreover, HyVE achieves 2.83x energy reduction compared with the previous ReRAM-based graph processing architecture.

The training of neural networks (NN) is usually time-consuming and resource intensive. The emerging metaloxide resistive random-access memory (RRAM) device has shown potential for the computation of NN. RRAM crossbar structure and multibit characteristics can perform the matrix-vector product in high energy efficiency, which is the most common operation of NN. Two challenges exist for realizing training NN based on RRAM. First, the current architectures based on RRAM only support the inference in training NN and cannot perform the backpropagation (BP) and the weight update of training NN. Second, training NN requires enormous iterations to constantly update the weights for reaching the convergence. However, this weight update leads to large energy consumption because of the nonideal factors of RRAM. In this paper, we propose a training-in-memory based on RRAM (TIME) architecture and the peripheral circuit design to enable training NN on RRAM. TIME supports the BP and the weight update while maximizing the re-usage of peripheral circuits of the inference operation on RRAM. Meanwhile, a set of optimization strategies focusing on the nonideal factors are designed to reduce the cost of tuning RRAM. We explore the performance of both supervised learning (SL) and deep reinforcement learning (DRL) on TIME. A specific mapping method of DRL is also introduced to further improve energy efficiency. Simulation results show that in SL, TIME can achieve 5.3× higher energy efficiency on average compared with DaDianNao, an application-specific integrated circuits (ASIC) in CMOS technology. In DRL, TIME can perform an average 126× higher than GPU in energy efficiency. If the cost of tuning RRAM can be further reduced, TIME has the potential to boost the energy efficiency by two orders of magnitudes compared with ASIC.

Large-scale graph processing requires the high bandwidth of data access. However, as graph computing continues to scale, it becomes increasingly challenging to achieve a high bandwidth on generic computing architectures. The primary reasons include: the random access pattern causing local bandwidth degradation, the poor locality leading to unpredictable global data access, heavy conflicts on updating the same vertex, and unbalanced workloads across processing units. Processing-in-memory (PIM) has been explored as a promising solution to providing high bandwidth, yet open questions of graph processing on PIM devices remain in: 1) how to design hardware specializations and the interconnection scheme to fully utilize bandwidth of PIM devices and ensure locality and 2) how to allocate data and schedule processing flow to avoid conflicts and balance workloads. In this paper, we propose GraphH, a PIM architecture for graph processing on the hybrid memory cube array, to tackle all four problems mentioned above. From the architecture perspective, we integrate SRAM-based on-chip vertex buffers to eliminate local bandwidth degradation. We also introduce reconfigurable double-mesh connection to provide high global bandwidth. From the algorithm perspective, partitioning and scheduling methods like index mapping interval-block and round interval pair are introduced to GraphH, thus workloads are balanced and conflicts are avoided. Two optimization methods are further introduced to reduce synchronization overhead and reuse on-chip data. The experimental results on graphs with billions of edges demonstrate that GraphH outperforms DDR-based graph processing systems by up to two orders of magnitude and $5.12 {\times }$ speedup against the previous PIM design.

Thin-film transistor (TFT) circuits are important for flexible electronics which are promising in the area of wearable devices and Internet of Things. However, most flexible TFT technologies only have unipolar devices and the process variation and defective rate are relatively high, which impose challenges to TFT circuit design. In this paper, we propose a novel logic array design based on pseudo-CMOS logic to address the problems of unipolar TFT circuit design. A multilayer interconnection architecture is presented to improve the routability of circuit and the area efficiency. Cell mapping and wire routing algorithms, which aim to map the logic gates of circuit to logic array and then route the interconnection wires, are devised to improve the performance of circuit in consideration of parameter variations of TFT and meanwhile enhance the routability. The experimental results show that the proposed logic array along with design methodologies can reduce more than 80% area compared with transistor level scheme and help to improve performance significantly.

In line with the rapid development of the Internet of Things (IoT), the maintenance of on-board batteries for a trillion sensor nodes has become prohibitive both in time and costs. Energy harvesting is a promising solution to this problem. However, conventional energy-harvesting systems with storage suffer from low efficiency because of conversion loss and storage leakage. Direct supply systems without energy buffer provide higher efficiency, but fail to satisfy quality of service (QoS) due to mismatches between input power and workloads. Recently, a novel dual-channel photovoltaic power system has paved the way to achieve both high energy efficiency and QoS guarantee. This article focuses on the design-time and run-time co-optimization of the dual-channel solar power system. At the design stage, we develop a task failure rate estimation framework to balance design costs and failure rate. At run-time, we propose a task failure rate aware QoS tuning algorithm to further enhance energy efficiency. Through the experiments on both a simulation platform and a prototype board, this study demonstrates a 27% task failure rate reduction compared with conventional architectures with identical design costs. And the proposed online QoS tuning algorithm brings up to 30% improvement in energy efficiency with nearly zero failure rate penalty.

Recent research on neural networks has shown a significant advantage in machine learning over traditional algorithms based on handcrafted features and models. Neural networks are now widely adopted in regions like image, speech, and video recognition. But the high computation and storage complexity of neural network inference poses great difficulty on its application. It is difficult for CPU platforms to offer enough computation capacity. GPU platforms are the first choice for neural network processes because of its high computation capacity and easy-to-use development frameworks. However, FPGA-based neural network inference accelerator is becoming a research topic. With specifically designed hardware, FPGA is the next possible solution to surpass GPU in speed and energy efficiency. Various FPGA-based accelerator designs have been proposed with software and hardware optimization techniques to achieve high speed and energy efficiency. In this article, we give an overview of previous work on neural network inference accelerators based on FPGA and summarize the main techniques used. An investigation from software to hardware, from circuit level to system level is carried out to complete analysis of FPGA-based neural network inference accelerator design and serves as a guide to future work.

Convolutional Neural Networks (CNNs) have achieved excellent performance on various artificial intelligence (AI) applications, while a higher demand on energy efficiency is required for future AI. Resistive Random-Access Memory (RRAM)-based computing system provides a promising solution to energy-efficient neural network training. However, it's difficult to support high-precision CNN in RRAM-based hardware systems. Firstly, multi-bit digital-analog interfaces will take up most energy overhead of the whole system. Secondly, it's difficult to write the RRAM to expected resistance states accurately; only low-precision numbers can be represented. To enable CNN training based on RRAM, we propose a low-bitwidth CNN training method, using low-bitwidth convolution outputs (CO), activations (A), weights (W) and gradients (G) to train CNN models based on RRAM. Furthermore, we design a system to implement the training algorithms. We explore the accuracy under different bitwidth combinations of (A, CO, W, G), and propose a practical tradeoff between accuracy and energy overhead. Our experiments demonstrate that the proposed system perform well on low-bitwidth CNN training tasks. For example, training LeNet-5 with 4-bit convolution outputs, 4-bit weights, 4-bit activations and 4-bit gradients on MNIST can still achieve 97.67% accuracy. Moreover, the proposed system can achieve 23.0X higher energy efficiency than GPU when processing the training task of LeNet-5, and 4.4X higher energy efficiency when processing the training task of ResNet-20.

Thin-film transistor (TFT) circuits are facing the challenges of unipolar device, process variation, and yield problems, which can be addressed by pseudo-CMOS logic array with multi-layer interconnect. However, existing design methodology does not take mechanical strain and temperature into consideration which may seriously affect the carrier mobility of TFT and thus the performance of whole logic array circuits. This paper presents a novel cell mapping algorithm including intrarow mapping step and inter-row mapping step for flexible logic array to mitigate the mobility influence. Experimental results indicate that there is more than 40% performance improvement in critical path delay at best case with the proposed algorithm.

Deeper and larger Neural Networks (NNs) have made breakthroughs in many fields. While conventional CMOS-based computing platforms are hard to achieve higher energy efficiency. RRAM-based systems provide a promising solution to build efficient Training-In-Memory Engines (TIME). While the endurance of RRAM cells is limited, it's a severe issue as the weights of NN always need to be updated for thousands to millions of times during training. Gradient sparsification can address this problem by dropping off most of the smaller gradients but introduce unacceptable computation cost. We proposed an effective framework, SGS-ARS, including Structured Gradient Sparsification (SGS) and Aging-aware Row Swapping (ARS) scheme, to guarantee write balance across whole RRAM crossbars and prolong the lifetime of TIME. Our experiments demonstrate that 356× lifetime extension is achieved when TIME is programmed to train ResNet-50 on Imagenet dataset with our SGS-ARS framework.

Process Variation (PV) may cause accuracy loss of the analog neural network (ANN) processors, and make it hard to be scaled down, as well as feasibility degrading. This paper first analyses the impact of PV on the performance of ANN chips. Then proposes an in-situ transfer learning method at system level to reduce PV's influence with low-precision back-propagation. Simulation results show the proposed method could increase 50% tolerance of operating point drift and 70% ∼ 100% tolerance of mismatch with less than 1% accuracy loss of benchmarks. It also reduces 66.7% memories and has about 50× energy-efficiency improvement of multiplication in the learning stage, compared with the conventional full-precision (32bit float) training system.

High energy consumption of conventional memory modules (e.g., DRAMs) hinders the further improvement of large-scale graph processing's energy efficiency. The emerging metal-oxide resistive random-access memory (ReRAM) and ReRAM crossbar have shown great potential in providing the energy-efficient memory module. However, the performance of ReRAMs suffers from data access patterns with poor locality and large amounts of written data, which are common in graph processing. In this paper, we propose a Hybrid Vertex-Edge memory hierarchy, HyVE, to avoid random access and data written to ReRAM modules. With data allocation and scheduling over vertices and edges, HyVE reduces memory energy consumption by 69% compared with conventional memory system in graph processing. Moreover, we adopt a bank level power-gating scheme to further reduce the stand-by power. Our evaluations show that the optimized design achieves at least 2.0× improvement of energy efficiency compared with DRAM-based designs.

Emerging memristor devices like metal oxide resistive switching random access memory (RRAM) and memristor crossbar have shown great potential in computing matrix-vector multiplication. However, due to the nonlinear distribution of resistance levels in memristor devices, the state-of-the-art multi-bit cell cannot accomplish the multi-bit computing task accurately. In this paper, we propose fault-tolerant schemes to rescue memristor-based computation with nonlinear resistance levels. We classify the resistance level distributions in memristor devices into three types, and the corresponding models are proposed to analyze the computation characteristics. We propose two theoretical conditions to determine if a memristor device can support multi-bit matrix computation. For the deviated linear model, the least squares method is used to reduce the computing error. When the resistance distribution obeys the proposed power model, a logarithmic operation circuit is used to decode the multiplication results and then accomplish the computing accurately. For the exponential model, since the device cannot complete typical matrix-vector multiplication from hardware level, we propose online and offline quantization methods to make the neural computing algorithms friendly to memristor device. Simulation results show that the root-mean-square error improves around 4% with the linear model and more than 99% with the power model. After quantization, the accuracy of ResNet-18 using memristor with exponential conductance levels can be improved to the same accuracy with ideal linear devices.

In recent years, Convolutional Neural Network (CNN) has been widely applied in computer vision tasks and has achieved significant improvement in image object detection. The CNN methods consume more computation as well as storage, so GPU is introduced for real-time object detection. However, due to the high power consumption of GPU, it is difficult to adopt GPU in mobile applications like automatic driving. The previous work proposes some optimizing techniques to lower the power consumption of object detection on mobile GPU or FPGA. In the first Low-Power Image Recognition Challenge (LPIRC), our system achieved the best result with mAP/Energy on mobile GPU platforms. We further research the acceleration of detection algorithms and implement two more systems for real-time detection on FPGA with higher energy efficiency. In this paper, we will introduce the object detection algorithms and summarize the optimizing techniques in three of our previous energy efficient detection systems on different hardware platforms for object detection.

Large-scale graph processing has been widely required in various domains, including social network analysis, neural network modeling, database computing, etc. Performance of large-scale graph suffers from random and unpredictable data access pattern, which leads to drastic bandwidth degradation on caches, DRAMs, and disks. The support for high bandwidth random access makes SRAMs the promising solution for graph processing. Many FPGA based large-scale graph processing systems have been proposed in previous works and taken advantage of the SRAM resources.

Convolutional Neural Networks (CNNs) play a vital role in machine learning. CNNs are typically both computing and memory intensive. Emerging resistive random-access memories (RRAMs) and RRAM crossbars have demonstrated great potentials in boosting the performance and energy efficiency of CNNs. Compared with small crossbars, large crossbars show better energy efficiency with less interface overhead. However, conventional workload mapping methods for small crossbars cannot make full use of the computation ability of large crossbars. In this paper, we propose an Overlapped Mapping Method (OMM) and MIxed Size Crossbar based RRAM CNN Accelerator (MISCA) to solve this problem. MISCA with OMM can reduce the energy consumption caused by the interface circuits, and improve the parallelism of computation by leveraging the idle RRAM cells in crossbars. The simulation results show that MISCA with OMM can achieve 2.7× speedup, 30% utilization rate improvement, and 1.2× energy efficiency improvement on average compared with fixed size crossbars based accelerator using the conventional mapping method. In comparison with GPU platform, MISCA with OMM can perform 490.4× higher on average in energy efficiency and 20× higher on average in speedup. Compared with PRIME, an existing RRAM based accelerator, MISCA has 26.4× speedup and 1.65× energy efficiency improvement.

Focus stacking is a promising technique to extend depth of field in general photography by fusing images captured at different focusing distances. In this paper, we propose a round-trip scene-adaptive image acquisition system to automatically capture focal stack and fuse a high quality all-in-focus image. Based on scene analysis, we cover entire depth range of the scene in the forward optical scanning and refine all objects' focusing positions accurately in the backward scanning. With captured images, we firstly extract depthmap and all-in-focus image with combination of max-gradient flow and blur kernel estimation. Secondly, a superpixel-level Gaussian Fitting is proposed to determine the next location to capture. Experiments on simulated data show that our method attain high quality all-in-focus image with fewer captured images.

Convolutional Neural Network (CNN) is a powerful model for image classification, but it is insufficient to deal with the spatial variance of the input. This paper presents a Maxout-based and input-normalized transformation invariant neural network (MINTIN), which aims at addressing the nuisance variation of images and accumulating transformation invariance. We introduce an innovative module, the Normalization, and combine it with the Maxout operator. While the former focuses on each image itself, the latter pays attention to augmented versions of input, resulting in fully-utilized information. This combination, which can be inserted into existing CNN architectures, enables the network to learn invariance to rotation and scaling. While the authors of TI-POOLING acclaimed that they reached state-of-the-art results, ours reach a maximum decrease of 0.71%, 0.23% and 0.51% in error rate on MNIST-rot-12k, half-rotated MNIST and scaling MNIST, respectively. The size of the network is also significantly reduced, leading to high computational efficiency.

This paper proposes a direct digital frequency synthesizer (DDFS) based on Lookup-Table-Rotation (LUT-ROT) architecture. The DDFS takes the advantages of coarse-fine LUT and pipelined rotation units to achieve both high-speed and high-resolution. Based on the analysis of approximation error, the trade-off between accuracy and memory usage is achieved, which leads to the lowest amplitude of noise and the smallest size of LUT. Experimental results show that the DDFS achieves 11.7mW/GHz power consumption and 96dBc SFDR in 2-GHz clock frequency.

This paper proposes a novel auto loss compensation (ALC) system to enable non-contact operations for capacitive coupled body channel communication (CC-BCC). The system employs a time-division compensation mismatch indicator (CMI) to continuously monitor the compensation error, and dynamically adjust the compensation inductor through a PI controller. With the close-loop topology, the proposed ALC system has three advantages: First, the path loss induced by non-contact status and backward coupling effect can be compensated simultaneously; Second, this system can dynamically attenuate the path losses, even when the channel characteristics vary with time; Third, this system has high robustness, which is insusceptible to channel variations; The simulation results show that the loss reduction of the proposed ALC system is 18 dB higher than the conventional compensation technique in the worst case.

In modern integrated circuit (IC) designs, intellectual property (IP) cores are often outsourced and designed by third-party vendors, resulting in the partial relinquishment of the control over the IC design flow. Thus, reliable verifications are required to mitigate the threat of hardware Trojans (HTs) which may be inserted into IP cores by malicious vendors. Existing trustiness verification methods cannot take the merit of high efficiency and accuracy at the same time. In this paper, we propose a multilevel fast trustiness verification framework based on feature analysis to detect HTs in third-party digital IP cores. The proposed framework combines flip-flop level and combinational logic level feature analysis to achieve both high efficiency and accuracy. Experimental results demonstrate that both explicitly and implicitly triggered HTs can be detected in very short time with a negligible false positive rate. More importantly, our framework has the unique advantage of being scalable to defend against future and stealthier HTs by adding new features into the framework.

Convolutional neural network (CNN) has become a successful algorithm in the region of artificial intelligence and a strong candidate for many computer vision algorithms. But the computation complexity of CNN is much higher than traditional algorithms. With the help of GPU acceleration, CNN-based applications are widely deployed in servers. However, for embedded platforms, CNN-based solutions are still too complex to be applied. Various dedicated hardware designs on field-programmable gate arrays (FPGAs) have been carried out to accelerate CNNs, while few of them explore the whole design flow for both fast deployment and high power efficiency. In this paper, we investigate state-of-the-art CNN models and CNN-based applications. Requirements on memory, computation and the flexibility of the system are summarized for mapping CNN on embedded FPGAs. Based on these requirements, we propose Angel-Eye, a programmable and flexible CNN accelerator architecture, together with data quantization strategy and compilation tool. Data quantization strategy helps reduce the bit-width down to 8-bit with negligible accuracy loss. The compilation tool maps a certain CNN model efficiently onto hardware. Evaluated on Zynq XC7Z045 platform, Angel-Eye is 6× faster and 5× better in power efficiency than peer FPGA implementation on the same platform. Applications of VGG network, pedestrian detection and face alignment are used to evaluate our design on Zynq XC7Z020. NIVIDA TK1 and TX1 platforms are used for comparison. Angel-Eye achieves similar performance and delivers up to 16× better energy efficiency.

Memristor-based computation provides a promising solution to boost the power efficiency of the neuromorphic computing system. However, a behavior-level memristor-based neuromorphic computing simulator, which can model the performance and realize an early stage design space exploration, is still missing. In this paper, we propose a simulation platform for the memristor-based neuromorphic system, called MNSIM. A hierarchical structure for memristor-based neuromorphic computing accelerator is proposed to provides flexible interfaces for customization. A detailed reference design is provided for large-scale applications. A behavior-level computing accuracy model is incorporated to evaluate the computing error rate affected by interconnect lines and nonideal device factors. Experimental results show that MNSIM achieves over 7000 times speed-up than SPICE simulation. MNSIM can optimize the design and estimate the tradeoff relationships among different performance metrics for users.

With the coming of the “Big Data” era, a high-energy-efficiency database is demanded for the Internet of things (IoT) application scenarios. The emerging Resistive Random Access Memory (RRAM) has been considered as an energy-efficient replacement of DRAM for next-generation main memory. In this article, we propose an RRAM-based SQL query unit with process-in-memory (PIM) characteristics. A bidirectional storage structure for a database in RRAM crossbar array is proposed that avoids redundant data transfer to cache and reduces cache miss rate compared with the storage method in DRAM for an in-memory database. The proposed RRAM-based SQL query unit can support a representative subset of SQL queries in memory and thus can further reduce the data transfer cost. The corresponding query optimization method is proposed to fully utilize the PIM characteristics. Simulation results show that the energy efficiency of the proposed RRAM-based SQL query unit is increased by 4 to 6 orders of magnitudes compared with the traditional architecture.

In recent years, Convolutional Neural Networks (CNNs) have been widely applied in computer vision and have achieved significant improvements in object detection tasks. Although there are many optimizing methods to speed up CNN-based detection algorithms, it is still difficult to deploy detection algorithms on real-time low-power systems. Field-Programmable Gate Array (FPGA) has been widely explored as a platform for accelerating CNN due to its promising performance, high energy efficiency, and flexibility. Previous works show that the energy consumption of CNN accelerators is dominated by the memory access. By fusing multiple layers in CNN, the intermediate data transfer can be reduced. However, previous accelerators with the cross-layer scheduling are designed for a particular CNN model. In addition to the memory access optimization, the Winograd algorithm can greatly improve the computational performance of convolution. In this article, to improve the flexibility of hardware, we design an instruction-driven CNN accelerator, supporting the Winograd algorithm and the cross-layer scheduling, for object detection. We modify the loop unrolling order of CNN, so that we can schedule a CNN across different layers with instructions and eliminate the intermediate data transfer. We propose a hardware architecture to support the instructions with Winograd computation units and reach the state-of-the-art energy efficiency. To deploy image detection algorithms onto the proposed accelerator with fixed-point computation units, we adopt the fixed-point fine-tune method, which can guarantee the accuracy of the detection algorithms. We evaluate our accelerator and scheduling policy on the Xilinx KU115 FPGA platform. The intermediate data transfer can be reduced by more than 90% on the VGG-D CNN model with the cross-layer strategy. Thus, the performance of our hardware accelerator reaches 1700GOP/s on the classification model VGG-D. We also implement a framework for object detection algorithms, which achieves 2.3× and 50× in energy efficiency compared with GPU and CPU, respectively. Compared with floating-point algorithms, the accuracy of the fixed-point detection algorithms only drops by less than 1%.

The emerging metal-oxide resistive switching random-access memory (RRAM) devices and RRAM crossbar arrays have demonstrated their potential in enormously boosting the speed and energy-efficiency of analog matrix-vector multiplication. Unfortunately, due to the immature fabrication technology, commonly occurring Stuck-At-Faults (SAFs) seriously degrade the computational accuracy of RRAM crossbar based Computing System (RCS). In this paper, we propose a Mapping Algorithm with inner fault-tolerant ability (MAO) to convert matrix parameters into RRAM conductances in RCS by providing larger mapping space and fully exploring the available mapping space. Furthermore, we present two computation-oriented redundancy schemes — ‘Redundant Crossbars’ (RX) and ‘Independent Redundant Columns’ (IRC) to alleviate the loss of computational accuracy due to SAFs. RX adds redundant RRAM crossbar arrays and IRC introduces independent redundant RRAM columns to compensate the computational errors brought by SAFs.

Recent progress in the machine learning field makes low bit-level Convolutional Neural Networks (CNNs), even CNNs with binary weights and binary neurons, achieve satisfying recognition accuracy on ImageNet dataset. Binary CNNs (BCNNs) make it possible for introducing low bit-level RRAM devices and low bit-level ADC/DAC interfaces in RRAM-based Computing System (RCS) design, which leads to faster read-and-write operations and better energy efficiency than before. However, some design challenges still exist: (1) how to make matrix splitting when one crossbar is not large enough to hold all parameters of one layer; (2) how to design the pipeline to accelerate the whole CNN forward process. In this paper, an RRAM crossbar-based accelerator is proposed for BCNN forward process. Moreover, the special design for BCNN is well discussed, especially the matrix splitting problem and the pipeline implementation. In our experiment, BCNNs on RRAM show much smaller accuracy loss than multi-bit CNNs for LeNet on MNIST when considering device variation. For AlexNet on ImageNet, the RRAM-based BCNN accelerator saves 58.2% energy consumption and 56.8% area compared with multi-bit CNN structure.

The training of neural network (NN) is usually time-consuming and resource intensive. Memristor has shown its potential in computation of NN. Especially for the metal-oxide resistive random access memory (RRAM), its crossbar structure and multi-bit characteristic can perform the matrix-vector product in high precision, which is the most common operation of NN. However, there exist two challenges on realizing the training of NN. Firstly, the current architecture can only support the inference phase of training and cannot perform the backpropagation (BP), the weights update of NN. Secondly, the training of NN requires enormous iterations and constantly updates the weights to reach the convergence, which leads to large energy consumption because of lots of write and read operations. In this work, we propose a novel architecture, TIME, and peripheral circuit designs to enable the training of NN in RRAM. TIME supports the BP and the weights update while maximizing the reuse of peripheral circuits for the inference operation on RRAM. Meanwhile, a variability-free tuning scheme and gradually-write circuits are designed to reduce the cost of tuning RRAM. We explore the performance of both SL (supervised learning) and DRL (deep reinforcement learning) in TIME, and a specific mapping method of DRL is also introduced to further improve the energy efficiency. Experimental results show that, in SL, TIME can achieve 5.3x higher energy efficiency on average compared with the most powerful application-specific integrated circuits (ASIC) in the literature. In DRL, TIME can perform averagely 126x higher than GPU in energy efficiency. If the cost of tuning RRAM can be further reduced, TIME have the potential of boosting the energy efficiency by 2 orders of magnitude compared with ASIC.

Thin-film transistor (TFT) circuits are important for flexible electronics which are promising in the area of wearable devices. However, most TFT technologies only have unipolar devices and the process variation and defective rate are relatively high, which impose challenges to TFT circuit design. In this paper, we propose a novel logic array based on pseudo-CMOS logic to address the problem of unipolar TFT circuit design. A multi-layer interconnect architecture and wire routing methodology are presented to improve the routability and meanwhile the area efficiency. The experimental results show that the proposed logic array reduces more than 80% area compared with transistor level scheme.

In recent years, Convolutional Neural Network (CNN) has been widely applied in computer vision tasks. FPGAs have been widely explored to accelerate CNNs due to its high performance, high energy efficiency, and flexibility. By fusing multiple layers in CNN, the intermediate data transfer can be reduced. With a faster algorithm using Winograd transformation, the computation of convolution can be further accelerated. However, previous accelerators with cross-layer or Winograd algorithm are designed for a particular CNN model. The FPGA should be reprogrammed when running another CNN model on the hardware. In this work, we design an instruction driven CNN accelerator supporting Winograd algorithm and cross-layer scheduling. We firstly modify the cross-layer loop unrolling order to extract basic operations as instructions, and then improve the on-chip memory architecture for higher computation units utilization rate in Winograd. We evaluate the hardware architecture and scheduling policy on Xilinx Virtex-7 690t FPGA platform. As a case study, the intermediate data transfer can be reduced by over 90% on VGG-D CNN model with cross-layer policy. The performance of our hardware accelerator reaches 1500 GOP/s. Experimental results show that our design achieves a 7 χ speed-up than previous cross-layer FPGA accelerator on the same platform. The performance can be further improved by 78% if larger Winograd transformation sizes are used.

The Burrows-Wheeler Transform (BWT) has received special attention due to its effectiveness in lossless data compression algorithms. Because BWT is a time-consuming task, the efficient hardware accelerator that can yield high throughputs is required in real-time applications. This paper presents a novel BWT accelerator based on the streaming sorting network. The streaming sorting network performs the suffix sorting of large amount of data which is the most difficult task in BWT. Our BWT accelerator is implemented on a NetFPGA board. Experimental results show that it achieves 14.3X speedup compared with the state-of-art work when the data block size is 4KB. Furthermore, we design and implement a lossless data compression system based on the proposed BWT accelerator. The hardware system is composed of Burrows-Wheeler Transform module, the move-to-front encoding module, the run length encoding module, and the canonical Huffman encoding module. We evaluate the system performance on a NetFPGA board at the frequency of 155MHz. The throughput of the system could reach 179 MB/s on board when we use only one streaming sorting network for a 4KB block. The system throughput can be linearly improved up to 537 MB/s in simulation on a Virtex UltraScale xcvu440 chip if we use three streaming sorting networks to compute BWT.

Due to the large volumes of data that need to be processed, efficient memory access and data transmission are crucial for high-performance implementations of convolutional neural networks (CNNs). Approximate memory is a promising technique to achieve efficient memory access and data transmission in CNN hardware implementations. To assess the feasibility of applying approximate memory techniques, we propose a framework for the data resilience evaluation (DRE) of CNNs and verify its effectiveness on a suite of prevalent CNNs. Simulation results show that a high degree of data resilience exists in these networks. By scaling the bit-width of the first five dominant data subsets, the data volume can be reduced by 80.38% on average with a 2.69% loss in relative prediction accuracy. For approximate memory with random errors, all the synaptic weights can be stored in the approximate part when the error rate is less than 10--4, while 3 MSBs must be protected if the error rate is fixed at 10--3. These results indicate a great potential for exploiting approximate memory techniques in CNN hardware design.

Intelligent computing and maintenance-free powering are two desirable characteristics of wearable IoT devices. Energy harvesting nonvolatile intelligent processor (NIP) with neural network computation capability has the potential to advance these goals. Individual ultraviolet (UV) exposure monitoring progressively becomes one conspicuous application of wearable devices. In resource constrained wearable sensor nodes, we can alleviate the data transmission burden via convolutional neural networks (CNNs) based pattern recognition. Nevertheless, in spite of the substantially improved computing capability of NIP, typically computational and memory intensive CNNs are still too bulky for on-node implementation. We develop an CNN-based pattern recognition system for nonvolatile IoT platform for smart UV monitoring, and propose a optimization method to achieve extremely tiny and efficient CNNs. Experimental results show that the offline-trained CNN can recognize individual UV exposure patterns with accuracy of 85%, and the simplified on-node CNN can achieve 93.2% parameters reduction with only 5% accuracy loss.

Hand pose estimation from monocular depth images is an important and challenging problem for human-computer interaction. Recently deep convolutional networks (ConvNet) with sophisticated design have been employed to address it, but the improvement over traditional methods is not so apparent. To promote the performance of directly 3D coordinate regression, we propose a tree-structured Region Ensemble Network (REN), which partitions the convolution outputs into regions and integrates the results from multiple regressors on each regions. Compared with multi-model ensemble, our model is completely end-to-end training. The experimental results demonstrate that our approach achieves the best performance among state-of-the-arts on two public datasets.

A prototype of fully flexible intelligent contact lens, which are shown in the impressive action movie series of “MISSION: IMPOSSIBLE”, has become the Possible Mission in this work. Hereon, the system adopts analog-to-information processing method to build a specific Multi-Layer Perceptron network for image classification tasks with flexible devices and circuits, where the information is extracted from raw data of the sensing analog signal directly. Simulated with HSPICE of Level-62 TFT device model, for standard test image data set of MNIST, the classification accuracy of the presented flexible neural network circuit is up to 92.99%; meanwhile, the classification speed is as fast as 10k fps, and the energy consumption is low to only 15.16μJ. Additionally, for the imperfections of flexible devices of larger devices mismatch and process variations, the fault-tolerance of the system has been evaluated as well, which demonstrates the feasibility of the presented methods and lowers the barrier to integrated all kinds of FLEXIBLE Devices into a FULLY FLEXIBLE Systems with sensors, processing parts and even energy harvesting parts, etc., in the future wearable smart terminals.

Flexible electronic is a promising technology for flexible and large-area sensing IoT applications, where ADC is a fundamental component This paper proposes a configurable and flexible VCO-based ADC, implemented with Oxide Thin-Film Transistors(TFT) technology. A VCO with four connecting modes is designed to configure the VCO-based ADC working under different power and resolutions. An Inkjet printing interconnection technology is introduced to enable the configurability of ADC, even after all TFT transistors are fabricated. It allows the ADC to be customized for different applications and avoids fabrication failure of devices. Experimental results show that the proposed ADC achieves a 0.8kS/s sampling rate. Its power consumption ranges from 541 to 866uW with ENOB from 3 to 6b.

Convolutional Neural Network (CNN) has become one of the most successful technologies for visual classification and other applications. As CNN models continue to evolve and adopt different kernel sizes in various applications, it is necessary for the hardware architecture to support reconfigurability. Previous FPGAs and programmable ASICs are fine-grained reconfigurable but with energy efficiency compromise. Considering specific features of CNNs, this paper presents an energy efficient coarse-grained reconfigurable architecture, denoted as CORAL. An application-specific configuration neural block is proposed for convolution operations with reconfigurable data quantization to reduce both energy consumption and on-chip memory requirements. An optimal data loading strategy is presented for CORAL to achieve the best energy efficiency. Experimental results show that CORAL improves 80.0% energy efficiency while reduces 78.9% chip area and 81.0% reconfiguration time compared with the best up-to-date programmable ASIC solution.

Fingertip detection plays an important role in human computer interaction. Previous works transform binocular images into depth images. Then depth-based hand pose estimation methods are used to predict 3D positions of fingertips. Different from previous works, we propose a new framework, named Two-Stream Binocular Network (TSBnet) to detect fingertips from binocular images directly. TSBnet first shares convolutional layers for low level features of right and left images. Then it extracts high level features in two-stream convolutional networks separately. Further, we add a new layer: binocular distance measurement layer to improve performance of our model. To verify our scheme, we build a binocular hand image dataset, containing about 117k pairs of images in training set and 10k pairs of images in test set. Our methods achieve an average error of 10.9mm on our test set, outperforming previous work by 5.9mm (relatively 35.1%).

The continuous globalization of the semiconductor industry has significantly raised the vulnerability of chips under hardware Trojan (HT) attacks. It is extremely challenging to detect HTs in fabricated chips due to the existence of process variations (PVs), since PVs may cause larger impacts than HTs. In this paper, we propose a novel framework for HT detection in digital integrated circuits. The goal of this paper is to detect HTs inserted during fabrication. The HT detection problem is formulated as an under-determined linear system by a sparse gate profiling technique, and the existence of HTs is mapped to the sparse solution of the linear system. A Bayesian inference-based calibration technique is proposed to recover PVs for each chip for the sparse gate profiling technique. A batch of under-determined linear systems are solved together by the well-studied simultaneous orthogonal matching pursuit algorithm to get their common sparse solution. Experimental results show that even under big measurement errors, the proposed framework gets quite high HT detection rates with low measurement cost.

Nonvolatile static random access memory (nvSRAM) has been widely investigated as a promising on-chip memory architecture in energy harvesting sensor nodes, due to zero standby power, resilience to power failures, and fast read/write operations. However, conventional approaches back up all data from static random access memory into nonvolatile memory when power failures happen. It leads to significant energy overhead and peak inrush current, which has a negative impact on the system performance and circuit reliability. This paper proposes a holistic data backup optimization to mitigate these problems in nvSRAM, consisting of a partial backup algorithm and a run-time adaptive write policy. A statistic dead-block predictor is employed to achieve dead block identification with trivial hardware overhead. An adaptive policy is used to switch between write-back and write-through strategy to reduce the rollback induced by backup failures. Experimental results show that the proposed scheme improves the performance by 4.6% on average while the backup power consumption and the inrush current are reduced by 38.1% and 54% on average compared to the full backup scheme. What is more, the backup capacitor size for energy buffer can be reduced by 40% on average under the same performance constraint.

Solar-powered sensor nodes (SCSNs) with energy storages have the greatest potential and are widely used in the coming era of the Internet of Things, since they avoid tedious battery maintenance tasks. However, because the solar energy source is unstable and limited, the sensor nodes suffer from high deadline miss ratio (DMR). To achieve better DMR, the existing scheduling algorithms find the best scheduling scheme in a single period of the recurring task queue and, hence, ignore the long-term performance. To tackle this challenge, this paper proposes a three-level dynamic voltage-frequency scaling (DVFS)-based scheduling strategy to minimize long-term DMR for dual-channel SCSNs. This approach includes a day-level scheduler to achieve a coarse-grained task arrangement, two artificial neural networks to determine the task priorities, and a DVFS-based task selection algorithm for slot-level execution. Experiments show that the proposed scheduler reduces DMR by over 30% on average.

Field-programmable gate arrays (FPGAs) have drawn lots of attentions due to their programmability and high performance. Recently, ultralow-power FPGAs for Internet of Things, together with energy-harvesting technique, have become an emerging self-powered computing platform. However, volatile memory in FPGA will lose their states under unstable power supplies and cannot work efficiently. Nonvolatile FPGA becomes a promising alternative. This paper proposes a hardware/software codesign nonvolatile FPGA with efficient offline/online checkpointing strategy (CP-FPGA). Backup energy is reduced by offline selecting proper checkpointing locations to minimize backup data. An online scheduler is further proposed to balance computation rollback overhead against backup energy. Experimental results show that the proposed CP-FPGA reduces 39.5% energy consumption on average compared with the state-of-the-art techniques.

Nonvolatile processors (NVPs) preserve run-time information when power failure occurs by utilizing nonvolatile memory technologies. This feature enables NVPs to make forward progress continuously under intermittent power supply in energy harvesting systems. This paper builds a gem5 based NVP simulator named NVPsim, which is validated against measured results of a fabricated prototype with reasonable error rate. Furthermore, to demonstrate the capability of NVPsim for architecture exploration, we evaluated performance and energy consumption of different NVP designs varying in the choice of nonvolatile memory for on-chip caches, the backup strategy and the energy buffer size. Experimental results indicate that nvSRAM outperforms other types of nonvolatile memory as the on-chip cache for energy harvesting systems.

With the booming trend of internet of things (IoT), reconfigurable devices, such as FPGAs, have drawn lots of attentions due to their flexible and high-performance capability. However, commercial FPGAs suffer from high leakage power consumption, which makes zero-leakage nonvolatile FPGA (nvFPGA) promising. This paper proposes a hardware/ software co-design based nvFPGA with efficient checkpointing strategy. With nonvolatile checkpointing BRAM (CBRAM), it maintains both computation data as well as configuration when power-off to avoid expensive rollbacks due to data loss. A checkpointing location-aware technique is used to balance computation rollback overheads and backup energy. Experimental results show that the proposed checkpointing strategy can reduce 45.8% backup data of nvFPGA when system-level power gating happens.

Wearable devices begin to integrate into the daily lives along with recent technology development. One of such important applications is to accurately monitor ultraviolet (UV) radiation received by the human body. To compensate for the localized monitoring area of existing personal UV monitoring devices, this paper proposes a reconstruction method to estimate the UV dose over the entire body based on multiple discrete wearable UV sensor nodes. Ambient factors and individual factors are both considered in this paper. The proposed estimation method is validated by a range of UV data collection experiments in realistic scenarios. Experimental results show that the proposed method reduces 68.3% estimation errors on average compared with existing single sensor based methods.

Deep Learning (DL) is becoming popular in a wide range of domains. Many emerging applications, ranging from image and speech recognition to natural language processing and information retrieval, rely heavily on deep learning techniques, especially the Neural Networks (NNs). NNs have led to great advances in recognition accuracy compared with other traditional methods in recent years. NN-based methods demand much more computation and memory resource, and therefore a number of NN accelerators have been proposed on CMOS-based platforms, such as FPGA and GPU [1]. However, it becomes more and more difficult to obtain substantial power efficiency and gains directly through the scaling down of traditional CMOS technique. Meanwhile, the large data amount in DL applications also meets an ever-increasing "memory wall" challenge because of the efficiency of von Neumann architecture. Consequently, there is a growing research interest of exploring emerging nano-devices and new computing architectures to further improve power efficiency [2].

Nonvolatile processors have manifested strong vitality in battery-less energy harvesting sensor nodes due to their characteristics of zero standby power, resilience to power failures and fast read/write operations. However, I/O and sensing operations cannot store their system states after power off, hence they are sensitive to power failures and high power switching overhead is induced during power oscillation, which significantly degrades the system performance. In this paper, we propose a novel performance-aware task scheduling technique considering power switching overhead for energy harvesting nonvolatile processors. We first give the analysis of the power switching overhead on energy harvesting sensor nodes. Then, the scheduling problem is formulated by MILP (Mixed Integer Linear Programming). Furthermore, a task splitting strategy is adopted to improve the performance and an heuristic scheduling algorithm is proposed to reduce the problem complexity. Experimental results show that the proposed scheduling approach can improve the performance by 14% on average compared to the state-of-the-art scheduling strategy. With the employment of the task splitting approach, the execution time can be further reduced by 10.6%.

Energy harvesting has been widely investigated as a promising alternative for future wearable sensors or internet-of-things. However, power and performance overhead is induced when IO operations are interrupted by power failures because non-preemptive characteristic of IO operations causes expensive re-executions. Furthermore, the state-of-art IO devices need long and power hungry initializing process, which makes IO operations inefficient in transient powered systems. This paper proposed a HW/SW co-design approach for nonvolatile IO system to maximize data acquisition. A ferroelectric flip-flop based nonvolatile IO architecture is adopted to reduce IO initialization overhead by 3-4 orders of magnitude. Based on the nonvolatile IO interface, we further formulate the optimal data acquisition as an INLP problem and a risk-aware online scheduler is presented to solve the problem efficiently. Experimental results show that the proposed HW/SW co-design architecture improves data acquisition by 2-5 times compared with conventional HW/SW architecture.

Convolutional Neural Network (CNN) is a powerful technique widely used in computer vision area, which also demands much more computations and memory resources than traditional solutions. The emerging metal-oxide resistive random-access memory (RRAM) and RRAM crossbar have shown great potential on neuromorphic applications with high energy efficiency. However, the interfaces between analog RRAM crossbars and digital peripheral functions, namely Analog-to-Digital Converters (ADCs) and Digital-to-Analog Converters (DACs), consume most of the area and energy of RRAM-based CNN design due to the large amount of intermediate data in CNN. In this paper, we propose an energy efficient structure for RRAM-based CNN. Based on the analysis of data distribution, a quantization method is proposed to transfer the intermediate data into 1 bit and eliminate DACs. An energy efficient structure using input data as selection signals is proposed to reduce the ADC cost for merging results of multiple crossbars. The experimental results show that the proposed method and structure can save 80% area and more than 95% energy while maintaining the same or comparable classification accuracy of CNN on MNIST.

Large-scale graph processing is gaining increasing attentions in many domains. Meanwhile, FPGA provides a power-efficient and highly parallel platform for many applications, and has been applied to custom computing in many domains. In this paper, we describe FPGP (FPGA Graph Processing), a streamlined vertex-centric graph processing framework on FPGA, based on the interval-shard structure. FPGP is adaptable to different graph algorithms and users do not need to change the whole implementation on the FPGA. In our implementation, an on-chip parallel graph processor is proposed to both maximize the off-chip bandwidth of graph data and fully utilize the parallelism of graph processing. Meanwhile, we analyze the performance of FPGP and show the scalability of FPGP when the bandwidth of data path increases. FPGP is more power-efficient than single machine systems and scalable to larger graphs compared with other FPGA-based graph systems.

In recent years, convolutional neural network (CNN) based methods have achieved great success in a large number of applications and have been among the most powerful and widely used techniques in computer vision. However, CNN-based methods are com-putational-intensive and resource-consuming, and thus are hard to be integrated into embedded systems such as smart phones, smart glasses, and robots. FPGA is one of the most promising platforms for accelerating CNN, but the limited bandwidth and on-chip memory size limit the performance of FPGA accelerator for CNN.

Speed-Up Robust Feature (SURF) is an effective algorithm for feature extraction. We propose a novel Scaled-RAM Interpolator (SRI) on FPGA to deal with the high complexity of SURF by introducing two methods. 1) Interpolation of Integral Image (I3) restores the sub-pixel details of image to improve matching precision, and halves the memory access to achieve acceleration; 2) Multi-Scaled RAM (MSR) normalizes the storage allocation by scale to decrease complexity of memory access and reject redundant memory. With SRI, our system achieves better matching precision, higher processing speed, and lower storage occupation. The results of evaluation implemented on Stratix III EP3SL340 FPGA show that 1) our SRI-SURF system performs better in matching than OpenSURF; 2) the system is capable to process 241K feature points per second (PPS), which is about 7× of previous work on FPGA and is comparable to the recent ASIC solution; 3) the maximal frame rate reaches 488FPS at VGA and 72FPS at 1080P, which outperforms designs in other publications; 4) our design is compact, which only occupies about 22% logic resource and about 43% RAM resource at 1080P. The proposed SRI-SURF meets the demand of real-time embedded applications with better precision at high resolution.

Frequent Itemset Mining (FIM) is designed to find frequently occurring itemsets among a series of transactions. It is extremely memory and time expensive. Frequent Itemset Mining from a Data Stream (FIM-DS) is even more challenging since storing the infinite data to memory is infeasible. In recent years, researchers have proposed various approximation algorithms for FIM-DS. However, the computation complexity is still high, and these methods are difficult to be accelerated using hardware accelerators. In this paper, we propose a Space-Saving based approximate algorithm for FIM-DS. It avoids exponential candidates generation and comparisons. We realize a hardware accelerator design and implement it on an FPGA platform. Experimental results show that our algorithm in software implementation achieves up to 8.4× speedup for transactions with small item database, and our hardware accelerator achieves up to 50,000× speedup for transactions with small number of items, and 5.3× speedup for transactions with extremely large number of items.

Presents a collection of slides covering the following topics: FPGA; software-hardware co-design; neural network acceleration; DeePhi Tech; deep learning; CNN acceleration; efficient inference engine; processing element architecture; and LSTM.

A precision-improved processing architecture of physical computing for energy-efficient SIFT feature extraction algorithm has been proposed in this paper. With the novel physical computing technology of active resistor network (PC: ARN), the SIFT algorithm could be processed in analog signal domain without synchronizing clock signals, which means the complex algorithm could be completed within the setup time of the circuit. Especially for the multi-scale Gaussian convolution of SIFT algorithm, an architecture of two-layer 1-dimension PC: ARN has been adopted to compute the horizontal and vertical 1-dimension gaussian filter, in which way higher accuracy can be obtained when compared with the results processed by a 2D active circuit network. A circuit-level simulation with 65nm CMOS technology has been carried out, which shows the energy consumption of gaussian pyramid multi-scale-filtering hardware architecture is about 25.3pJ, where the size of input frame is assigned as 256×256 pixels. Additionally, the average matching ratio in different image pairs is around 80%. Moreover, integrated into the dominating CMOS image sensor with column-parallel readout technology of analog-to-digital convertor, about 20× speedup can be achieved comparing with previous implementations with FPGA, GPU, etc.

Recent studies show that graph processing systems on a single machine can achieve competitive performance compared with cluster-based graph processing systems. In this paper, we present NXgraph, an efficient graph processing system on a single machine. We propose the Destination-Sorted Sub-Shard (DSSS) structure to store a graph. To ensure graph data access locality and enable fine-grained scheduling, NXgraph divides vertices and edges into intervals and sub-shards. To reduce write conflicts among different threads and achieve a high degree of parallelism, NXgraph sorts edges within each sub-shard according to their destination vertices. Then, three updating strategies, i.e., Single-Phase Update (SPU), Double-Phase Update (DPU), and Mixed-Phase Update (MPU), are proposed in this paper. NXgraph can adaptively choose the fastest strategy for different graph problems according to the graph size and the available memory resources to fully utilize the memory space and reduce the amount of data transfer. All these three strategies exploit streamlined disk access patterns. Extensive experiments on three real-world graphs and five synthetic graphs show that NXgraph outperforms GraphChi, TurboGraph, VENUS, and GridGraph in various situations. Moreover, NXgraph, running on a single commodity PC, can finish an iteration of PageRank on the Twitter [1] graph with 1.5 billion edges in 2.05 seconds; while PowerGraph, a distributed graph processing system, needs 3.6s to finish the same task on a 64-node cluster.

Body channel communication (BCC) is one of the best candidates for communications in wireless body sensor networks as it uses the human body as transmission media to minimize transmission loss resulting better energy efficiency. The main issue of BCC is the loss in its backward path, which is formed by the capacitive coupling between two floated GND electrodes (GEs) of transmitter (TX) and receiver (RX). To mitigate the backward path loss, an off-chip inductor could be used to resonate with the backward capacitance to reduce the impedance of the backward path. However, this method is not suitable for wearable applications as the off-chip inductor only works for fixed communication distance. In this paper, we present a novel self-adaptive capacitive compensation (SACC) scheme to reduce the capacitive loss of the backward path. The proposed system automatically estimates the distance between GEs of TX and RX with the help of received signal strength indicator (RSSI). The backward capacitance is then calculated based on the estimated distance. And then the capacitance is compensated by a digitally controlled active inductor to reduce the backward path loss. Simulation shows that the proposed scheme achieves more than 15 dB channel enhancement at the IEEE 802.15.6 standard frequency.

Deep learning, and especially Convolutional Neural Network (CNN, is among the most powerful and widely used techniques in computer vision. Applications range from image classification to object detection, segmentation, Optical Character Recognition (OCR), etc. At the same time, CNNs are both computationally intensive and memory intensive, making them difficult to be deployed on low power lightweight embedded systems. In this work, we introduce an on-chip convoltional neural network implementation for low-power embedded system. We point out that the high precision of weights limits the low-power CNN implementation on both FPGA and RRAM platform. A dynamic quantization method is introduced to reduce the precision while maintaining the same or comparable accuracy at the same time. Finally, the de ailed designs of low-power FPGA-based CNN and RRAM-based CNN are provided and compared. The results show that FPGA-based design gets 2× energy efficiency compared with GPU implementation, and toe RRAM-based design can further obtain more than 40× energy efficiency gains.

Reliable and ultra-low power time synchronization becomes more and more important with the popularity of energy harvesting sensor nodes. This paper proposes an untethered and probabilistic ultra-lower power time synchronization method for energy intermittent sensor network. It avoids the frequent RF communications with the assistance of a solar clock. The SATS system consists of two main parts: the synchronizer, a low power solar clock module for time synchronization, and the S3-Mapping, an offline sequence matching algorithm. Furthermore, we develop an improved version of S3-Mapping, which reduces the computation complexity from exponential to linear using the redundancy models and the onion peeling method. The SATS system is validated by both simulations and a prototype, which shows that the second level synchronization precision can be achieved under reasonable probability. What's more, the energy consumption of time synchronization is reduced by over 1 ~ 2 magnitudes compared with the up-to-date low power time synchronization protocol.

This study presents a high‐performance audio watermarking scheme using spread spectrum modulation. Unlike conventional extractors which use simple correlation, this watermarking scheme exploits the perceptual characteristic of the watermarked audio before correlation. It is noted that although the watermark extractor works blindly neither which the original audio signal nor the embedded watermark signal is available, however, the spectral power structure of embedded watermark can be estimated using perceptual analysis methods. With this information, the watermark performance is improved by introducing an estimation‐equalisation‐correlation based extraction mechanism. The pre‐equaliser at the extractor is carefully designed to obtain optimised extraction performance. Moreover, the perceptual analysis and shaping method are improved to make sure the watermark estimation is accurate. The perceptual characteristic aware extraction‐based watermarking scheme achieves high embedding capacity up to 43 bps/channels, with low perceptual distortion to the host audio. Experiments on real audio signals show that the proposed watermarking scheme achieves high performance and is robust against various types of attacks.

Abstract This paper addresses the audio piracy problem over Internet by tweaking data redundancy of host media signal for optimized embedding and extraction of spread spectrum watermarking. In particular, we take into account a special feature that host audio signals have the short‐time stationary property and their major power can be removed using linear prediction filter. By combining the redundancy removing and the improved spread spectrum modulation, host interference is canceled with minimized embedding distortion to the host audio. The data redundancy removing is also applied at the receiver to achieve matched filtering and improved performance. Experiments based on real audio signals show that our proposed scheme performs robustly against various kinds of channel attacks while maintaining high extraction performance. Copyright © 2016 John Wiley & Sons, Ltd.

The random telegraph noise (RTN) is becoming more serious in advanced technologies. Due to the unpredictability of the physical phenomenon, RTN is a good randomness source for true random number generators (TRNG). In this paper, we build fundamental randomness models for TRNGs based on single trap- and multiple traps-induced RTN. We theoretically derive the autocorrelation coefficient, bias, and bit rate for RTN-based TRNGs. Two representative RTN-based TRNG schemes are simulated to verify the proposed randomness models. An oscillator-based TRNG is also studied based on the theoretical randomness model of multiple traps-induced RTN. We also provide basic guidelines for designing RTN-based TRNGs.

Energy harvesting from natural environment gives range of benefits for the Internet of things. Scavenging energy from photovoltaic (PV) cells is one of the most practical solutions in terms of power density among existing energy harvesting sources. PV power systems mandate the maximum power point tracking (MPPT) to scavenge the maximum possible solar energy. In general, a switching-mode power converter, an MPPT charger, controls the charging current to the energy storage element (a battery or equivalent), and the energy storage element provides power to the load device. The mismatch between the maximum power point (MPP) current and the load current is managed by the energy storage element. However, such architecture causes significant energy loss (typically over 20%) and a significant weight/volume and a high cost due to the cascaded power converters and the energy storage element. This paper pioneers a converter-less PV power system with the MPPT that directly supplies power to the load without the power converters or the energy storage element. The proposed system uses a nonvolatile microprocessor to enable an extremely fine-grain dynamic power management in a few hundred microseconds. This makes it possible to match the load current with the MPP current. We present detailed modeling, simulation, and optimization of the proposed energy harvesting system including the radio frequency transceiver. Experiments show that the proposed setup achieves an 87.1% of overall system efficiency during a day, 30.6% higher than the conventional MPPT methods in actual measurements, and thus a significantly higher duty cycle under a weak solar irradiance.

With the advent of the era of trillion sensors, solar-powered sensor nodes are widely used as they do not require battery charging or replacement. However, the limited and intermittent solar energy supply seriously affects deadline miss rate (DMR) of tasks. Furthermore, traditional solar-powered sensor nodes also suffer from energy loss of battery charging and voltage conversion. Recently, a storage-less and converter-less power supply architecture has been proposed to achieve higher energy efficiency by removing the leaky energy storage and dc voltage conversion. Without energy storages, a node using inter-task scheduling is more sensitive to solar variations, which results in high DMRs. This paper proposes an intra-task scheduling scheme for the storage-less and converter-less solar-powered sensor nodes, whose features include power prediction based on classified solar profiles, a trigger mechanism to select scheduling points, an artificial neural network to calculate task priorities and a fine-grained task selection algorithm. Experimental results show that the proposed algorithm reduces DMR by up to 30% and improves energy utilization efficiency by 20% with trivial energy overheads.

Developing circuits for streaming applications written in C (or its variants) can benefit greatly from C-to-RTL (C2RTL) synthesis. Yet, most existing C2RTL tools lack system-level options to trade off various design constraints, such as delay and area. This article introduces a systematic way to accomplish C2RTL synthesis for streaming applications containing thousands of lines of C (or its variants) codes. Synthesizing circuits for such large applications presents serious challenges for existing C2RTL tools. Specifically, the proposed approach determines simultaneously the number of pipeline stages and the number of times that each functional block is duplicated in each pipeline stage. A mixed integer linear programming-based solution is formulated for obtaining the optimal solution. Furthermore, a heuristic algorithm is developed for large-scale problems. To accommodate the differences of the data rates between the adjacent hardware modules, first-in-first-out (FIFO) buffers are indispensable, but their overheads are nonnegligible. A parallelism-aware FIFO sizing method is also introduced to determine the optimal sizes of FIFOs. Experimental results on seven real-world applications demonstrate that the algorithms in the synthesis flow can make effective design trade-offs and find superior solutions in a short time compared with existing approaches. Furthermore, the algorithms achieve optimal results in most cases with subsecond running time.

The matrix-vector multiplication is the key operation for many computationally intensive algorithms. In recent years, the emerging metal oxide resistive switching random access memory (RRAM) device and RRAM crossbar array have demonstrated a promising hardware realization of the analog matrix-vector multiplication with ultra-high energy efficiency. In this paper, we analyze the impact of nonlinear voltage-current relationship of RRAM devices and the interconnect resistance as well as other crossbar array parameters on the circuit performance and present a design guide. On top of that, we propose a technological exploration flow for device parameter configuration to overcome the impact of nonideal factors and achieve a better trade-off among performance, energy and reliability for each specific application. The simulation results of a support vector machine (SVM) and MNIST pattern recognition dataset show that the RRAM crossbar array-based SVM is robust to the input signal fluctuation but sensitive to the tunneling gap deviation. A further resistance resolution test presents that a 4-bit RRAM device is able to realize a recognition accuracy of ∼ 90%, indicating the physical feasibility of RRAM crossbar array-based SVM. In addition, the proposed technological exploration flow is able to achieve 10.98% improvement of recognition accuracy on the MNIST dataset and 26.4% energy savings compared with previous work.

The emerging nonvolatile memory (NVM) technology can potentially change the landscape of future IC designs with numerous benefits, such as high performance, low leakage power, and data retention. These advantages motivate designers to exploit utilizing NVM in in ASIC and FPGA. However, unique challenges such as large write energy and asymmetric read/write operations, lead to extra design knobs. This paper focuses on the NVM allocation and hierarchy optimization in high-level synthesis. A hierarchical hybrid memory architecture is presented. The proposed framework optimizes the memory hierarchy, type (NVM or SRAM) and capacity. Both an mixed-integer linear programming (MILP) and a branch-and-bound heuristic are developed. Experimental results demonstrate up to 69.3% power reduction compared with designs without NVM.

Power consumption is becoming more critical in modern integrated circuit (IC) designs and clock network is one of the major contributors for on-chip power. Resonant clock has been investigated as a potential solution to reduce the power consumption in clock network by recycling the energy with on-chip inductors. Most of the previous resonant clock work focuses on H-tree structures, while in this work, we propose a modeling and optimization method for the mesh structure, which suffers from the high power consumption more seriously than the tree structure. Closed-form expressions for the transfer function, skew, and power are derived. Based on these expressions, impacts of design factors, such as the buffer size, LC tank location, grid size, wire width, and the sparsity of buffers and LC tanks, are fully explored to make trade-offs among power, skew, and area, which can be used as design guidelines for top level resonant clock mesh in early design stages. The exploration is also extended to 3D ICs and different mesh structures are evaluated. Matlab-based implementation of the proposed simplified circuit model can achieve over 105 times speedup compared to SPICE-based simulation.

Approximate computing technology has aroused growing interest in circuit and system design for its well-performed trade-off between output quality and performance. Numerous basic circuits and system design methodologies for approximate computing have been proposed. Considering that the existing methodologies for the evaluation of tradeoff between output quality and performance is time-consuming, this paper presents a fast design methodology for approximate accumulator based on statistical error model, in which the inexact multistage speculative adder is adopted and modeled for its advantage of compact error pattern. To validate the proposed methodology, Support Vector Machine(SVM) algorithm is analyzed and mapped to a hardware system composed of inexact and accurate computing circuits. Results show that our time for searching the optimal mapping circuits has been saved by 22.08% than functional-based simulation where the final approximate system design achieves 1.57× speedups with 8.56% accuracy degradation.

PM2.5has already been a major pollutant in many cities in China. It is a kind of harmful pollutant which may cause several kinds of lung diseases. However, the existing methods to monitor PM2.5with high accuracy are too expensive to popularize. The high cost also limits the further researches about PM2.5. This paper implements a method to estimate PM2.5with low cost and high accuracy by Artificial Neural Network (ANN) technique using other pollutants and meteorological factors that are easy to be monitored. An Entropy Maximization step is proposed to avoid the over-fitting related to the data distribution of pollutant data. Also, how to choose the input attributes is abstracted to an optimization problem. An iterative greedy algorithm is proposed to solve it, which reduces the cost and increases the estimation accuracy at the same time. The experiment shows that the linear correlation coefficient between the estimated value and real value is 0.9488. Our model can also classify PM2.5levels with a high accuracy. Additionally, the trade-off between accuracy and cost is investigated according to the price and error rate of each sensor.

In the near future, self-powered wearable sensor nodes are becoming as an emerging trend in healthcare applications. However, several design challenges exist before their wide usage, such as the limited and unstable power output as well as unpredictable power profiles. This paper shows an overview of candidate solutions and proposes that a cross-layer approach is needed.

Recently, general-purpose graphics processing units (GPGPUs) have been widely used to accelerate computing in various applications. To store the contexts of thousands of concurrent threads on a GPU, a large static random-access memory (SRAM)-based register file is employed. Due to high leakage power of SRAM, the register file consumes 20% to 40% of the total GPU power consumption. Thus, hybrid memory system, which combines SRAM and the emerging non-volatile memory (NVM), has been employed for register file design on GPUs. Although it has shown strong potential to alleviate the power issue of GPUs, existing hybrid memory solutions might not exploit the intrinsic feature of GPU register file. By leveraging the warp schedule on GPU, this paper proposes a hybrid register architecture which consists of a NVM-based register file and mixed SRAM-based write buffers with a warp-aware write back strategy. Simulation results show that our design can eliminate 64% of write accesses to NVM and reduce power of register file by 66% on average, with only 4.2% performance degradation. After we apply the power gating technique, the register power is further reduced to 25% of SRAM counterpart on average.

The invention of resistive-switching random access memory (RRAM) devices and RRAM crossbar-based computing system (RCS) demonstrate a promising solution for better performance and power efficiency. The interfaces between analog and digital units, especially AD/DAs, take up most of the area and power consumption of RCS and are always the bottleneck of mixed-signal computing systems. In this work, we propose a novel architecture, MEI, to minimize the overhead of AD/DA by MErging the Interface into the RRAM crossbar. An optional ensemble method, the Serial Array Adaptive Boosting (SAAB), is also introduced to take advantage of the area and power saved by MEI and boost the accuracy and robustness of RCS. On top of these two methods, a design space exploration is proposed to achieve trade-offs among accuracy, area, and power consumption. Experimental results on 6 diverse benchmarks demonstrate that, compared with the traditional architecture with AD/DAs, MEI is able to save 54.63%~86.14% area and reduce 61.82%~86.80% power consumption under quality guarantees; and SAAB can further improve the accuracy by 5.76% on average and ensure the system performance under noisy conditions.

Energy harvesting is gaining more and more attentions due to its characteristics of ultra-long operation time without maintenance. However, frequent unpredictable power failures from energy harvesters bring performance and reliability challenges to traditional processors. Nonvolatile processors are promising to solve such a problem due to their advantage of zero leakage and efficient backup and restore operations. To optimize the nonvolatile processor design, this paper proposes new metrics of nonvolatile processors to consider energy harvesting factors for the first time. Furthermore, we explore the nonvolatile processor design from circuit to system level. A prototype of energy harvesting nonvolatile processor is set up and experimental results show that the proposed performance metric meets the measured results by less than 6.27% average errors. Finally, the energy consumption of nonvolatile processor is analyzed under different benchmarks.

Solar-powered sensor nodes with energy storages are widely used today and promising in the coming trillion sensor era, as they do not require manual battery charging or replacement. The changeable and limited solar power supply seriously affects the deadline miss rates (DMRs) of tasks on these nodes and therefore energy-driven task scheduling is necessary. However, current algorithms focus on the single period (or the current task queue) for high energy utilization and suffer from bad long term DMR. To get better long term DMR, we propose a long term deadline-aware scheduling algorithm with energy migration strategies for distributed super capacitors. Experimental results show that the proposed algorithm reduces the DMR by 27.8% and brings less than 3% of the total energy consumption.

Simultaneous localization and mapping (SLAM) is a key algorithm in localization tasks. Considering the limited payload and power on mobile robots, FPGA-based SLAM is a promising onboard solution. This paper presents an FPGA-based SLAM system, which can recover the indoor moving trajectory of the stereo cameras in real-time. We propose a low computational complexity VO-SLAM (Visual Odometry based SLAM) algorithm, and implement the algorithm on a matrix processor based on DE3 develop board. Dedicated matrix accelerators are designed to support application requirements, and a hierarchical matrix computing mechanism is proposed. The algorithm accuracy in the real scenario test is comparable to more complex EKF-SLAM algorithm. Onboard experiments demonstrate the system achieves a processing speed of 31 fps with 30000 features in the global map, which outperforms designs in other publications. We compare the onboard implementation with Intel i7 and achieve 10x energy saving for each frame.

With the exponential growth of data size, data storage and analysis have been exposed to more challenges due to the lack of disk capacity and the limited network bandwidth. Data compression technique provides a good solution to mitigate these effects. In this paper, we propose a self-aware data compression system on FPGA for typical data warehousing, such as Hive, with column stored data and multi-threading requirements. The hardware accelerators can change the degree and hierarchy of parallelism depending on the data to be compressed (during the runtime). We test the system performance on a Xilinx VC707 FPGA board and the experimental results show that, up to 16 3-parallelism accelerators can be implemented and the throughput could be improved up to 432 MB/s. It is 6.25X speedup compared with the software solution under the same number of threads.

Inspired by the human brain's function and efficiency, neuromorphic computing offers a promising solution for a wide set of tasks, ranging from brain machine interfaces to real-time classification. The spiking neural network (SNN), which encodes and processes information with bionic spikes, is an emerging neuromorphic model with great potential to drastically promote the performance and efficiency of computing systems. However, an energy efficient hardware implementation and the difficulty of training the model significantly limit the application of the spiking neural network. In this work, we address these issues by building an SNN-based energy efficient system for real time classification with metal-oxide resistive switching random-access memory (RRAM) devices. We implement different training algorithms of SNN, including Spiking Time Dependent Plasticity (STDP) and Neural Sampling method. Our RRAM SNN systems for these two training algorithms show good power efficiency and recognition performance on realtime classification tasks, such as the MNIST digit recognition. Finally, we propose a possible direction to further improve the classification accuracy by boosting multiple SNNs.

Physical computing scheme of active resistor network is proposed in this paper to set up a multi-scale Gaussian filter, which is also called Gaussian Pyramid in image signal processing. The analog output signal of each image photodiode could be directly processed by the circuit topology of an active resistor network, which has been pre-designed to meet the requirement of the complicated Gaussian Pyramid in SIFT algorithm. Since it is operated with no clock, the physical computing scheme, with only setup time of the whole processing circuits, is much faster than various current digital realizations. The circuit-level simulation with 65nm CMOS technology has been carried out, which shows the energy consumption of the Gaussian Pyramid processing circuit is around 74.79pJ to filter a frame of 256 × 256 pixel image, and the circuit setting time of the processing is about 138.03ps, considering the parasitic capacitances of each nodes of the circuits. Furthermore, the presented circuit is integrated into a smart CMOS image sensor architecture. With the same processing procedure, the new method of active resistor network could achieve 1.58X speedup when compared with its counterpart of FPGA implementation, in which the column-parallel readout technology of analog-to-digital convertor is used.

A 14-bit 1.0-GS/s current-steering digital-to-analog converter (DAC) was designed in a 65-nm CMOS process. For such current-steering DACs with a high sampling rate, the code-dependent load variations and switching glitches are a main bottleneck which limits the spurious-free dynamic range (SFDR). Dynamic element matching (DEM) has been an effective solution to randomize these glitches for a higher SFDR and also to reduce the matching requirement of the current cells for an area-efficient design which also improves the SFDR with reduced parasitic capacitance. An effective method named TRI-DEMRZ is proposed in this paper, consisting of time-relaxed interleaving, DEM and return-to-zero encoding. We also apply TRI-DEMRZ in synergy with complementary switched current sources (CSCS) to design the DAC for the purpose of a small die size and enhanced SFDR performance. Post-layout simulations show >80 dB SFDR up to the Nyquist. This DAC has a mixed 1.2 V / 2.5 V power supply and an active area of 0.48 mm2.

Human body communication (HBC) has significant advantage over wireless communication schemes in wireless body area networks (WBANs) in terms of power efficiency due to the high conductivity of human body. An accurate circuit model for transmission channel is necessary for optimizing the HBC transceiver performance. Conventional models achieve limited accuracy because of incomplete body tissue model or the use of tranmission-line at circuit level. In this paper, we proposed a comprehensive HBC circuit model which is based on 5 human-surface tissue layers representing the physiological characteristics of living tissues and the frequency dependence of their dielectric properties. Instead of using transmission-line, our model is based on lumped-element analysis, which is more accurate at the 21 MHz frequency band specified by the IEEE 802.15.6 HBC standard. We verified the proposed model by actual measurement on human body at various of communication distances. Experimental results show that the proposed model achieved the minimum error among all the modeling works, i.e., 1.80% minimum error and 2.24% maximal error at various communication distances.

With the fast development of Internet of Things (IoTs) in recent years, many IoT applications, such as structure health monitoring, surveillance camera and etc, require both extensive computation for burst-mode signal processing as well as ultra low power continuous operations. However, most of conventional IoT processors focus on ultra low power consumption and cannot satisfy those demands. This paper proposes a novel energy-efficient heterogenous dual-core processor, which includes both an ultra low power near-threshold CoreL and a fast CoreH to meet those emerging requirements. Furthermore, an optimal framework is proposed to realize energy efficient task mapping and scheduling. The processor is fabricated and its energy consumption in low power mode is as low as 7.7pJ/cycle and outperforms related work. Detailed analysis under several real applications shows that up to 2.62× energy efficiency improvements can be achieved without deadline miss compared with the high-performance-only signle core architecture.

To mitigate the “Power Wall” challenges for both mobile devices and data centers, accelerator-rich architecture with normally-off mode has been intensively studied recently. Power/energy optimization in high-level synthesis for accelerator design is critical for such accelerator-rich architecture. The emerging nonvolatile memory (NVM), offers many benefits such as ultra-low leakage power, high density, and instant power-on/off, and therefore is a promising alternative for the hardware accelerator design to achieve further power reduction. However, such NVM suffers from large write energy and latency, which brings new challenges for the buffer allocation in the custom accelerator design. This paper presents the first framework that optimizes NVM allocation in high-level synthesis for custom accelerator design, considering loop transformations. It solves the loop transformation, buffer allocation, and buffer type selection to minimize the memory power consumption, while under area, bandwidth, and performance constraints. This paper formulates the optimization problem, and solves it with a problem-specific designed stimulated annealing solution. Experiments demonstrate 32% extra power reduction compared with the previous method without optimizing loop transformations.

Third-party intellectual property (3PIP) cores are widely used in integrated circuit designs. It is essential and important to ensure their trustworthiness. Existing hardware trust verification techniques suffer from high computational complexity, low extensibility, and inability to detect implicitly-triggered hardware trojans (HTs). To tackle the above problems, in this paper, we present a novel 3PIP trust verification framework, named FASTrust, which conducts HT feature analysis on the flip-flop level control-data flow graph (CDFG) of the circuit. FASTrust is not only able to identify existing explicitly-triggered and implicitly-triggered HTs appeared in the literature in an efficient and effective manner, but more importantly, it also has the unique advantage of being scalable to defend against future and more stealthy HTs by adding new features to the system.

Approximate computing is a promising design paradigm for better performance and power efficiency. In this paper, we propose a power efficient framework for analog approximate computing with the emerging metal-oxide resistive switching random-access memory (RRAM) devices. A programmable RRAM-based approximate computing unit (RRAM-ACU) is introduced first to accelerate approximated computation, and an approximate computing framework with scalability is then proposed on top of the RRAM-ACU. In order to program the RRAM-ACU efficiently, we also present a detailed configuration flow, which includes a customized approximator training scheme, an approximator-parameter-to-RRAM-state mapping algorithm, and an RRAM state tuning scheme. Finally, the proposed RRAM-based computing framework is modeled at system level. A predictive compact model is developed to estimate the configuration overhead of RRAM-ACU and help explore the application scenarios of RRAM-based analog approximate computing. The simulation results on a set of diverse benchmarks demonstrate that, compared with a x86-64 CPU at 2 GHz, the RRAM-ACU is able to achieve 4.06-196.41× speedup and power efficiency of 24.59-567.98 GFLOPS/W with quality loss of 8.72% on average. And the implementation of hierarchical model and X application demonstrates that the proposed RRAM-based approximate computing framework can achieve 12.8× power efficiency than its pure digital implementation counterparts (CPU, graphics processing unit, and field- programmable gate arrays).

The sparse matrix solver by LU factorization is a serious bottleneck in Simulation Program with Integrated Circuit Emphasis (SPICE)-based circuit simulators. The state-of-the-art Graphics Processing Units (GPU) have numerous cores sharing the same memory, provide attractive memory bandwidth and compute capability, and support massive thread-level parallelism, so GPUs can potentially accelerate the sparse solver in circuit simulators. In this paper, an efficient GPU-based sparse solver for circuit problems is proposed. We develop a hybrid parallel LU factorization approach combining task-level and data-level parallelism on GPUs. Work partitioning, number of active thread groups, and memory access patterns are optimized based on the GPU architecture. Experiments show that the proposed LU factorization approach on NVIDIA GTX580 attains an average speedup of 7.02$\times$(geometric mean) compared with sequential PARDISO, and 1.55$\times$compared with 16-threaded PARDISO. We also investigate bottlenecks of the proposed approach by a parametric performance model. The performance of the sparse LU factorization on GPUs is constrained by the global memory bandwidth, so the performance can be further improved by future GPUs with larger memory bandwidth.

Through-silicon-via (TSV) could provide vertical connections among different dies in 3-D integrated circuits (3-D ICs), but the significant silicon area occupied by TSVs may bring great challenge to designers in 3-D clock tree synthesis (CTS), because only a few whitespace blocks can be used for clock TSV insertion after floorplan and placement are determined, specifically in the area-efficient 3-D IC designs. This paper proposes a whitespace-aware TSV arrangement algorithm in 3-D CTS, which mainly consists of three stages: sink preclustering, whitespace-aware 3-D method of means and medians (3-D-MMMs) topology generation, and deferred-merge embedding merging segment reconstruction. By leveraging the TSV-to-TSV coupling model, we also propose an efficient clock TSV arrangement method to alleviate the coupling effect of adjacent TSVs. Compared with the traditional 3-D-MMM-based CTS with TSV moving adjustment, the experimental results show that our proposed algorithm is more practical and efficient, achieving 49.2% reduction on the average skew and 1.9% reduction on the average power.

As different tiers are stacked together in 3-D integrated circuits, the power/ground (P/G) network simulation becomes more challenging than that of 2-D cases. In this brief, we propose a hierarchical simulation method suitable for 3-D P/G network (HS3-DPG), which takes advantage of the inherent hierarchical structure of 3-D P/G network. The port equivalent model (PEM) is introduced to mask the details of P/G grid in each tier. Besides, we introduce the locality property to further simplify the simulation. Some 3-D P/G network benchmarks extracted from industrial designs are used to verify the correctness of our method. Experimental results show that, HS3-DPG can achieve considerable speedup, while maintaining high accuracy. Simplified PEMs considering the locality property can save nearly 80% memory allocation compared with the full PEMs when the number of through-silicon-vias between the adjacent tiers becomes quite large.

Behavioral models are necessary to shorten the design turn-around time of phase-locked loops (PLLs). For general cases in a system-on-chip, the PLL can resist the supply noise from other modules by a low-dropout regulator. Therefore, interactions among the supply ports of PLL submodules play a much more important part in the overall noise performance. In this brief, we propose a time-domain charge-pump PLL model including supply-noise interactions inside PLL. Described by Verilog-A, the behavioral results are compared with the simulated results of transistor circuits. It can be seen that the proposed model has reduced the simulation time to about 1.7% when compared with transistor-level simulating by SpectreRF. Under a nonideal power grid, our model can improve the jitter simulation accuracy when compared with the conventional model, e.g., simulated jitter error is lowered by 52.6% under a power grid with 8-Ω power-line segment and 1-pF decoupling capacitance.

Random telegraph noise (RTN) has become an important reliability issue at the sub-65nm technology node. Existing RTN simulation approaches mainly focus on single trap induced RTN and transient response of RTN, which are usually time-consuming for circuit-level simulation. This paper proposes a statistical algorithm to study multiple traps induced RTN in digital circuits, to show the temporal distribution of circuit delay under RTN. Based on the simulation results we show how to protect circuit from RTN. Bias dependence of RTN is also discussed.

The artificial neural network (ANN) is among the most widely used methods in data processing applications. The memristor-based neural network further demonstrates a power efficient hardware realization of ANN. Training phase is the critical operation of memristor-based neural network. However, the traditional training method for memristor-based neural network is time consuming and energy inefficient. Users have to first work out the parameters of memristors through digital computing systems and then tune the memristor to the corresponding state. In this work, we introduce a mixed-signal training acceleration framework, which realizes the self-training of memristor-based neural network. We first modify the original stochastic gradient descent algorithm by approximating calculations and designing an alternative computing method. We then propose a mixed-signal acceleration architecture for the modified training algorithm by equipping the original memristor-based neural network architecture with the copy crossbar technique, weight update units, sign calculation units and other assistant units. The experiment on the MNIST database demonstrates that the proposed mixed-signal acceleration is 3 orders of magnitude faster and 4 orders of magnitude more energy efficient than the CPU implementation counterpart at the cost of a slight decrease of the recognition accuracy (<; 5%).

This paper pioneers the maximum power point tracking (MPPT) of photovoltaic (PV) cells that directly supply power to a microprocessor without an energy storage element (a battery or a large-size capacitor) nor power converters. The maximum power point tracking is conventionally performed by an MPPT charger that stores in the energy storage element, and a voltage regulator (typically a DC-DC converter) produces a proper voltage level for the microprocessor. The energy storage element is an energy buffer and makes it possible to perform MPPT of the PV cells and power management of the microprocessor independently. However, the energy storage element, MPPT charger and DC-DC converter cause seriously limited lifetime (when a typical battery is adopted), significant energy loss (typically over 20%), increased weight/volume and high cost, etc. The proposed method enables extremely fine-grain dynamic power management (DPM) in every a few hundred microseconds and performs the MPPT without using an MPPT charger and a DC-DC converter as well as an energy storage element. We achieve 84.5% of energy harvesting efficiency using the proposed setup with huge reduction in cost, weight and volume, and extended lifetime, which is not even numerically comparable with conventional MPPT methods.

High-performance general-purpose graphics processing units (GPGPUs) may suffer from serious power and negative bias temperature instability (NBTI) problems. In this paper, we propose a framework for run-time aging and power optimization. Our technique is based on the observation that many GPGPU applications achieve optimal performance with only a portion of cores due to either bandwidth saturation or shared resource contention. During run-time, given the dynamically tracked NBTI-induced threshold voltage shift and the problem size of GPGPU applications, our algorithm returns the optimal number of cores using detailed performance modeling. The unused cores are power-gated for power saving and NBTI recovery. Experiments show that our proposed technique achieves on average 34% reduction in NBTI-induced threshold voltage shift and 19% power reduction, while the average performance degradation is less than 1%.

Three-dimensional (3D) integration technology has been proposed as a promising technology to provide small footprint, reduced wirelength, and the capability of heterogeneous integration. In particular, 3D IC is a good candidate to address the design issues in conventional analog/digital mixed-signal IC designs. In this work, we focus on modeling and analyzing the impacts of through silicon vias (TSVs) on mixed-signal ICs. Based on the analysis, a set of design methodologies for 3D mixed-signal ICs are proposed. The design methodologies are verified with a case study, in which a 12-bit successive approximation register analog-to-digital converter (SAR ADC) is re-designed by partitioning it into three stacked layers for 3D integration. The experimental results show that, compared to the traditional 2D counterpart, our 3D SAR ADC with optimized TSV placement can achieve significant area and power reduction, and performance improvement. Specifically, due to the isolation of substrate noise disturbance in our 3D design, the signal-to-noise-plus-distortion ratio (SNDR) is improved from 68.74 dB to 74.12 dB.

The world is experiencing a data revolution to discover knowledge in big data. Large scale neural networks are one of the mainstream tools of big data analytics. Processing big data with large scale neural networks includes two phases: the training phase and the operation phase. Huge computing power is required to support the training phase. And the energy efficiency (power efficiency) is one of the major considerations of the operation phase. We first explore the computing power of GPUs for big data analytics and demonstrate an efficient GPU implementation of the training phase of large scale recurrent neural networks (RNNs). We then introduce a promising ultrahigh energy efficient implementation of neural networks' operation phase by taking advantage of the emerging memristor technique. Experiment results show that the proposed GPU implementation of RNNs is able to achieve 2 ~ 11× speed-up compared with the basic CPU implementation. And the scaled-up recurrent neural network trained with GPUs realizes an accuracy of 47% on the Microsoft Research Sentence Completion Challenge, the best result achieved by a single RNN on the same dataset. In addition, the proposed memristor-based implementation of neural networks demonstrates power efficiency of > 400 GFLOPS/W and achieves energy savings of 22× on the HMAX model compared with its pure digital implementation counterpart.

The emerging neuromorphic computation provides a revolutionary solution to the alternative computing architecture and effectively extends Moore's Law. The discovery of the memristor presents a promising hardware realization of neuromorphic systems with incredible power efficiency, allowing efficiently executing the analog matrix-vector multiplication on the memristor crossbar architecture. However, during computations, the memristor will slowly drift from its initial programmed state, leading to a gradual decline of the computation precision of memristor crossbar-based computing engine (MCE). In this paper, we propose an inline calibration mechanism to guarantee the computation quality of the MCE. The inline calibration mechanism collects the MCE's computation error through ‘interrupt-and-benchmark (I&B)’ operations and predicts the best calibration time through polynomial fitting of the computation error data. We also develop an adaptive technique to adjust the time interval between two neighbor I&B operations and minimize the negative impact of the I&B operation on system performance. The experiment results demonstrate that the proposed inline calibration mechanism achieves a calibration efficiency of 91.18% on average and negligible performance overhead (i.e., 0.439%).

Frequent item counting is one of the most important operations in time series data mining algorithms, and the space saving algorithm is a widely used approach to solving this problem. With the rapid rising of data input speeds, the most challenging problem in frequent item counting is to meet the requirement of wire-speed processing. In this paper, we propose a streaming oriented PE-ring framework on FPGA for counting frequent items. Compared with the best existing FPGA implementation, our basic PE-ring framework saves 50% lookup table resources cost and achieves the same throughput in a more scalable way. Furthermore, we adopt SIMD-like cascaded filter for further performance improvements, which outperforms the previous work by up to 3.24 times in some data distributions.

The popularization and application of Cloud Computing have provided a new approach for users to get computing resources in recent years. Meanwhile, due to the advantages including programmability and power-efficiency, FPGAs have been applied to custom computing in many domains. Previous work has made resources of FPGA available under the cloud environment. However, the effective usage of FPGAs in the cloud requires efficient online task scheduling: to properly assign as many tasks from different tenants as possible to the FPGAs. In this paper, we propose a benefit-based scheduling metric to evaluate the task assignment Based on the metric, we accelerate task execution according to our benefit-based scheduling algorithms. By applying our benefit-based scheduling metric to a real OpenStack-based cloud environment, 60.32% computing resources are saved compared with the conventional throughput-based metric. Furthermore, a Replacement-Considering algorithm, which considers the task replacement, is proposed taking the characteristics of cloud into account. The results show that our FPGA accelerated cloud system is 1.386 times faster than using the previous algorithm.

Solar-powered sensor nodes without specific energy maintenance have shown great promise in many applications, but they suffer from large energy storage and power converter loss. The storage-less and converter-less architecture with nonvolatile processing units has been proposed to reduce the energy loss. However, the architecture is sensitive to solar variations, since there is no energy buffering. Traditional inter-task scheduling methods may not work well due to large variations of task execution time. To tackle the challenge, we develop an algorithm for intra-task scheduling to achieve better quality of service. The experimental results show that the intra-task scheduling algorithm reduces deadline miss rate by as much as 35% and improves energy utilization by close to 20%.

Large scale artificial neural networks (ANNs) have been widely used in data processing applications. The recurrent neural network (RNN) is a special type of neural network equipped with additional recurrent connections. Such a unique architecture enables the recurrent neural network to remember the past processed information and makes it an expressive model for nonlinear sequence processing tasks. However, the large computation complexity makes it difficult to effectively train a recurrent neural network and therefore significantly limits the research on the recurrent neural network in the last 20 years. In recent years, the use of graphics processing units (GPUs) becomes a significant advance to speed up the training process of large scale neural networks by taking advantage of the massive parallelism capabilities of GPUs. In this paper, we propose an efficient GPU implementation of the large scale recurrent neural network and demonstrate the power of scaling up the recurrent neural network with GPUs. We first explore the potential parallelism of the recurrent neural network and propose a fine-grained two-stage pipeline implementation. Experiment results show that the proposed GPU implementation can achieve 2 ~ 11 x speed-up compared with the basic CPU implementation with the Intel Math Kernel Library. We then use the proposed GPU implementation to scale up the recurrent neural network and improve its performance. The experiment results of the Microsoft Research Sentence Completion Challenge demonstrate that the large scale recurrent network without class layer is able to beat the traditional class-based modest-size recurrent network and achieve an accuracy of 47%, the best result achieved by a single recurrent neural network on the same dataset.

Nonvolatile processors (NVP) have been an emerging topic in recent years due to its zero standby power, data retention and instant-on features. The conventional full replacement architecture in NVP has drawbacks of large area overhead and high backup energy. This paper provides a partial replacement based hybrid register architecture to significantly abate above problems. However, the hybrid register architecture can induce potential critical data loss and backup errors. In this paper, we propose a critical-data overflow aware register allocation (CORA). Different from other register allocation methods, CORA efficiently reduces the possibility of critical data spilling and backup errors. The experiment results show that CORA reduces the critical data overflow rate by up to 52%. The hybrid register architecture reduces the chip area by 45.1% and backup energy by 82.8% when using CORA.

Multi-stage latency adders based on different prediction schemes have been proved promising to enhance the circuit performance with negligible overhead. This paper presents a novel predictor exploiting both the detection and the sequence-dependence between the successive calculations. The detection of carry-kill pattern of the input data can lower the probability of the operation with multiple clock cycles and the sequence-dependence between the successive calculations is adapted to eliminate redundant cycles. The improved predictors have been inserted into Ripple Carry Adder (RCA) and a multistage latency structure has been setup. Compared with the previous predictors, the proposed one could have the same function with less prediction bits, which results in more energy-efficiency. Simulation results show that 2.41X-3.05X speedups can be achieved than the non-prediction counterpart. Furthermore, a design flow and a method for error control are proposed when applying the adder to approximate computation so that more performance improvement could be obtained after trading off certain precision.

Body channel communication (BCC) is a most power efficient way for communications among sensors in a wireless body-area network (WBAN). In BCC, the forward signal of the quasi-static field is conducted by the body surface, whereas the backward path is formed by the electrostatic coupling between the GND electrodes (GEs) of transmitter and receiver. As a result, the transmission loss is dominated by the backward path, which has high impedance due to small air capacitance between two compact GEs. Conventional backward path enhancement techniques make use of a large inductor to resonate with the air capacitance in order to reduce the impedance. Such approach is not suitable for integrated solution and not reconfigurable for varying communication distances. In this paper, we propose a novel active channel enhancer to compensate the loss in backward path, which is integratable and reconfigurable for variable distances and frequencies. Designed with 0.13 µm CMOS process, the proposed active enhancer improves the quasi-static coupling by more than 15 dB for a wide frequency band of 40 MHz–120 MHz compared to the 4 dB enhancement of conventional method; and the power consumption is only 0.6 mW.

With the miniaturization of electronic devices, small-size but high-capacity power supply systems appear to be more and more important. A hybrid power source, which consists of a fuel cell (FC) and a rechargeable battery, has the advantages of long lifetime and good load-following capabilities. In this article, we propose the schematic of a hybrid power supply system that can be integrated on a chip compatible with present CMOS processes. For the on-chip, fuel-cell-based hybrid power system in wireless sensor node design, we propose a two steps optimization: (1) dynamic power management (DPM), and (2) adaptive fuel cell optimal power point tracking (AOPPT). Simulation results demonstrate that the on-chip FC-Bat hybrid power system can be used for wireless sensor nodes under different usage scenarios. Our proposed DPM method can achieve 12.9% more energy savings than the method without DPM. Meanwhile, implementing our AOPPT approach can save about 17% energy compared with the fixed architecture for the fuel cell system. For an on-chip power system with 1cm 2 area consumption, the wafer-level battery can power a typical sensor node for only about five months, while our on-chip hybrid power system will supply the same sensor node for two years steadily.

Domain of stereo vision is highly important in the fields of autonomous cars, video tolling, robotics, and aerial surveys. The specific feature of this domain is that we should handle not only the pixel-by-pixel 2D processing in one image but also the 3D processing for depth estimation by comparing information about a scene from several images with different perspectives. This feature brings challenges to memory resource utilization, because an extra dimension of data has to be buffered. Due to the memory limitation, few of previous stereo vision implementations provide both accurate and high-speed processing for high-resolution images at the same time. To achieve domain-specific acceleration for stereo vision, the memory limitation has to be addressed. This article uses a Mini-Census ADaptive Support Region (MCADSR) stereo matching algorithm as a case study due to its high accuracy and representative operations in this domain. To relieve the memory limitation and achieve high-speed processing, the article proposes several efficient optimization methods including vertical-first cost aggregation, hybrid parallel processing, and hardware-friendly integral image. The article also presents a customizable system which provides both accurate and high-speed stereo matching for high-resolution images. The benefits of applying the optimization methods to the system are highlighted. With the aforesaid optimization and specific customization implemented on FPGA, the demonstrated system can process 47.6 fps (frames per second) and 129 fps for video size of 1920 × 1080 with a large disparity range of 256 and 1024 × 768 with a disparity range of 128, respectively. Our results are up to 1.64 times better than previous work in terms of Million Disparity Estimation per second (MDE/s). For accuracy, the 7.65% overall average error rate outperforms current work which can provide real-time processing with this high-resolution and large disparity range.

Nonvolatile (NV) processors have attracted much attention in recent years due to their zero standby power, resilience to power failures, and instant-on feature. One design challenge of NV processors is the excess area needed by NV registers. This paper introduces a parallel compare and compress (PaCC) architecture to reduce such excess area. A key component of the PaCC architecture is a new codec which effectively balances area and performance. In addition, the PaCC architecture includes a configurable state table to support reference vector selection for different applications. With the proposed vector selection algorithm, the PaCC architecture can outperform other vector selection approaches by over 59% in terms of reduction in the number of NV registers. The proposed architecture has been fully realized at the circuit level and synthesized for the Rohm's 0.13-μm ferroelectric-CMOS hybrid process. Results demonstrate that the design can reduce the number of NV registers by 70%-80% with less than 1% overflow possibility, which leads to up to 30% processor area saving. The overall approach is applicable to any NV processor design regardless of the NV material used.

Developing FPGA solutions for streaming applications written in C (or its variants) can benefit greatly from automatic C-to-RTL (C2RTL) synthesis. Yet, the complexity and stringent throughput/cost constraints of such applications are rather challenging for existing C2RTL synthesis tools. This paper considers automatic partition and block-level parallelization to address these challenges. An MILP-based approach is introduced for finding an optimal partition of a given program into blocks while allowing block-level parallelization. In order to handle extremely large problem instances, a heuristic algorithm is also discussed. Experimental results based on seven well known multimedia applications demonstrate the effectiveness of both solutions.

As different chips are stacked together in 3D ICs, the power/ground (P/G) network simulation becomes more challenging than that of 2D cases. In this paper, we propose a hierarchical simulation method suitable for 3D P/G network (HS3DPG), which can ensure full parallelism and good scalability with the number of tiers. In the IR drop analysis, when there are 9 tiers, the hierarchical method can be 6.5 times faster than the direct full network simulation. The accuracy of HS3DPG has been verified by a 3D P/G network from the industrial design. Besides, we introduce the “locality” property into HS3DPG to further simplify the simulation. Finally, HS3DPG is used to analyze the voltage distribution of a 3D P/G network with clustered TSVs.

Automatic C-to-RTL (C2RTL) synthesis can greatly benefit hardware design for streaming applications. However, stringent through-put/area constraints, especially the demand for power optimization at the system level is rather challenging for existing C2RTL synthesis tools. This paper considers a power-aware C2RTL framework using voltage-frequency islands (VFIs) to address these challenges. Given the throughput, area, and power constraints, an MILP-based approach is introduced to synthesize C-code into an RTL design by simultaneously considering three design knobs, i.e., partition, parallelization, and VFI assignment to get the global optimal solution. A heuristic solution is also discussed to deal with the scalability challenge facing the MILP formulation. Experimental results based on four well known multimedia applications demonstrate the effectiveness of both solutions.

Nonvolatile processor (NVP) has become an emerging topic in recent years. The conventional NV processor equips each flip-flop with a nonvolatile storage for data backup, which results in much faster backup speed with significant area overheads. A compression based architecture (PRLC) solved the area problem but with a nontrivial increasing on backup time. This paper provides a segment-based parallel compression (SPaC) architecture to achieve tradeoffs between area and backup speed. Furthermore, we use an off-line and online hybrid method to balance the workloads of different compression modules in SPaC. Experimental results show that SPaC can achieve 76% speed up against PRLC and meanwhile reduces the area by 16% against conventional NV processors.

Subsequence search, especially subsequence similarity search, is one of the most important subroutines in time series data mining algorithms, and there is increasing evidence that Dynamic Time Warping (DTW) is the best distance metric. However, in spite of the great effort in software speedup techniques, including early abandoning strategies, lower bound, indexing, computation-reuse, DTW still cost too much time for many applications, e.g. 80% of the total time. Since DTW is a 2-Dimension sequential dynamic search with quite high data dependency, it is hard to use parallel hardware to accelerate it. In this work, we propose a novel framework for FPGA based subsequence similarity search and a novel PE-ring structure for DTW calculation. This framework utilizes the data reusability of continuous DTW calculations to reduce the bandwidth and exploit the coarse-grain parallelism; meanwhile guarantees the accuracy with a two-phase precision reduction. The PE-ring supports on-line updating patterns of arbitrary lengths, and utilizes the hard-wired synchronization of FPGA to realize the fine-grained parallelism. It also achieves flexible parallelism degree to do performance-cost trade-off. The experimental results show that we can achieve several orders of magnitude speedup in accelerating subsequence similarity search compared with the best software and current GPU/FPGA implementations in different datasets.

The cessation of Moore's Law has limited further improvements in power efficiency. In recent years, the physical realization of the memristor has demonstrated a promising solution to ultra-integrated hardware realization of neural networks, which can be leveraged for better performance and power efficiency gains. In this work, we introduce a power efficient framework for approximated computations by taking advantage of the memristor-based multilayer neural networks. A programmable memristor approximated computation unit (Memristor ACU) is introduced first to accelerate approximated computation and a memristor-based approximated computation framework with scalability is proposed on top of the Memristor ACU. We also introduce a parameter configuration algorithm of the Memristor ACU and a feedback state tuning circuit to program the Memristor ACU effectively. Our simulation results show that the maximum error of the Memristor ACU for 6 common complex functions is only 1.87% while the state tuning circuit can achieve 12-bit precision. The implementation of HMAX model atop our proposed memristor-based approximated computation framework demonstrates 22× power efficiency improvements than its pure digital implementation counterpart.

The Modified Discrete Cosine Transform (MDCT) is widely used in audio signals compression, but mostly limited to representing audio signals. This is because the MDCT is a real transform: Phase information is missing and spectral power varies frame to frame even for pure sine waves. We have a key observation concerning the structure of the MDCT spectrum of a sine wave: Across frames, the complete spectrum changes substantially, but if separated into even and odd subspectra, neither changes except scaling. Inspired by this observation, we find that the MDCT spectrum of a sine wave can be represented as an envelope factor times a phase-modulation factor. The first one is shift-invariant and depends only on the sine wave's amplitude and frequency, thus stays constant over frames. The second one has the form of sinθ for all odd bins and cosθ for all even bins, leading to subspectra's constant shapes. But this θ depends on the start point of a transform frame, therefore, changes at each new frame, and then changes the whole spectrum. We apply this formulation of the MDCT spectral structure to frequency estimation in the MDCT domain, both for pure sine waves and sine waves with noises. Compared to existing methods, ours are more accurate and more general (not limited to the sine window). We also apply the spectral structure to stereo coding. A pure tone or tone-dominant stereo signal may have very different left and right MDCT spectra, but their subspectra have similar shapes. One ratio for even bins and one ratio for odd bins will be enough to reconstruct the right from the left, saving half bitrate. This scheme is simple and at the same time more efficient than the traditional Intensity Stereo (IS).

The sparse matrix solver has become a bottleneck in simulation program with integrated circuit emphasis (SPICE)-like circuit simulators. It is difficult to parallelize the solver because of the high data dependency during the numeric LU factorization and the irregular structure of circuit matrices. This paper proposes an adaptive sparse matrix solver called NICSLU, which uses a multithreaded parallel LU factorization algorithm on shared-memory computers with multicore/multisocket central processing units to accelerate circuit simulation. The solver can be used in all the SPICE-like circuit simulators. A simple method is proposed to predict whether a matrix is suitable for parallel factorization, such that each matrix can achieve optimal performance. The experimental results on 35 matrices reveal that NICSLU achieves speedups of$2.08\times\sim 8.57\times~({\rm on~the~geometric~mean})$, compared with KLU, with 1–12 threads, for the matrices which are suitable for the parallel algorithm. NICSLU can be downloaded from http://nicslu.weebly.com.

Reducing feature sizes and power supply voltage allows integrating more processing units (PUs) on multiprocessor system on chip (MPSoC) to satisfy the increasing demands of applications. However, it also makes MPSoC more susceptible to various reliability threats, such as high temperature and power/ground (P/G) noise. As the scale and complexity of MPSoC continuously increase, monitoring and mitigating reliability threats at runtime could offer better performance, scalability, and flexibility for MPSoC designs. In this paper, we propose a systematic approach, on-chip sensor network (SENoC), to collaboratively predict, detect, report, and alleviate runtime threats in MPSoC. SENoC not only detects reliability threats and shares related information among PUs, but also plans and coordinates the reactions of related PUs in MPSoC. SENoC is used to alleviate the impacts of simultaneous switching noise in MPSoC's P/G network during power gating. Based on the detailed noise behaviors under different scenarios derived by our circuit-level MPSoC P/G noise simulation and analysis platform, simulation results show that SENoC helps to achieve on average 26.2 percent performance improvement compared with the traditional stop-go method with 1.4 percent area overhead in an 8*8-core MPSoC in 45 nm. An architecture-level cycle-accurate simulator based on SystemC is implemented to study the performance of the proposed SENoC. By applying sophisticated scheduling techniques to optimize the total system performance, a higher performance improvement of 43.5 percent is achieved for a set of real-life applications.

Sparse matrix solver has become the bottleneck in SPICE simulator. It is difficult to parallelize the solver because of the high data-dependency during the numerical LU factorization. This paper proposes a parallel LU factorization (with partial pivoting) algorithm on shared-memory computers with multi-core CPUs, to accelerate circuit simulation. Since not every matrix is suitable for parallel algorithm, a predictive method is proposed to decide whether a matrix should use parallel or sequential algorithm. The experimental results on 35 circuit matrices reveal that the developed algorithm achieves speedups of 2.11×∼8.38× (on geometric-average), compared with KLU, with 1∼8 threads, on the matrices which are suitable for parallel algorithm. Our solver can be downloaded from http://nicslu.weebly.com.

In modern embedded systems, the C2RTL (high-level synthesis) technology helps the designer to greatly reduce time-to-market, while satisfying the performance and cost constraints. To attack the performance challenges in complex designs, we propose a FIFO-connected hierarchical approach to replace the traditional flatten one in stream applications. Furthermore, we develop an analytical algorithm to find the optimal FIFO capacity to connect multiple modules efficiently. Finally, we prove the advantages of the proposed method and the feasibility of our algorithm in seven real applications. Experimental results show that the hierarchical approach can have an up to 10.43 times speedup compared to the flatten design, while our analytical FIFO sizing algorithm shrinks design time from hours to seconds with the same accuracy compared to the simulation based approach.

The Mote architecture is the most popular platform used in wireless sensor network applications. In this architecture, microcontroller is responsible for all jobs, such as scheduling, sampling, computing, and communication. In the past one year, two practical applications: bridge structural health monitoring system and rare animal monitoring system are developed and deployed in Wuxi and Beijing, China. It is found that Mote architecture faces many problems in these applications. First, sampling, computing, and communication conflicts with each other if they are not carefully scheduled; second, some jobs are very difficult even impossible to be implemented in the microcontroller; third, low power, one of the most fundamental design principles in wireless sensor networks, is sometimes violated with all jobs implemented in the microcontroller. Software optimization is attempted to solve these problems. However, the effect is very limited. Application specific sensor node architecture is necessary for implementing these applications efficiently. In this paper, we propose new application specific sensor node architecture and corresponding design principles and then applied them in the field deployments. Experimental and field tests show that these architectures are more efficient than Mote architecture in these applications.

Sparse solver has become the bottleneck of SPICE simulators. There has been few work on GPU-based sparse solver because of the high data-dependency. The strong data-dependency determines that parallel sparse LU factorization runs efficiently on shared-memory computing devices. But the number of CPU cores sharing the same memory is often limited. The state of the art Graphic Processing Units (GPU) naturally have numerous cores sharing the device memory, and provide a possible solution to the problem. In this paper, we propose a GPU-based sparse LU solver for circuit simulation. We optimize the work partitioning, the number of active thread groups, and the memory access pattern, based on GPU architecture. On matrices whose factorization involves many floating-point operations, our GPU-based sparse LU factorization achieves 7.90x speedup over 1-core CPU and 1.49x speedup over 8-core CPU. We also analyze the scalability of parallel sparse LU factorization and investigate the specifications on CPUs and GPUs that most influence the performance.

Nonvolatile processor has become an emerging topic in recent years due to its zero standby power, resilience to power failures and instant on feature. This paper first demonstrated a fabricated nonvolatile 8051-compatible processor design, which indicates the ferroelectric nonvolatile version leads to over 90% area overhead compared with the volatile design. Therefore, we proposed a compare and compress recovery architecture, consisting of a parallel run-length codec (PRLC) and a state table logic, to reduce the area of nonvolatile registers. Experimental results demonstrate that it can reduce the number of nonvolatile registers by 4 times with less than 1% overflow possibility, which leads to 43% overall processor area savings. Furthermore, we implemented the novel PRLC and defined the method to optimize the optimal parallel degree to accelerate the compressions. Finally, we proposed a reconfigurable state table architecture, which supports the reference vector selecting for different applications. With our heuristic vector selecting algorithm, the optimal vector can provide over 42% better register number reduction than other vector selecting approaches. Our method is also applicable to designs with other nonvolatile materials based registers.

This paper presents an FPGA based stereo vision system for future video tolling, which can achieve real-time processing for high resolution video streams. The key component for the system is SAD (Sum of Absolute Differences) based stereo matching. Although simple and effective, this method usually needs much computation power to satisfy real-time requirement. We propose a Hybrid-D Box-Filtering algorithm in hardware to explore disparity-level and row-level parallelism for SAD computation. This method enables processing of high resolution images with limited on-chip memory resources. The experimental results show that the system can process 46 fps (frames per second) for video of 1280*1024 resolution with a large disparity range of 256, and 400 fps for a video of 640*480 resolution with a disparity range of 128. Our results are up to 3 times better than previous work in the metric of points times disparity per second (PDS).

This paper presents an all-digital Gaussian frequency shift keying (GFSK) demodulator with robust clock data recovery (CDR) for low-intermediate-frequency (low-IF) receivers in wireless sensor networks (WSN). The proposed demodulator can detect and adapt to the intermediate frequency of the received signal automatically. In addition, the CDR can tolerate the frequency deviation of the input clock. An implementation of the demodulator with CDR is realized with HJTC 0.18 ¼m CMOS technology. The chip is designed for GFSK signals with a center frequency of 200 kHz, a modulation index of 1 and a data rate of 100 kbps. Experimental results show that the chip consumes 0.53 mA from a 1.8 V power supply, and only a 11 dB input signal to noise ratio (SNR) is required for 10-3 bit error rate (BER). The tolerance range for IF offset is \pm12.5% at 11 dB input SNR, and the CDR can tolerate frequency deviation of the input clock of \pm0.1%.

This paper describes an informed unipolar spread spectrum (SS) modulation to improve the performance of SS-based watermark extraction with self-synchronization. Compared with bipolar SS, unipolar SS, realized by employing a pair of orthogonal codes to represent the watermark bits, is more robust against the distorted correlation between the received watermarked signal and the reference. The informed embedding utilizes the host interference to improve the performance of unipolar watermark synchronization and decoding. Experiments on audio signals show that the proposed scheme is robust against most of the attacks while preserving the media fidelity, and effective self-synchronization is achieved.

Energy harvesting sensor nodes based on real nonvolatile processors are demonstrated to show the desirable characteristics of those systems, such as no battery, zero stand-by power, microsecond-scale sleep and wake-up time, high resilience to random power failures and fine-grained power management. Furthermore, we show its applications to a distributed moving object detection system, one of novel nonvolatile computing systems.

For the node chips of wireless sensor networks (WSN), low power and fast settling are the two most important factors. In this paper, a low-power fast-settling phase-locked loop (PLL) frequency synthesizer working at 1.72 GHz∼1.74 GHz is designed for a 100 kb/s gauss frequency shift keying (GFSK) WSN transceiver. Low power consumption is realized by a bond-wire voltage-controlled oscillator (VCO) and a multi-stage power-scaling prescaler. Instead of conventional diode-based electro-static discharge (ESD) protection, resistor-based ESD protection is proposed for the bond-wire VCO to decrease the parasitic capacitance so that the automatic frequency calibration (AFC) range is enlarged by 50%. In addition, a dynamic-bandwidth scheme is proposed to meet the requirements of time-division half-duplex WSN systems. The chip is implemented with HJTC 0.18 µm CMOS technology. Measured results show that the PLL consumes 10.6 mW and settles within 18 µs including the AFC process; the phase noise is −91.9 dBc/Hz@10 kHz and −119.3 dBc/Hz@1 MHz under the receiving (Rx) state, and −95.2 dBc/Hz@10 kHz and −116.8 dBc/Hz@1 MHz under the transmitting (Tx) state.

Three-dimensional (3D) stacking technology enables integration of more memory on top of chip multiprocessors (CMPs). As the number of cores and the capacity of on-chip memory increase, the Non-Uniform Cache Architecture (NUCA) becomes more attractive. Compared to 2D cases, 3D stacking provides more options for the design of on-chip memory due to numerous advantages, such as the extra layout dimension, low latency across layers, etc. On the other hand, 3D stacking aggravates the thermal problem due to the increase of power density. In this work, we first study the design of 3D-stacked set-associative L2 caches through managing the placement of cache ways. The evaluation results show that the placement and corresponding management of 3D cache ways have an impact on the performance of CMPs. Then, we show that the efficiency of thermal control is also related to the placement of cache ways. For caches implemented with different memory technologies, the placement and management of cache ways have different effects on power consumption and power distribution. Consequently, we propose techniques to improve the efficiency of thermal control for different memory technologies. The evaluation results show the trade-off between performance and thermal control efficiency.

As technology scales, negative bias temperature instability (NBTI) has become a major reliability concern for circuit designers. And the growing process variations can no longer be ignored. Meanwhile, reducing power consumption remains to be one of the design goals. In this paper, a variation-aware supply voltage assignment (SVA) technique combining dual$V_{dd}$assignment and dynamic$V_{dd}$scaling is proposed on a statistical platform, to minimize circuit power under an aging-aware timing constraint. The experimental results show that our SVA technique can mitigate on average 62% of the NBTI-induced circuit delay degradation. Compared with guard-banding and single$V_{dd}$scaling approaches, our approach saves more energy.

With the miniaturization of electronic devices, small size but high capacity power supply system appears to be more and more important. A hybrid power source, which consists of a fuel cell (FC) and a rechargeable battery, has the advantages of long lifetime and good load following capabilities. In this paper, we propose the schematic of a hybrid power supply system, that can be integrated on a chip compatible with present CMOS process. Besides, considering the problem of maximizing the on-chip fuel cell's lifetime, we propose a modified dynamic power management (DPM) algorithm for on-chip fuel cell based hybrid power system in wireless sensor node design. Taking the wireless sensor node powered by this hybrid power system as an example, we analyze the improvement of the FC-Bat hybrid power system. The simulation results demonstrate that the on-chip FC-Bat hybrid power system can be used for wireless sensor node under different usage scenarios. Meanwhile, for an on-chip power system with 1cm2area consumption, the wafer-level battery can power a typical sensor node for only about 5 months, while our on-chip hybrid power system will supply the same sensor node for 2 years steadily.

Nowadays, Graphics Processing Unit (GPU), as a kind of massive parallel processor, has been widely used in general purposed computing tasks. Although there have been mature development tools, it is not a trivial task for programmers to write GPU programs. Based on this consideration, we propose a novel parallel computing architecture. The architecture includes a parallel programming model, named Gemma, and a programming framework, named April. Gemma is based on generalized matrix operations, and helps to alleviate the difficulty of describing parallel algorithms. April is a high-level framework that can compile and execute tasks described in Gemma with OpenCL. In particular, April can automatically 1) choose the best parallel algorithm and mapping scheme, and generate OpenCL kernels, 2) schedule Gemma tasks based on execution costs such as data storing and transferring. Our experimental results show that with competitive performance, April considerably reduces the programs' code length compared with OpenCL.

The research on understanding the human brain has attracted more and more attention. A promising method is to model the brain as a network based on modern imaging technologies and then to apply graph theory algorithms for analysis. In this work, we examine the computing bottleneck of this method, and propose a CPU-GPU heterogeneous platform to accelerate the process. We construct a statistical brain network from a sample of 198 people and get characteristics such as nodal degree and modularity. This is the first study of voxel-based brain networks on large samples. We also illustrate that domain-specific hardware platform can have a significant impact on neuroscience studies.

Multi-core processor Simulation Platform is always a very important tool in modern multi-core processor design for the system-level design and evaluation. In this paper, a multi-core processor simulator is proposed by modifying Simple Scalar v3.0to simulate parallelized multi-core programs. Shared memory is used for the communication between different cores, which is the communication network among several different parts of the parallelized program separately. Two simulators are designed for different kinds of usage, one for functional simulation and the other for the simulation of the system with two-level cache. The mismatch of such simulator is less than 10% on average, and the presented simulator is used to evaluate the high-performance video processing systems.

In this paper, a simple, efficient, low power off-chip memory design is proposed, which fully exploits the features of DRAM memory and video application, as well as overcomes the drawbacks of algorithm complexity and system modification of embedded compression, which is a popular way to decrease power consumption of the off-chip memory. The integration of the scheme into video decoder will not involve any extra video decoding complexity. It adopts the simple bus-invert encoding scheme. Based on the fact that the power consumption of logic `0' bit is less than that of logic `1', bus-invert encoding scheme is applied to the transferring data between video decoder and off-chip memory. Meanwhile, the features of fault tolerance of human eyes and lossy processing of video decoding application are exploited to solve the extra flag-bit of encoder scheme in off-chip SDARM memory, which has the fixed bit width and is less flexible than on-chip SRAM. This scheme is integrated into MPEG-2 decoder system. The experiment results show that this scheme can archive 20%-35% reduction in power consumption of logic `1' bit, and the objective quality of image has about 1.5db PSNR improvement on average.

As variable delays are observed in the integrated circuits under different data inputs, it is expected to enhance the performance of the circuit using the average-case design methodology. This paper presents a novel approach using the time-domain multistage speculation to realize a variable-latency circuit, in which speculation points with double-sampling and check-recovery units are inserted into the critical path to enhance the performance. Furthermore, a design framework is implemented to convert a original circuit into the new one automatically. Experimental results showed that a 1.79 - 4.42 speedup in a 64-bit ripple carry adder and up to 30.5% throughput enhancements in several ISCAS and MCNC benchmarks with reasonable area overheads.

In this study, the authors propose a new method for constructing irregular low-density parity-check codes (LDPC codes) with midterm or short block lengths. It is a Monte-Carlo simulation-based optimisation procedure called downhill simplex which uses progressive edge growth method for Tanner graph construction in cost function evaluations. It outperforms the density evolution (DE) method in both efficiency and performance. The efficiency of the authors' proposal mostly depends on the Monte-Carlo simulation time which owing to the midterm block lengths is relatively small compared to the time delay of the conventional DE method. Simulation results show that for block length of 3200 and a relatively low code rate 3/8, LDPC codes constructed using the authors' proposal noticeably outperform those constructed with DE method by more than 0.2 dB in the entire signal to noise ratio (SNR) range under AWGN channel.

As technology scales, Negative Bias Temperature Instability (NBTI), which causes temporal performance degradation in digital circuits by affecting PMOS threshold voltage, is emerging as one of the major circuit reliability concerns. In this paper, we first investigate the impact of NBTI on PMOS devices and propose a temporal performance degradation model that considers the temperature variation between active and standby mode. We then discuss the resemblance between NBTI and leakage mechanisms, and find out that the impact of input vector and internal node on leakage and NBTI is different; hence, leakage and NBTI should be optimized simultaneously. Based on this, we study the impact of standby leakage reduction techniques (including input vector control and sleep transistor insertion) on circuit performance degradation considering active and standby temperature differences. We demonstrate the potential mitigation of the circuit performance degradation by these techniques.

As technology scales, the aging effect caused by negative bias temperature instability (NBTI) has become a major reliability concern. In the mean time, reducing leakage power remains to be one of the key design goals. Because both NBTI-induced circuit degradation and standby leakage power have a strong dependency on the input vectors, input vector control (IVC) technique could be adopted to reduce the leakage power and mitigate NBTI-induced degradation. The IVC technique, however, is ineffective for larger circuits. Consequently, in this paper, we propose two gate replacement algorithms [direct gate replacement (DGR) algorithm and divide and conquer-based gate replacement (DCBGR) algorithm], together with optimal input vector selection, to simultaneously reduce the leakage power and mitigate NBTI-induced degradation. Our experimental results on 23 benchmark circuits reveal the following. 1) Both DGR and DCBGR algorithms outperform pure IVC technique by 15%–30% with 5% delay relaxation for three different design goals: leakage power reduction only, NBTI mitigation only, and leakage/NBTI cooptimization. 2) The DCBGR algorithm leads to better optimization results and save on average more than 10$\times$runtime compared to the DGR algorithm. 3) The area overhead for leakage reduction is much more than that for NBTI mitigation.

Shrinking the feature size allows more and better functions on a single chip. However, it makes multiprocessor system-on-chip (MPSoC) more susceptible to various reliability threats. Power supply noise is a major reliability problem faced by low power MPSoCs using power gating techniques. Powering on and off a processing unit in MPSoCs will induce large power/ground (P/G) noise and can cause timing divergence and even functional errors in surrounding processing units. Previous work on resilient architectures mainly focused on power/thermal management and neglected the important side-effect: P/G noise induced by power gating. In this paper, for the first time, we formulate a task scheduling problem with the consideration of P/G noise based on our detailed P/G noise analysis platform for MPSoC. Two efficient algorithms are proposed to reduce noise protection penalty and improve MPSoC performance. Our experiments show that both simulated annealing and heuristic algorithms can achieve on average 25% performance improvement together with up to 80% noise protection penalty saving compared with the conservative stop-go method for short tasks (shorter than 20 K clock cycles). For longer tasks up to 200 K clock cycles, the performance improvement of our methods will become relatively low. However, we can still achieve at least 35.2% noise protection penalty saving. Furthermore, a lightweight online adjustment strategy accompanying the offline scheduling method is proposed to adapt to runtime variations and improve reliability.

Machine learning and data mining are gaining increasing attentions of the computing society. FPGA provides a highly parallel, low power, and flexible hardware platform for this domain, while the difficulty of programming FPGA greatly limits its prevalence. MapReduce is a parallel programming framework that could easily utilize inherent parallelism in algorithms. In this paper, we describe FPMR, a MapReduce framework on FPGA, which provides programming abstraction, hardware architecture, and basic building blocks to developers.

The spectrum of a sinusoid using the Modified Discrete Cosine Transform (MDCT), when separated into an even subspectrum and an odd subspectrum by bin parity, gives rise to a distinctive property-subspectral shapes are independent of the sinusoid phase, which contributes only to scaling. Based on this finding, we propose an Even-Odd (EO) scheme for stereo coding: partitioning the even and odd subspectra separately into subbands to capture the fine spectral structures of sinusoidal and rich tone signals. The scheme reduces the coding noises by 0-20 dB for music signals. When integrated into a MDCT domain KLT-based stereo coder, the scheme boosts subjective listening test (MUSHRA) scores. This coder, called KLT-EO, competes the Parametric Stereo (PS) in quality by a slightly higher bitrate but without the algorithmic delay of 20 ms resulted from the stereo processing.

This paper presents a linear operation called Maximal Coherence Rotation (MCR) on paired vectors for stereo audio coding. Intrigued by the idea that stronger coherence between paired channels will lead to higher stereo coding efficiency, we develop MCR to maximize the coherence restricted by being invertible and energy-conserving. It results in equal energy, minimized difference, and always non-negative coherence for the pair of channels processed. In binaural hearing, this can be viewed as turning a physical sound source at any azimuth to a virtual one on the median plane. A prototype MCR stereo coder shows significantly higher quality for some test sequences than that of AMR-WB+, one of the best low bitrate stereo coders in the public domain. And as a preprocessing tool for MPEG-4 Parametric Stereo (PS), MCR avoids out-of-phase inter-channel cancellation during 2-to-1 channel downmixing without any additional bandwidth requirement, thanks to the maximized coherence.

The research on complex Brain Networks plays a vital role in understanding the connectivity patterns of the human brain and disease-related alterations. Recent studies have suggested a noninvasive way to model and analyze human brain networks by using multi-modal imaging and graph theoretical approaches. Both the construction and analysis of the Brain Networks require tremendous computation. As a result, most current studies of the Brain Networks are focused on a coarse scale based on Brain Regions. Networks on this scale usually consist around 100 nodes. The more accurate and meticulous voxel-base Brain Networks, on the other hand, may consist 20K to 100K nodes. In response to the difficulties of analyzing large-scale networks, we propose an acceleration framework for voxel-base Brain Network Analysis based on Graphics Processing Unit (GPU). Our GPU implementations of Brain Network construction and modularity achieve 24x and 80x speedup respectively, compared with single-core CPU. Our work makes the processing time affordable to analyze multiple large-scale Brain Networks.

Reducing feature sizes and power supply voltage allows integrating more processing units (PUs) on multiprocessor system-on-chip (MPSoC) to satisfy the increasing demands of applications. However, it also makes MPSoC more susceptible to various reliability threats, such as high temperature and power/ground (P/G) noise. As the scale and complexity of MPSoC continuously increase, monitoring and mitigating reliability threats at run time could offer better performance, scalability, and flexibility for MPSoC designs. In this paper, we propose a systematic approach, on-chip sensor network (SENoC), to collaboratively detect, report, and alleviate run-time threats in MPSoC. SENoC not only detects reliability threats and shares related information among PUs, but also plans and coordinates the reactions of related PUs in MPSoC. SENoC is used and explained in our case study to alleviate the impacts of simultaneous switching noise in MPSoC's P/G network during power gating. Based on the detailed noise behaviors under different scenarios derived by our circuit-level MPSoC P/G noise simulation and analysis platform, simulation results show that SENoC helps to achieve on average 26.12% performance improvement compared with the traditional stop-go method with 1.4% area overhead in an 8*8-core MPSoC in 45nm.

As technology scales, the aging effect caused by Negative Bias Temperature Instability (NBTI) has become a major reliability concern for circuit designers. On the other hand, reducing leakage power remains to be one of the design goals. Because both NBTI-induced circuit degradation and standby leakage power have a strong dependency on the input vectors, Input Vector Control (IVC) technique may be adopted to mitigate leakage and NBTI. However, IVC technique is in-effective for larger circuits. Therefore, in this paper, we propose two fast gate replacement algorithms together with optimal input vector selection to simultaneously mitigate leakage power and NBTI induced circuit degradation: Direct Gate Replacement (DGR) algorithm and Divide and Conquer Based Gate Replacement (DCBGR) algorithm. Our experimental results on 20 benchmark circuits at 65nm technology node reveal that: 1) Both DGR and DCBGR algorithms outperform pure IVC about on average 20% for three different object functions: leakage power reduction only, NBTI mitigation only, and leakage/NBTI co-optimization. 2) The DCBGR algorithm leads to better optimization results and save on average 100X runtime compared with the DGR algorithm.

Wireless sensor networks hold the potential to open new domains to distributed data acquisition. However, low-cost battery-powered nodes are often used to implement such networks, resulting in tight energy and communication bandwidth constraints. Cluster-based data compression and aggregation helps to reduce communication energy consumption. However, neglecting to adapt cluster sizes to local network conditions has limited the efficiency of previous clustering schemes. We have found that sensor node distances and densities are key factors in clustering. To the best of our knowledge, this is the first work taking these factors into consideration when adaptively forming data aggregation clusters. Compared with previous uniform-size clustering techniques, the proposed algorithm achieves up to 24% communication energy savings in uniform density networks and 36% savings in non-uniform density networks.

In this paper, we propose an energy efficient architecture of wireless sensor network node. It consists of a general-purpose processor and several compression accelerators. To verify the low energy consumption of this architecture, we implement a baseband chip of sensor node by 1-poly 6-metal 0.18um CMOS technology, in which a hardware accelerator is realized based on a distributed wavelet compression algorithm. Our measurements show that the compression accelerator based architecture reduces over 98% energy consumption compared with the traditional solution.

Wireless sensor networks hold the potential to open new domains to distributed data acquisition. However, such networks are prone to premature failure because some nodes deplete their batteries more rapidly than others due to workload variations, non-uniform communication, and heterogenous hardware. Many-to-one traffic patterns are common in sensor networks, further increasing node power consumption heterogeneity. Most previous sensor network lifetime enhancement techniques focused on balancing power distribution, based on the assumption of uniform battery capacity allocation among homogeneous nodes.

Abstract—As technology scales, negative bias temperature instability (NBTI) has become a major reliability concern for circuit designers. And the growing process variations can no longer be ignored. Meanwhile, reducing power consumption remains to be one of the design goals. In this paper, a variation-aware supply voltage assignment (SVA) technique combining dual assignment and dynamic scaling is proposed on a statistical platform, to minimize circuit power under an aging-aware timing constraint. The experimental results show that our SVA technique can mitigate on average 62 % of the NBTI-induced circuit delay degrada-tion. Compared with guard-banding and single scaling approaches, our approach saves more energy. Index Terms—Dynamic power, leakage power, negative bias temperature instability (NBTI), supply voltage assignment (SVA). I.

In this paper, we propose a new capacitive boosted buffer technique that can be used in high speed interconnect for ultra-dynamic voltage scaling (UDVS) application with the process variation effect mitigated. The circuit is simple and fully compatible with digital CMOS technology. Implemented in a standard 0.18 μm CMOS technology, the circuit is shown applicable for both sub-threshold circuit and above threshold circuit without the problem of short current. Simulation results demonstrate the conclusion that the proposed new buffer is more robust to load, process, voltage, and temperature (PVT) variations. When applied to a simple H-tree clock network, the proposed buffer can reduce the skew by 5.5x when compared to that of the traditional buffer.

There are many algorithms to solve large sparse linear systems in parallel; however, most of them acquire synchronization and thus are lack of scalability. In this paper, we propose a new distributed numerical algorithm, called Directed Transmission Method (DTM). DTM is a fully asynchronous, scalable and continuous-time iterative algorithm to solve the arbitrarily-large sparse linear system whose coefficient matrix is symmetric-positive-definite (SPD). DTM is able to be freely running on the heterogeneous parallel computer with arbitrary number of processors, which might be manycore microprocessors, clusters, grids, clouds, and the Internet. We proved that DTM is convergent by making use of the final value theorem of Laplacian Transformation. Numerical experiments show that DTM is efficient.

In order to save the limited bandwidth resource, high-order M-ary amplitude and Phase Shift Keying (APSK) signal constellation as quadrature amplitude modulation (QAM) is widely used in modern bandwidth-limited communication and broadcasting systems. However, such dense constellation tends to be very sensitive to the disturbance of noise, including not only white Gaussian noise but also phase noise which is caused by many nonideal factors in transmitter, receiver and channel. This paper extended a classical constellation optimization algorithm in white Gaussian noise to the context where phase noise is considered as well. And through the proposed algorithm, a 64 points signal constellation with outstanding performance is obtained. Simulation indicates the symbol error performance of the new constellation is not only 6dB better than QAM at error rate of 10-3in phase noise of 0.1rad, but also comparative with QAM when only white Gaussian noise is considered.

In this paper, we propose some soft-error-tolerant latches and flip-flops that can be used in dual-VDD systems. By utilizing local redundancy and inner feedback techniques, the latches and flip-flops can recover from soft errors caused by cosmic rays and particle strikes. The proposed flip-flop can be used as a level shifter without the problems of static leakage and redundant switching activity. Implemented in a standard 0.18- mum technology, the proposed latches and flip-flops show superior performance compared to conventional ones in terms of delay and power while keeping the soft-error-tolerant characteristic. Experimental results show that compared to the traditional built-in soft-error-tolerant D latch, the D-QN delay of the new D latch is 29.1% less than that of the traditional built-in soft-error-tolerant D latch while consuming 16.5% less power as well. The D-Q delay and power of the new flip-flop are about 47.7% and 54% less than those of the traditional high speed level-converting flip-flop, respectively. In addition, the proposed flip-flop is more robust to soft errors. The critical charge which represents the minimum charge at the D input required to cause an error of the flip-flop can be increased by more than 46.4%. The time window during which the flip-flop will be erroneous caused by single-event upsets at the D input is reduced by more than 22.2%.

Sleep transistor (ST) insertion is a valuable leakage reduction technique in circuit standby mode. Fine-grain sleep transistor insertion (FGSTI) makes it easier to guarantee circuit functionality and improve circuit noise margins. In this paper, we introduce a novel two-phase FGSTI technique which consists of ST placement and ST sizing. These two phases are formally modeled using mixed integer linear programming (MILP) models. When the circuit timing relaxation is not large enough to assign ST everywhere, leakage feedback (LF) gates, which are used to avoid floating states, induce large area and dynamic power overhead. An extended multi-object ST placement model is further proposed to reduce the leakage current and the LF gate number simultaneously. Finally, heuristic algorithms are developed to speed up the ST placement phase. Our experimental results on the ISCAS'85 benchmarks reveal that: 1) the two-phase FGSTI technique achieves better results than the simultaneous ST placement and sizing method; 2) when the circuit timing relaxation varies from 0% to 5%, the multi-object ST placement model can achieve on average 4times-9times LF gate number reduction, while the leakage difference is only about 8% of original circuit leakage; 3) our heuristic algorithm is 1000times faster than the MILP method within an acceptable loss of accuracy.

As technology scales, negative bias temperature instability (NBTI), which causes temporal performance degradation in digital circuits by affecting PMOS threshold voltage, is emerging as one of the major circuit reliability concerns. In this paper, the authors first investigate the impact of NBTI on PMOS devices and propose a novel temporal performance degradation model for digital circuits considering the temperature difference between active and standby mode. For the first time, the impact of input vector control (to minimize standby leakage) on the NBTI is investigated. Minimum leakage vectors, which lead to minimum circuit performance degradation and remains maximum leakage reduction rate, are selected and used during the standby mode. Furthermore, the potential to save the circuit performance degradation by internal node control techniques during circuit standby mode is discussed. Our simulation results show that: 1) the active and standby time ratio and the standby mode temperature have considerable impact on the circuit performance degradation; 2) the NBTI-aware IVC technique leads to an average 3% savings of the total circuit degradation; while the potential of internal node control may lead to 10% savings of the total circuit degradation

Power has become an important concern for nanometer circuit design as well as timing characteristic. In this paper, a novel low power interface circuit design technique was proposed for multiple voltage islands scheme by using output feedback, conditional switch, and pulsed clock technique. The method was applied to new types of flip-flops and combinational logics to eliminate level converters and remove redundant switching activities. Combined with multiple VTH technique, a low clock swing flip-flop is designed to verify our new method. Experimental results show that the leakage power of the new flip-flop can be reduced by an average of 58.14% in standby mode and the total power consumption can be reduced by an average of 55.76% in active mode, while the delay time stays the same

This paper presents a noise reduced Digitally Controlled Oscillator (DCO) using complementary varactor pairs for PHS transceivers. Due to some modifications on the varactors, reduced phase noise and increased frequency resolution are obtained for our DCO. The DCO is designed in a 0.18μm CMOS process with a central running frequency of 3.8GHzand over 1GHz tuning range. Simulation results show that the phase noise at 1.2MHzoffset frequency is below -123dBc/Hzwhile drawing only 2.8mA of current from a 1.8V supply. It demonstrates that our DCO achieves improved phase noise and power consumption while its performance has no dependence on the feature size of the given process.

Short-wave together with medium-and longwave broadcasting still has large listenership worldwide. With the newly developed digital system DRM will overcome these reception problems and bring high audio quality, which will change the whole broadcasting scenario. This paper will give an overview of DRM including its development history, the standardization, the receiving technology, the key features and state of the art. The emergence of DRM lights the way of digital radio ahead.

In deep-submicrometer integrated circuits, inductance effects have become increasingly significant, and interconnects are often modeled as transmission lines. An equivalent circuit model, which is called the γd/n model and regularly constructed by two elementary resistance–capacitance–inductance–conductance cells, is proposed for transmission line modeling in this paper. Unlike those moment-matching techniques, the γd/n model is a physical model in nature, and it can be applied to generated high-accuracy, stable, passive, and reduced-order models of transmission lines. In addition, RC(L) interconnect loads are also considered in this paper. Experimental results show that the proposed method can accurately capture the transmission line effect and the capacitive load effect. The waveform obtained by this method differs from that by the exact model in SPICE simulation with the average voltage difference less than 0.9%. For a wide range of interconnect parameters, it has been shown that this method can achieve less than 6% in average error of the 50% delays for both the near-end response and the far-end response.

A clock and data recovery (CDR) circuit is proposed based on the sigma-delta quantization. The phase of the new CDR circuit is adjusted by a sigma-delta modulated reference clock that increases the stability of the system and can easily interface with PLL cores embedded in FPGAs. The approximate linear model of the proposed CDR is analyzed for SONET/SDH applications to evaluate its performance. The measurement shows that the jitter tolerance meets the ITU-T requirement with a high margin of 0.3UI. The commercial equipment has been developed using a single FPGA chip based on the SDM-CDR

Multi-threshold CMOS is a valuable leakage reduction method in circuit standby mode. Reducing leakage current through fine-grain sleep transistor insertion (FGSTI) makes it easier to guarantee circuit functionality and improves circuit noise margins. In this paper, we first indicate the negligible dependence of ST size on the amount of leakage saving which makes the two-phase FGSTI reasonable based on our leakage current and delay models. Then we introduce a novel two-phase FGSTI technique: a) ST placement and b) ST sizing, which are formally modeled as two linear programming (LP) models respectively. Our experimental results show that the two-phase FGSTI technique can achieve 78.91%, 92.55%, 97.97% leakage saving when the circuit slowdown is 0%, 3%, 5% respectively. Comparing to the simultaneous ST placement and sizing method using mix integer linear programming (MLP) [1], our technique leads to on average 2% more leakage current reduction while at least 10X runtime saving since fewer variables and constraints with less approximation are used in the LP models. When the circuit slowdown is large enough to perform conventional fixed slowdown method, our technique can still achieve 75.48% ST area saving. Moreover, we show that when the circuit slowdown is 0%, it should be carefully considered to use FGSTI technique due to a large amount of leakage feedback gates.

Power grid plays an important role in determining circuit performance, and the accuracy and efficiency of power grid analysis algorithm has become critical in timing, power and noise estimation of modern integrated circuits. In this paper a stochastic algorithm based on Gibbs sampling is proposed to solve the problem of power grid analysis, and the test results shows that it reaches a good accuracy with linear complexity. The method has incremental property of localizing computation, a desirable property favoring in modern CAD. Therefore it can be embedded at all the design and verification levels of integrated circuits.

In deep submicron (DSM) integrated circuits (IC), coupling capacitors between interconnects become dominant over grounded capacitors. As a result, the dynamic power dissipation of one node is no longer only in relation to the signal on that node, and it also depends on signals on its neighbor nodes through coupling capacitors. Thus, for their limitation in dealing with capacitively coupled nets, past jobs on power estimation are facing rigorous challenges and need to be ameliorated. This paper proposes and proves a simple and fast approach to predicting dynamic power dissipation of coupled interconnect networks: a coupling capacitor in dynamic CMOS logic circuits is decoupled and mapped into an equivalent cell containing an XOR gate and a grounded capacitor, and the whole circuit after mapping, consuming the same power as the original one, could be easily managed by generally-used gate-level power estimation tools. This paper also investigates the correlation coefficient method (CCM). Given the signal probabilities and the correlation coefficients between signals, the dynamic power of interconnect networks can be calculated by using CCM. It can be proved that the decoupling method and CCM draw identical results, that is to say, the decoupling method implicitly preserves correlation properties between signals and there is no accuracy loss in the decoupling process. Moreover, it is addressed that the coupling capacitors in static CMOS circuits could be decoupled and mapped into an equivalent cell containing a more complicated logic block, and the power can be obtained by the probability method for dynamic CMOS logic circuits.

Nonlinear functions can be approximated by the linear combination of base functions, which provides a road towards the analog synthesis. An improved Simulated Annealing Algorithm (SA) for nonlinear function approximation and a universal implementation of analog circuits are presented in this paper. Synthesis results demonstrate the validity and efficiency of the proposed approach.