Neural network (NN) accelerators with multi-chipmodule (MCM) architectures enable integration of massive computation capability; however, they face challenges of computing resource underutilization and off-chip communication overheads. Traditional parallelization schemes for NN inference on MCM architectures, such as intra-layer parallelism and inter-layer pipelining, show incompetency in breaking through both challenges, limiting the scalability of MCM architectures. We observed that existing works typically deploy layers separately rather than considering them jointly. This underexploited dimension leads to compromises between system computation and communication, thus hindering optimal utilization, especially as hardware/software scale. To address this limitation, we propose Scope, a merged pipeline framework incorporating this overlooked multi-layer dimension, thereby achieving improved throughput and scalability by relaxing tradeoffs between computation, communication and memory costs. This new dimension, however, adds to the complexity of design space exploration (DSE). To tackle this, we develop a series of search algorithms that achieves exponential-to-linear complexity reduction, while identifying solutions that rank in the top 0.05% of performance. Experiments show that Scope achieves up to $1.73 \times$ throughput improvement while maintaining similar energy consumption for ResNet-152 inference compared to state-of-the-art approaches.

Diffusion Policy (DP) has demonstrated strong potential in robotic visuomotor control, offering robust generalization and seamless integration of multi-modal data. However, its complex model structure and increasing multi-modal inputs have brought latency and power challenges for edge resource-constrained robotic platforms. To address the above challenges, we identify the potential intra-model and inter-model redundancies in DP. We observe that DP relies on frequent multi-modal inputs such as images and text during execution. However, the demands of fine-grained robotic manipulation result in substantial intra-modal similarity across consecutive image frames, which, combined with inter-modal semantic redundancy between images and language, indicates that much of the input information is repetitive and potentially compressible. Yet prior works have not exploited these characteristics for targeted optimization. We therefore propose a hardware–software co-design accelerator. On the algorithmic side, we introduce self- and cross-guided modal compression, leveraging intra- and inter-modality similarity to reduce redundant computation within the key DP modules. On the hardware side, we design a tailored architecture that supports multiple operators with optimized sparse memory access, lightweight computation engines, and reconfigurable on-chip dataflow, substantially reducing energy cost. Experimental results demonstrate a 26× speedup over a high-performance GPU while consuming only 1.5 W, enabling low-power and real-time robotic control on edge robotic devices.

While large Transformer models have exhibited outstanding performance on multimodal tasks, the underlying massive parameters land up with memory-wall issues. To address this bottleneck, SRAM-based compute-in-memory (CiM) is a promising technique. However, frequent off-chip weight loading due to limited on-chip capacity could severely limit the systemlevel energy efficiency. Recently, a high-density CiM structure at 16.4Mb/mm2, YOLoC, has shown the potential of complete on-chip deployment of a large detection model using transistor-based read-only-memory (ROM). However, it is still challenging to deploy even larger Transformer models. With opportunities provided by LoRA for finetuning large pretrained models on ROM-CiM with very light SRAM-CiMs, this work achieves ultra-high density up to 165.6Mb/mm2 by eliminating the use of transistors for ROM-CiM with a proposed 3D-METRO and a 3D stacking array on the mature CMOS process. Unlike the usual belief that parasitics have negative impacts, this work observes that parasitics can be utilized for data storage. Furthermore, a local recovering unit (LRU) is proposed for addressing the interference due to the transistor-less structure. 3D-METRO achieves ultra-high density improvement over the previous YOLoC, which is hundreds of times higher than that of SRAM-CiM, enabling the opportunity for large language model (LLM) deployment on a single chip with 28x energy efficiency improvement.

Recent studies have proved the feasibility of ultra-low-precision (≤ 8-bit) training. However, most of the existing operational circuits support only a few higher precisions (FP16, FP32, etc.) for multiplication, and the bit width for accumulation cannot be reduced, which results in low area and power efficiencies. In this paper, we propose MPICC, a multiple-precision Multiply-Accumulate (MAC) unit designed for ultra-low-precision training. It supports inter-combined computations across 18 different precision combinations, including LOG4 (radix-4 FP4), FP4, FP6, FP8, and INT4. It also reduces the accumulation precision from FP16 to FP12 through an optimized Stochastic Rounding (SR) strategy to further save logic resources. Moreover, a low-cost emulating controller, which time-division multiplexed the low-precision MAC unit, is also designed to accomplish high-precision computations for critical DNN layers. Compared with the existing multiple-precision computing units, the area/energy efficiencies of this design are improved by 1.17×/1.19× at FP8, and 4.69×/3.64× at FP4, respectively. The SR strategy further reduces the area/power consumption by 15.6%/14.9% of the floating-point accumulator.

Owing to mature fabrication support and high flexibility, static random-access memory (SRAM) has become a very promising candidate for compute-in-memory (CiM) in accelerating deep neural networks (DNNs). However, SRAM-based CiM has low memory density and thus very limited total on-chip capacity, resulting in frequent weights reloading and additional power consumption during end-to-end inference tasks. Analog ROM CiM increases memory density but suffers from low computing density caused by A/D converter (ADC) limitation. To address these challenges, for the first time, a fully digital compute-in-read-only-memory (DCiROM) design approach is proposed in this paper. DCiROM introduces a novel ROM-logic fusion CiM that successfully reduces CiM area by 51% while maintaining high memory density and computing performance. By reusing multiply-and-accumulation (MAC) resources, DCiROM further achieves flexibility with a minimal area cost. We have implemented a DCiROM chip loaded 3024Kb ResNet-56 parameters using 65nm CMOS technology. This macro achieves 10.2x-55.7x higher normalized FoM (memory density x computing density) than the state-of-the-art CiM works. It also reduces 2.9x-9.9x energy consumption per image inference than SRAM CiM works when considering off-chip access.

In recent years, Neural Networks developed rapidly to deal with tasks in the field of Computer Vision and Natural Language Process, etc. With the development of AI Generated Content, U-Net based Diffusion Models (DM) take image synthesis to new heights. U-Net performs the noise prediction of DM, the latency of which accounts for the majority of the end-to-end latency. Although FPGA has been proven to be a high performance platform to deploy NN, a series of facts still pose challenges for efficient U-Net based DMs deployment based on FPGA. The input vector length and type of the special function vary between different layers. The absence of model periodicity increases the granularity and complexity of operator scheduling. Skip-connection and residual connection inside model cause meta-data retaining in the memory, which is not conductive to avoiding memory overflow and decreasing total off-chip memory access.

Compute-in-Memory (CiM) is a promising technique for energy-efficient deep neural network (DNN) inference to miti-gate the memory bottleneck. Unfortunately, conventional SRAM-based CiM has a low density and limited on-chip capacity, resulting in undesired weight reloading from off-chip DRAM. The emerging high-density ROM-based CiM architecture has recently revealed the opportunity of deploying large-scale DNNs on-chip, with optional assisting SRAM to ensure moderate flexibility. However, prior analog-domain ROM CiM still suffers from limited memory density improvement and low computing area efficiency due to stringent array structure and large A/D converter (ADC) overhead. This paper presents DSC-ROM, a fully digital sparsity-compressed compute-in-ROM architecture to address these challenges. DSC-ROM introduces a fully synthesizable macro-level design methodology that achieves a record-high memory density of 27.9 Mb/mm2in a 28nm CMOS technology. Experimental results show that the macro area efficiency of DSC-ROM improves by 5.6-6.6x compared with prior analog-based ROM CiM. Furthermore, a novel weight fine-tuning technique is proposed to ensure task transfer flexibility and reduce required assisting SRAM cells by 94.4%. Experimental results show that DSC-ROM designed for ResNet-18 pre-trained on ImageNet dataset achieves <0.5% accuracy loss in CIFAR-10 and FER2013, compared with the fully SRAM-based CiM.

Compound AI systems showcase impressive performance and versatility compared to single AI models by combining large language models (LLMs) with various smaller expert models. The major bottleneck of compound AI lies in frequent data movement due to the massive parameters and dynamic routing mechanisms. Compute-In-Memory (CiM) has demonstrated great potential to mitigate the memory wall. However, constrained by the rigid array structure, existing CiM accelerators struggle to meet more general and diverse model compression demands of compound AI, such as fine-grained pruning for expert models and outlier-aware quantization for LLM-based router models. The lack of support for agile model compression hinders the deployment of compound AI systems on CiM accelerators.To fully unlock the potential of CiM in accelerating compound AI, we present DANCE, a dual-side N:M sparse compressed digital CiM architecture with cross-layer co-optimizations: (i) At the circuit level, DANCE introduces a customized set-associative selection circuit to extract N:M sparse patterns for both weights and activations, maintaining high parallelism; (ii) At the architecture level, DANCE explores a novel design paradigm that integrates fine-grained pruning and outlier-aware quantization into a unified N:M sparsity compression framework. Experimental results show that DANCE achieves up to 4.36× energy efficiency improvement with <1% accuracy loss for ResNet-18 on CIFAR-100, and up to 2.59× energy efficiency improvement with <0.5 perplexity increase for Llama-7B on WikiText-2, compared to the conventional digital CiM baseline.

The rapid growth in the parameter size of large language models (LLMs) has introduced significant challenges for deployment on edge devices. To address these challenges, this paper focuses on developing a post-training quantization (PTQ) framework tailored for LLM deployment on edge devices. Our framework introduces an enhanced channel smoothing technique based on channel value ranges, combined with channel reordering to mitigate quantization errors associated with large channel value range differences and activation outliers, reducing the need for quantization and dequantization steps during inference, making it more suitable for edge deployment. Our approach achieves full integer quantization for all model operations, reducing model size by 4×. Through extensive experiments on various language tasks using the OPT model, we demonstrate that our framework surpasses state-of-the-art methods under the W4A4 configuration.

Organic thin-film transistor (OTFT) is a promising device technology for flexible large-area high-channel-count active-matrix neurostimulation system due to its flexibility and biocompatibility. However, circuits made by OTFT might be sensitive to device variation. As a result, it is difficult to achieve precise neurostimulation without any compensation structure in the pixel circuits. This work proposes a 6T2C threshold voltage compensation circuit for neurostimulation, which has low output current variation of 10.53%, reduced from the variation of 17.85% without compensation. We also improve the OTFT fabrication process with encapsulation to allow the circuits to operate under an electrolyte environment. Using the pixel circuits, we implement a 256-channel active-matrix neurostimulation system. The system can output stimulation with any pattern and allow each channel to output independently and simultaneously.

Large Language Models (LLMs) have achieved remarkable success across many applications, with Mixture of Experts (MoE) models demonstrating great potential. Compared to traditional dense models, MoEs achieve better performance with less computation. Speculative decoding (SD) is a widely used technique to accelerate LLM inference without accuracy loss, but it has been considered efficient only for dense models. In this work, we first demonstrate that, under medium batch sizes, MoE surprisingly benefits more from SD than dense models. Furthermore, as MoE becomes sparser -- the prevailing trend in MoE designs -- the batch size range where SD acceleration is expected to be effective becomes broader. To quantitatively understand tradeoffs involved in SD, we develop a reliable modeling based on theoretical analyses. While current SD research primarily focuses on improving acceptance rates of algorithms, changes in workload and model architecture can still lead to degraded SD acceleration even with high acceptance rates. To address this limitation, we introduce a new metric 'target efficiency' that characterizes these effects, thus helping researchers identify system bottlenecks and understand SD acceleration more comprehensively. For scenarios like private serving, this work unveils a new perspective to speed up MoE inference, where existing solutions struggle. Experiments on different GPUs show up to 2.29x speedup for Qwen2-57B-A14B at medium batch sizes and validate our theoretical predictions.

Transformers have recently shown significant performance across various tasks, such as natural language processing (NLP) and computer vision (CV). However, the performance comes at the cost of large memory and computation overhead. Existing researches primarily focus on accelerating matrix multiplication (MatMul) through techniques like quantization and pruning, notably increasing the proportion of nonlinear operations in inference runtime. Meanwhile, previous approaches designed for nonlinear operations struggle with inefficient implementation as they are incapable of achieving both computation and memory efficiency. Additionally, these methods often require retraining or fine-tuning leading to substantial costs and inconveniences. To overcome these problems, we propose efficient implementation of nonlinear operations with quantization-approximation-based strategy. Through an in-depth analysis of the dataflow and data distribution of nonlinear operations, we design distinct quantization and approximation strategies tailored for different operations. Specifically, log2 quantization and power-of-two factor quantization have been employed in Softmax and LayerNorm, complemented by logarithmic function and low-precision statistic calculation as approximation strategies. Furthermore, the proposed efficient GeLU implementation integrates a nonuniform lookup procedure alongside low-bit-width quantization. Experimental results demonstrate negligible accuracy drops without the need for retraining or fine-tuning. By implementing the hardware design, it achieves$3.14\times - 6.34\times $energy-efficiency and$3.01\times - 10.1\times $area-efficiency improvements compared to state-of-the-art application-specific-integrated-circuit (ASIC) designs. In system-level evaluation, substantial speedup and reductions in energy consumption of 15% to 35% are achieved for end-to-end inference across both GPU and ASIC accelerator platforms.

AdderNet is an innovative neural network (NN) structure that substitutes multiplications with additions in convolutional operations, while computing-in-memory (CIM) is an efficient architecture that tackles the memory bottleneck for von Neumann architectures. Previous work has explored the SRAM-based CIM AdderNet circuits and demonstrates high energy efficiency. However, it still suffers low storage density, repetitive readout, and redundant comparisons. In this brief, an RRAM-based CIM macro is proposed for efficient AdderNet with the following innovations. First, RRAM cells are adopted to replace SRAM for high-density weight storage. A low-power readout and hold circuit is proposed to save redundant read power of weight data held for multiple cycles. Second, an 8-bit comparator with an early-stop strategy is proposed to compare 8-bit activations and weights in one cycle. Third, an activation (ACT) differential strategy is proposed to reduce redundant comparisons. The proposed 28-nm RRAM CIM macro achieves 12.8-TOPS/mm2peak area efficiency and 126-TOPS/W peak energy efficiency, which is$3.0\times $and$1.2\times $compared with the state-of-the-art AdderNet CIM macro.

Implantable neuroprostheses require stimulators with high channel counts and mechanical flexibility. Organic thin-film transistor (OTFT), an essential building block for flexible circuits and system, is a promising candidate. However, the development of photolithographic OTFTs for complete bioelectronic system integration remains a challenge, due to their limited yield and uniformity. This paper reports a 4-mask photolithographic OTFT circuit integration technology, which shows a high device yield of 100% (50/50) and small device variation in threshold voltage of 0.64 V and in mobility of 4.9%. Using a device-circuit-system co-design approach, we demonstrate an active-matrix neurostimulation array comprised of 1024 pixels of a 4T1C stimulation circuits, in which independent stimulation intensity levels can be programmed and current stimulus at all channels can output simultaneously. The electrical function of the complete neurostimulation system is verified, showing a small variation of 15.59% for the output stimulation currents among pixels. This OTFT-based neurostimulation system provides a potential solution for the next-generation neurostimulators with high channel counts and mechanical flexibility.

Computing-in-memory (CIM) architectures based on various devices, such as resistive random access memory, SRAM, DRAM, etc., have demonstrated promising energy efficiency. Single-device-based CIM chips show different advantages on performance, power, or area metrics under different workload/operators sizes and application requirements. Some nonidealities, such as the write endurance of some nonvolatile devices, also influence the design choices. Motivated by the emerging 2.5-D/3-D chiplet integration, this work aims to combine the advantages of CIM/storage chips based on different devices, and proposes a design exploration framework to combine the advantages of CIM chips based on these devices in a 3-D-stack architecture. This work proposes: 1) an evaluation method for the power, performance, and area metrics of the multichiplet CIM architecture; 2) an abstraction for the single-device-based CIM chiplets and artificial intelligence algorithm operators; and 3) a mapping and optimization strategy to explore the 2.5-D/3-D CIM chiplet set. The effectiveness of the mapping strategy is verified with a small-scale brute-force search. The proposed design exploration framework can help to find a better-multichiplet CIM architecture. Under a simple design case, the proposed 3-D CIM architecture shows$4.68\times $–$53.32\times $energy efficiency compared with the single CIM chip baselines. The abstracted chiplet library is open-source available in the open-sourcehttps://github.com/dai0dai/3D_CIM_Chiplet_Architecture_Exploration.

Transformer networks have been increasingly successful in various fields. The input sequence lengths have become much larger as the algorithm and task complexity develops, which is challenging due to high computational and storage cost. Softmax and LayerNorm are bottleneck nonlinear operators in ultra-long sequence Transformer networks. To improve the efficiency of Softmax, assumption-based and quantization-based Softmax approaches are introduced. However, the sparsity potential to accelerate Softmax itself is not fully discovered. To improve the efficiency of LayerNorm, some works reduce the input size, and some works explore the pipeline. However, the sparsity potential is also not yet explored. To address these challenges, this article presents the ULSeq-TA software–hardware co-design framework. The software includes 1) the grouped sparse Softmax method to leverage the data magnifying characteristic to explore the middle and post-Softmax sparse processing and 2) the dual-path sparse LayerNorm method which explores the dimensional significance for sparse calculation. The hardware includes 1) an attention fusion architecture which reduces the on-chip memory with fused operators; 2) the grouped sparse Softmax core; and 3) the dual-path sparse LayerNorm core. Experiments show that the software achieves$4.45\times $and$7.59\times $computation reduction with little output difference for Softmax and LayerNorm, respectively. The hardware architecture supports at most 32768 sequence length with only 186-kB on-chip memory and achieves$1.75\times -1.98\times $and$3.22\times -4.32\times $speedups for sparse Softmax core and sparse LayerNorm core with little accuracy loss, respectively.

As domain-specific training data is recognized as valuable intellectual property, acquiring well-trained weights in Convolutional Neural Networks (CNN) has emerged as a new threat to the neural network design community. To design a CNN accelerator that is resilient to side-channel threats, it is crucial to have an accurate and efficient security-driven framework at the early design stage. However, there is no standard way to perform root-cause analysis on the power side channel that exists in FPGA-based CNN accelerators. Therefore, we build RE-Specter, a framework that facilitates security-driven design space exploration (DSE) across various building components, combination patterns, and parallelism configurations in CNNs. The goal is to fully understand the power side-channel effects resulting from architectural modifications or optimization decisions. We further compare the benchmarks considering precision, resource utilization, and power side-channel leakage. Finally, we experimentally explore the design space of various architectural features. The experimental results show that low-bit precision delivers more secure architectures (68.9× among DSPs, 2439× among LUTs) in Measurement-To-Disclosure (MTD), but mixed-precision strategies are necessary to maintain the model accuracy. For loop optimization, in 16-parallel scenario, accumulator-based architecture outperforms the architecture featuring an adder tree with the improvements of 8.28× in MTD and 1.38× in PST.

In the era of intelligent IoT, huge amount of sensor data is collected and then transmitted to processor elements in edge devices or cloud servers. The latency and energy consumption in this process have been a bottleneck and are becoming more severe. To mitigate this problem, the idea of combining sensors, memory and processors for collectively handling the data, has been proposed and explored actively in recent efforts. In this work, thin-film transistor (TFT), which has been widely adopted in display devices and flexible sensors, is exploited. It is shown that, while TFT is promising for large-area sensing, it also shows a great potential for computing and storing data for large-area and low-cost edge sensors. More specifically, we have fabricated and measured two large-area TFT-based near-sensor computing-in-memory (CiM) chips adopting embedded DRAM (eDRAM) and ROM structure respectively. We further give a detailed analysis of the integration of CiM arrays and sensor arrays to realize a sensing and data pre-process system. Measurement and simulation results show that such TFT-based solutions can accomplish real-time sensing and multiply-accumulate (MAC) processing in the analog field, which simplifies the system design with lowered energy and latency in our neural network evaluations.

Logic camouflage is a widely adopted technique that mitigates the threat of intellectual property (IP) piracy and overproduction in the integrated circuit (IC) supply chain. Camouflaged logic achieves functional obfuscation through physical-level ambiguity and post-manufacturing programmability. However, discussions on programmability are confined to the level of logic cells/gates, limiting the broader-scale application of logic camouflage. In this work, we propose a novel module-level configuration methodology for programmable camouflaged logic that can be implemented without additional hardware ports and with negligible resources. We prove theoretically that the configuration of the programmable camouflaged logic cells can be achieved through the inputs and netlist of the original module. Further, we propose a novel lightweight ferroelectric FET (FeFET)-based reconfigurable logic gate (rGate) family and apply it to the proposed methodology. With the flexible replacement and the proposed configuration-aware conversion algorithm, this work is characterized by the input-only programming scheme as well as the combination of high output error rate and point-function-like defense. Evaluations show an average of >95% of the alternative rGate location for camouflage, which is sufficient for the security-aware design. We illustrate the exponential complexity in function state traversal and the enhanced defense capability of locked blackbox against Boolean Satisfiability (SAT) attacks compared with key-based methods. We also preserve an evident output Hamming distance and introduce negligible hardware overheads in both gate-level and module-level evaluations under typical benchmarks.

Effectively preserving and encoding structure features from objects in irregular and sparse LiDAR points is a crucial challenge to 3D object detection on the point cloud. Recently, Transformer has demonstrated promising performance on many 2D and even 3D vision tasks. Compared with the fixed and rigid convolution kernels, the self-attention mechanism in Transformer can adaptively exclude the unrelated or noisy points and is thus suitable for preserving the local spatial structure in the irregular LiDAR point cloud. However, Transformer only performs a simple sum on the point features, based on the self-attention mechanism, and all the points share the same transformation for value. A such isotropic operation cannot capture the direction-distance-oriented local structure, which is essential for 3D object detection. In this work, we propose a Structure-Embedding transFormer (SEFormer), which can not only preserve the local structure as a traditional Transformer but also have the ability to encode the local structure. Compared to the self-attention mechanism in traditional Transformer, SEFormer learns different feature transformations for value points based on the relative directions and distances to the query point. Then we propose a SEFormer-based network for high-performance 3D object detection. Extensive experiments show that the proposed architecture can achieve SOTA results on the Waymo Open Dataset, one of the most significant 3D detection benchmarks for autonomous driving. Specifically, SEFormer achieves 79.02% mAP, which is 1.2% higher than existing works. https://github.com/tdzdog/SEFormer.

Unlike the grid-paced RGB images, network compression, i.e.pruning and quantization, for the irregular and sparse 3D point cloud face more challenges. Traditional quantization ignores the unbalanced semantic distribution in 3D point cloud. In this work, we propose a semantic-guided adaptive quantization framework for 3D point cloud. Different from traditional quantization methods that adopt a static and uniform quantization scheme, our proposed framework can adaptively locate the semantic-rich foreground points in the feature maps to allocate a higher bitwidth for these "important" points. Since the foreground points are in a low proportion in the sparse 3D point cloud, such adaptive quantization can achieve higher accuracy than uniform compression under a similar compression rate. Furthermore, we adopt a block-wise fine-grained compression scheme in the proposed framework to fit the larger dynamic range in the point cloud. Moreover, a 3D point cloud based software and hardware co-evaluation process is proposed to evaluate the effectiveness of the proposed adaptive quantization in actual hardware devices. Based on the nuScenes dataset, we achieve 12.52% precision improvement under average 2-bit quantization. Compared with 8-bit quantization, we can achieve 3.11× energy efficiency based on co-evaluation results.

Data quantization is an effective method to accelerate neural network training and reduce power consumption. However, it is challenging to perform low-bit quantized training: the conventional equal-precision quantization will lead to either high accuracy loss or limited bit-width reduction, while existing mixed-precision methods offer high compression potential but failed to perform accurate and efficient bit-width assignment. In this work, we propose DYNASTY, a block-wise dynamic-precision neural network training framework. DYNASTY provides accurate data sensitivity information through fast online analytics, and maintains stable training convergence with an adaptive bit-width map generator. Network training experiments on CIFAR-100 and ImageNet dataset are carried out, and compared to 8-bit quantization baseline, DYNASTY brings up to 5.1× speedup and 4.7× energy consumption reduction with no accuracy drop and negligible hardware overhead.

Multi-level cell (MLC) NVM-based CiM has become a promising candidate in computing-in-memory (CiM) designs because of its non-volatility, high cell density, and improving compatibility with the CMOS process. However, most MLC CiM faces the challenges of non-ideal device limitations, including the low on/off ratio, large device-to-device variations, and read disturbances, which limit the computing accuracy, reliability, and throughput performance. This work proposes Victor, a variation-resilient approach using cell-clustered charge-domain computing for high-density and high-throughput MLC CiM. A cell-clustered-computing with local recovery unit (LRU) design methodology is proposed to improve matrix-vector-multiplication (MVM) reliability and throughput. To showcase the capability of Victor, 2b-4b MLC Resistive RAM (RRAM) is taken as an example for design and evaluation. Results show that Victor reaches 3.56x energy efficiency, 4x variation tolerance compared with the prior ratio-based MLC CiM. In addition, the throughput is improved by 3.1x with less than 1% DNN accuracy loss. Moreover, a dynamic boundary adaption approach is proposed to restore the accuracy loss of state drifting, which in return reduces the energy and latency overhead by 100x and 1.25x, respectively, compared with the conventional write-and-verify approach.

Genome sequence analysis is a powerful tool in medical and scientific research. Considering the inevitable sequencing errors and genetic variations, approximate string matching (ASM) has been adopted in practice for genome sequencing. However, with exponentially increasing bio-data, ASM hardware acceleration is facing severe challenges in improving the throughput and energy efficiency with the accuracy constraint.This paper presents ASMCap, an ASM acceleration approach for genome sequence analysis with hardware-algorithm co-optimization. At the circuit level, ASMCap adopts charge-domain computing based on the capacitive multi-level content addressable memories (ML-CAMs), and outperforms the state-of-the-art ML-CAM-based ASM accelerators EDAM with higher accuracy and energy efficiency. ASMCap also has misjudgment correction capability with two proposed hardware-friendly strategies, namely the Hamming-Distance Aid Correction (HDAC) for the substitution-dominant edits and the Threshold-Aware Sequence Rotation (TASR) for the consecutive indels. Evaluation results show that ASMCap can achieve an average of 1.2x (from 74.7% to 87.6%) and up to 1.8x (from 46.3% to 81.2%) higher F1score (the key metric of accuracy), 1.4x speedup, and 10.8x energy efficiency improvement compared with EDAM. Compared with the other ASM accelerators, including ResMA based on the comparison matrix, and SaVI based on the seeding strategy, ASMCap achieves an average improvement of 174x and 61x speedup, and 8.7e3x and 943x higher energy efficiency, respectively.

Transformers have shown remarkable performance in both natural language processing (NLP) and computer vision (CV) tasks. However, their real-time inference speed and efficiency are limited due to the inefficiency in Softmax and Layer Normalization (LayerNorm). Previous works based on function approximation suffer from inefficient implementation as they place emphasis on computation while disregarding memory overhead concerns. Moreover, such methods rely on retraining to compensate for approximation error which can be costly and inconvenient. In this paper, we present SOLE, a hardware-software co-design for Softmax and LayerNorm which is composed of E2Softmax and AILayerNorm. E2Softmax utilizes log2 quantization of exponent function and log-based division to approximate Softmax while AILayerNorm adopts low-precision statistic calculation. Compared with state-of-the-art designs, we achieve both low-precision calculation and low bit-width storage on Softmax and LayerNorm. Experiments show that SOLE maintains inference accuracy without retraining while offering orders of magnitude speedup and energy savings over GPU, achieving 3.04×, 3.86× energy-efficiency improvements and 2.82×, 3.32× area-efficiency improvements over prior state-of-the-art custom hardware for Softmax and LayerNorm, respectively.

The concept of multi-level cell (MLC) enabled by emerging memory device technologies has introduced new opportunities for memory density improvement, including in the cache scenarios with some high-endurance technologies. However, the access latency of different bits within an MLC memory cell is inherently nonuniform, which raises challenges in utilizing the MLC technology for low-latency cache. To exploit the access performance of the MLC cache, the key is identifying the hot data blocks and mapping them to fast MLC bits. Prior works perform the hot/cold data management based on block-wise access patterns with considerable hardware overheads. Inspired by the memory hierarchy, this work proposes a new concept of in-cell hierarchical victim cache as embedded memory and systematically presents the cache architecture, operating mechanism, design space exploration, optimizations, and evaluations. By utilizing the slow bits as the victim buffer, lower hit latency with low implementation overheads is achieved. Based on the in-cell victim cache, two optimization techniques, namely selective victim retrieval, and victim-bypassing write (VBW) are proposed, to further improve performance and prolong cache endurance, respectively. Evaluation results show that the MLC victim cache significantly improves the average system performance by 20.2% over conventional MLC cache and achieves 98% performance of the upper bound implemented with 2x memory cells SLC. The proposed VBW also reduces energy consumption by 21% and improves lifetime by over 80%, showing a new promising dimension for future MLC cache design.

It has recently become trendy for neural network accelerators to scale from single-core to chiplet-based multichip architecture, as the growth of neural network depth and complexity are calling for the promotion of computation and memory capabilities. However, the unintended extensive inter-chip communication of chiplet-based accelerator may bottleneck the parallelism of deep learning inference, which is undesirable for many real-time applications and energy-efficient devices. Although it is imperative for novel schemes to be devised to alleviate this problem, related works are scarce. In this work, we present CampQ, a fine-grained communication-aware mixed-precision quantization method to accelerate inference parallelization by reducing the major inter-chiplet communication overhead. By leveraging the AutoML technique, CampQ is capable of determining different bit-width to activation groups according to thier transmission distances in on-package network. The experimental results show 1.4×-2.6× performance benefits and 29%-60% energy reduction over the 16-bit models for various neural networks and parallelism approaches.

Computing-in-memory (CIM) architecture utilizing emerging non-volatile devices is promising for energy-efficient neural network (NN) applications. However, the non-ideal factors of non-volatile devices and analog circuits may incur severe accuracy loss, which cuts the algorithm and hardware design apart. The algorithm/hardware designers are skilled in either macro-scope NN models or detailed circuit/device errors, while sophisticated research of the joint effect on accuracy loss is urgently needed. In this paper, we propose a user-friendly, fast, and accurate simulation framework (CIMUFAS) to explore the impact of various non-ideal devices/circuits on algorithm accuracy. Based on multiple architecture-level CIM mapping/scheduling workflow, sophisticated non-ideal factors with different error models are established. The CIMUFAS also provides easy-to-use interfaces to flexibly support user-defined models/parameters for specified devices/circuits. Besides, the CIMUFAS framework achieves reasonable simulation time. Compared with MNSIM 2.0, the simulation time is reduced by 45% even after adding a more realistic hardware configuration. This CIMUFAS framework is verified with two fabricated CIM chips with <0.04% accuracy mismatch. The source code of CIMUFAS is publicly available at https://github.com/Hlal/CIMUFAS.

SRAM-based computing-in-memory (CIM) has been widely explored to accelerate neural networks (NNs). However, it is challenging to store all weights of many modern NNs due to limited on-chip SRAM capacity. This bottleneck induces a large amount of off-chip DRAM accesses and impedes the improvement of performance and energy efficiency. This paper proposes a new approach of computing in resistive random-access memory (ReRAM)-assisted energy- and area-efficient SRAM (CREAM) for accelerating large-scale NNs while eliminating the DRAM access. The NN weights are all stored in high-density on-chip ReRAMs and restored to the proposed non-volatile SRAM (nvSRAM) CIM cells with array-level parallelism. Furthermore, to deal with the influence of ReRAM and CMOS variations, a novel layer-wise and bit-wise weight-configuration search algorithm is proposed by leveraging different sensitivity of each layer in NN models. A data-aware weight-mapping method is also presented to efficiently map NN models to ReRAMs in CREAM for high computation parallelism. The experiment results show$10.3\times $weight storage density over the standard 6T SRAM array. Evaluations of ResNet-18 and VGG-9 on CIFAR-10/CIFAR-100 datasets show up to$3.47\times $and$1.70\times $energy efficiency over two baseline designs of SRAM-CIM and ReRAM-CIM, respectively, in addition to 15.6% higher accuracy than ReRAM-CIM under device variations.

Performing data-intensive tasks in the von Neumann architecture is challenging to achieve both high performance and energy efficiency due to the memory wall bottleneck. Compute-in-memory (CiM) is a promising mitigation approach by enabling parallel and in-situ multiply-accumulate (MAC) operations within the memory array. Thanks to the good matching of capacitors, SRAM-based charge-domain CiM (Q-CiM) has shown its potential for higher row-wise parallelism. However, the peripheral circuits of Q-CiM, such as the input drivers and analog-digital converters (ADCs), limit further improvement of throughput and area efficiency. This paper proposes a single-ADC multi-bit accumulation CiM macro architecture SAMBA, which can perform multi-bit MAC operation with ReLU of two vectors in one CiM cycle by only a single A/D conversion to mitigate the ADC overhead. In addition, post-correction methods are proposed to compensate the non-linearity of sensitive circuit modules in SAMBA to recover the accuracy drop due to the capacitor mismatch. A proof-of-concept macro is fabricated in a 65nm process and achieves 51.2GOPS throughput and 10.3TOPS/W energy efficiency, while showing 88.6% accuracy on CIFAR-10 and 64.8% accuracy on the CIFAR-100 with VGG-8 model.

The rapid development of edge artificial intelligence (AI) raises high requirements for data-intensive neural network (NN) computing and storage of edge devices, under a limited chip footprint and energy supply source. As a promising approach for energy-efficient processing, computing-in-memory (CiM) has been widely explored in recent efforts to mitigate the data transmission bottleneck. However, CiM with small on-chip memory capacity results in expensive data reloads, limiting its deployment in large-scale NN applications. Moreover, the increased leakage under advanced CMOS scaling lowers the energy efficiency. In this work, device-circuit synergy based on the indium-gallium-zinc-oxide (IGZO) thin-film transistor (TFT) is adopted to address these challenges. First, 4-transistor-1-capacitor (4T1C) IGZO eDRAM CiM is proposed with higher density than SRAM-based CiM and enhanced data retention by both lower device leakage and a differential cell structure. Second, exploiting the back-end-of-line (BEOL) compatibility and vertical integration of emerging channel-all-around (CAA) IGZO devices, 3D eDRAM CiM is proposed, which paves the way for IGZO-based CiM with ultra-high density. Circuit techniques including time-interleaved computing and differential refresh are proposed to guarantee accuracy under large-capacity 3D CiM. As a proof of concept, a$128 \times 32$CiM array is fabricated under a foundry low-temperature poly-crystalline and oxide (LTPO) technology, demonstrating high computing linearity and long data retention. Benchmarks on scaled 45nm IGZO technology show energy efficiency of 686 TOPS/W for array only, and 138 TOPS/W while considering peripheral overheads.

Bitwise logic-in-memory (BLiM) is a promising approach to efficient computing in data-intensive applications by reducing data movement between memory and processing units. However, existing BLiM techniques have challenges towards higher energy efficiency and speed: (i) DC power in computing and result sensing is significant in most existing RRAM and MRAM based BLiM solutions; (ii) before the computation result could be stored back to the same memory array, existing BLiM has to sense the result first, at the cost of extra power and latency due to the sense amplifiers (SAs). Targeting at higher energy efficiency and speed, this work proposes a new BLiM approach in 2-transistor/ cell (2T/C) and 3T/C topologies based on ferroelectric field-effect transistors (FeFETs), supporting a variety of computing functions. For the first time, this new approach supports SA-free direct write-back, and consumes no static power for computing and sensing with proposed fully dynamic computing and sensing schemes. Another highlight is that this work further minimizes the dynamic power by (i) reducing the chance of bitline charging activities and (ii) recycling the bitline charge in sensing multi-operand operations. Compared with prior BLiM methods based on nonvolatile memories, evaluation shows 3.0x–100x latency and 1.3x–200x energy improvement for typical in- memory XOR operation, which further leads to 3.0x–58x and 3.2x–78x savings of latency and energy, respectively, for the application of advanced-encryption standard (AES).

Intermittent systems powered by ambient energy harvesting are becoming popular for the benefits of an infinite lifetime and minimum maintenance requirements. Nonvolatile processors (NVPs) enable continual task executions under an unstable power supply with an efficient reactive checkpointing strategy. However, recovering concurrent peripherals in an intermittent system may incur significant overhead once power failures take place, and the recovery of interrupts also lacks discussion in existing works. Noticing the different optimization directions between responsive checkpointing within NVPs and proactive checkpointing required to recover concurrent peripherals, this paper proposes REMARK, an NVP architecture enabling hybrid checkpointing and efficient peripheral recovery. REMARK expands the current NVP structure with a hybrid backup/restore module, a peripheral handler and an interrupt handler, which addresses the recovery problem of both peripherals and interrupts efficiently. A REMARK chip is fabricated to verify the proposed architecture. Results show that the execution efficiency is improved by$13\times $compared with the state-of-the-art. With programming optimization, another 36.5% performance improvement can be achieved.

Non-volatile computing-in-memory (nvCIM) is a novel architecture used for deep neural networks (DNNs) because it can reduce the movement of data between computing units and memory units. As sparsity has made great progress in DNNs, the existing nvCIM architecture is only optimized for structured sparsity but little for unstructured sparsity. To solve this problem, the sparsity-aware nvCIM macro is proposed to improve the computing performance and network classification accuracy, and to support both structured and unstructured sparsity. First, the analog switch array is used to take advantage of the structured sparsity and to improve the computing parallelism. Second, the low-resolution current-mode analog-to-digital converter (CMADC) is designed to optimize the unstructured sparsity. Experimental results show that the peak equivalent energy efficiency of the proposed nvCIM macro is 9.1 TOPS/W (A8W8, 8-bit activations and 8-bit weights) with only 0.51% accuracy loss, and 584.9 TOPS/W (A1W1), which is 4.8 -$7.5\times$compared to the state-of-the-art nvCIM macros.

Deep neural network achieves exciting performance in several tasks with heavy storing and computing costs. Previous works adopt pruning-based methods to slim deep network. For traditional pruning, either the convolution kernel or the network inference is static, which cannot fully compress the model parameter and restrains their performance. In this paper, we propose an online pruning algorithm to support dynamic kernel generation and dynamic network inference at the same time. Two novel techniques including the filter generator and the importance-level based channel pruning are proposed. Moreover, we validate the success of the proposed method by the implementation on Ultra96-v2 FPGA. Compared with state-of-art static or dynamic pruning methods, our method can reduce the top-5 accuracy drop by nearly 50% for ResNet model on ImageNet at similar compressing level. It can also achieve better accuracy while up to 50% fewer weights are reduced to be saved on chip.

Low-bit quantization is a big challenge for neural network training. Conventional training hardware adopts FP32 to accumulate the partial-sum result, which seriously degrades energy efficiency. In this paper, a technology called dynamic group accumulation (DGA) is proposed to reduce the accumulation error. First, we model the proposed group accumulation method and give the optimal DGA algorithm. Second, we design a training architecture and implement a hardware-efficient DGA unit. Third, we make a comprehensive analysis of the DGA algorithm and training architecture. The proposed method is evaluated on CIFAR and ImageNet datasets, and results show that DGA can reduce accumulation bit-width by 6 bits while achieving the same precision as the static group method. With the FP12 DGA, the CNN algorithm only loses 0.11% accuracy in ImageNet training, and our architecture saves 32% of power consumption compared to the FP32 baseline.

Computing-in-memory (CiM) is a promising technique to achieve high energy efficiency in data-intensive matrix-vector multiplication (MVM) by relieving the memory bottleneck. Unfortunately, due to the limited SRAM capacity, existing SRAM-based CiM needs to reload the weights from DRAM in large-scale networks. This undesired fact weakens the energy efficiency significantly. This work, for the first time, proposes the concept, design, and optimization of computing-in-ROM to achieve much higher on-chip memory capacity, and thus less DRAM access and lower energy consumption. Furthermore, to support different computing scenarios with varying weights, a weight fine-tune technique, namely Residual Branch (ReBranch), is also proposed. ReBranch combines ROM-CiM and assisting SRAM-CiM to achieve high versatility. YOLoC, a ReBranch-assisted ROM-CiM framework for object detection is presented and evaluated. With the same area in 28nm CMOS, YOLoC for several datasets has shown significant energy efficiency improvement by 14.8x for YOLO (DarkNet-19) and 4.8x for ResNet-18, with <8% latency overhead and almost no mean average precision (mAP) loss (−0.5% ~ +0.2%), compared with the fully SRAM-based CiM.

Motivated by reducing the data transfer activities in data-intensive neural network computing, SRAM-based compute-in-memory (CiM) has made significant progress. Unfortunately, SRAM has low density and limited on-chip capacity. This makes the deployment of large models inefficient due to the frequent DRAM access to update the weight in SRAM. Recently, a ROM-based CiM design, YOLoC, reveals the unique opportunity of deploying a large-scale neural network in CMOS by exploring the intriguing high density of ROM. However, even though assisting SRAM has been adopted in YOLoC for task transfer within the same domain, it is still a big challenge to overcome the read-only limitation in ROM and enable more flexibility. Therefore, it is of paramount significance to develop new ROM-based CiM architectures and provide broader task space and model expansion capability for more complex tasks.

Previous RRAM-based computing-in-memory works mainly focus on the current-domain approach. However, the performance and accuracy of current computation are limited by large read currents of RRAM cells and their variations. This work presents a novel RRAM-based charge-domain design, C-RRAM, to resolve these limitations. A 3TlRlC cell is proposed to execute MAC operations by capacitor discharging. The resistance variations can be tolerated with reasonable discharging time, which is accelerated by a positive feedback loop. Also, the output from each cell is accumulated by charge sharing instead of summing currents to eliminate the static current path in readout circuits. In this way, robust and efficient RRAM-based CIM operation is enabled with fully input parallelism. A $512\times 514$ RRAM array is implemented to evaluate the benefits of the proposed charge-domain approach. The experiment results show that C-RRAM can suppress the output variations by $41\times$ and incur negligible accuracy loss for ResNet-18 on Cifar10 dataset. Compared to previous ITIR current-domain RRAM designs, it achieves $1.2\times$ energy efficiency and $127\times$ area efficiency due to improved parallelism.

Exploiting neural network sparsity is one of the most important directions to accelerate CNN executions. Plenty of techniques are proposed to exploit neural network sparsity, where spatial-wise pruning is quite effective for input image. However, previous spatial-wise pruning methods need nontrivial hardware overhead for dynamic execution, due to layer-by-layer binary sampling and online scheduling. This paper proposes a structured configured, spatial-wise pruning technique. Numerous computation will be saved by skipping unimportant region. By using a unified importance map, the computing graph could be compiled in advance to make it more hardware friendly. Additionally, due to multi-level measurement of importance for each region, our method can have a better performance on various tasks. On image classification task, the method can have around 50% fewer top-1 accuracy drop than previous spatialwise pruning methods at similar sparse level. On super resolution and image deraining task, the method can bring $5 \times$ to $19 \times$ acceleration while causing neglectable effect on reconstruction quality. Hardware implementation is also included.

introduction Share on Guest Editorial: ACM JETC Special Issue on Hardware-Aware Learning for Medical Applications Editors: Yiyu Shi University of Notre Dame, Notre Dame, Indiana, USA University of Notre Dame, Notre Dame, Indiana, USAView Profile , Yongpan Liu Tsinghua University, Beijing, China Tsinghua University, Beijing, ChinaView Profile , Jianxu Chen Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V, Dortmund, Germany Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V, Dortmund, GermanyView Profile , Steve Jiang University of Texas Southwestern Medical Center Dallas, Texas, USA University of Texas Southwestern Medical Center Dallas, Texas, USAView Profile Authors Info & Claims ACM Journal on Emerging Technologies in Computing SystemsVolume 18Issue 2April 2022 Article No.: 24pp 1–3https://doi.org/10.1145/3503262Online:31 December 2021Publication History 0citation60DownloadsMetricsTotal Citations0Total Downloads60Last 12 Months60Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

In recent years, convolutional neural networks (CNNs) have achieved significant advancements in various fields. However, the computation and storage overheads of CNNs are overwhelming for Internet-of-Things devices. Both network pruning algorithms and hardware accelerators have been introduced to empower CNN inference at the edge. Network pruning algorithms reduce the size and computational cost of CNNs by regularizing unimportant weights to zeros. However, existing works lack intrakernel structured types to tradeoff between sparsity and hardware efficiency, and the index storage for irregularly pruned networks is significant. Hardware accelerators leverage the sparsity of pruned CNNs to improve energy efficiency. However, their process element (PE) utilization rate is low because of uneven sparsity among input convolutional kernels. To overcome these problems, we propose PACA: a Pattern pruning Algorithm and Channel-fused high PE utilization Accelerator for CNNs. It includes three parts: a pattern pruning algorithm to explore the intrakernel sparsity type and reduce the index storage, a channel-fused hardware architecture to reduce the PEs’ idle rate and improve the performance, and a heuristic and taboo search-based smart fusion scheduler to analyze the idle PE problem and schedule the channel fusion in hardware. To demonstrate the effectiveness of PACA, we have implemented the software parts by Python and the hardware architecture by RTL codes. Experimental results on various datasets show that compared with an existing work, PACA can reduce the index storage overhead by$3.47\times $–$5.63\times $with 3.85–9.12 average patterns, and it can improve the hardware performance by$2.01\times $–$5.53\times $because of PEs’ idle rate reduction.

Computing-in-memory (CIM) is a new architecture which is more energy-efficient than the Von Neumann architecture due to the fact that it performs calculation in the memory units which can reduce a large amount of data movement. Nowadays, CIM with non-volatile memory (nvCIM), such as resistive random access memory (RRAM), has become a research frontier to further improve computing performance. Recent works mainly explored how to improve the computing performance of nvCIM, but seldom paid attention to the problem of accuracy loss. In this paper, we propose the nvCIM framework, which can systematically analyze the relationship between the classification accuracy of network models and main analog factors. Based on the nvCIM framework, we further provide detailed optimization methods, including RRAM features, array properties, and ADC parameters. The adaptive voltage-controlled SET and pulse-controlled RESET (VSPR) program-verify scheme is proposed to achieve high-resolution RRAM. And the margin enhancement based current-mode sense amplifier (MECSA) and offset reduction based analog-to-digital converter (ORADC) are proposed to improve the accuracy of analog part computing. Experimental results show that the macro-level and system-level energy efficiency is 112.1 TOPS/W and 9.86 TOPS/W respectively with less than 3.51% loss in contrast to the ideal accuracy, which is$2.9\times $-$25.9\times $compared to the energy efficiency of existing RRAM based nvCIM accelerators.

Weight pruning methods of deep neural networks (DNNs) have been demonstrated to achieve a good model pruning rate without loss of accuracy, thereby alleviating the significant computation/storage requirements of large-scale DNNs. Structured weight pruning methods have been proposed to overcome the limitation of irregular network structure and demonstrated actual GPU acceleration. However, in prior work, the pruning rate (degree of sparsity) and GPU acceleration are limited (to less than 50%) when accuracy needs to be maintained. In this work, we overcome these limitations by proposing a unified, systematic framework of structured weight pruning for DNNs. It is a framework that can be used to induce different types of structured sparsity, such as filterwise, channelwise, and shapewise sparsity, as well as nonstructured sparsity. The proposed framework incorporates stochastic gradient descent (SGD; or ADAM) with alternating direction method of multipliers (ADMM) and can be understood as a dynamic regularization method in which the regularization target is analytically updated in each iteration. Leveraging special characteristics of ADMM, we further propose a progressive, multistep weight pruning framework and a network purification and unused path removal procedure, in order to achieve higher pruning rate without accuracy loss. Without loss of accuracy on the AlexNet model, we achieve 2.58× and 3.65× average measured speedup on two GPUs, clearly outperforming the prior work. The average speedups reach 3.15× and 8.52× when allowing a moderate accuracy loss of 2%. In this case, the model compression for convolutional layers is 15.0× , corresponding to 11.93× measured CPU speedup. As another example, for the ResNet-18 model on the CIFAR-10 data set, we achieve an unprecedented 54.2× structured pruning rate on CONV layers. This is 32× higher pruning rate compared with recent work and can further translate into 7.6× inference time speedup on the Adreno 640 mobile GPU compared with the original, unpruned DNN model. We share our codes and models at the link http://bit.ly/2M0V7DO.

As one type of associative memory, content-addressable memory (CAM) has become a critical component in several applications, including caches, routers, and pattern matching. Compared with the conventional CAM that could only deliver a “matched or not-matched” result, emerging multilevel CAM (ML-CAM) is capable of delivering “the degree of match” with multilevel distance calculation. This feature has been desired in applications that need beyond-Boolean matching results. However, existing ML-CAM designs are limited by the bit-cell device discharging current mismatch and vulnerability to the timing of sensing operations for distance calculation. This inherent constraint makes it difficult to further improve the accuracy and scalability toward higher accuracy and higher dimension matching. In this work, we propose CapCAM, a multilevel Capacitive Content Addressable Memory. It could be implemented based on either static random-access memory (SRAM) or emerging technologies, e.g., the ferroelectric field-effect transistor (FeFET). CapCAM could provide linear and stable voltage drop scaled by the match degree and need no strict timing for result sensing, which embraces the high-accuracy and high-scalability search. The inherent enabler of CapCAM is the charge-domain computing mechanism. This article will present the basic concept, operating mechanisms, detailed circuit designs, and circuit-level simulations of CapCAM. Besides, we apply CapCAM to few-shot learning applications and compare CapCAM with the current-domain TCAM designs. Results show 99.2% accuracy for a five-way five-shot classification task with our proposed CapCAM design while considering 1-fF capacitors, 20-domain FeFETs, and 256 columns. In contrast, the prior work based on discharging dynamics requires strict timing controls and suffers from accuracy degradation under the same configuration, which demonstrates CapCAM’s capability of low-power, accurate, and scalable multilevel CAM (ML-CAM) computing.

Block-circulant based compression is a popular technique to accelerate neural network inference. Though storage and computing costs can be reduced by transforming weights into block-circulant matrices, this method incurs uneven data distribution in the frequency domain and imbalanced workload. In this paper, we propose RAB: a Reconfigurable Architecture Block-Circulant Neural Network Accelerator to solve the problems via two techniques. First, a fine-grained frequency-domain quantization is proposed to accelerate MAC operations. Second, a reconfigurable architecture is designed to transform FFT/IFFT modules into MAC modules, which alleviates the imbalanced workload and further improves efficiency. Experimental results show that RAB can achieve 1.9x/1.8x area/energy efficiency improvement compared with the state-of-the-art block-circulant compression based accelerator.

Nowadays, deep neural network (DNN) has played an important role in machine learning. Non-volatile computingin-memory (nvCIM) for DNN has become a new architecture to optimize hardware performance and energy efficiency. However, the existing nvCIM accelerators focus on system-level performance but ignore analog factors. In this paper, the sense margin and offset are considered in the proposed nvCIM framework. The margin enhancement based current-mode sense amplifier (MECSA) and the offset reduction based analog-to-digital converter (ORADC) are proposed to improve the accuracy of the ADC. Based on the above methods, the nvCIM framework is displayed and the experiment results show that the proposed framework has an improvement on area, power, and latency with the high accuracy of network models, and the energy efficiency is 2.3 - 20.4x compared to the existing RRAM based nvCIM accelerators.

Neural networks running on cloud servers have been widely used in smart surveillance, but they require high bandwidth to upload videos. Edge-cloud collaborative encoding based on ROI (Region-Of-Interest) can reduce bandwidth requirement, but it suffers from inaccurate ROI detection due to feedback latency and undetected new targets. To address the above challenges, we propose an object detection system named PETRI. It adopts a latency-hiding pipeline workflow with adaptive keyframe interval selection for different input videos, and utilizes a retro-tracking method to find undetected targets. While achieving negligible impact on model accuracy, the proposed PETRI can save up to 66.44% and 30.25% bandwidth compared with the cloud only method and the previous state-of-art work respectively.

In the era of Intelligent IoT, huge amount of sensor data is collected and then transmitted to processor elements in edge devices or cloud servers. The latency and energy consumption in this process have been a bottleneck and are becoming more severe. To mitigate this problem, the idea of combining sensors, memory and processors for collectively handling the data, has been proposed and explored actively in recent efforts. In this work, thin-film transistor (TFT), which has been widely adopted in display devices and flexible sensors, is exploited. It is shown that, while TFT is promising for near-sensor processing architecture, it also shows a great potential for computing and storage for large-area and low-cost edge sensors. More specifically, we propose an almost-nonvolatile near-sensor computing-in-memory (CiM) array based on indium-gallium- zinc-oxide (IGZO) TFT, and further, integrate the CiM array with a sensor array to be a sensing and data pre-process system. We show that such a TFT-based solution can accomplish realtime sensing and multiply-and-accumulate (MAC) processing in the analog field, which simplifies the system design with lowered energy and latency in our neural network evaluations.

There is an ongoing trend to increasingly offload inference tasks, such as CNNs, to edge devices in many IoT scenarios. As energy harvesting is an attractive IoT power source, recent ReRAM-based CNN accelerators have been designed for operation on harvested energy. When addressing the instability problems of harvested energy, prior optimization techniques often assume that the load is fixed, overlooking the close interactions among input power, computational load, and circuit efficiency, or adapt the dynamic load to match the just-in-time incoming power under a simple harvesting architecture with no intermediate energy storage. Targeting a more efficient harvesting architecture equipped with both energy storage and energy delivery modules, this paper is the first effort to target whole system, end-to-end efficiency for an energy harvesting ReRAM-based accelerator. First, we model the relationships among ReRAM load power, DC-DC converter efficiency, and power failure overhead. Then, a maximum computation progress tracking scheme ( MaxTracker ) is proposed to achieve a joint optimization of the whole system by tuning the load power of the ReRAM-based accelerator. Specifically, MaxTracker accommodates both continuous and intermittent computing schemes and provides dynamic ReRAM load according to harvesting scenarios. We evaluate MaxTracker over four input power scenarios, and the experimental results show average speedups of 38.4%/40.3% (up to 51.3%/84.4%), over a full activation scheme (with energy storage) and order-of-magnitude speedups over the recently proposed (energy storage-less) ResiRCA technique. Furthermore, we also explore MaxTracker in combination with the Capybara reconfigurable capacitor approach to offer more flexible tuners and thus further boost the system performance.

Ternary content addressable memory (TCAM) is one type of associative memory and has been widely used in caches, routers, and many other mapping-aware applications. While the conventional SRAM-based TCAM is high speed and bulky, there have been denser but slower and less reliable nonvolatile TCAMs using nonvolatile memory (NVM) devices. Meanwhile, some CMOS TCAMs using dynamic memories have been also proposed. Although dynamic TCAM could be denser than the 16T SRAM TCAM and more reliable than the nonvolatile TCAMs, CMOS dynamic TCAMs still suffer from the row-by-row refresh energy and time overheads. In this article, we propose dynamic TCAM using nanoelectromechanical (NEM) relays (DyTAN), and utilize one-shot refresh (OSR) to solve the memory refresh problem. By exploiting the unique NEM relay characteristics, DyTAN outperforms the existing works in the balance between density, speed, and power efficiency. Compared with the 16T SRAM-based TCAM, the 5T CMOS dynamic TCAM, the 2T2R TCAM, and the 2FeFET TCAM, evaluations show that the proposed DyTAN reduces the write energy by up to$2.3\times $,$1.3\times $,$131\times $, and$13.5\times $, and improves the search energy-delay-product (EDP) by up to$12.7\times $,$1.7\times $,$1.3\times $, and$2.8\times $, respectively.

Ferroelectric FETs (FeFETs) have emerged as a promising multi-level/cell (MLC) nonvolatile memory (NVM) candidate for low-power applications. This originates from the advantages of both efficient memory access and intrinsic device-level in-memory computing flexibilities. However, there still exist challenges for FeFET MLC NVM: (i) high power consumption in read operations due to high-gain requirement for sense amplifiers during sensing, and (ii) high latency and energy consumption in write operations with conventional recursive program-and-verify. Targeting at lower power, less latency, and higher density, this work investigates and optimizes the read and write approaches to MLC FeFET NVM design: (i) Adaptive FeFET memory State Mapping (ASM) between the FeFET drain-source current and the digital states to increase the sensing margin; (ii) Adaptive FeFET Gate Biasing (AGB) read methods that adopt the optimized FeFET gate voltage to boost the sensible dynamic range and to store more levels of states per cell; (iii) Adaptive Prediction-based Direct (APD) write methods that minimize the program-andverify activities. Evaluations show significant latency and energy improvement. Furthermore, the number of sensible levels of states per cell is also increased with an enhanced dynamic sensing range and an enhanced sensing margin.

Recently CNN-based methods have made remarkable progress in broad fields. Both network pruning algorithms and hardware accelerators have been introduced to accelerate CNN. However, existing pruning algorithms have not fully studied the pattern pruning method, and current index storage scheme of sparse CNN is not efficient. Furthermore, the performance of existing accelerators suffers from no-load PEs on sparse networks. This work proposes a software-hardware co-design to address these problems. The software includes an ADMM-based method which compresses the patterns of convolution kernels with acceptable accuracy loss, and a Huffman encoding method which reduces index storage overhead. The hardware is a fusion-enabled systolic architecture, which can reduce PEs' no-load rate and improve performance by supporting the channel fusion. On CIFAR-10, this work achieves 5.63× index storage reduction with 2-7 patterns among different layers with 0.87% top-1 accuracy loss. Compared with the state-of-art accelerator, this work achieves 1.54×-1.79× performance and 25%-34% reduction of no-load rate with reasonable area and power overheads.

There is growing interest in deploying energy harvesting processors and accelerators in Internet of Things (IoT). Energy harvesting harnesses the energy scavenged from the environment to power a system. Although it has many advantages over battery-operated systems such as lightweight, compact size, and no necessity of recharging and maintenance, it may suffer frequently power-down and a fluctuating power supply even with power on. Non-volatile processor (NVP) is a promising architecture for effective computing in energy harvesting scenarios. Recently, non-volatile accelerators (NVA) have been proposed to perform computations of deep learning algorithms. In this paper, we overview the recent studies of NVP and NVA across the layers of hardware, architecture, software and their co-design. Especially, we present the design insights of how the state-of-the-art works adapt their specific designs to the intermittent and fluctuating power conditions with the energy harvesting technology. Finally, we discuss recent trends using NVP and NVA in energy harvesting scenarios.

Although compression techniques like pruning or quantization are beneficial for accelerators' energy efficiency, the large search space makes finding the appropriate compression scheme difficult. Besides, most existing works ignore the combination of both pruning and quantization. In this paper, we propose a reinforcement learning (RL) based joint compression framework to find the appropriate pruning ratio and quantization bit-width for accelerators. By interacting with the energy model of the target accelerator, the RL agent can learn the effect of compression scheme on both accuracy and energy efficiency. Through a long trial-and-error process, the agent can finally reach an optimal trade-off between accuracy and energy efficiency. Compared with control groups whose compression hyper-parameters are not jointly optimized, the proposed framework can achieve at least 25% energy reduction with higher accuracy or much higher accuracy with small disadvantages on energy. Compared with 8-bit quantized baseline, the framework can achieve 90% and 85% energy reduction on Cifar10 and Cifar100 respectively.

Flexible capacitive pressure sensing system (FCPSS) is promising in the area of healthcare, robotics, and Internet of Things (IoT). As the size of the sensing array increases, designing energy-efficient FCPSS is getting challenging. This work provides a comprehensive solution for low-power FCPSS design, where major contributions are as follows. 1) Crosstalk-induced measurement error in a crossbar structure FCPSS is first studied and an accurate and low-power linear iterative algorithm is proposed for on-chip sensing array calibration (SAC). 2) Binary Neural Network (BNN)-based spatial-temporal adaptive sensing scheme for FCPSS is first proposed to utilize the sparsity of sampling and to further improve energy efficiency. Combined with the clock-gating-friendly low-power sensor interface, the system consumes 31.39 μJ energy and gains 95.04% capacitor measurement accuracy for each sensing operation on a 10×10 array, achieving 116× energy reduction compared with the state-of-the-art technology.

Compute-in-memory (CiM) is a promising method for mitigating the memory wall problem in data-intensive applications. The proposed bitwise logic-in-memory (BLiM) is targeted at data intensive applications, such as database, data encryption. This work proposes a low-power BLiM approach using the emerging nonvolatile ferroelectric FETs with direct write-back and data-adaptive dynamic sensing interface. Apart from general-purpose random-access memory, it also supports BLiM operations such as copy, not, nand, xor, and full adder (FA). The novel features of the proposed architecture include: (i) direct result-write-back based on the remnant bitline BLiM charge that avoids bitline sensing and charging operations; (ii) a fully dynamic sensing interface that needs no static reference current, but adopts data-adaptive voltage references for certain multi-operand operations, and (iii) selective bitline charging from wordline (instead of pre-charging all bitlines) to save power and also enable direct write-back. Detailed BLiM operations and benchmarking against conventional approaches show the promise of low-power computing with the FeFET-based circuit techniques.

Activation I/O traffic is a critical bottleneck of video neural network processor. Recent works adopted an inter-frame difference method to reduce activation size. However, current methods can't fully adapt to the various precision and sparsity in differential data. In this paper, we propose the multi-channel precision-sparsity-adapted codec, which will separate the differential activation and encode activation in multiple channels. We analyze the most adapted encoding of each channel, and select the optimal channel number with the best performance. A two-channel codec hardware has been implemented in the ASIC accelerator, which can encode/decode activations in parallel. Experiment results show that our coding achieves 2.2x-18.2x compression rate in three scenarios with no accuracy loss, and the hardware has 42x/174x improvement on speed and energy-efficiency compared with the software codec.

Energy harvesting is an attractive way to power future Internet of Things (IoT) devices since it can eliminate the need for battery or power cables. However, harvested energy is intrinsically unstable. While Field-programmable Gate Array (FPGAs) have been widely adopted in various embedded systems, it is hard to survive unstable power since all the memory components in FPGA are based on volatile Static Random-access Memory (SRAMs). The emerging non-volatile memory-based FPGAs provide promising potentials to keep configuration data on the chip during power outages. Few works have considered implementing efficient runtime intermediate data checkpoint on non-volatile FPGAs. To realize accumulative computation under intermittent power on FPGA, this article proposes a low-cost design framework, Data-Flow-Tracking FPGA (DFT-FPGA), which utilizes binary counters to track intermediate data flow. Instead of keeping all on-chip intermediate data, DFT-FPGA only targets on necessary data that is labeled by off-line analysis and identified by an online tracking system. The evaluation shows that compared with state-of-the-art techniques, DFT-FPGA can realize accumulative computing with less off-line workload and significantly reduce online roll-back time and resource utilization.

Convolutional neural networks (CNNs) have become powerful algorithms in various tasks. Application-specific integrated circuit (ASIC) has been widely used to accelerate CNN on mobile platforms because of its tremendous energy efficiency and performance. Meanwhile, CNNs have become much sparser with the development of network pruning algorithms. Recent works have employed different methods to improve the energy efficiency and performance of ASIC accelerators by utilizing the sparsity character of CNN. However, some of these methods suffer from tremendous output memory overhead and performance degradation induced by hash collisions. To overcome the aforementioned problem, we propose GAAS: an efficient group associated architecture and scheduler module for sparse CNN accelerators. It achieves smaller output memory overhead and higher performance compared with the state-of-the-art accelerator. Our proposed method GAAS mainly consists of two parts: 1) an$n$-way group associated architecture to reduce the output memory overhead and 2) a scheduler module to improve the performance. Besides, a load-balancing algorithm is proposed and implemented in the scheduler module to improve the performance by reducing the hash collision rate. To demonstrate the efficiency of GAAS, we implement a 4-way image-principal associated architecture with a$16\times 16$PE array and the scheduler module on our proposed method. The experimental results on AlexNet, VGG16, ResNet18, and MobileNet show that GAAS can reduce the output memory overhead by 50%, and it can surely improve the performance of them by$1.53\times $,$1.62\times $,$1.46\times $, and$1.55\times $, respectively.

In recent years, ASIC CNN Accelerators have attracted great attention among researchers for the high performance and energy efficiency. Some former works utilize the sparsity of CNN networks to improve the performance and the energy efficiency. However, these methods bring tremendous overhead to the output memory, and the performance suffers from the hash collision. This paper presents: 1) an N-Way Group Association Architecture to reduce the memory overhead for Sparse CNN Accelerators; 2) a Sparse Data Group Association Load Balancing Algorithm which is implemented by the Scheduler module in the architecture to reduce the collision rate and improve the performance. Compared with the state-of-art accelerator, this work achieves either 1) 1.74x performance with 50% memory overhead reduction in the 4-way associated design or 2) 1.91x performance without memory overhead reduction the 2-way associated design, which is close to the theoretical performance limit (without collision).

ReRAM-based processing-in-memory (PIM) architecture is a promising solution for deep neural networks (NN), due to its high energy efficiency and small footprint. However, traditional PIM architecture has to use a separate crossbar array to store either positive or negative (P/N) weights, which limits both energy efficiency and area efficiency. Even worse, imbalance running time of different layers and idle ADCs/DACs even lower down the whole system efficiency. This paper proposes AERIS, an Area/Energy-efficient 1T2R ReRAM based processing-In-memory NN System-on-a-chip to enhance both energy and area efficiency. We propose an area-efficient 1T2R ReRAM structure to represent both P/N weights in a single array, and a reference current cancelling scheme (RCS) is also presented for better accuracy. Moreover, a layer-balance scheduling strategy, as well as the power gating technique for interface circuits, such as ADCs/DACs, is adopted for higher energy efficiency. Experiment results show that compared with state-of-the-art ReRAM-based architectures, AERIS achieves 8.5x/1.3x peak energy/area efficiency improvements in total, due to layer-balance scheduling for different layers, power gating of interface circuits, and 1T2R ReRAM circuits. Furthermore, we demonstrate that the proposed RCS compensates the non-ideal factors of ReRAM and improves NN accuracy by 5.2% in the XNOR net on CIFAR-10 dataset.

Making embedded memory symmetric provides the capability of memory access in both rows and columns, which brings new opportunities of significant energy and time savings if only a portion of data in the words need to be accessed. This work investigates the use of ferroelectric field-effect transistors (FeFETs), an emerging nonvolatile, low-power, deeply-scalable, CMOS-compatible transistor technology, and proposes a new 3-transistor/cell symmetric nonvolatile memory (SymNVM). With ~1.67x higher density as compared with the prior FeFET design, significant benefits of energy and latency improvement have been achieved, as evaluated and discussed in depth in this paper.

Large-scale graph processing requires the high bandwidth of data access. However, as graph computing continues to scale, it becomes increasingly challenging to achieve a high bandwidth on generic computing architectures. The primary reasons include: the random access pattern causing local bandwidth degradation, the poor locality leading to unpredictable global data access, heavy conflicts on updating the same vertex, and unbalanced workloads across processing units. Processing-in-memory (PIM) has been explored as a promising solution to providing high bandwidth, yet open questions of graph processing on PIM devices remain in: 1) how to design hardware specializations and the interconnection scheme to fully utilize bandwidth of PIM devices and ensure locality and 2) how to allocate data and schedule processing flow to avoid conflicts and balance workloads. In this paper, we propose GraphH, a PIM architecture for graph processing on the hybrid memory cube array, to tackle all four problems mentioned above. From the architecture perspective, we integrate SRAM-based on-chip vertex buffers to eliminate local bandwidth degradation. We also introduce reconfigurable double-mesh connection to provide high global bandwidth. From the algorithm perspective, partitioning and scheduling methods like index mapping interval-block and round interval pair are introduced to GraphH, thus workloads are balanced and conflicts are avoided. Two optimization methods are further introduced to reduce synchronization overhead and reuse on-chip data. The experimental results on graphs with billions of edges demonstrate that GraphH outperforms DDR-based graph processing systems by up to two orders of magnitude and $5.12 {\times }$ speedup against the previous PIM design.

As power failures often occur in energy harvesting powered nonvolatile processors (NVPs), checkpointing is needed during program execution. It is observed that checkpointing is implemented with high overhead in applications with loops, because a large amount of data needs backup during loop execution. As such, we are motivated to reduce the amount of checkpointing data by analyzing data locality and shortening data lifetime in loops. This paper proposes a checkpointing-aware loop tiling technique which targets to reduce the checkpointing and recovering overheads for loops. Specifically, we first derive the optimal tile size for nested loops considering checkpointing distance and data dependencies. Then, the implementations of checkpointing and recovering for tiled loops are presented. Finally, the experiments are conducted to evaluate the effectiveness of the proposed method. The experimental results show that compared to the no-tiling method, the checkpointing-aware loop tiling method reduces the checkpointing and recovering data by 36.2% on average and reduces the total execution time and dynamic energy for checkpointing and recovering by 27.2% and 22.9% on average, respectively.

Thin-film transistor (TFT) circuits are important for flexible electronics which are promising in the area of wearable devices and Internet of Things. However, most flexible TFT technologies only have unipolar devices and the process variation and defective rate are relatively high, which impose challenges to TFT circuit design. In this paper, we propose a novel logic array design based on pseudo-CMOS logic to address the problems of unipolar TFT circuit design. A multilayer interconnection architecture is presented to improve the routability of circuit and the area efficiency. Cell mapping and wire routing algorithms, which aim to map the logic gates of circuit to logic array and then route the interconnection wires, are devised to improve the performance of circuit in consideration of parameter variations of TFT and meanwhile enhance the routability. The experimental results show that the proposed logic array along with design methodologies can reduce more than 80% area compared with transistor level scheme and help to improve performance significantly.

In line with the rapid development of the Internet of Things (IoT), the maintenance of on-board batteries for a trillion sensor nodes has become prohibitive both in time and costs. Energy harvesting is a promising solution to this problem. However, conventional energy-harvesting systems with storage suffer from low efficiency because of conversion loss and storage leakage. Direct supply systems without energy buffer provide higher efficiency, but fail to satisfy quality of service (QoS) due to mismatches between input power and workloads. Recently, a novel dual-channel photovoltaic power system has paved the way to achieve both high energy efficiency and QoS guarantee. This article focuses on the design-time and run-time co-optimization of the dual-channel solar power system. At the design stage, we develop a task failure rate estimation framework to balance design costs and failure rate. At run-time, we propose a task failure rate aware QoS tuning algorithm to further enhance energy efficiency. Through the experiments on both a simulation platform and a prototype board, this study demonstrates a 27% task failure rate reduction compared with conventional architectures with identical design costs. And the proposed online QoS tuning algorithm brings up to 30% improvement in energy efficiency with nearly zero failure rate penalty.

Thin-film transistor (TFT) circuits are facing the challenges of unipolar device, process variation, and yield problems, which can be addressed by pseudo-CMOS logic array with multi-layer interconnect. However, existing design methodology does not take mechanical strain and temperature into consideration which may seriously affect the carrier mobility of TFT and thus the performance of whole logic array circuits. This paper presents a novel cell mapping algorithm including intrarow mapping step and inter-row mapping step for flexible logic array to mitigate the mobility influence. Experimental results indicate that there is more than 40% performance improvement in critical path delay at best case with the proposed algorithm.

Nonvolatile processors have emerged as one of the promising solutions for energy harvesting scenarios, among which Wireless Sensor Networks (WSN) provide some of the most important applications. In a typical distributed sensing system, due to difference in location, energy harvester angles, power sources, etc. different nodes may have different amount of energy ready for use. While prior approaches have examined these challenges, they have not done so in the context of the features offered by nonvolatile computing approaches, which disrupt certain foundational assumptions. We propose a new set of nonvolatility-exploiting optimizations and embody them in the NEOFog system architecture. We discuss shifts in the tradeoffs in data and program distribution for nonvolatile processing-based WSNs, showing how non-volatile processing and non-volatile RF support alter the benefits of computation and communication-centric approaches. We also propose a new algorithm specific to nonvolatile sensing systems for load balancing both computation and communication demands. Collectively, the NV-aware optimizations in NEOFog increase the ability to perform in-fog processing by 4.2X and can increase this to 8X if virtualized nodes are 3X multiplexed.

To meet the needs of the Internet of Things (IoTs) devices, energy harvesting systems are proposed to power the systems instead of battery. Addressing the problem that harvested energy is unstable, nonvolatile processors (NVPs) have been proposed to hold intermediate data and avoid frequent program restarting from the beginning. However, NVPs often suffer a lot of waste on energy and system sources that can not be used for program execution owing to the frequent backup and recovery operations. To further improve the performance of NVPs, the paper proposes a dual-threshold method to maximize execution progress by enabling a system to hibernate to wait for power resumption instead of backing up data directly upon power interruptions. In particular, the optimal high and low thresholds, and the switches of system hibernation and backup, are discussed in details in order to achieve the goal of maximizing computation progress. The evaluation results show an average of up to 82.3% reduction on power failures and 1.5x speedup for forwarding progress by the proposed dual-threshold method compared to the conventional single threshold scheme.

While the Internet of Things (IoT) keeps advancing, its full adoption is continually blocked by power delivery problems. One promising solution is Non-Volatile (NV) processors, which harvest energy for themselves and employ a NV memory hierarchy. This allows them to perform computations when power is available, checkpoint and hibernate when power is scarce, and resume their work at a later time. However, utilizing NV memory creates new security vulnerabilities in the form of wear out attacks in the register file. This paper explores the dangers of this design oversight and proposes a mitigation strategy that takes advantage of the unique properties and operating characteristics of NV processors. The proposed defense integrates the power management unit and a two-level register rotation approach, which improves NV processor endurance by 30.1x in attack situations and an average of 7.1x in standard workloads.

This live demonstration shows a self-powered hardware platform for healthcare application on accurately monitoring ultraviolet(UV) radiation. Nonvolatile processor(NVP)[1] based sensor nodes harvest energy from solar panels and are connected to a Rohm ZigBee chip as the gateway node to support the network level communication. The gateway node finally uploads UV radiation data to a PC or a workstation for data analysis and display. Our platform provides two sensing modes (Data-First and Delay-First) under different UV radiation patterns to achieve high performance.

This paper proposes a direct digital frequency synthesizer (DDFS) based on Lookup-Table-Rotation (LUT-ROT) architecture. The DDFS takes the advantages of coarse-fine LUT and pipelined rotation units to achieve both high-speed and high-resolution. Based on the analysis of approximation error, the trade-off between accuracy and memory usage is achieved, which leads to the lowest amplitude of noise and the smallest size of LUT. Experimental results show that the DDFS achieves 11.7mW/GHz power consumption and 96dBc SFDR in 2-GHz clock frequency.

Energy harvesting is a promising solution to power ubiquitous Internet-of-Things (IoT) devices. But the frequent and inevitable power failure incurs significant backup overhead, greatly degrading performance and energy efficiency. Nonvolatile processor (NVP), which can checkpoint processor states, is designed to tackle this problem. The conventional system-level design method involves repeated system modification and verification on hardware, in which measurement on hardware consumes the majority time. To expedite the NVP-based system design process, we propose a rapid system prototyping flow to eliminate repeated hardware measurement in the design flow. This method involves an NVP system-level simulator, which takes the harvester power trace, system characteristics extracted from hardware, and user design as the input, and analyzes system energy and time profile under this power trace. Iterative system optimization and verification are conducted on the simulator, with only the final verification on hardware. We demonstrate the advantages of this method by two design cases, in which time, energy efficiency and the impact of different capacitor size are optimized.

In modern integrated circuit (IC) designs, intellectual property (IP) cores are often outsourced and designed by third-party vendors, resulting in the partial relinquishment of the control over the IC design flow. Thus, reliable verifications are required to mitigate the threat of hardware Trojans (HTs) which may be inserted into IP cores by malicious vendors. Existing trustiness verification methods cannot take the merit of high efficiency and accuracy at the same time. In this paper, we propose a multilevel fast trustiness verification framework based on feature analysis to detect HTs in third-party digital IP cores. The proposed framework combines flip-flop level and combinational logic level feature analysis to achieve both high efficiency and accuracy. Experimental results demonstrate that both explicitly and implicitly triggered HTs can be detected in very short time with a negligible false positive rate. More importantly, our framework has the unique advantage of being scalable to defend against future and stealthier HTs by adding new features into the framework.

Energy harvesting is becoming a favorable alternative to power future generation embedded systems, as it is more environmentally and user friendly. However, energy harvesting powered embedded systems suffer from frequent execution interruption due to unstable energy supply. To tackle this problem, nonvolatile memory has been deployed to save the whole volatile state for computation. When power resumes, the processor can restore the state back to volatile memories and continue execution. However, without careful consideration, the process of checkpointing and resuming could cause inconsistency between volatile and nonvolatile memories, which leads to irreversible errors. In this article, we propose a consistency-aware adaptive checkpointing scheme that ensures correctness for all checkpoints. The proposed technique efficiently identifies all possible inconsistency positions in programs and inserts auxiliary code to ensure correctness by offline analysis. In addition, adaptive checkpointing assisted register file profiling and online tracking techniques further reduce the overhead of each checkpoint. Evaluation results show that the proposed checkpointing strategy can successfully eliminate inconsistency errors and greatly reduce the checkpointing overhead.

Batteryless energy harvesting systems face a twofold challenge in converting incoming energy into forward progress. Not only must such systems contend with inherently weak and fluctuating power sources, but they have very limited temporal windows for capitalizing on transitory periods of above-average power. To maximize forward progress, such systems should aggressively consume energy when it is available, rather than optimizing for peak averagecase efficiency. However, there are multiple ways that a processor can trade between consumption and performance. In this paper, we examine two approaches, frequency scaling and resource scaling, and develop a predictor-driven scheme for dynamically allocating future power budgets between the two techniques. We show that our solution can achieve forward progress equal to 2.08X of the baseline Out-of-Order (OoO) processor with the best static configuration of frequency and resources. The combined technique outperforms either technique in isolation, with frequency-only and resource-only approaches achieving 1.43X and 1.61X forward progress improvements, respectively.

Thin-film transistor (TFT) circuits are important for flexible electronics which are promising in the area of wearable devices. However, most TFT technologies only have unipolar devices and the process variation and defective rate are relatively high, which impose challenges to TFT circuit design. In this paper, we propose a novel logic array based on pseudo-CMOS logic to address the problem of unipolar TFT circuit design. A multi-layer interconnect architecture and wire routing methodology are presented to improve the routability and meanwhile the area efficiency. The experimental results show that the proposed logic array reduces more than 80% area compared with transistor level scheme.

Energy harvesting has become a promising solution to power up Internet-of-Things (IoT) devices. In this scenario, the constrained power budget and frequent absence of ambient energy cause severe reliability issues and performance degradation on conventional CMOS computing circuits. Fortunately, the advent of nonvolatile processor (NVP) opens the possibility to compute continuously using an intermittent power supply. It is considered as a key component of the next generation IoT edge devices. In this work, we provide insights to the evolution of the NVP and its application in real world scenarios. Efforts on improving the performance of NVP and future research prospects are also discussed in this paper.

Intelligent computing and maintenance-free powering are two desirable characteristics of wearable IoT devices. Energy harvesting nonvolatile intelligent processor (NIP) with neural network computation capability has the potential to advance these goals. Individual ultraviolet (UV) exposure monitoring progressively becomes one conspicuous application of wearable devices. In resource constrained wearable sensor nodes, we can alleviate the data transmission burden via convolutional neural networks (CNNs) based pattern recognition. Nevertheless, in spite of the substantially improved computing capability of NIP, typically computational and memory intensive CNNs are still too bulky for on-node implementation. We develop an CNN-based pattern recognition system for nonvolatile IoT platform for smart UV monitoring, and propose a optimization method to achieve extremely tiny and efficient CNNs. Experimental results show that the offline-trained CNN can recognize individual UV exposure patterns with accuracy of 85%, and the simplified on-node CNN can achieve 93.2% parameters reduction with only 5% accuracy loss.

Flexible electronic is a promising technology for flexible and large-area sensing IoT applications, where ADC is a fundamental component This paper proposes a configurable and flexible VCO-based ADC, implemented with Oxide Thin-Film Transistors(TFT) technology. A VCO with four connecting modes is designed to configure the VCO-based ADC working under different power and resolutions. An Inkjet printing interconnection technology is introduced to enable the configurability of ADC, even after all TFT transistors are fabricated. It allows the ADC to be customized for different applications and avoids fabrication failure of devices. Experimental results show that the proposed ADC achieves a 0.8kS/s sampling rate. Its power consumption ranges from 541 to 866uW with ENOB from 3 to 6b.

Convolutional Neural Network (CNN) has become one of the most successful technologies for visual classification and other applications. As CNN models continue to evolve and adopt different kernel sizes in various applications, it is necessary for the hardware architecture to support reconfigurability. Previous FPGAs and programmable ASICs are fine-grained reconfigurable but with energy efficiency compromise. Considering specific features of CNNs, this paper presents an energy efficient coarse-grained reconfigurable architecture, denoted as CORAL. An application-specific configuration neural block is proposed for convolution operations with reconfigurable data quantization to reduce both energy consumption and on-chip memory requirements. An optimal data loading strategy is presented for CORAL to achieve the best energy efficiency. Experimental results show that CORAL improves 80.0% energy efficiency while reduces 78.9% chip area and 81.0% reconfiguration time compared with the best up-to-date programmable ASIC solution.

Energy harvesting techniques become increasingly popular as power supplies for embedded systems. However, the harvested energy is intrinsically unstable. Thus, the program execution may be interrupted frequently. Although the development of non-volatile processors (NVP) can save and restore execution states, both hardware and software challenges exist for energy harvesting powered embedded systems. On the hardware side, existing power detector only signals the ``poor'' quality of the harvested power based on a preset threshold voltage. The inappropriate setting of this threshold will make the NVP based embedded system suffer from either unnecessary checkpointing or checkpointing failures. On the software side, not all tasks can be checkpointed. Once the power is off, these tasks will have to restart from the beginning. In this paper, a task scheduler is proposed to maximize task progress by prioritizing tasks which cannot be checkpointed when power is weak so that they can finish before the power outage. To assist task scheduling, three additional modules including voltage monitor, checkpointing handler, and routine handler, are proposed. Experimental results show increased overall task progress and reduced energy consumption.

Batteryless IoT devices powered through energy harvesting face a fundamental imbalance between the potential volume of collected data and the amount of energy available for processing that data locally. However, many such devices perform similar operations across each new input record, which provides opportunities for mining the potential information in buffered historical data, at potentially lower effort, while processing new data rather than abandoning old inputs due to limited computational energy. We call this approach incidental computing, and highlight synergies between this approach and approximation techniques when deployed on a non-volatile processor platform (NVP). In addition to incidental computations, the backup and restore operations in an incidental NVP provide approximation opportunities and optimizations that are unique to NVPs.

Energy harvesting instead of battery is a better power source for wearable devices due to many advantages such as long operation time without maintenance and comfort to users. However, harvested energy is naturally unstable and program execution will be interrupted frequently. To solve this problem, nonvolatile processor (NVP) has been proposed because it can back up volatile state before the system energy is depleted. However, this backup process also introduces non-negligible energy and area overhead. To improve the performance of NVP, retention state has been proposed recently which can enable a system to retain the volatile data to wait for power resumption instead of saving data immediately. The goal of this paper is to forward program execution as much as possible by exploiting retention state. Specifically, two objectives are achieved. The first objective is to minimize power failures of the system if there is a great probability to get power resumption during retention state. The second objective of this paper is to achieve maximum computation efficiency if it is unlikely to avoid power failure. Compared to the instant backup scheme, evaluation results report that power failure can be reduced by 81.6% and computation efficiency can be increased by 2.5x by the proposed retention state-aware energy management strategy.

The continuous globalization of the semiconductor industry has significantly raised the vulnerability of chips under hardware Trojan (HT) attacks. It is extremely challenging to detect HTs in fabricated chips due to the existence of process variations (PVs), since PVs may cause larger impacts than HTs. In this paper, we propose a novel framework for HT detection in digital integrated circuits. The goal of this paper is to detect HTs inserted during fabrication. The HT detection problem is formulated as an under-determined linear system by a sparse gate profiling technique, and the existence of HTs is mapped to the sparse solution of the linear system. A Bayesian inference-based calibration technique is proposed to recover PVs for each chip for the sparse gate profiling technique. A batch of under-determined linear systems are solved together by the well-studied simultaneous orthogonal matching pursuit algorithm to get their common sparse solution. Experimental results show that even under big measurement errors, the proposed framework gets quite high HT detection rates with low measurement cost.

Nonvolatile static random access memory (nvSRAM) has been widely investigated as a promising on-chip memory architecture in energy harvesting sensor nodes, due to zero standby power, resilience to power failures, and fast read/write operations. However, conventional approaches back up all data from static random access memory into nonvolatile memory when power failures happen. It leads to significant energy overhead and peak inrush current, which has a negative impact on the system performance and circuit reliability. This paper proposes a holistic data backup optimization to mitigate these problems in nvSRAM, consisting of a partial backup algorithm and a run-time adaptive write policy. A statistic dead-block predictor is employed to achieve dead block identification with trivial hardware overhead. An adaptive policy is used to switch between write-back and write-through strategy to reduce the rollback induced by backup failures. Experimental results show that the proposed scheme improves the performance by 4.6% on average while the backup power consumption and the inrush current are reduced by 38.1% and 54% on average compared to the full backup scheme. What is more, the backup capacitor size for energy buffer can be reduced by 40% on average under the same performance constraint.

Wearable devices gain increasing popularity since they can collect important information for healthcare and well-being purposes. Compared with battery, energy harvesting is a better power source for these wearable devices due to many advantages. However, harvested energy is naturally unstable and program execution will be interrupted frequently. Nonvolatile processors demonstrate promising advantages to back up volatile state before the system energy is depleted. However, it also introduces non-negligible energy and area overhead. In this paper, we aim to reduce the amount of data that need to be backed up during a power failure. Based on the observation that stack size varies along program execution, we propose to analyze the application program and identify efficient backup positions, by which the stack content to back up can be significantly reduced. The evaluation results show an average of 45.7% reduction on nonvolatile stack size for stack backup, with 0.58% storage overhead. In the mean time, with the proposed schemes, the energy utilization and program forward progress can be greatly improved compared with instant backup.

Self-powered systems running on scavenged energy will be a key enabler for pervasive computing across the Internet of Things. The variability of input power in energy-harvesting systems limits the effectiveness of static optimizations aimed at maximizing the input-energy-to-computation ratio. We show that the resultant gap between available and exploitable energy is significant, and that energy storage optimizations alone do not significantly close the gap. We characterize these effects on a real, fabricated energy-harvesting system based on a nonvolatile processor. We introduce a unified energy-oriented approach to first optimize the number of backups, by more aggressively using the stored energy available when power failure occurs, and then optimize forward progress via improving the rate of input energy to computation via dynamic voltage and frequency scaling and self-learning techniques. We evaluate combining these schemes and show capture of up to 75.5% of all input energy toward processor computation, an average of 1.54 × increase over the best static “Forward Progress” baseline system. Notably, our energy-optimizing policy combinations simultaneously improve both the rate of forward progress and the rate of backup events (by up to 60.7% and 79.2% for RF power, respectively, and up to 231.2% and reduced to zero, respectively, for solar power). This contrasts with static frequency optimization approaches in which these two metrics are antagonistic.

Solar-powered sensor nodes (SCSNs) with energy storages have the greatest potential and are widely used in the coming era of the Internet of Things, since they avoid tedious battery maintenance tasks. However, because the solar energy source is unstable and limited, the sensor nodes suffer from high deadline miss ratio (DMR). To achieve better DMR, the existing scheduling algorithms find the best scheduling scheme in a single period of the recurring task queue and, hence, ignore the long-term performance. To tackle this challenge, this paper proposes a three-level dynamic voltage-frequency scaling (DVFS)-based scheduling strategy to minimize long-term DMR for dual-channel SCSNs. This approach includes a day-level scheduler to achieve a coarse-grained task arrangement, two artificial neural networks to determine the task priorities, and a DVFS-based task selection algorithm for slot-level execution. Experiments show that the proposed scheduler reduces DMR by over 30% on average.

Field-programmable gate arrays (FPGAs) have drawn lots of attentions due to their programmability and high performance. Recently, ultralow-power FPGAs for Internet of Things, together with energy-harvesting technique, have become an emerging self-powered computing platform. However, volatile memory in FPGA will lose their states under unstable power supplies and cannot work efficiently. Nonvolatile FPGA becomes a promising alternative. This paper proposes a hardware/software codesign nonvolatile FPGA with efficient offline/online checkpointing strategy (CP-FPGA). Backup energy is reduced by offline selecting proper checkpointing locations to minimize backup data. An online scheduler is further proposed to balance computation rollback overhead against backup energy. Experimental results show that the proposed CP-FPGA reduces 39.5% energy consumption on average compared with the state-of-the-art techniques.

Nonvolatile processors (NVPs) preserve run-time information when power failure occurs by utilizing nonvolatile memory technologies. This feature enables NVPs to make forward progress continuously under intermittent power supply in energy harvesting systems. This paper builds a gem5 based NVP simulator named NVPsim, which is validated against measured results of a fabricated prototype with reasonable error rate. Furthermore, to demonstrate the capability of NVPsim for architecture exploration, we evaluated performance and energy consumption of different NVP designs varying in the choice of nonvolatile memory for on-chip caches, the backup strategy and the energy buffer size. Experimental results indicate that nvSRAM outperforms other types of nonvolatile memory as the on-chip cache for energy harvesting systems.

The key to high performance for GPU architecture lies in massive threading to drive the large number of cores and enable overlapping of threading execution. However, in reality, the number of threads that can simultaneously execute is often limited by the size of the register file on GPUs. The traditional SRAM-based register file costs so large amount of chip area that it cannot scale to meet the increasing demand of massive threading for GPU applications. Racetrack memory is a promising technology for designing large capacity register file on GPUs due to its high data storage density. However, without careful deployment of registers, the lengthy shift operation of racetrack memory may hurt the performance. In this paper, we explore racetrack memory for designing high performance register file for GPU architecture. High storage density racetrack memory helps to improve the thread level parallelism, i.e., the number of threads that simultaneously execute. However, if the bits of the registers are not aligned to the ports, shift operations are required to move the bits to the ports. To mitigate the shift operation overhead problem, we develop a register file preshifting strategy and a compile-time managed register mapping algorithm. Experimental results demonstrate that our technique achieves up to 24% (19% on average) improvement in performance for a variety of GPU applications.

With the booming trend of internet of things (IoT), reconfigurable devices, such as FPGAs, have drawn lots of attentions due to their flexible and high-performance capability. However, commercial FPGAs suffer from high leakage power consumption, which makes zero-leakage nonvolatile FPGA (nvFPGA) promising. This paper proposes a hardware/ software co-design based nvFPGA with efficient checkpointing strategy. With nonvolatile checkpointing BRAM (CBRAM), it maintains both computation data as well as configuration when power-off to avoid expensive rollbacks due to data loss. A checkpointing location-aware technique is used to balance computation rollback overheads and backup energy. Experimental results show that the proposed checkpointing strategy can reduce 45.8% backup data of nvFPGA when system-level power gating happens.

Wearable devices begin to integrate into the daily lives along with recent technology development. One of such important applications is to accurately monitor ultraviolet (UV) radiation received by the human body. To compensate for the localized monitoring area of existing personal UV monitoring devices, this paper proposes a reconstruction method to estimate the UV dose over the entire body based on multiple discrete wearable UV sensor nodes. Ambient factors and individual factors are both considered in this paper. The proposed estimation method is validated by a range of UV data collection experiments in realistic scenarios. Experimental results show that the proposed method reduces 68.3% estimation errors on average compared with existing single sensor based methods.

Nonvolatile processors have manifested strong vitality in battery-less energy harvesting sensor nodes due to their characteristics of zero standby power, resilience to power failures and fast read/write operations. However, I/O and sensing operations cannot store their system states after power off, hence they are sensitive to power failures and high power switching overhead is induced during power oscillation, which significantly degrades the system performance. In this paper, we propose a novel performance-aware task scheduling technique considering power switching overhead for energy harvesting nonvolatile processors. We first give the analysis of the power switching overhead on energy harvesting sensor nodes. Then, the scheduling problem is formulated by MILP (Mixed Integer Linear Programming). Furthermore, a task splitting strategy is adopted to improve the performance and an heuristic scheduling algorithm is proposed to reduce the problem complexity. Experimental results show that the proposed scheduling approach can improve the performance by 14% on average compared to the state-of-the-art scheduling strategy. With the employment of the task splitting approach, the execution time can be further reduced by 10.6%.

Energy harvesting has been widely investigated as a promising alternative for future wearable sensors or internet-of-things. However, power and performance overhead is induced when IO operations are interrupted by power failures because non-preemptive characteristic of IO operations causes expensive re-executions. Furthermore, the state-of-art IO devices need long and power hungry initializing process, which makes IO operations inefficient in transient powered systems. This paper proposed a HW/SW co-design approach for nonvolatile IO system to maximize data acquisition. A ferroelectric flip-flop based nonvolatile IO architecture is adopted to reduce IO initialization overhead by 3-4 orders of magnitude. Based on the nonvolatile IO interface, we further formulate the optimal data acquisition as an INLP problem and a risk-aware online scheduler is presented to solve the problem efficiently. Experimental results show that the proposed HW/SW co-design architecture improves data acquisition by 2-5 times compared with conventional HW/SW architecture.

Energy harvesting is becoming a preferred choice for future wearable embedded systems compared to batteries because of size, longevity, and maintenance convenience. However, harvested energy is intrinsically unstable. In order to overcome this drawback, non-volatile processors (NVPs) have been proposed to bridge intermittent program execution. However, the harvested power is limited even with multiple energy harvesters when they are in-door. Therefore, a near-threshold processor is ideal to maintain low power consumption. One of the biggest challenges in realizing near-threshold non-volatile processor is to provide a required high write voltage to non-volatile memories when there is a power failure and checkpoint is needed. In order to address this challenge, in this paper, we propose a dynamic converter reconfiguration for ambient energy harvesting-based NVPs to support near-threshold computing. We further investigate thorough optimization techniques to achieve high robustness in reconfiguration and checkpointing, high conversion efficiency, and low ripple magnitude. Experimental results demonstrate that the proposed techniques can significantly reduce the power consumption and improve the performance of energy harvesters and NVPs.

Processing-in-memory (PIM) is a promising solution to address the "memory wall" challenges for future computer systems. Prior proposed PIM architectures put additional computation logic in or near memory. The emerging metal-oxide resistive random access memory (ReRAM) has showed its potential to be used for main memory. Moreover, with its crossbar array structure, ReRAM can perform matrix-vector multiplication efficiently, and has been widely studied to accelerate neural network (NN) applications. In this work, we propose a novel PIM architecture, called PRIME, to accelerate NN applications in ReRAM based main memory. In PRIME, a portion of ReRAM crossbar arrays can be configured as accelerators for NN applications or as normal memory for a larger memory space. We provide microarchitecture and circuit designs to enable the morphable functions with an insignificant area overhead. We also design a software/hardware interface for software developers to implement various NNs on PRIME. Benefiting from both the PIM architecture and the efficiency of using ReRAM for NN computation, PRIME distinguishes itself from prior work on NN acceleration, with significant performance improvement and energy saving. Our experimental results show that, compared with a state-of-the-art neural processing unit design, PRIME improves the performance by ~2360x and the energy consumption by ~895x, across the evaluated machine learning benchmarks.

Due to size, longevity, safety, and recharging concerns, energy harvesting is becoming a better choice for many wearable embedded systems. However, harvested energy is intrinsically unstable. In order to overcome this drawback, nonvolatile processors (NVPs) was proposed to bridge intermittent program execution. However, even with NVPs, frequent power interruption will severely degrade system performance. In this paper, we will propose a multi-source energy harvesting system to combine multiple harvesting sources to provide a more stable power supply using real-life ambient in-door irradiation level and thermal energy power trace. Maximum power extraction and converter parameter optimization techniques will be discussed. Experimental results show that by optimizing the MOSFET switch size in DC-DC converters between the harvesting system and NVPs, the average output power of the multi-source harvesting system can be increased. The proposed architecture is very promising in providing a stable energy source for NVPs.

Reliable and ultra-low power time synchronization becomes more and more important with the popularity of energy harvesting sensor nodes. This paper proposes an untethered and probabilistic ultra-lower power time synchronization method for energy intermittent sensor network. It avoids the frequent RF communications with the assistance of a solar clock. The SATS system consists of two main parts: the synchronizer, a low power solar clock module for time synchronization, and the S3-Mapping, an offline sequence matching algorithm. Furthermore, we develop an improved version of S3-Mapping, which reduces the computation complexity from exponential to linear using the redundancy models and the onion peeling method. The SATS system is validated by both simulations and a prototype, which shows that the second level synchronization precision can be achieved under reasonable probability. What's more, the energy consumption of time synchronization is reduced by over 1 ~ 2 magnitudes compared with the up-to-date low power time synchronization protocol.

Energy harvesting is under intense investigation as a promising substitute for batteries. However, given the erratic nature of ambient energy sources, temporary status in conventional CMOS circuits can be lost upon a sudden power outage. Taking advantage of emerging nonvolatile memories (NVMs), nonvolatile processor (NVP) backs up system contexts when power failure occurs, and recalls pre-stored data on resumption. It has become a hot topic for the capability to survive power variations and to guarantee forward progress on computation tasks. This paper acts as a brief guide introducing the concepts, the current status, the challenges and opportunities, as well as emerging applications of NVPs. Through this paper, we expect to help researchers who are new in this area better understand the development trends and future research prospects. We also hope to attract researchers to join and explore more innovative applications of NVP.

The random telegraph noise (RTN) is becoming more serious in advanced technologies. Due to the unpredictability of the physical phenomenon, RTN is a good randomness source for true random number generators (TRNG). In this paper, we build fundamental randomness models for TRNGs based on single trap- and multiple traps-induced RTN. We theoretically derive the autocorrelation coefficient, bias, and bit rate for RTN-based TRNGs. Two representative RTN-based TRNG schemes are simulated to verify the proposed randomness models. An oscillator-based TRNG is also studied based on the theoretical randomness model of multiple traps-induced RTN. We also provide basic guidelines for designing RTN-based TRNGs.

Energy harvesting from natural environment gives range of benefits for the Internet of things. Scavenging energy from photovoltaic (PV) cells is one of the most practical solutions in terms of power density among existing energy harvesting sources. PV power systems mandate the maximum power point tracking (MPPT) to scavenge the maximum possible solar energy. In general, a switching-mode power converter, an MPPT charger, controls the charging current to the energy storage element (a battery or equivalent), and the energy storage element provides power to the load device. The mismatch between the maximum power point (MPP) current and the load current is managed by the energy storage element. However, such architecture causes significant energy loss (typically over 20%) and a significant weight/volume and a high cost due to the cascaded power converters and the energy storage element. This paper pioneers a converter-less PV power system with the MPPT that directly supplies power to the load without the power converters or the energy storage element. The proposed system uses a nonvolatile microprocessor to enable an extremely fine-grain dynamic power management in a few hundred microseconds. This makes it possible to match the load current with the MPP current. We present detailed modeling, simulation, and optimization of the proposed energy harvesting system including the radio frequency transceiver. Experiments show that the proposed setup achieves an 87.1% of overall system efficiency during a day, 30.6% higher than the conventional MPPT methods in actual measurements, and thus a significantly higher duty cycle under a weak solar irradiance.

With the advent of the era of trillion sensors, solar-powered sensor nodes are widely used as they do not require battery charging or replacement. However, the limited and intermittent solar energy supply seriously affects deadline miss rate (DMR) of tasks. Furthermore, traditional solar-powered sensor nodes also suffer from energy loss of battery charging and voltage conversion. Recently, a storage-less and converter-less power supply architecture has been proposed to achieve higher energy efficiency by removing the leaky energy storage and dc voltage conversion. Without energy storages, a node using inter-task scheduling is more sensitive to solar variations, which results in high DMRs. This paper proposes an intra-task scheduling scheme for the storage-less and converter-less solar-powered sensor nodes, whose features include power prediction based on classified solar profiles, a trigger mechanism to select scheduling points, an artificial neural network to calculate task priorities and a fine-grained task selection algorithm. Experimental results show that the proposed algorithm reduces DMR by up to 30% and improves energy utilization efficiency by 20% with trivial energy overheads.

Developing circuits for streaming applications written in C (or its variants) can benefit greatly from C-to-RTL (C2RTL) synthesis. Yet, most existing C2RTL tools lack system-level options to trade off various design constraints, such as delay and area. This article introduces a systematic way to accomplish C2RTL synthesis for streaming applications containing thousands of lines of C (or its variants) codes. Synthesizing circuits for such large applications presents serious challenges for existing C2RTL tools. Specifically, the proposed approach determines simultaneously the number of pipeline stages and the number of times that each functional block is duplicated in each pipeline stage. A mixed integer linear programming-based solution is formulated for obtaining the optimal solution. Furthermore, a heuristic algorithm is developed for large-scale problems. To accommodate the differences of the data rates between the adjacent hardware modules, first-in-first-out (FIFO) buffers are indispensable, but their overheads are nonnegligible. A parallelism-aware FIFO sizing method is also introduced to determine the optimal sizes of FIFOs. Experimental results on seven real-world applications demonstrate that the algorithms in the synthesis flow can make effective design trade-offs and find superior solutions in a short time compared with existing approaches. Furthermore, the algorithms achieve optimal results in most cases with subsecond running time.

The emerging nonvolatile memory (NVM) technology can potentially change the landscape of future IC designs with numerous benefits, such as high performance, low leakage power, and data retention. These advantages motivate designers to exploit utilizing NVM in in ASIC and FPGA. However, unique challenges such as large write energy and asymmetric read/write operations, lead to extra design knobs. This paper focuses on the NVM allocation and hierarchy optimization in high-level synthesis. A hierarchical hybrid memory architecture is presented. The proposed framework optimizes the memory hierarchy, type (NVM or SRAM) and capacity. Both an mixed-integer linear programming (MILP) and a branch-and-bound heuristic are developed. Experimental results demonstrate up to 69.3% power reduction compared with designs without NVM.

In the near future, self-powered wearable sensor nodes are becoming as an emerging trend in healthcare applications. However, several design challenges exist before their wide usage, such as the limited and unstable power output as well as unpredictable power profiles. This paper shows an overview of candidate solutions and proposes that a cross-layer approach is needed.

Recently, general-purpose graphics processing units (GPGPUs) have been widely used to accelerate computing in various applications. To store the contexts of thousands of concurrent threads on a GPU, a large static random-access memory (SRAM)-based register file is employed. Due to high leakage power of SRAM, the register file consumes 20% to 40% of the total GPU power consumption. Thus, hybrid memory system, which combines SRAM and the emerging non-volatile memory (NVM), has been employed for register file design on GPUs. Although it has shown strong potential to alleviate the power issue of GPUs, existing hybrid memory solutions might not exploit the intrinsic feature of GPU register file. By leveraging the warp schedule on GPU, this paper proposes a hybrid register architecture which consists of a NVM-based register file and mixed SRAM-based write buffers with a warp-aware write back strategy. Simulation results show that our design can eliminate 64% of write accesses to NVM and reduce power of register file by 66% on average, with only 4.2% performance degradation. After we apply the power gating technique, the register power is further reduced to 25% of SRAM counterpart on average.

Wearable devices are becoming increasingly important in our daily lives. Energy harvesting instead of battery is a better power source for these wearable devices due to many advantages. However, harvested energy is often unstable and program execution will be frequently interrupted. Non-volatile processors demonstrate promising advantages to back up volatile state before the system energy is depleted. But Non-volatile processors require additional memory for backing up, thus introducing non-negligible overhead in terms of energy, runtime as well as chip area. In this work, we target at non-volatile register reduction for energy harvesting based wearable devices. This paper proposes to stack trimming the memory footprint via a novel compiler directed method. The evaluation results deliver on average 28.6% reduction of non-volatile register files for backing up stack area, with ultra low runtime overhead.

Energy harvesting is gaining more and more attentions due to its characteristics of ultra-long operation time without maintenance. However, frequent unpredictable power failures from energy harvesters bring performance and reliability challenges to traditional processors. Nonvolatile processors are promising to solve such a problem due to their advantage of zero leakage and efficient backup and restore operations. To optimize the nonvolatile processor design, this paper proposes new metrics of nonvolatile processors to consider energy harvesting factors for the first time. Furthermore, we explore the nonvolatile processor design from circuit to system level. A prototype of energy harvesting nonvolatile processor is set up and experimental results show that the proposed performance metric meets the measured results by less than 6.27% average errors. Finally, the energy consumption of nonvolatile processor is analyzed under different benchmarks.

Energy harvesting has become a favorable alternative to batteries for wearable embedded systems since it is more environmental and user friendly. However, harvested energy is intrinsically unstable, which could frequently interrupt a processor's execution. To tackle this problem, non-volatile processors have been proposed to checkpoint the whole volatile processor state into attached non-volatile memories periodically. When power resumes, the processor can copy the checkpointed state back to volatile memories and continue execution. However, without careful consideration, the process of checkpointing and resuming could cause inconsistency among different memory addresses and lead to irreversible errors. In this paper, we present a consistency aware checkpointing scheme that ensures correctness for all checkpoints. The proposed technique efficiently identifies all possible inconsistency positions in programs and inserts auxiliary code to ensure correctness. Evaluation results show that the proposed checkpointing technique can successfully eliminate inconsistency errors and greatly reduce the checkpointing overhead.

Solar-powered sensor nodes with energy storages are widely used today and promising in the coming trillion sensor era, as they do not require manual battery charging or replacement. The changeable and limited solar power supply seriously affects the deadline miss rates (DMRs) of tasks on these nodes and therefore energy-driven task scheduling is necessary. However, current algorithms focus on the single period (or the current task queue) for high energy utilization and suffer from bad long term DMR. To get better long term DMR, we propose a long term deadline-aware scheduling algorithm with energy migration strategies for distributed super capacitors. Experimental results show that the proposed algorithm reduces the DMR by 27.8% and brings less than 3% of the total energy consumption.

Energy harvesting has been widely investigated as a promising method of providing power for ultra-low-power applications. Such energy sources include solar energy, radio-frequency (RF) radiation, piezoelectricity, thermal gradients, etc. However, the power supplied by these sources is highly unreliable and dependent upon ambient environment factors. Hence, it is necessary to develop specialized systems that are tolerant to this power variation, and also capable of making forward progress on the computation tasks. The simulation platform in this paper is calibrated using measured results from a fabricated nonvolatile processor and used to explore the design space for a nonvolatile processor with different architectures, different input power sources, and policies for maximizing forward progress.

Energy harvesting systems without an energy storage device have to efficiently harness the fluctuating and weak power sources to ensure the maximum computational progress. While a simpler processor enables a higher turn-on potential with a weak source, a more powerful processor can utilize more energy that is harvested. Earlier work shows that different complexity levels of nonvolatile microarchitectures provide best fit for different power sources, and even different trails within same power source. In this work, we propose a dynamic nonvolatile microarchitecture by integrating all non-pipelined (NP), N-stage-pipeline (NSP), and Out of Order (OoO) cores together. Neural network machine learning algorithms are also integrated to dynamically adjust the microarchitecture to achieve the maximum forward progress. This integrated solution can achieve forward progress equal to 2.4× of the baseline NP architecture (1.82× of an OoO core).

Racetrack memory is an emerging non-volatile memory based on spintronic domain wall technology. It can achieve ultra-high storage density. Also, its read/write speed is comparable to that of SRAM. Due to the tape-like structure of its storage cell, a "shift" operation is introduced to access racetrack memory. Thus, prior research mainly focused on minimizing shift latency/energy of racetrack memory while leveraging its ultra-high storage density. Yet the reliability issue of a shift operation, however, is not well addressed. In fact, racetrack memory suffers from unsuccessful shift due to domain misalignment. Such a problem is called "position error" in this work. It can significantly reduce mean-time-to-failure (MTTF) of racetrack memory to an intolerable level. Even worse, conventional error correction codes (ECCs), which are designed for "bit errors", cannot protect racetrack memory from the position errors.

With the fast development of Internet of Things (IoTs) in recent years, many IoT applications, such as structure health monitoring, surveillance camera and etc, require both extensive computation for burst-mode signal processing as well as ultra low power continuous operations. However, most of conventional IoT processors focus on ultra low power consumption and cannot satisfy those demands. This paper proposes a novel energy-efficient heterogenous dual-core processor, which includes both an ultra low power near-threshold CoreL and a fast CoreH to meet those emerging requirements. Furthermore, an optimal framework is proposed to realize energy efficient task mapping and scheduling. The processor is fabricated and its energy consumption in low power mode is as low as 7.7pJ/cycle and outperforms related work. Detailed analysis under several real applications shows that up to 2.62× energy efficiency improvements can be achieved without deadline miss compared with the high-performance-only signle core architecture.

To mitigate the “Power Wall” challenges for both mobile devices and data centers, accelerator-rich architecture with normally-off mode has been intensively studied recently. Power/energy optimization in high-level synthesis for accelerator design is critical for such accelerator-rich architecture. The emerging nonvolatile memory (NVM), offers many benefits such as ultra-low leakage power, high density, and instant power-on/off, and therefore is a promising alternative for the hardware accelerator design to achieve further power reduction. However, such NVM suffers from large write energy and latency, which brings new challenges for the buffer allocation in the custom accelerator design. This paper presents the first framework that optimizes NVM allocation in high-level synthesis for custom accelerator design, considering loop transformations. It solves the loop transformation, buffer allocation, and buffer type selection to minimize the memory power consumption, while under area, bandwidth, and performance constraints. This paper formulates the optimization problem, and solves it with a problem-specific designed stimulated annealing solution. Experiments demonstrate 32% extra power reduction compared with the previous method without optimizing loop transformations.

This paper studies the performance analysis problem of energy-harvesting real-time network systems in the Real-Time Calculus (RTC) framework. The behavior of an energy-harvesting node turns out to be a generalization of two known components in RTC: it behaves like an AND connector if the capacitor used to temporally store surplus energy has unlimited capacity and there is no energy loss, while it behaves like a greedy processing component (GPC) if the size of the capacitor is zero and thus surplus energy is lost or passed to other nodes immediately. In this paper, methods are developed to analyze the worst-case performance, in terms of delay and backlog, of energy-harvesting nodes as well as compute upper/lower bounds of their data and energy outputs. Moreover, with the proposed analysis methods, we disclose some interesting properties of the worst-case behaviors of energy-harvesting systems, which provide useful information to guide system design. Experiments are conducted to evaluate our theoretical contributions and also confirm that the disclosed properties are not just the result of our analysis, but indeed hold in realistic system behaviors.

This paper pioneers the maximum power point tracking (MPPT) of photovoltaic (PV) cells that directly supply power to a microprocessor without an energy storage element (a battery or a large-size capacitor) nor power converters. The maximum power point tracking is conventionally performed by an MPPT charger that stores in the energy storage element, and a voltage regulator (typically a DC-DC converter) produces a proper voltage level for the microprocessor. The energy storage element is an energy buffer and makes it possible to perform MPPT of the PV cells and power management of the microprocessor independently. However, the energy storage element, MPPT charger and DC-DC converter cause seriously limited lifetime (when a typical battery is adopted), significant energy loss (typically over 20%), increased weight/volume and high cost, etc. The proposed method enables extremely fine-grain dynamic power management (DPM) in every a few hundred microseconds and performs the MPPT without using an MPPT charger and a DC-DC converter as well as an energy storage element. We achieve 84.5% of energy harvesting efficiency using the proposed setup with huge reduction in cost, weight and volume, and extended lifetime, which is not even numerically comparable with conventional MPPT methods.

Solar-powered sensor nodes without specific energy maintenance have shown great promise in many applications, but they suffer from large energy storage and power converter loss. The storage-less and converter-less architecture with nonvolatile processing units has been proposed to reduce the energy loss. However, the architecture is sensitive to solar variations, since there is no energy buffering. Traditional inter-task scheduling methods may not work well due to large variations of task execution time. To tackle the challenge, we develop an algorithm for intra-task scheduling to achieve better quality of service. The experimental results show that the intra-task scheduling algorithm reduces deadline miss rate by as much as 35% and improves energy utilization by close to 20%.

Nonvolatile processors (NVP) have been an emerging topic in recent years due to its zero standby power, data retention and instant-on features. The conventional full replacement architecture in NVP has drawbacks of large area overhead and high backup energy. This paper provides a partial replacement based hybrid register architecture to significantly abate above problems. However, the hybrid register architecture can induce potential critical data loss and backup errors. In this paper, we propose a critical-data overflow aware register allocation (CORA). Different from other register allocation methods, CORA efficiently reduces the possibility of critical data spilling and backup errors. The experiment results show that CORA reduces the critical data overflow rate by up to 52%. The hybrid register architecture reduces the chip area by 45.1% and backup energy by 82.8% when using CORA.

Nonvolatile (NV) processors have attracted much attention in recent years due to their zero standby power, resilience to power failures, and instant-on feature. One design challenge of NV processors is the excess area needed by NV registers. This paper introduces a parallel compare and compress (PaCC) architecture to reduce such excess area. A key component of the PaCC architecture is a new codec which effectively balances area and performance. In addition, the PaCC architecture includes a configurable state table to support reference vector selection for different applications. With the proposed vector selection algorithm, the PaCC architecture can outperform other vector selection approaches by over 59% in terms of reduction in the number of NV registers. The proposed architecture has been fully realized at the circuit level and synthesized for the Rohm's 0.13-μm ferroelectric-CMOS hybrid process. Results demonstrate that the design can reduce the number of NV registers by 70%-80% with less than 1% overflow possibility, which leads to up to 30% processor area saving. The overall approach is applicable to any NV processor design regardless of the NV material used.

Developing FPGA solutions for streaming applications written in C (or its variants) can benefit greatly from automatic C-to-RTL (C2RTL) synthesis. Yet, the complexity and stringent throughput/cost constraints of such applications are rather challenging for existing C2RTL synthesis tools. This paper considers automatic partition and block-level parallelization to address these challenges. An MILP-based approach is introduced for finding an optimal partition of a given program into blocks while allowing block-level parallelization. In order to handle extremely large problem instances, a heuristic algorithm is also discussed. Experimental results based on seven well known multimedia applications demonstrate the effectiveness of both solutions.

Automatic C-to-RTL (C2RTL) synthesis can greatly benefit hardware design for streaming applications. However, stringent through-put/area constraints, especially the demand for power optimization at the system level is rather challenging for existing C2RTL synthesis tools. This paper considers a power-aware C2RTL framework using voltage-frequency islands (VFIs) to address these challenges. Given the throughput, area, and power constraints, an MILP-based approach is introduced to synthesize C-code into an RTL design by simultaneously considering three design knobs, i.e., partition, parallelization, and VFI assignment to get the global optimal solution. A heuristic solution is also discussed to deal with the scalability challenge facing the MILP formulation. Experimental results based on four well known multimedia applications demonstrate the effectiveness of both solutions.

Nonvolatile processor (NVP) has become an emerging topic in recent years. The conventional NV processor equips each flip-flop with a nonvolatile storage for data backup, which results in much faster backup speed with significant area overheads. A compression based architecture (PRLC) solved the area problem but with a nontrivial increasing on backup time. This paper provides a segment-based parallel compression (SPaC) architecture to achieve tradeoffs between area and backup speed. Furthermore, we use an off-line and online hybrid method to balance the workloads of different compression modules in SPaC. Experimental results show that SPaC can achieve 76% speed up against PRLC and meanwhile reduces the area by 16% against conventional NV processors.

In modern embedded systems, the C2RTL (high-level synthesis) technology helps the designer to greatly reduce time-to-market, while satisfying the performance and cost constraints. To attack the performance challenges in complex designs, we propose a FIFO-connected hierarchical approach to replace the traditional flatten one in stream applications. Furthermore, we develop an analytical algorithm to find the optimal FIFO capacity to connect multiple modules efficiently. Finally, we prove the advantages of the proposed method and the feasibility of our algorithm in seven real applications. Experimental results show that the hierarchical approach can have an up to 10.43 times speedup compared to the flatten design, while our analytical FIFO sizing algorithm shrinks design time from hours to seconds with the same accuracy compared to the simulation based approach.

Nonvolatile processor has become an emerging topic in recent years due to its zero standby power, resilience to power failures and instant on feature. This paper first demonstrated a fabricated nonvolatile 8051-compatible processor design, which indicates the ferroelectric nonvolatile version leads to over 90% area overhead compared with the volatile design. Therefore, we proposed a compare and compress recovery architecture, consisting of a parallel run-length codec (PRLC) and a state table logic, to reduce the area of nonvolatile registers. Experimental results demonstrate that it can reduce the number of nonvolatile registers by 4 times with less than 1% overflow possibility, which leads to 43% overall processor area savings. Furthermore, we implemented the novel PRLC and defined the method to optimize the optimal parallel degree to accelerate the compressions. Finally, we proposed a reconfigurable state table architecture, which supports the reference vector selecting for different applications. With our heuristic vector selecting algorithm, the optimal vector can provide over 42% better register number reduction than other vector selecting approaches. Our method is also applicable to designs with other nonvolatile materials based registers.

Energy harvesting sensor nodes based on real nonvolatile processors are demonstrated to show the desirable characteristics of those systems, such as no battery, zero stand-by power, microsecond-scale sleep and wake-up time, high resilience to random power failures and fine-grained power management. Furthermore, we show its applications to a distributed moving object detection system, one of novel nonvolatile computing systems.

As variable delays are observed in the integrated circuits under different data inputs, it is expected to enhance the performance of the circuit using the average-case design methodology. This paper presents a novel approach using the time-domain multistage speculation to realize a variable-latency circuit, in which speculation points with double-sampling and check-recovery units are inserted into the critical path to enhance the performance. Furthermore, a design framework is implemented to convert a original circuit into the new one automatically. Experimental results showed that a 1.79 - 4.42 speedup in a 64-bit ripple carry adder and up to 30.5% throughput enhancements in several ISCAS and MCNC benchmarks with reasonable area overheads.

Wireless sensor networks hold the potential to open new domains to distributed data acquisition. However, low-cost battery-powered nodes are often used to implement such networks, resulting in tight energy and communication bandwidth constraints. Cluster-based data compression and aggregation helps to reduce communication energy consumption. However, neglecting to adapt cluster sizes to local network conditions has limited the efficiency of previous clustering schemes. We have found that sensor node distances and densities are key factors in clustering. To the best of our knowledge, this is the first work taking these factors into consideration when adaptively forming data aggregation clusters. Compared with previous uniform-size clustering techniques, the proposed algorithm achieves up to 24% communication energy savings in uniform density networks and 36% savings in non-uniform density networks.

In this paper, we propose an energy efficient architecture of wireless sensor network node. It consists of a general-purpose processor and several compression accelerators. To verify the low energy consumption of this architecture, we implement a baseband chip of sensor node by 1-poly 6-metal 0.18um CMOS technology, in which a hardware accelerator is realized based on a distributed wavelet compression algorithm. Our measurements show that the compression accelerator based architecture reduces over 98% energy consumption compared with the traditional solution.

Wireless sensor networks hold the potential to open new domains to distributed data acquisition. However, such networks are prone to premature failure because some nodes deplete their batteries more rapidly than others due to workload variations, non-uniform communication, and heterogenous hardware. Many-to-one traffic patterns are common in sensor networks, further increasing node power consumption heterogeneity. Most previous sensor network lifetime enhancement techniques focused on balancing power distribution, based on the assumption of uniform battery capacity allocation among homogeneous nodes.

Multi-threshold CMOS is a valuable leakage reduction method in circuit standby mode. Reducing leakage current through fine-grain sleep transistor insertion (FGSTI) makes it easier to guarantee circuit functionality and improves circuit noise margins. In this paper, we first indicate the negligible dependence of ST size on the amount of leakage saving which makes the two-phase FGSTI reasonable based on our leakage current and delay models. Then we introduce a novel two-phase FGSTI technique: a) ST placement and b) ST sizing, which are formally modeled as two linear programming (LP) models respectively. Our experimental results show that the two-phase FGSTI technique can achieve 78.91%, 92.55%, 97.97% leakage saving when the circuit slowdown is 0%, 3%, 5% respectively. Comparing to the simultaneous ST placement and sizing method using mix integer linear programming (MLP) [1], our technique leads to on average 2% more leakage current reduction while at least 10X runtime saving since fewer variables and constraints with less approximation are used in the LP models. When the circuit slowdown is large enough to perform conventional fixed slowdown method, our technique can still achieve 75.48% ST area saving. Moreover, we show that when the circuit slowdown is 0%, it should be carefully considered to use FGSTI technique due to a large amount of leakage feedback gates.