Grace Li Zhang

dblp:179/3068 · DBLP profile ↗
← Back
55ranked-venue papers
14as first author
41since 2021 · last 2026
0000-0002-8289-9288ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Systems, architecture and hardware · 54 · 14 first-author · 40 since 2021Software engineering, systems software and programming languages · 19 · 3 first-author · 15 since 2021Artificial intelligence and machine learning · 1 · 1 since 2021
YearPublicationVenuePosition
2026 GCPT: Gradient-aware Clustering Method for Efficient Post-Training Quantization in Large Neural Networks
abstract
Large-scale neural network models have achieved outstanding performance across diverse tasks, but often come with expensive computational costs. In this paper, we propose a gradient-aware clustering method for post-training quantization (GCPT) in order to effectively reduce the computational overhead. Our key idea is to cluster the weights of linear layers based on their gradient-aware contributions to the overall loss function. Afterwards, all weights are replaced by a small set of cluster centroids to minimize the variation of the loss function due to quantization. To further accelerate inference, the inputs associated with those weights in the same cluster are first aggregated and then the sum is multiplied with the shared centroid, thereby reducing the number of scalar multiplications. Experiments on three large-scale models demonstrate that the proposed GCPT method achieves up to 93.8% computational cost reduction, while preserving memory usage and inference accuracy, compared to other state-of-the-art methods.
Chuyi Dai, Grace Li Zhang
DATE6
2026 Late Breaking Results: Conversion of Neural Networks into Logic Flows for Edge Computing
abstract
Neural networks have been successfully applied in various resource-constrained edge devices, where usually central processing units (CPUs) instead of graphics processing units exist due to limited power availability. State-of-the-art research still focuses on efficiently executing enormous numbers of multiply-accumulate (MAC) operations. However, CPUs themselves are not good at executing such mathematical operations on a large scale, since they are more suited to execute control flow logic, i.e., computer algorithms. To enhance the computation efficiency of neural networks on CPUs, in this paper, we propose to convert them into logic flows for execution. Specifically, neural networks are first converted into equivalent decision trees, from which decision paths with constant leaves are then selected and compressed into logic flows. Such logic flows consist of if and else structures and a reduced number of MAC operations. Experimental results demonstrate that the latency can be reduced by up to 14.9 % on a simulated RISC-V CPU without any accuracy degradation. - The code is open source at https://github.com/TUDa-HWAI/NN2Logic
Daniel Stein, Shaoyi Huang, Rolf Drechsler, Bing Li 0005, Grace Li Zhang
DATE5
2026 HLSRewriter: Efficient Refactoring and Optimization of C/C++ Code with LLMs for High-Level Synthesis
abstract
In High-Level Synthesis (HLS), refactoring a standard C/C++ code into its HLS-compatible version (HLS-C) still requires significant human effort. While various program scripts have been introduced to automate this process, the resulting code still contains many HLS-incompatible issues that need to be manually refactored and optimized by developers. Since Large Language Models (LLMs) have the ability to automate code generation, they can also be used for automated code refactoring and optimization in HLS. However, due to the limited training of LLMs, considering hardware and software simultaneously, hallucinations may occur when using LLMs for HLS, leading to synthesis failures. To address these challenges, we introduce HLSRewriter , an LLM-aided code refactoring and optimization framework that takes regular C/C++ code as input and automatically generates its corresponding optimized HLS-C code for hardware synthesis with minimal human intervention. To mitigate LLM hallucinations, a step-wise reasoning process is employed to analyze and detect HLS-incompatible errors. Afterwards, a repair library containing reference templates is efficiently created by scanning the HLS tool manual, followed by cooperation with a Retrieval-Augmented Generation (RAG) paradigm to guide the LLMs toward correct refactoring. In addition, a pipeline-aware decomposition strategy is introduced to progressively break down complex loop structures into smaller tasks with a balanced trade-off between latency and area, thereby enabling efficient pipelining and parallel execution. To further improve hardware efficiency, a bit width adjuster module is incorporated into this framework to optimize the precision of floating-point variables. Moreover, LLM-aided HLS optimization strategies are introduced to add/tune hardware directives in HLS-C code, thereby enhancing the performance of the final synthesized hardware. Experimental results demonstrate that the proposed LLM-aided framework can achieve higher refactoring pass rates and superior hardware performance in 24 real-world tasks compared with traditional approaches and the direct application of LLMs for code refactoring and optimization. The codes are open-sourced at this link: https://github.com/code-source1/catapult .
Kangwei Xu, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Ulf Schlichtmann, Bing Li 0005
ACM Trans. Design Autom. Electr. Syst.2
2025 An Efficient General-Purpose Optical Accelerator for Neural Networks
abstract
General-purpose optical accelerators (GOAs) have emerged as a promising platform to accelerate deep neural networks (DNNs) due to their low latency and energy consumption. Such an accelerator is usually composed of a given number of interleaving Mach-Zehnder-Interferometers (MZIs). This interleaving architecture, however, has a low efficiency when accelerating neural networks of various sizes due to the mismatch between weight matrices and the GOA architecture. In this work, a hybrid GOA architecture is proposed to enhance the mapping efficiency of neural networks onto the GOA. In this architecture, independent MZI modules are connected with microring resonators (MRRs), so that they can be combined to process large neural networks efficiently. Each of these modules implements a unitary matrix with inputs adjusted by tunable coefficients. The parameters of the proposed architecture are searched using genetic algorithm. To enhance the accuracy of neural networks, selected weight matrices are expanded to multiple unitary matrices applying singular value decomposition (SVD). The kernels in neural networks are also adjusted to use up the on-chip computational resources. Experimental results show that with a given number of MZIs, the mapping efficiency of neural networks on the proposed architecture can be enhanced by 21.87%, 21.20%, 24.69%, and 25.52% for VGG16 and Resnet18 on datasets Cifar10 and Cifar100, respectively. The energy consumption and computation latency can also be reduced by over 67% and 21%, respectively.
Sijie Fei, Amro Eldebiky, Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann
ASP-DAC3
2025 CorrectBench: Automatic Testbench Generation with Functional Self-Correction using LLMs for HDL Design
abstract
Functional simulation is an essential step in digital hardware design. Recently, there has been a growing interest in leveraging Large Language Models (LLMs) for hardware testbench generation tasks. However, the inherent instability associated with LLMs often leads to functional errors in the generated testbenches. Previous methods do not incorporate automatic functional correction mechanisms without human intervention and still suffer from low success rates, especially for sequential tasks. To address this issue, we propose CorrectBench, an automatic testbench generation framework with functional self-validation and self-correction. Utilizing only the RTL specification in natural language, the proposed approach can validate the correctness of the generated testbenches with a success rate of 88.85 %. Furthermore, the proposed LLM-based corrector employs bug information obtained during the self-validation process to perform functional self-correction on the generated testbenches. The comparative analysis demonstrates that our method achieves a pass ratio of 70.13 % across all evaluated tasks, compared with the previous LLM-based testbench generation framework's 52.18% and a direct LLM-based generation method's 33.33%. Specifically in sequential circuits, our work's performance is 62.18 % higher than previous work in sequential tasks and almost 5 times the pass ratio of the direct method. The codes and experimental results are open-sourced at the link: https://github.com/AutoBench/CorrectBench.
Ruidi Qiu, Grace Li Zhang, Rolf Drechsler, Ulf Schlichtmann, Bing Li 0005
DATE2
2025 Large Language Models (LLMs) for Verification, Testing, and Design
Chandan Kumar Jha 0001, Muhammad Hassan 0001, Khushboo Qayyum, Sallar Ahmadi-Pour, Kangwei Xu, Ruidi Qiu, Jason Blocklove, Luca Collini, Andre Nakkab, Ulf Schlichtmann, Grace Li Zhang, Ramesh Karri, Bing Li 0005, Siddharth Garg, Rolf Drechsler
ETS11
2025 HLSTester: Efficient Testing of Behavioral Discrepancies with LLMs for High-Level Synthesis
Kangwei Xu, Bing Li 0005, Grace Li Zhang, Ulf Schlichtmann
ICCAD3
2025 Revolution or Hype? Seeking the Limits of Large Models in Hardware Design
abstract
Recent breakthroughs in Large Language Models (LLMs) and Large Circuit Models (LCMs) have sparked excitement across the electronic design automation (EDA) community, promising a revolution in circuit design and optimization. Yet, this excitement is met with significant skepticism: Are these AI models a genuine revolution in circuit design, or a temporary wave of inflated expectations? This paper serves as a foundational text for the corresponding ICCAD 2025 panel, bringing together perspectives from leading experts in academia and industry. It critically examines the practical capabilities, fundamental limitations, and future prospects of large AI models in hardware design. The paper synthesizes the core arguments surrounding reliability, scalability, and interpretability, framing the debate on whether these models can meaningfully outperform or complement traditional EDA methods. The result is an authoritative overview offering fresh insights into one of today’s most contentious and impactful technology trends.
Qiang Xu 0001, Leon Stok, Rolf Drechsler, Xi Wang 0009, Grace Li Zhang, Igor L. Markov
ICCAD5
2025 Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression
abstract
Large Language Models (LLMs) have achieved remarkable breakthroughs. However, the huge number of parameters in LLMs require significant amount of memory storage in inference, which prevents their practical deployment in many applications. To reduce memory storage of LLMs, singular value decomposition (SVD) provides a promising solution to approximate weight matrices for compressing LLMs. In this paper, we take a step further to explore parameter sharing across different layers with SVD to achieve more effective compression for LLMs. Specifically, weight matrices in different layers are decomposed and represented with a linear combination of a set of shared basis vectors and unique coefficients. The types of weight matrices and the layer selection for basis sharing are examined when compressing LLMs to maintain the performance. Comprehensive experiments demonstrate that Basis-Sharing outperforms state-of-the-art SVD-based compression approaches, especially at large compression ratios.
Jingcun Wang, Yu-Guang Chen, Ing-Chao Lin, Bing Li 0005, Grace Li Zhang
ICLR5
2025 A Scalable 2T-1FeFET-Based Content Addressable Memory Design for Energy Efficient Data Search
abstract
Content addressable memory (CAM) is widely used in advanced machine learning models and data-intensive applications for associative search tasks, thanks to the highly parallel pattern matching capability. Most state-of-the-art CAM designs primarily aim to reduce the CAM cell area by utilizing nonvolatile memories (NVMs). However, there has been limited research on optimizing the design and energy efficiency of NVM-based CAMs for practical deployment in edge devices and AI hardware. This article introduces a general compact and energy efficient CAM design scheme that minimizes design overhead by using only one NVM device per cell. Our proposed CAM design realizes both binary CAM (BCAM) and multibit CAM (MCAM) by leveraging the binary and multilevel storage property of NVM devices without additional cell overheads. Additionally, we propose an adaptive matchline (ML) precharge and discharge scheme to further optimize search energy by significantly reducing the ML voltage swing. Ferroelectric field-effect transistors (FeFETs) serve as representative NVMs in our proposed design, and we present a 2T-1FeFET CAM array incorporating a sense amplifier that implements the proposed ML scheme. Evaluation results show that our proposed 2T-1FeFET BCAM design achieves energy efficiency improvements of$6.64\times $/$4.74\times $/$9.14\times $/$3.02\times $compared to CMOS/ReRAM/STT-MRAM/2FeFET BCAM arrays, while 2T-1FeFET MCAM design achieves$8.25\times $/$5.68\times $/$56.35\times $better-energy efficiency compared to ReRAM/3T-1FeFET/1FeFET-1R MACM arrays. Benchmarking results demonstrate that our BCAM/MCAM approach provides$3.2\times $/$3.7\times $and$2.0\times $/$2.2\times $energy-delay product improvement over the 2T-2R and 2FeFET CAM in accelerating query processing applications.
Jiahao Cai, Hamza Errahmouni Barkam, Mohsen Imani, Kai Ni 0004, Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann, Cheng Zhuo, Xunzhao Yin
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.5
2025 Efficient Model Switching in RRAM-Based DNN Accelerators
abstract
Resistive random access memory (RRAM) has emerged as a promising technology for deep neural network (DNN) accelerators, but programming every weight in a DNN onto RRAM cells for inference can be both time-consuming and energy-intensive, especially when switching between different DNN models. This article introduces a hardware-aware multimodel merging (HA3M) framework designed to minimize the need for reprogramming by maximizing weight reuse, while taking into account the hardware constraints of the accelerator. The framework includes three key approaches: 1) crossbar (XB)-aware model mapping (XAMM); 2) block-based layer matching (BLM); and 3) multimodel retraining (MMR). XAMM reduces the XB usage of the preprogrammed model on RRAM XBs while preserving the model’s structure. BLM reuses preprogrammed weights in a block-based manner, ensuring the inference process remains unchanged. MMR then equalizes the block-based matched weights across multiple models. Experimental results show that the proposed framework significantly reduces programming cycles in multi-DNN switching scenarios while maintaining or even enhancing accuracy, and eliminating the need for reprogramming.
Fang-Yi Gu, Ing-Chao Lin, Bing Li 0005, Ulf Schlichtmann, Grace Li Zhang
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.5
2024 Logic Design of Neural Networks for High-Throughput and Low-Power Applications
abstract
Neural networks (NNs) have been successfully deployed in various fields. In NNs, a large number of multiply-accumulate (MAC) operations need to be performed. Most existing digital hardware platforms rely on parallel MAC units to accelerate these MAC operations. However, under a given area constraint, the number of MAC units in such platforms is limited, so MAC units have to be reused to perform MAC operations in a neural network. Accordingly, the throughput in generating classification results is not high, which prevents the application of traditional hardware platforms in extreme-throughput scenarios. Besides, the power consumption of such platforms is also high, mainly due to data movement. To overcome this challenge, in this paper, we propose to flatten and implement all the operations at neurons, e.g., MAC and ReLU, in a neural network with their corresponding logic circuits. To improve the throughput and reduce the power consumption of such logic designs, the weight values are embedded into the MAC units to simplify the logic, which can reduce the delay of the MAC units and the power consumption incurred by weight movement. The retiming technique is further used to improve the throughput of the logic circuits for neural networks. In addition, we propose a hardware-aware training method to reduce the area of logic designs of neural networks. Experimental results demonstrate that the proposed logic designs can achieve high throughput and low power consumption for several high-throughput applications.
Kangwei Xu, Grace Li Zhang, Ulf Schlichtmann, Bing Li 0005
ASPDAC2
2024 Computational and Storage Efficient Quadratic Neurons for Deep Neural Networks
abstract
Deep neural networks (DNNs) have been widely deployed across diverse domains such as computer vision and natural language processing. However, the impressive accomplishments of DNNs have been realized alongside extensive computational demands, thereby impeding their applicability on resource-constrained devices. To address this challenge, many researchers have been focusing on basic neuron structures, the fundamental building blocks of neural networks, to alleviate the computational and storage cost. In this work, an efficient quadratic neuron architecture distinguished by its enhanced utilization of second-order computational information is introduced. By virtue of their better expressivity, DNNs employing the proposed quadratic neurons can attain similar accuracy with fewer neurons and computational cost. Experimental results have demonstrated that the proposed quadratic neuron structure exhibits superior computational and storage efficiency across various tasks when compared with both linear and non-linear neurons in prior work.
Chuangtao Chen 0001, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Ulf Schlichtmann, Bing Li 0005
DATE2
2024 A FeFET-based Time-Domain Associative Memory for Multi-bit Similarity Computation
abstract
The exponential growth of data across various domains of human society necessitates the rapid and efficient data processing. In many contemporary data-intensive applications, similarity computation (SC) is one of the most fundamental and indispensable operations. In recent years, In-memory computing (IMC) architectures have been designed to accelerate SC by reducing data movement costs, however, they encounter challenges with signal domain conversion, variation sensitivity, and limited precision. This paper proposes a ferroelectric FET (FeFET) based time-domain (TD) associative memory (AM) for energy efficient SC. Such TD design can convert its output (i.e., time interval) to digits with relatively simple sensing circuitry thus saves large amount of area and energy compared with conventional IMC designs that process analog voltage/current signals. The variable-capacitance (VC) delay chain structure in our design supports quantitative SC and enhances robustness against variations. Furthermore, by exploiting multi-domain ferroelctric FET (FeFET), our design is capable of performing SC on vectors with multi-bit element, enabling support for higher-precision algorithms. Simulation results show that the proposed TD-AM achieves 13.8x/1.47x energy saving of our design compared to CMOS/NVM based TD-IMC designs. Additionally, our design exhibits good robustness in monte carlo simulation with variation extracted from experimental measurements. Investigation on precision of hyperdimensional computing (HDC) show that higher element precision reduces the size of HDC model when considering to achieve same accuracy, indicating an improved efficiency. Benchmarkings against GPU demonstrate in general 2/3 orders of magnitude speedup/energy efficiency improvement of our design. Our proposed multi-bit TD-AM promises energy-efficient quantitative SC for diverse intensive data processing application, especially in energy-constrained scenarios.
Qingrong Huang, Hamza Errahmouni Barkam, Jianyi Yang 0003, Thomas Kämpfe, Kai Ni 0004, Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann, Mohsen Imani, Cheng Zhuo, Xunzhao Yin
DATE7
2024 ScanCamouflage: Obfuscating Scan Chains with Camouflaged Sequential and Logic Gates
abstract
Scan chain is a commonly used technique in testing integrated circuits as it provides observability and controllability of the internal states of circuits. However, its presence can make circuits vulnerable to attacks and potentially result in confidential internal data leakage. In this paper, we propose a novel technique for obfuscating scan chains using camouflaged flip-flops, which are designed with the same layout as the original flip-flops but have the actual functionality of a buffer. Furthermore, we employ camouflaged logic gates interconnected in special configurations to increase the difficulty of SAT attack. Experimental results demonstrate that circuits with only a small number of flip-flops can already be protected by the proposed technique while incurring only a minimal area overhead.
Tarik Ibrahimpasic, Grace Li Zhang, Michaela Brunner, Georg Sigl, Bing Li 0005, Ulf Schlichtmann
DATE2
2024 Class-Aware Pruning for Efficient Neural Networks
abstract
Deep neural networks (DNNs) have demonstrated remarkable success in various fields. However, the large number of floating-point operations (FLOPs) in DNNs poses challenges for their deployment in resource-constrained applications, e.g., edge devices. To address the problem, pruning has been introduced to reduce the computational cost in executing DNNs. Previous pruning strategies are based on weight values, gradient values and activation outputs. Different from previous pruning solutions, in this paper, we propose a class-aware pruning technique to compress DNNs, which provides a novel perspective to reduce the computational cost of DNNs. In each iteration, the neural network training is modified to facilitate the class-aware pruning. Afterwards, the importance of filters with respect to the number of classes is evaluated. The filters that are only important for a few number of classes are removed. The neural network is then retrained to compensate for the incurred accuracy loss. The pruning iterations end until no filter can be removed anymore, indicating that the remaining filters are very important for many classes. This pruning technique outperforms previous pruning solutions in terms of accuracy, pruning ratio and the reduction of FLOPs. Experimental results confirm that this class-aware pruning technique can significantly reduce the number of weights and FLOPs, while maintaining a high inference accuracy. Our code is available at https://github.com/HWAI-TUDa/Class-Aware-Pruning
Mengnan Jiang, Jingcun Wang, Amro Eldebiky, Xunzhao Yin, Cheng Zhuo, Ing-Chao Lin, Grace Li Zhang
DATE7
2024 OplixNet: Towards Area-Efficient Optical Split-Complex Networks with Real-to-Complex Data Assignment and Knowledge Distillation
abstract
Having the potential for high speed, high throughput, and low energy cost, optical neural networks (ONN s) have emerged as a promising candidate for accelerating deep learning tasks. In conventional ONNs, light amplitudes are modulated at the input and detected at the output. However, the light phases are still ignored in conventional structures, although they can also carry information for computing. To address this issue, in this paper, we propose a framework called OplixNet to compress the areas of ONNs by modulating input image data into the amplitudes and phase parts of light signals. The input and output parts of the ONN s are redesigned to make full use of both amplitude and phase information. Moreover, mutual learning across different ONN structures is introduced to maintain the accuracy. Experimental results demonstrate that the proposed framework significantly reduces the areas of ONNs with the accuracy within an acceptable range. For instance, 75.03 % area is reduced with a 0.33% accuracy decrease on fully connected neural network (FCNN) and 74.88% area is reduced with a 2.38% accuracy decrease on ResNet-32.
Ruidi Qiu, Amro Eldebiky, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Ulf Schlichtmann, Bing Li 0005
DATE3
2024 BasisN: Reprogramming-Free RRAM-Based In-Memory-Computing by Basis Combination for Deep Neural Networks
abstract
Deep neural networks (DNNs) have made breakthroughs in various fields including image recognition and language processing. DNNs execute hundreds of millions of multiply-and-accumulate (MAC) operations. To efficiently accelerate such computations, analog in-memory-computing platforms have emerged leveraging emerging devices such as resistive RAM (RRAM). However, such accelerators face the hurdle of being required to have sufficient on-chip crossbars to hold all the weights of a DNN. Otherwise, RRAM cells in the crossbars need to be reprogramed to process further layers, which causes huge time/energy overhead due to the extremely slow writing and verification of the RRAM cells. As a result, it is still not possible to deploy such accelerators to process large-scale DNNs in industry. To address this problem, we propose the BasisN framework to accelerate DNNs on any number of available crossbars without reprogramming. BasisN introduces a novel representation of the kernels in DNN layers as combinations of global basis vectors shared between all layers with quantized coefficients. These basis vectors are written to crossbars only once and used for the computations of all layers with marginal hardware modification. BasisN also provides a novel training approach to enhance computation parallelization with the global basis vectors and optimize the coefficients to construct the kernels. Experimental results demonstrate that cycles per inference and energy-delay product were reduced to below 1% compared with applying reprogramming on crossbars in processing large-scale DNNs such as DenseNet and ResNet on ImageNet and CIFAR100 datasets, while the training and hardware costs are negligible.
Amro Eldebiky, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Ing-Chao Lin, Ulf Schlichtmann, Bing Li 0005
ICCAD2
2024 CorrectNet+: Dealing With HW Non-Idealities in In-Memory-Computing Platforms by Error Suppression and Compensation
abstract
The last decade has witnessed the breakthrough of deep neural networks (DNNs) in many fields. With the increasing depth of DNNs, hundreds of millions of multiply-and-accumulate (MAC) operations need to be executed. To accelerate such operations efficiently, analog in-memory computing platforms based on emerging devices, e.g., resistive RAM (RRAM), have been introduced. These acceleration platforms rely on analog properties of the devices and thus suffer from process variations. Consequently, weights in neural networks configured into these platforms can deviate from the nominal trained values, which may lead to feature errors and a significant degradation of the inference accuracy. Besides, additional HW aspects represent key controlling factors for such computing platforms, namely, the limited RRAM cell programmable conductance levels, which limits the number of bits stored in one RRAM cell, the ADC noise converting analog values to digital domain and the ADC power scaling with the number of bits of its output. To address these points, in this article, we propose a framework to enhance the robustness of neural networks under variations. First, an enhanced Lipschitz constant regularization is adopted during neural network training to suppress the amplification of errors propagated through network layers. Additionally, the quantization setting of a NN model is optimized considering robustness against weight variations and total ADC power consumption. Afterward, error compensation is introduced at necessary locations determined by reinforcement learning (RL) to rescue the feature maps with remaining errors. Experimental results demonstrate that inference accuracy of neural networks can be recovered from as low as 1.69% under variations back to more than 95% of their original accuracy at the highest level of variations and reducing total ADC power consumption by 55% while the training and hardware cost are negligible.
Amro Eldebiky, Grace Li Zhang, Georg Böcherer, Bing Li 0005, Ulf Schlichtmann
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.2
2024 A 3D Hybrid Optical-Electrical NoC Using Novel Mapping Strategy Based DCNN Dataflow Acceleration
abstract
A large number of multiply-accumulate operations and memory accesses required in deep convolutional neural networks (DCNN) leads to high latency and energy consumption (EC), that hinder their further applications. Dataflow-based acceleration schemes reduce memory accesses by leveraging reusable data in DCNNs. Row Stationary (RS) dataflow is a more advanced dataflow. In the convolutional layer acceleration of RS dataflow, the flexibility of mapping from logical processing element (LPE) sets to physical PE sets is relatively poor. The utilization of processing elements (PEs) is low. In this paper, a novel mapping strategy based on genetic algorithm (GAMS) with the goal of optimizing EC is proposed. GAMS is designed to address the energy inefficiencies faced when mapping RS dataflow. A 3D hybrid optical-electrical Network-on-Chip (3DHOENoC) is proposed to further improve the communication efficiency, energy efficiency and the processing speed of DCNN. Simulation and evaluation results show that GAMS can achieve better mapping flexibility, higher PEs utilization and 15.9% improvement of execution speed on average. In addition, the execution time (ET) performance of processing the DCNN can be further improved by adopting the 3DHOENoC architecture with better communication parallelism.
Bowen Zhang 0004, Huaxi Gu, Grace Li Zhang, Yintang Yang, Ziteng Ma, Ulf Schlichtmann
IEEE Trans. Parallel Distributed Syst.3
2023 PowerPruning: Selecting Weights and Activations for Power-Efficient Neural Network Acceleration
abstract
Deep neural networks (DNNs) have been successfully applied in various fields. A major challenge of deploying DNNs, especially on edge devices, is power consumption, due to the large number of multiply-and-accumulate (MAC) operations. To address this challenge, we propose PowerPruning, a novel method to reduce power consumption in digital neural network accelerators by selecting weights that lead to less power consumption in MAC operations. In addition, the timing characteristics of the selected weights together with all activation transitions are evaluated. The weights and activations that lead to small delays are further selected. Consequently, the maximum delay of the sensitized circuit paths in the MAC units is reduced even without modifying MAC units, which thus allows a flexible scaling of supply voltage to reduce power consumption further. Together with retraining, the proposed method can reduce power consumption of DNNs on hardware by up to 73.9% with only a slight accuracy loss.
Richard Petri 0002, Grace Li Zhang, Yiran Chen 0001, Ulf Schlichtmann, Bing Li 0005
DAC2
2023 CorrectNet: Robustness Enhancement of Analog In-Memory Computing for Neural Networks by Error Suppression and Compensation
Amro Eldebiky, Grace Li Zhang, Georg Böcherer, Bing Li 0005, Ulf Schlichtmann
DATE2
2023 Countering Uncertainties in In-Memory-Computing Platforms with Statistical Training, Accuracy Compensation and Recursive Test
abstract
In-memory-computing (IMC) has become an efficient solution for implementing neural networks on hardware. However, IMC platforms request weights in neural networks to be programmed to exact values. This is a very demanding task due to programming complexity, process variations, noise, as well as thermal effects. Accordingly, new methods should be introduced to counter such uncertainties. In this paper, we first discuss a method to train neural networks statistically with process variations modeled as correlated random variables. The statistical effect is incorporated in the cost function during training. Consequently, a neural network after statistical training becomes robust to uncertainties. To deal with variations and noise further, we also introduce a compensation method with extra layers for neural networks. These extra layers are trained offline again after the weights in the original neural network are determined to enhance the inference accuracy. Finally, we will discuss a method for testing the effect of process variations in an optical acceleration platform for neural networks. This optical platform uses Mach-Zehnder Interferometers (MZIs) to implement the multiply-accumulate operations. However, trigonometric functions in the transformation matrix of an MZI make it very sensitive to process variations. To address this problem, we apply a recursive test procedure to determine the properties of MZIs inside an optical acceleration module, so that process variations can be compensated accordingly to maintain the inference accuracy of neural networks.
Amro Eldebiky, Grace Li Zhang, Bing Li 0005
DATE2
2023 Class-based Quantization for Neural Networks
abstract
In deep neural networks (DNNs), there are a huge number of weights and multiply-and-accumulate (MAC) operations. Accordingly, it is challenging to apply DNNs on resource- constrained platforms, e.g., mobile phones. Quantization is a method to reduce the size and the computational complexity of DNNs. Existing quantization methods either require hardware overhead to achieve a non-uniform quantization or focus on model-wise and layer-wise uniform quantization, which are not as fine-grained as filter-wise quantization. In this paper, we propose a class-based quantization method to determine the minimum number of quantization bits for each filter or neuron in DNNs individually. In the proposed method, the importance score of each filter or neuron with respect to the number of classes in the dataset is first evaluated. The larger the score is, the more important the filter or neuron is and thus the larger the number of quantization bits should be. Afterwards, a search algorithm is adopted to exploit the different importance of filters and neurons to determine the number of quantization bits of each filter or neuron. Experimental results demonstrate that the proposed method can maintain the inference accuracy with low bit-width quantization. Given the same number of quantization bits, the proposed method can also achieve a better inference accuracy than the existing methods.
Grace Li Zhang, Huaxi Gu, Bing Li 0005, Ulf Schlichtmann
DATE2
2023 SteppingNet: A Stepping Neural Network with Incremental Accuracy Enhancement
abstract
Deep neural networks (DNNs) have successfully been applied in many fields in the past decades. However, the in-creasing number of multiply-and-accumulate (MAC) operations in DNNs prevents their application in resource-constrained and resource-varying platforms, e.g., mobile phones and autonomous vehicles. In such platforms, neural networks need to provide ac-ceptable results quickly and the accuracy of the results should be able to be enhanced dynamically according to the computational resources available in the computing system. To address these challenges, we propose a design framework called SteppingNet. SteppingNet constructs a series of sub nets whose accuracy is incrementally enhanced as more MAC operations become avail-able. Therefore, this design allows a trade-off between accuracy and latency. In addition, the larger sub nets in SteppingNet are built upon smaller subnets, so that the results of the latter can directly be reused in the former without recomputation. This property allows SteppingNet to decide on-the-fly whether to enhance the inference accuracy by executing further MAC operations. Experimental results demonstrate that SteppingNet provides an effective incremental accuracy improvement and its inference accuracy consistently outperforms the state-of-the-art work under the same limit of computational resources.
Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Huaxi Gu, Bing Li 0005, Ulf Schlichtmann
DATE2
2023 A Novel and Efficient Block-Based Programming for ReRAM-Based Neuromorphic Computing
abstract
ReRAM-based accelerators have emerged as promising accelerators for deep neural networks (DNNs). How-ever, programming every ReRAM cell to its corresponding conductance before inference can be time-consuming and energy-intensive using existing one-by-one/row-by-row programming mechanisms. Although a two-phase multi-row programming scheme has been proposed to enhance programming efficiency, there are situations where multiple rows cannot be programmed together and only row-by-row programming can be employed. Therefore, this paper proposes a new block-based programming architecture for ReRAM crossbars that enables precise control of wordline and bitline transistors. In addition, a block-based programming framework, including the approximation phase and the fine-tuning phase, along with a multi-line programming algorithm and a programming-aware model retraining are proposed to reduce programming cycles and energy consumption. Experimental results demonstrate that our proposed method can reduce programming cycles and energy consumption by 46%-49 % and 63 % -64 %, respectively, compared to the state of the art. Additionally, the area and power overhead are negligible.
Wei-Lun Chen, Fang-Yi Gu, Ing-Chao Lin, Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann
ICCAD4
2023 NearUni: Near-Unitary Training for Efficient Optical Neural Networks
abstract
Optical neural networks with Mach-Zender interferometers (MZIs) have demonstrated advantages over their electronic counterparts in computing efficiency and power consumption. However, implementing the computation with a weight matrix in DNNs using this technique requires the decomposition of the weight matrix into two unitary matrices, because an optical network can only realize a single unitary matrix due to its structural property. Accordingly, a direct implementation of DNNs onto optical networks suffer from a low area efficiency. To address this challenge, in this paper, a near-unitary training framework is proposed. In this framework, a weight matrix in DNNs is first partitioned into square submatrices to reduce the number of MZIs in the optical networks. Afterwards, training is adjusted to make the partitioned submatrices as close to unitary as possible. Such a matrix is then represented further by the sum of a unitary matrix and a sparse matrix. The latter implements the difference between the unitary matrix and the near-unitary matrix after training. In this way, only one optical network is needed to implement this unitary matrix and the low computation load in the sparse matrix can be implemented with area-efficient microring resonators (MRRs). Experimental results show that the area footprint can be reduced by 81.81%, 85.51%, 48.6% for ResNet34, VGG16, and fully connected neural networks, respectively, while the inference accuracy is still maintained on CIFAR100 and MNIST datasets.
Amro Eldebiky, Bing Li 0005, Grace Li Zhang
ICCAD3
2023 BRoCoM: A Bayesian Framework for Robust Computing on Memristor Crossbar
abstract
Memristor crossbar arrays are considered to be a promising platform for neuromorphic computing. To deploy a trained neural network (NN) model on memristor crossbars, memristors need to be programmed to the corresponding weight values. In fact, due to device-based process variation and noise, deviations of the stored weights from the trained weights are inevitable, thereby causing the degradation of the actual inference performance. This article proposes a unified Bayesian inference-based framework, BRoCoM, which connects device nonidealities and algorithmic training together for robust computing on memristor crossbars. BRoCoM is able to incorporate different levels of nonidealities into prior weight distribution, and transform robustness optimization to Bayesian NN (BNN) training, the weights of NNs are optimized to accommodate uncertainties and minimize inference degradation. Experimental results confirm the capability of the proposed BRoCoM to achieve stable inference performance while tolerating the nonideal effects of process variation and noise.
Qingrong Huang, Grace Li Zhang, Xunzhao Yin, Bing Li 0005, Ulf Schlichtmann, Cheng Zhuo
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.4
2023 Ferroelectric Ternary Content Addressable Memories for Energy-Efficient Associative Search
abstract
A fast and efficient search function across the database has been a core component for a number of data-intensive tasks in machine learning, IoT applications, and inference. However, the conventional digital machines implementing the search functionality with repetitive arithmetic operations suffer from the energy efficiency and performance degradation due to the significant data transfer between the storage and processing units in the Von Neumann architecture. Ternary content addressable memories (TCAMs) are an essential hardware form of computing-in-memory (CiM) designs that aim to overcome the data transfer bottlenecks by implementing the parallel associative search function within the memory blocks. While most state-of-the-art TCAM designs focus on improving the information density by harnessing compact nonvolatile memories (NVMs), little efforts have been spent on optimizing the energy efficiency of the NVM-based TCAM. In this article, by exploiting the ferroelectric FET (FeFET) as a representative NVM, we propose an NOR-type 2FeFET-1T and an NAND-type 2FeFET-2T TCAM designs that enable highly energy-efficient associative search by reducing the associated precharge overheads. We then propose a hybrid ferroelectric NAND-NOR (HFNN) TCAM design to further improve the energy efficiency. An HFNN-based segmented architecture is proposed to reduce the search delay and energy by search operation pipeline. Evaluation results suggest that the proposed 2FeFET-1T, 2FeFET-2T and HFNN TCAM design consume$3.03\times $,$8.08\times $, and$226.92\times $less search energy than the conventional 16T complementary metal oxide semiconductor (CMOS) TCAM, respectively. Application benchmarking shows that our proposed 2FeFET-1T/2FeFET-2T/HFNN TCAM can save, on average, 45.2%/50.6%/57.5% the GPU energy consumption as compared to the conventional GPU.
Xunzhao Yin, Yu Qian 0002, Mohsen Imani, Kai Ni 0004, Chao Li 0065, Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann, Cheng Zhuo
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.6
2022 Energy efficient data search design and optimization based on a compact ferroelectric FET content addressable memory
abstract
Content Addressable Memory (CAM) is widely used for associative search tasks in advanced machine learning models and data-intensive applications due to the highly parallel pattern matching capability. Most state-of-the-art CAM designs focus on reducing the CAM cell area by exploiting the nonvolatile memories (NVMs). There exists only little research on optimizing the design and energy efficiency of NVM based CAMs for practical deployment in edge devices and AI hardware. In this paper, we propose a general compact and energy efficient CAM design scheme that alleviates the design overhead by employing just one NVM device in the cell. We also propose an adaptive matchline (ML) precharge and discharge scheme that further optimizes the search energy by fully reducing the ML voltage swing. We consider Ferroelectric field effect transistors (FeFETs) as the representative NVM, and present a 2T-1FeFET CAM array including a sense amplifier implementing the proposed ML scheme. Evaluation results suggest that our proposed 2T-1FeFET CAM design achieves 6.64×/4.74×/9.14×/3.02× better energy efficiency compared with CMOS/ReRAM/STT-MRAM/2FeFET CAM arrays. Benchmarking results show that our approach provides 3.3×/2.1× energy-delay product improvement over the 2T-2R/2FeFET CAM in accelerating query processing applications.
Jiahao Cai, Mohsen Imani, Kai Ni 0004, Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann, Cheng Zhuo, Xunzhao Yin
DAC4
2022 RRAM-based Neuromorphic Computing: Data Representation, Architecture, Logic, and Programming
abstract
RRAM crossbars provide a promising hardware plat-form to accelerate matrix-vector multiplication in deep neural networks (DNNs). To exploit the efficiency of RRAM crossbars, extensive research ex-amining architecture, data representation, logic de-sign as well as device programming should be conducted. This extensive scope of research aspects is enabled and required by the versatility of RRAM cells and their organization in a computing system. These research aspects affect or benefit each other. Therefore, they should be considered systematically to achieve an efficient design in terms of design complexity and computational performance in accelerating DNNs. In this paper, we illustrate study exam-ples on these perspectives on RRAM crossbars, in-cluding data representation with pulse widths, archi-tecture improvement, implementation of logic functions using RRAM cells, and efficient programming of RRAM devices for accelerating DNNs.
Grace Li Zhang, Shuhang Zhang, Hai Li 0001, Ulf Schlichtmann
DSD1
2022 Aging Aware Retraining for Memristor-based Neuromorphic Computing
abstract
Memristor-based crossbars, which can achieve 1-2 orders of magnitude energy efficiency improvement over digital machines, have been introduced to accelerate the neural networks of machine learning tasks. Due to the high voltage pulses repeatedly applied onto memristors during programming and online tuning, the effective resistance ranges of the memristors actually decrease as a result of aging, which eventually impair the inference accuracy of the neural network running on the memristor-based crossbar. In this paper, we propose an algorithm-hardware co-design framework combining aging aware retraining and gradient sparsification to mitigate the impact of aging and extend the lifetime of the crossbar. Experimental results show that the proposed method can effectively increase the inference accuracy by up to 16% even with severe aging, while the crossbar lifetime can be extended by up to $2.7\times$.
Wenwen Ye, Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann, Cheng Zhuo, Xunzhao Yin
ISCAS2
2022 PathDriver+: Enhanced Path-Driven Architecture Design for Flow-Based Microfluidic Biochips
abstract
Continuous-flow microfluidic biochips have attracted high research interest over the past years. Inside such a chip, fluid samples of milliliter volumes are efficiently transported between devices (e.g., mixers, heaters, etc.) to automatically perform various laboratory procedures in biology and biochemistry. Each transportation task, however, requires an exclusive flow path composed of multiple contiguous microchannels during its execution period. Excess/waste fluids, in the meantime, should be discarded by independent flow paths connected to waste ports. All these paths are etched in a very tiny chip area using multilayer soft lithography and driven by flow ports connecting with external pressure sources, forming a highly integrated chip architecture that determines the final performance of biochips. In this article, we propose a new and practical design flow called PathDriver+ (PD+) for the architecture design of microfluidic biochips, integrating the actual fluid manipulations into both high-level synthesis and physical design, which has never been considered in prior work. With this design flow, highly efficient chip architectures with a flow-path network that enables the actual fluid transportation and removal can be constructed automatically. Meanwhile, fluid volume management between devices and flow-path minimization are realized for the first time, thus, ensuring the correctness of assay outcomes while reducing the complexity of chip architectures. Additionally, diagonal channel routing is implemented to fundamentally improve the chip performance. The tradeoff between the numbers of channel intersections and fluidic ports is evaluated to further reduce the fabrication cost of biochips. The experimental results on multiple benchmarks confirm that the proposed design flow leads to high assay execution efficiency and low overall chip cost.
Xing Huang 0001, Youlin Pan, Grace Li Zhang, Bing Li 0005, Wenzhong Guo, Tsung-Yi Ho, Ulf Schlichtmann
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.3
2022 VirtualSync+: Timing Optimization With Virtual Synchronization
abstract
In digital circuit designs, sequential components such as flip-flops are used to synchronize signal propagations. Logic computations are aligned at and thus isolated by flip-flop stages. Although this fully synchronous style can reduce design efforts significantly, it may affect circuit performance negatively, because sequential components can only introduce delays into signal propagations but never accelerate them. In this article, we propose a new timing model, VirtualSync+, in which signals, specially those along critical paths, are allowed to propagate through several sequential stages without flip-flops. Timing constraints are still satisfied at the boundary of the optimized circuit to maintain a consistent interface with existing designs. By removing clock-to-q delays and setup time requirements of flip-flops on critical paths, the performance of a circuit can be pushed even beyond the limit of traditional sequential designs. In addition, we further enhance the optimization with VirtualSync+ by fine-tuning with commercial design tools, e.g., design compiler from Synopsys, to achieve more accurate result. To achieve this fine-tuning, we first optimize the circuits by reallocating sequential components with sequential and combinational components as delay units. Afterward, the removal locations of flip-flops with respect to the circuits under optimization are extracted and the corresponding wave-pipelining timing constraints compatible with commercial design tools are established. These timing constraints are then incorporated into the optimization flow of commercial tools to generate the optimized circuits. The experimental results demonstrate that circuit performance can be improved by up to 4% (average 1.5%) compared with that after extreme retiming and sizing, while the increase of area is still negligible. This timing performance is enhanced beyond the limit of traditional sequential designs. It also demonstrates that compared with those after retiming and sizing, the circuits with VirtualSync+ can achieve better timing performance under the same area cost or smaller area cost under the same clock period, respectively.
Grace Li Zhang, Bing Li 0005, Xing Huang 0001, Xunzhao Yin, Cheng Zhuo, Masanori Hashimoto, Ulf Schlichtmann
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.1
2021 Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks
abstract
RRAM-based crossbars and optical neural networks are attractive platforms to accelerate neuromorphic computing. However, both accelerators suffer from hardware uncertainties such as process variations. These uncertainty issues left unaddressed, the inference accuracy of these computing platforms can degrade significantly. In this paper, a statistical training method where weights under process variations and noise are modeled as statistical random variables is presented. To incorporate these statistical weights into training, the computations in neural networks are modified accordingly. For optical neural networks, we modify the cost function during software training to reduce the effects of process variations and thermal imbalance. In addition, the residual effects of process variations are extracted and calibrated in hardware test, and thermal variations on devices are also compensated in advance. Simulation results demonstrate that the inference accuracy can be improved significantly under hardware uncertainties for both platforms.
Grace Li Zhang, Bing Li 0005, Ying Zhu 0008, Yiyu Shi 0001, Xunzhao Yin, Cheng Zhuo, Huaxi Gu, Tsung-Yi Ho, Ulf Schlichtmann
ASP-DAC1
2021 Bayesian Inference Based Robust Computing on Memristor Crossbar
abstract
Memristor based crossbars are a promising platform for neural network acceleration. To deploy a trained network model on a memristor crossbar, memristors need to be programmed to realize the trained weights of the network. However, due to process and dynamic variations, deviation of weights from the trained value is inevitable and inference accuracy thus degrades. In this paper, we propose a unified Bayesian inference based framework which connects hardware variations and algorithmic training together for robust computing on memristor crossbars. The framework incorporates different levels of variations into priori weight distribution, and transforms robustness optimization to Bayesian neural network training, where weights of neural networks are optimized to accommodate variations and minimize inference degradation. Simulation results with the proposed framework confirm stable inference accuracy under process and dynamic variations.
Qingrong Huang, Grace Li Zhang, Xunzhao Yin, Bing Li 0005, Ulf Schlichtmann, Cheng Zhuo
DAC3
2021 Energy-Aware Designs of Ferroelectric Ternary Content Addressable Memory
abstract
Ternary content addressable memories (TCAMs) are a special form of computing-in-memory (CiM) circuits that aim to address the so-called memory wall issues by merging the parallel search function with memory blocks. Due to the content addressing nature, TCAMs have been widely utilized for search intensive tasks in low-power, data analytic applications, such as IP routers, associative memories, and learning models. While most state-of-the-art TCAM designs focus on improving the TCAM density by harnessing compact nonvolatile memories (NVMs), little efforts have been spent on reducing and optimizing the energy consumption of the NVM based TCAM. In this paper, by exploiting the Ferroelectric FET (FeFET) as a representative NVM, we propose two compact and energy-aware designs of ferroelectric TCAMs for low power applications. We first introduce a novel 2FeFET based XOR-like gate structure that can also be adopted to other NVMs, and then leverage the structure to propose two TCAM designs that achieve high energy efficiency by either reducing the associated precharge overhead (2FeFET-1T cell), or eliminating the precharge phase typically required by TCAMs (2FeFET-2T cell). We evaluate and compare the designs w.r.t area, search energy and delay at array level with other existing designs, and benchmark the proposed TCAM designs in an associative memory based GPU architecture. The results suggest that the proposed 2FeFET-1T/2FeFET-2T TCAM design consumes 3.03X/8.08X less search energy than the conventional 16T CMOS TCAM, while the proposed design cell area is only 32.1%/39.3% of the latter. Compared with the state-of-the-art 2FeFET only TCAM array, our proposed designs still achieve 1.79X and 4.79X search energy reduction, respectively. Moreover, our proposed designs can achieve, on average, 45.2%/51.5% energy saving compared with the conventional GPU based architecture at the application level.
Yu Qian 0002, Zhenhao Fan, Chao Li 0065, Mohsen Imani, Kai Ni 0004, Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann, Cheng Zhuo, Xunzhao Yin
DATE7
2021 Hardware-Software Codesign of Weight Reshaping and Systolic Array Multiplexing for Efficient CNNs
abstract
The last decade has witnessed the breakthrough of deep neural networks (DNNs) in various fields, e.g., image/speech recognition. With the increasing depth of DNNs, the number of multiply-accumulate operations (MAC) with weights explodes significantly, preventing their applications in resource-constrained platforms. The existing weight pruning method is considered to be an effective method to compress neural networks for acceleration. However, weights after pruning usually exhibit irregular patterns. Implementing MAC operations with such irregular weight patterns on hardware platforms with regular designs, e.g., GPUs and systolic arrays, might result in an underutilization of hardware resources. To utilize the hardware resource efficiently, in this paper, we propose a hardware-software codesign framework for acceleration on systolic arrays. First, weights after unstructured pruning are reorganized into a dense cluster. Second, various blocks are selected to cover the cluster seamlessly. To support the concurrent computations of such blocks on systolic arrays, a multiplexing technique and the corresponding systolic architecture is developed for various CNNs. The experimental results demonstrate that the performance of CNN inferences can be improved significantly without accuracy loss.
Jingyao Zhang 0002, Huaxi Gu, Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann
DATE3
2021 An Efficient Programming Framework for Memristor-based Neuromorphic Computing
abstract
Memristor-based crossbars are considered to be promising candidates to accelerate vector-matrix computation in deep neural networks. Before being applied for inference, mem-ristors in the crossbars should be programmed to conductances corresponding to the network weights after software training. Existing programming methods, however, adjust conductances of memristors individually with many programming-reading cycles. In this paper, we propose an efficient programming framework for memristor crossbars, where the programming process is partitioned into the predictive phase and the fine-tuning phase. In the predictive phase, multiple memristors are programmed simultaneously with a memristor programming model and IR-drop estimation. To deal with the programming inaccuracy resulting from process variations, noise and IR-drop and move conductances to target values, memristors are fine-tuned afterwards to reach a specified programming accuracy. Simulation results demonstrate that the proposed method can reduce the number of programming-reading cycles by up to 94.77% and 90.61% compared to existing one-by-one and row-by-row programming methods, respectively.
Grace Li Zhang, Bing Li 0005, Xing Huang 0001, Shuhang Zhang, Florin Burcea, Helmut E. Graeb, Tsung-Yi Ho, Hai Li 0001, Ulf Schlichtmann
DATE1
2021 Reliable Memristor-based Neuromorphic Design Using Variation- and Defect-Aware Training
abstract
The memristor crossbar provides a unique opportunity to develop a neuromorphic computing system (NCS) with high scalability and energy efficiency. However, the reliability issues that arise from the immature fabrication process and physical device limitations, i.e., variations and stuck-at-faults (SAF), dramatically prevent its wide application in practice. Specifically, variations make the programmed weights deviate from their expected values. On the other hand, defective mem-ristors cannot even represent the weights effectively. In this work, we propose a variation- and defect-aware framework to improve the reliability of memristor-based NCS while minimizing the inference performance loss. We propose to develop analytical weight models to characterize the non-ideal effects of variations and SAFs, which can then be incorporated into a Bayesian neural network as priori and constraint. We then convert the reliability improvement to the neural network training for optimal weights that can accommodate variations and defects across the chips, which does not require computation-intensive retraining or cost-expensive testing. Extensive experimental results with the proposed framework confirm its effective capability of improving the reliability of NCS, while significantly mitigating the inference accuracy degradation under even severe variations and SAFs.
Grace Li Zhang, Xunzhao Yin, Bing Li 0005, Ulf Schlichtmann, Cheng Zhuo
ICCAD2
2021 A Reconfigurable Multiplier for Signed Multiplications with Asymmetric Bit-Widths
abstract
Multiplications have been commonly conducted in quantized CNNs, filters, and reconfigurable cores, and so on, which are widely deployed in mobile and embedded applications. Most multipliers are designed to perform multiplications with symmetric bit-widths, i.e., n - by n -bit multiplication. Such features would cause extra area overhead and performance loss when m - by n -bit multiplications ( m > n ) are deployed in the same hardware design, resulting in inefficient multiplication operations. It is highly desired and challenging to propose a reconfigurable multiplier design to accommodate operands with both symmetric and asymmetric bit-widths. In this work, we propose a reconfigurable approximate multiplier to support multiplications at various precisions, i.e., bit-widths. Unlike prior works of approximate adders assuming a uniform weight distribution with bit-wise independence, scenarios like a quantized CNN may have a centralized weight distribution and hence follow a Gaussian-like distribution with correlated adjacent bits. Thus, a new block-based approximate adder is also proposed as part of the multiplier to ensure energy-efficient operation with an awareness of the bit-wise correlation. Our experimental results show that the proposed approximate adder significantly reduces the error rate by 76% to 98% over a state-of-the-art approximate adder for Gaussian-like distribution scenarios. Evaluation results show that the proposed multiplier is 19% faster and 22% more power saving than a Xilinx multiplier IP at the same bit precision and achieves a 23.94-dB peak signal-to-noise ratio, which is comparable to the accurate one of 24.10 dB when deployed in a Gaussian filter for image processing tasks.
Chuliang Guo, Li Zhang 0021, Grace Li Zhang, Bing Li 0005, Weikang Qian, Xunzhao Yin, Cheng Zhuo
ACM J. Emerg. Technol. Comput. Syst.4
2020 Timing Resilience for Efficient and Secure Circuits
abstract
In this paper, we will cover several techniques that can enhance the resilience of timing of digital circuits. Using post-silicon tuning components, the clock arrival times at flip-flops can be modified after manufacturing to balance delays between flip-flops. The actual delay properties of flip-flops will be examined to exploit the natural flexibility of such components. Wave-pipelining paths spanning several flip-flop stages can be integrated into a synchronous design to improve the circuit performance and to reduce area. In addition, with this technique, it cannot be taken for granted anymore that all the combinational paths in a circuit work with respect to one clock period. Therefore, a netlist alone does not represent all the design information. This feature enables the potential to embed wave-pipelining paths into a circuit to increase the complexity of reverse engineering. In order to replicate a design, attackers therefore have to identify the locations of the wave-pipelining paths, in addition to the netlist extracted from reverse engineering. Therefore, the security of the circuit against counterfeiting can be improved.
Grace Li Zhang, Michaela Brunner, Bing Li 0005, Georg Sigl, Ulf Schlichtmann
ASP-DAC1
2020 Statistical Training for Neuromorphic Computing using Memristor-based Crossbars Considering Process Variations and Noise
abstract
Memristor-based crossbars are an attractive platform to accelerate neuromorphic computing. However, process variations during manufacturing and noise in memristors cause significant accuracy loss if not addressed. In this paper, we propose to model process variations and noise as correlated random variables and incorporate them into the cost function during training. Consequently, the weights after this statistical training become more robust and together with global variation compensation provide a stable inference accuracy. Simulation results demonstrate that the mean value and the standard deviation of the inference accuracy can be improved significantly, by even up to 54% and 31%, respectively, in a two-layer fully connected neural network.
Ying Zhu 0008, Grace Li Zhang, Bing Li 0005, Yiyu Shi 0001, Tsung-Yi Ho, Ulf Schlichtmann
DATE2
2020 Reliable and Robust RRAM-based Neuromorphic Computing
abstract
RRAM-based crossbars are a promising hardware platform to accelerate computations in neural networks. Before such a crossbar can be used as an accelerator for neural networks, RRAM cells should be programmed to target resistances to represent weights in neural networks. However, this process degrades the valid range of the resistances of RRAM cells from the fresh state, called aging effect. Therefore, after a certain number of programming iterations, these RRAM cells cannot be programmed reliably anymore, affecting the classification accuracy of neural networks negatively. In addition, process variations during manufacturing and noise during programming of RRAM cells also lead to significant accuracy degradation. To solve the problems described above, in this paper, we introduce a software/hardware codesign framework to reduce the aging effect in RRAM crossbars. To counter process variations and noise, we first model them as random variables and then modify the computations in software training considering these variables. Simulation results show that the lifetime of RRAM crossbars can be extended by up to 11 times with the codesign framework and the mean value and the standard deviation of the inference accuracy under process variations and noise can be improved significantly.
Grace Li Zhang, Bing Li 0005, Ying Zhu 0008, Shuhang Zhang, Yiyu Shi 0001, Tsung-Yi Ho, Hai Li 0001, Ulf Schlichtmann
ACM Great Lakes Symposium on VLSI1
2020 PathDriver: A Path-Driven Architectural Synthesis Flow for Continuous-Flow Microfluidic Biochips
abstract
Continuous-flow microfluidic biochips have attracted high research interest over the past years. Inside such a chip, fluid samples of milliliter volumes are efficiently transported between devices (e.g., mixers, etc.) to automatically perform various laboratory procedures in biology and biochemistry. Each transportation task, however, requires an exclusive flow path composed of multiple contiguous microchannels during its execution period. Excess/waste fluids, in the meantime, should be discarded by independent flow paths connected to waste ports. All these paths are etched in a very tiny chip area using multilayer soft lithography and driven by flow ports connecting with external pressure sources, forming a highly integrated chip architecture that dominates the performance of biochips. In this paper, we propose a practical synthesis flow called PathDriver for the design automation of microfluidic biochips, integrating the actual fluid manipulations into both high-level synthesis and physical design, which has never been considered in prior work. Given the protocols of biochemical applications, PathDriver aims to generate highly efficient chip architectures with a flow-path network that enables the manipulation of actual fluid transportation and removal. Additionally, fluid volume management between devices and flow-path minimization are realized for the first time, thus ensuring the correctness of assay outcomes while reducing the complexity of chip architectures. Experimental results on multiple benchmarks demonstrate the effectiveness of the proposed synthesis flow.
Xing Huang 0001, Youlin Pan, Grace Li Zhang, Bing Li 0005, Wenzhong Guo, Tsung-Yi Ho, Ulf Schlichtmann
ICCAD3
2020 Countering Variations and Thermal Effects for Accurate Optical Neural Networks
abstract
Optical neural networks (ONNs) have emerged as a promising high-performance computing platform to accelerate deep neural networks. In ONNs, phases of light are modulated through Mach-Zehnder Interferometers (MZIs), and MZIs are connected in a gridlike layout to implement multiply-accumulate operations. However, ONNs are very sensitive to process variations and thermal effects. This sensitivity leads to a significant degradation of inference accuracy of ONNs and thus renders them unusable in practice. In this paper, we propose a framework to calibrate process variations and counter thermal effects by power compensation. Experimental results demonstrate that the proposed framework can recover the inference accuracy under variations and thermal effects, e.g., from as low as 11.05% back to 74.11% for LeNet-5 on Cifar10, so that ONNs can achieve an inference accuracy similar to the accuracy after software training while providing their high bandwidth in neuromorphic computing.
Ying Zhu 0008, Grace Li Zhang, Bing Li 0005, Xunzhao Yin, Cheng Zhuo, Huaxi Gu, Tsung-Yi Ho, Ulf Schlichtmann
ICCAD2
2020 TimingCamouflage+: Netlist Security Enhancement With Unconventional Timing
abstract
With recent advances in reverse engineering, attackers can reconstruct a netlist to counterfeit chips by opening the die and scanning all layers of authentic chips. This relatively easy counterfeiting is made possible by the use of the standard simple clocking scheme, where all combinational blocks function within one clock period, so that a netlist of combinational logic gates and flip-flops is sufficient to duplicate a design. In this article, we propose to invalidate the assumption that a netlist completely represents the function of a circuit with unconventional timing. With the introduced wave-pipelining (WP) paths, attackers have to capture gate and interconnect delays during reverse engineering, or to test a huge number of combinational paths to identify the WP paths. To hinder the test-based attack, we construct false paths with WP to increase the counterfeiting challenge. The experimental results confirm that WP true paths and false paths can be constructed in benchmark circuits successfully with only a negligible cost, thus thwarting the potential attack techniques.
Grace Li Zhang, Bing Li 0005, Meng Li 0004, Bei Yu 0001, David Z. Pan, Michaela Brunner, Georg Sigl, Ulf Schlichtmann
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.1
2019 Aging-aware Lifetime Enhancement for Memristor-based Neuromorphic Computing
abstract
Memristor-based crossbars have been applied successfully to accelerate vector-matrix computations in deep neural networks. During the training process of neural networks, the conductances of the memristors in the crossbars must be updated repetitively. However, memristors can only be programmed reliably for a given number of times. Afterwards, the working ranges of the memristors deviate from the fresh state. As a result, the weights of the corresponding neural networks cannot be implemented correctly and the classification accuracy drops significantly. This phenomenon is called aging, and it limits the lifetime of memristor-based crossbars. In this paper, we propose a co-optimization framework combining software training and hardware mapping to reduce the aging effect. Experimental results demonstrate that the proposed framework can extend the lifetime of such crossbars up to 11 times, while the expected accuracy of classification is maintained.
Shuhang Zhang, Grace Li Zhang, Bing Li 0005, Hai Li 0001, Ulf Schlichtmann
DATE2
2019 EffiTest2: Efficient Delay Test and Prediction for Post-Silicon Clock Skew Configuration Under Process Variations
abstract
At nanometer manufacturing technology nodes, process variations affect circuit performance significantly. This trend leads to a large timing margin and thus overdesign in the traditional worst-case circuit design flow. To combat this pessimism, post-silicon clock tuning buffers can be deployed to balance timing slacks of consecutive combinational paths in individual chips by tuning clock skews after manufacturing. A challenge of this method is that path delays of each chip with timing failures should be measured to gather the information for clock skew configuration. However, current methods for delay measurement rely on path-wise frequency stepping, which requires much time from expensive testers. In this paper, we propose an efficient delay test framework (EffiTest2) to solve the post-silicon testing problem by testing only representative paths with delay alignment using the already-existing tunable buffers in the circuit. Experimental results demonstrate that EffiTest2 can reduce the number of frequency stepping iterations by more than 94% with only a slight yield loss.
Grace Li Zhang, Bing Li 0005, Yiyu Shi 0001, Jiang Hu 0001, Ulf Schlichtmann
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.1
2018 Virtualsync: timing optimization by synchronizing logic waves with sequential and combinational components as delay units
abstract
In digital circuit designs, sequential components such as flip-flops are used to synchronize signal propagations. Logic computations are aligned at and thus isolated by flip-flop stages. Although this fully synchronous style can reduce design efforts significantly, it may affect circuit performance negatively, because sequential components can only introduce delays into signal propagations instead of accelerating them. In this paper, we propose a new timing model, VirtualSync, in which signals, specially those along critical paths, are allowed to propagate through several sequential stages without flip-flops. Timing constraints are still satisfied at the boundary of the optimized circuit to maintain a consistent interface with existing designs. By removing clock-to-q delays and setup time requirements of lip-lops on critical paths, the performance of a circuit can be pushed even beyond the limit of traditional sequential designs. Experimental results demonstrate that circuit performance can be improved by up to 11.5% (average 3.1%) compared with that after thorough sizing and retiming, while the increase of area is still negligible.
Grace Li Zhang, Bing Li 0005, Masanori Hashimoto, Ulf Schlichtmann
DAC1
2018 TimingCamouflage: Improving circuit security against counterfeiting by unconventional timing
abstract
With recent advances in reverse engineering, attackers can reconstruct a netlist to counterfeit chips by opening the die and scanning all layers of original chips. This relatively easy counterfeiting is made possible by the use of the standard simple clocking scheme where all combinational blocks function within one clock period. In this paper, we propose a method to invalidate the assumption that a netlist completely represents the function of a circuit. With the help of wave-pipelining paths, this method forces attackers to capture delay information from manufactured chips, which is a very challenging task because we also introduce false paths. Experimental results confirm that wave-pipelining paths and false paths can be constructed in benchmark circuits successfully with only a negligible cost, while the potential attack techniques can be thwarted.
Grace Li Zhang, Bing Li 0005, Bei Yu 0001, David Z. Pan, Ulf Schlichtmann
DATE1
2018 Design-Phase Buffer Allocation for Post-Silicon Clock Binning by Iterative Learning
abstract
At submicrometer manufacturing technology nodes, process variations affect circuit performance significantly. To counter these variations, engineers are reserving more timing margin to maintain yield, leading to an unaffordable overdesign. Most of these margins, however, are wasted after manufacturing, because process variations cause only some chips to be really slow, while other chips can easily meet given timing specifications. To reduce this pessimism, we can reserve less timing margin and tune failed chips after manufacturing with clock buffers to make them meet timing specifications. With this post-silicon clock tuning, critical paths can be balanced with neighboring paths in each chip specifically to counter the effect of process variations. Consequently, chips with timing failures can be rescued and the yield can thus be improved. This is specially useful in high-performance designs, e.g., high-end CPUs, where clock binning makes chips with higher performance much more profitable. In this paper, we propose a method to determine where to insert post-silicon tuning buffers during the design phase to improve the overall profit with clock binning. This method learns the buffer locations with a Sobol sequence iteratively and reduces the buffer ranges afterward with tuning concentration and buffer grouping. Experimental results demonstrate that the proposed method can achieve a profit improvement of about 14% on average and up to 26%, with only a small number of tuning buffers inserted into the circuit.
Grace Li Zhang, Bing Li 0005, Jinglan Liu, Yiyu Shi 0001, Ulf Schlichtmann
IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.1
2016 EffiTest: efficient delay test and statistical prediction for configuring post-silicon tunable buffers
abstract
At nanometer manufacturing technology nodes, process variations significantly affect circuit performance. To combat them, post-silicon clock tuning buffers can be deployed to balance timing budgets of critical paths for each individual chip after manufacturing. The challenge of this method is that path delays should be measured for each chip to configure the tuning buffers properly. Current methods for this delay measurement rely on path-wise frequency stepping. This strategy, however, requires too much time from expensive testers. In this paper, we propose an efficient delay test framework (EffiTest) to solve the post-silicon testing problem by aligning path delays using the already-existing tuning buffers in the circuit. In addition, we only test representative paths and the delays of other paths are estimated by statistical delay prediction. Experimental results demonstrate that the proposed method can reduce the number of frequency stepping iterations by more than 94% with only a slight yield loss.
Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann
DAC1
2016 Sampling-based buffer insertion for post-silicon yield improvement under process variability
Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann
DATE1
2016 PieceTimer: a holistic timing analysis framework considering setup/hold time interdependency using a piecewise model
abstract
In static timing analysis, clock-to-q delays of flip-flops are considered as constants. Setup times and hold times are characterized separately and also used as constants. The characterized delays, setup times and hold times, are applied in timing analysis independently to verify the performance of circuits. In reality, however, clock-to-q delays of flip-flops depend on both setup and hold times. Instead of being constants, these delays change with respect to different setup/hold time combinations. Consequently, the simple abstraction of setup/hold times and constant clock-to-q delays introduces inaccuracy in timing analysis. In this paper, we propose a holistic method to consider the relation between clock-to-q delays and setup/hold time combinations with a piecewise linear model. The result is more accurate than that of traditional timing analysis, and the incorporation of the interdependency between clock-to-q delays, setup times and hold times may also improve circuit performance.
Grace Li Zhang, Bing Li 0005, Ulf Schlichtmann
ICCAD1