Modern neural networks have undergone extensive hardware optimization to address the increasing computational demands. While most existing acceleration strategies concentrate on linear operations, the relative cost of these nonlinear operations has become a critical efficiency bottleneck. This work presents QUNF+, a hardware-centric, quadratic-based approximation framework that offers a universal, scalable, and accurate approach for accelerating a wide range of nonlinear functions. Unlike conventional piecewise linear methods, QUNF+ segments functions uniformly and applies second-order Taylor expansions, yielding superior accuracy with fewer segments. QUNF+ also introduces a hardware-efficient approximation scheme with adjustable precision and an optional remainder compensation mechanism. In addition, we propose a set of novel function remapping techniques further reducing approximation error with minimal overhead. We design a fully integrated hardware architecture incorporating Canonical Signed Digit encoding and logic pruning to minimize resource usage without accuracy loss. Experimental results demonstrate that QUNF+ achieves up to 4.1x, 1.5x and 1.7x improvements in power, area, and latency, respectively, over state-of-the-art PWL methods. Application to real-world Transformer models results in an accuracy degradation of less than 0.72% and 0.10% on NLP and CV tasks, with system-level evaluations showing up to 24.38% reduction in total energy consumption and 92.39% in nonlinear operation energy. These results establish QUNF+ as a robust and scalable solution for nonlinear acceleration in modern AI hardware.

Large language models (LLMs) have been the driving force behind significant advancements in artificial intelligence. However, their unique self-attention mechanism leads to difficulties in accelerating the inference. Softmax, with its complex nonlinear operations and low parallelism, significantly limits the efficiency of LLMs for long sequences. This work proposes a novel high-parallelism hardware/software co-design Softmax solution. By incorporating a sum estimation algorithm, we eliminate the need for complex exponential on all elements. A high-speed low-energy hardware architecture is introduced by applying high-parallelism statistical module and simplified division module. Experimental results demonstrate that our approach achieves a 9.90–44.75% reduction in latency and up to a 17.36% reduction in energy consumption compared with state-of-the-art Softmax hardware, with minimal impact on model inference accuracy.

Nonlinear functions (NFs) in Transformers require high-precision computation consuming significant time and energy, despite the aggressive quantization schemes for other components. Piece-wise Linear (PWL) approximation-based methods offer more efficient processing schemes for NFs but fall short in dealing with functions with high nonlinearities. Moreover, PWL-based methods still suffer from inevitably high latency introduced by the Multiply-And-Add (MADD) unit. To address these issues, this paper proposes a novel quadratic approximation scheme and a highly integrated, multiplier-less hardware structure, as a unified method to accelerate any unary nonlinear function. We also demonstrate implementation examples for GELU, Softmax, and LayerNorm. The experimental results show that the proposed method achieves up to 5.41% higher inference accuracy and 60.12% lower area-delay product.

Approximate computing emerges as a viable solution to enhance energy efficiency in applications sensitive to human perception, particularly on edge devices. This work introduces a novel piece-wise approximate floating-point divider that boasts resource efficiency and runtime configurability. Our method leverages a piece-wise approximation algorithm for computing 1/ y by exploiting powers of 2, complemented by an error compensation technique grounded in thorough mathematical analysis. This approach facilitates the realization of a reciprocal-based floating-point divider devoid of multipliers, which not only mitigates hardware resource consumption but also reduces latency. Additionally, we unveil a multi-level runtime configurable hardware architecture that significantly improves flexibility across diverse application contexts. Compared to the existing state-of-the-art approximate dividers and truncated exact dividers, our proposed solution achieves a superior compromise between precision and resource efficiency. Application-level evaluations reveal that our design provides over 87.7% energy saving while maintaining a negligible impact on output quality.

Approximate computing is a promising alternative to improve energy efficiency for human perception related applications on the edge. This work proposes a piece-wise approximate floating-point divider, which is resource-efficient and run-time configurable. We provide a piece-wise approximation algorithm for 1/ y, utilizing powers of 2. This approach enables the implementation of a reciprocal-based floating-point divider that is independent of multipliers, which not only reduces hardware consumption but also results in shorter latency. Furthermore, a multi-level run-time configurable hardware structure is intro-duced, enhancing the adaptability to various application scenarios. When compared to the prior state-of-the-art approximate divider, the proposed divider strikes an advantageous balance between accuracy and resource efficiency. The application-level evaluation of the proposed dividers demonstrates manageable and minimal degradation of the output quality when compared to the exact divider.

Large Language Models (LLMs) have revolutionized many domains with exceptional performance while their large sizes hinder their broad applicability, especially in the edge computation scenarios. Designing large-scale LLM-specific accelerators is also challenging, suffering from the complicated, cumbersome, and time-consuming design, simulation, and optimization process. This paper meticulously proposes an agile framework for accelerator development, supporting efficient LLM inference. Firstly, we investigate the architecture of LLMs, uncover performance bottlenecks, and design an optimized binarized accelerator and a configurable RISC-V-based SoC to boost the inference of binary LLMs. Further, a novel fidelity-driven method is proposed to learn the multi-fidelity representation, solving the modeling and accuracy issues due to the lack of accurate later-stage data in the EDA flow, by capturing complex relationships among simulation metrics in and across different fidelities. Tailored strategies across model preparation, backend kernel implementations, agile accelerator and SoC design, and inference simulation are incorporated into our framework to refine the development workflow. Our method significantly accelerates the hardware design, simulation, and optimization processes. Experimental results illustrate the impressive speed and effectiveness of our framework in designing edge LLM accelerators and optimizing LLM inference.

Given the stringent requirements of energy efficiency for Internet-of-Things edge devices, approximate multipliers, as a basic component of many processors and accelerators, have been constantly proposed and studied for decades, especially in error-resilient applications. The computation error and energy efficiency largely depend on how and where the approximation is introduced into a design. Thus, this article aims to provide a comprehensive review of the approximation techniques in multiplier designs ranging from algorithms and architectures to circuits. We have implemented representative approximate multiplier designs in each category to understand the impact of the design techniques on accuracy and efficiency. The designs can then be effectively deployed in high-level applications, such as machine learning, to gain energy efficiency at the cost of slight accuracy loss.

Fast Fourier Transform (FFT) is a key digital signal processing algorithm that is widely deployed in mobile and portable devices. Recently, with the popularity of human perception related tasks, it is noted that the requirements of full precision and exactness are not always necessary for FFT computation. We propose a top-down approximate Floating-Point FFT design methodology to fully exploit the error-tolerance nature of the FFT algorithm. An efficient error modeling of the configurable approximate multiplier is proposed to link the multiplier approximation to the FFT algorithm precision. Then an approximation optimization flow is formulated to maximize the energy efficiency. Experimental results show that the proposed approximate FFT can achieve up to 52% Area-Delay-Product improvement and 23% energy saving when compared to the exact FFT. The proposed approximate FFT is also found to cover almost 2X wider precision range with higher energy efficiency in comparison with the prior state-of-the-art approximate FFT.

The conventional on-chip spiral inductor consumes a significant top-metal routing area, thereby preventing its popularity in many on-chip applications. Recently through-silicon-via– (TSV) based inductor (also known as a TSV-inductor) with a magnetic core has been proved to be a viable option for the on-chip DC-DC converter. The operating conditions of these inductors play a major role in maximizing the performance and efficiency of the DC-DC converter. However, there is a critical need to study the design and optimization details of magnetic core TSV-inductors with the unique three-dimensional structure embedding magnetic core. This article aims to provide a clear understanding of the modeling details of a magnetic core TSV-inductor and a design and optimization methodology to assist efficient inductor design. Moreover, a machine learning–assisted model combining physical details and artificial neural network is also proposed to extract the equivalent circuit to further facilitate DC-DC converter design. Experimental results show that the optimized TSV-inductor with the magnetic core and air-gap can achieve inductance density improvement of up to 7.7 \( \times \) and quality factor improvements of up to 1.6 \( \times \) for the same footprint compared with the TSV-inductor without a magnetic core. For on-chip DC-DC converter applications, the converter efficiency can be improved by up to 15.9% and 6.8% compared with the conventional spiral and TSV-inductor without magnetic core, respectively.