The growing prevalence and computational demands of Artificial Intelligence (AI) workloads has led to widespread use of hardware accelerators in their execution. Scaling the performance of AI accelerators across generations is pivotal to their success in commercial deployments. The intrinsic error-resilient nature of AI workloads present a unique opportunity for performance/energy improvement through precision scaling. Motivated by the recent algorithmic advances in precision scaling for inference and training, we designed RaPiD1, a 4-core AI accelerator chip supporting a spectrum of precisions, namely, 16 and 8-bit floating-point and 4 and 2-bit fixed-point. The 36mm2RaPiD chip fabricated in 7nm EUV technology delivers a peak 3.5 TFLOPS/W in HFP8 mode and 16.5 TOPS/W in INT4 mode at nominal voltage. Using a performance model calibrated to within 1% of the measurement results, we evaluated DNN inference using 4-bit fixed-point representation for a 4-core 1 RaPiD chip system and DNN training using 8-bit floating point representation for a 768 TFLOPs AI system comprising 4 32-core RaPiD chips. Our results show INT4 inference for batch size of 1 achieves 3 - 13.5 (average 7) TOPS/W and FP8 training for a mini-batch of 512 achieves a sustained 102 - 588 (average 203) TFLOPS across a wide range of applications.

Advances in deep neural networks (DNNs) and the availability of massive real-world data have enabled superhuman levels of accuracy on many AI tasks and ushered the explosive growth of AI workloads across the spectrum of computing devices. However, their superior accuracy comes at a high computational cost, which necessitates approaches beyond traditional computing paradigms to improve their operational efficiency. Leveraging the application-level insight of error resilience, we demonstrate how approximate computing (AxC) can significantly boost the efficiency of AI platforms and play a pivotal role in the broader adoption of AI-based applications and services. To this end, we present RaPiD, a multi-tera operations per second (TOPS) AI hardware accelerator core (fabricated at 14-nm technology) that we built from the ground-up using AxC techniques across the stack including algorithms, architecture, programmability, and hardware. We highlight the workload-guided systematic explorations of AxC techniques for AI, including custom number representations, quantization/pruning methodologies, mixed-precision architecture design, instruction sets, and compiler technologies with quality programmability, employed in the RaPiD accelerator.

The resilience of Deep Learning (DL) training and inference workloads to low-precision computations, coupled with the demand for power-and area-efficient hardware accelerators for these workloads, has led to the emergence of 16-bit floating point formats as the precision of choice for DL hardware accelerators. This paper describes our optimized 16-bit format that has 6 exponent bits and 9 fraction bits, derived from a study of the range of values encountered in DL applications. We demonstrate that our format preserves the accuracy of DL networks, and we compare its ease-of-use for DL against IEEE-754 half-precision (5 exponent bits and 10 fraction bits) and bfloat16 (8 exponent bits and 7 fraction bits). Further, our format eliminated sub-normals and simplifies rounding modes and handling of corner cases. This streamlines floating-point unit logic and enables realization of a compact power-efficient computation engine.

The combination of growth in compute capabilities and availability of large datasets has led to a re-birth of deep learning. Deep Neural Networks (DNNs) have become state-of-the-art in a variety of machine learning tasks spanning domains across vision, speech, and machine translation. Deep Learning (DL) achieves high accuracy in these tasks at the expense of 100s of ExaOps of computation; posing significant challenges to efficient large-scale deployment in both resource-constrained environments and data centers.

Local variation of delay paths has a significant impact on modern microprocessor performance and yield. A critical path monitor is reported which extracts timing variability information on various critical paths, including sample processor paths. The very compact circuit contains 256 copies of 15 different delay paths, enabling measurement of the statistics of delay variation, as a function of threshold voltage, supply voltage, fanout, temperature, and circuit topology. Measurements of 14nm SOI finFET [1] circuit path delays are presented. The reported sensor can offer a variety of advantages on a processor chip, ranging from testing time improvement to power savings.

A detailed synthesis study has been performed on a functional unit from a recent IBM microprocessor to explore the voltage-frequency space for energy-efficient design points across a wide performance spectrum ranging from 625 MHz at 0.48V to 5.6 GHz at 0.95V. It is found that the optimal operating voltage depends strongly on frequency for an energy-efficient design. Circuit characteristics, as represented by the combination of the average gate width, effective VT, and buffering scheme, differ significantly between designs optimized for low voltage-frequency and for high voltage-frequency operations and suggest a distinct application dependence in the selection of standard cell images and optimal design points. In particular, for optimal energy efficiency at a given frequency, low voltage designs should utilize smaller gate width and lower VT. Though a design energy-optimized near 1V is more scalable over a wide frequency range when operating at low voltages, designs optimized at a lower voltage-frequency point can be leveraged to offer better solutions in both performance and energy efficiency within a narrow frequency range near the design point.

We present a nonintrusive concurrent error detection (CED) method for protecting the control logic of a contemporary floating-point unit (FPU). The proposed method is based on the observation that control logic errors lead to extensive data path corruption and affect, with high probability, the exponent part of the IEEE-754 floating-point representation. Thus, exponent monitoring can be utilized to detect errors in the control logic of the FPU. Predicting the exponent involves relatively simple operations; therefore, our method incurs significantly lower overhead than the classical approach of duplicating the control logic of the FPU. Indeed, experimental results on the openSPARC T1 processor using SPEC2006FP benchmarks show that as compared to control logic duplication, which incurs an area overhead of 17.9 percent of the FPU size, our method incurs an area overhead of only 5.8 percent yet still achieves detection of over 93 percent of transient errors in the FPU control logic. Moreover, the proposed method offers the ancillary benefit of also detecting 98.1 percent of the data path errors that affect the exponent, which cannot be detected via duplication of control logic. Finally, when combined with a classical residue code-based method for the fraction, our method leads to a complete CED solution for the entire FPU which provides a coverage of 94.1 percent of all errors at an area cost of 16.32 percent of the FPU size.

We present a non-intrusive concurrent error detection (CED) method for protecting the control logic of a contemporary floating point unit (FPU). The proposed method is based on the observation that control logic errors lead to extensive datapath corruption and affect, with high probability, the exponent part of the IEEE 754 floating point representation. Thus, exponent monitoring can be utilized to detect errors in the control logic of the FPU. Predicting the exponent involves relatively simple operations, therefore our method incurs significantly lower overhead than the classical approach of duplicating the control logic of the FPU. Indeed, experimental results on the openSPARC T1 processor show that, as compared to control logic duplication, which incurs an area overhead of 17.9% of the FPU size, our method incurs an area overhead of only 5.8% yet still achieves detection of over 95% of transient errors in the FPU control logic. Moreover, the proposed method offers the ancillary benefit of also detecting 98.1% of datapath errors that affect the exponent, which cannot be detected via duplication of control logic. Finally, when combined with a classical residue code-based method for the fraction, our method leads to a complete CED solution for the entire FPU which provides a coverage of 94.4% of all errors at an area cost of 16.32% of the FPU size.

Static timing analysis techniques are widely used to verify the timing behavior of large digital designs implemented predominantly in conventional static CMOS. These techniques, however, are not sufficient to completely verify the dynamic circuit families now finding favor in high-performance designs. In this paper, we describe an approach that extends static timing analysis to a high-performance dynamic CMOS logic family called self-resetting CMOS (SRCMOS). Due to the circuit structure employed in SRCMOS, designs naturally decompose into a hierarchy of gates and macros; timing analysis must address and preferably exploit this hierarchy. At the gate level, three categories of constraints on pulse timing arise from considering the effects of pulse width, overlap, and collisions. Timing analysis is performed at the macro level, by a) performing timing tests at macro boundaries and b) using macro-level delay models. We define various macro-level timing tests which ensure that fundamental gate-level timing constraints are satisfied. We extend the standard delay model to handle leading and trailing edges of signal pulses, across-chip variations, trading of signals, and slow and fast operating conditions. We have developed an SRCMOS timing analyzer based on this approach; the analyzer implemented as extensions to a standard static timing analysis program, thus facilitating its integration into an existing design system and methodology.