The growing prevalence and computational demands of Artificial Intelligence (AI) workloads has led to widespread use of hardware accelerators in their execution. Scaling the performance of AI accelerators across generations is pivotal to their success in commercial deployments. The intrinsic error-resilient nature of AI workloads present a unique opportunity for performance/energy improvement through precision scaling. Motivated by the recent algorithmic advances in precision scaling for inference and training, we designed RaPiD1, a 4-core AI accelerator chip supporting a spectrum of precisions, namely, 16 and 8-bit floating-point and 4 and 2-bit fixed-point. The 36mm2RaPiD chip fabricated in 7nm EUV technology delivers a peak 3.5 TFLOPS/W in HFP8 mode and 16.5 TOPS/W in INT4 mode at nominal voltage. Using a performance model calibrated to within 1% of the measurement results, we evaluated DNN inference using 4-bit fixed-point representation for a 4-core 1 RaPiD chip system and DNN training using 8-bit floating point representation for a 768 TFLOPs AI system comprising 4 32-core RaPiD chips. Our results show INT4 inference for batch size of 1 achieves 3 - 13.5 (average 7) TOPS/W and FP8 training for a mini-batch of 512 achieves a sustained 102 - 588 (average 203) TFLOPS across a wide range of applications.

Advances in deep neural networks (DNNs) and the availability of massive real-world data have enabled superhuman levels of accuracy on many AI tasks and ushered the explosive growth of AI workloads across the spectrum of computing devices. However, their superior accuracy comes at a high computational cost, which necessitates approaches beyond traditional computing paradigms to improve their operational efficiency. Leveraging the application-level insight of error resilience, we demonstrate how approximate computing (AxC) can significantly boost the efficiency of AI platforms and play a pivotal role in the broader adoption of AI-based applications and services. To this end, we present RaPiD, a multi-tera operations per second (TOPS) AI hardware accelerator core (fabricated at 14-nm technology) that we built from the ground-up using AxC techniques across the stack including algorithms, architecture, programmability, and hardware. We highlight the workload-guided systematic explorations of AxC techniques for AI, including custom number representations, quantization/pruning methodologies, mixed-precision architecture design, instruction sets, and compiler technologies with quality programmability, employed in the RaPiD accelerator.

The combination of growth in compute capabilities and availability of large datasets has led to a re-birth of deep learning. Deep Neural Networks (DNNs) have become state-of-the-art in a variety of machine learning tasks spanning domains across vision, speech, and machine translation. Deep Learning (DL) achieves high accuracy in these tasks at the expense of 100s of ExaOps of computation; posing significant challenges to efficient large-scale deployment in both resource-constrained environments and data centers.

A detailed synthesis study has been performed on a functional unit from a recent IBM microprocessor to explore the voltage-frequency space for energy-efficient design points across a wide performance spectrum ranging from 625 MHz at 0.48V to 5.6 GHz at 0.95V. It is found that the optimal operating voltage depends strongly on frequency for an energy-efficient design. Circuit characteristics, as represented by the combination of the average gate width, effective VT, and buffering scheme, differ significantly between designs optimized for low voltage-frequency and for high voltage-frequency operations and suggest a distinct application dependence in the selection of standard cell images and optimal design points. In particular, for optimal energy efficiency at a given frequency, low voltage designs should utilize smaller gate width and lower VT. Though a design energy-optimized near 1V is more scalable over a wide frequency range when operating at low voltages, designs optimized at a lower voltage-frequency point can be leveraged to offer better solutions in both performance and energy efficiency within a narrow frequency range near the design point.

Ring-oscillator-based test structures that can separately measure the negative bias temperature instability (NBTI) and positive bias temperature instability (PBTI) degradation effects in digital circuits are presented for high-k metal gate devices. The mathematical derivation also shows that the structure for frequency degradation measurement can directly be used for estimating the portion of the NBTI and PBTI in the conventional ring oscillator. The proposed test structures including frequency degradation sensing circuitry have been implemented in an experimental high-k/metal gate SoI process.

Power consumption has become a major consideration in nanometer chip design. Since the dynamic power is proportional to V dd2, and the static power is proportional to V dd, lowering power supply voltage is an efficient method to reduce the power usage.

Ring oscillator based test structures that can separately measure the NBTI and PBTI degradation effects in digital circuits are presented for high-k metal-gate devices. The proposed test structures enable simultaneous stress of all devices under test in either NBTI or PBTI mode and measure frequency or threshold voltage shifts. The mathematical derivation also shows that the structure for frequency degradation measurement can directly be used for estimating the portion of the NBTI and PBTI in the conventional ring oscillator. The proposed test structures including beat frequency sensing circuitry have been designed in a 0.9V, 45nm SOI technology.

We have reported previously [1] a low-swing latch (LSL) with superior performance-power tradeoff compared to the conventional pass-gate master-slave latch. In this paper, hardware results are presented for the proposed LSL with pulsed clock waveforms. The motivation is to combine low-voltage swing with pulsed signals to further reduce overall system power in high-frequency microprocessors. We have designed a 65-bit accumulator loop experiment to mimic a microprocessor pipeline stage. The local clock buffer design features a mode switch to toggle between two-phase (c1/c2) master-slave clocking and one-phase pulsed (c2 only) clocking. Our data show that 15-25% system power saving can be achieved in pulsed mode compared to non-pulsed mode. Power contribution from individual components is also presented.

In this paper we present techniques shown to significantly enhance the custom circuit design process typical of high-performance microprocessors. This methodology combines flexible custom circuit design with automated tuning and physical design tools to provide new opportunities to optimized design throughout the development cycle.

This paper reviews the recent advances of silicon-on-insulator (SOI) technology for complementary metal-oxide-semiconductor (CMOS) very-large-scale-integration memory and logic applications. Static random access memories (SRAMs), dynamic random access memories (DRAMs), and digital CMOS logic circuits are considered. Particular emphases are placed on the design issues and advantages resulting from the unique SOI device structure. The impact of floating-body in partially depleted devices on the circuit operation, stability, and functionality are addressed. The use of smart-body contact to improve the power and delay performance is discussed, as are global design issues.

This paper presents a detailed study on the impact of floating body in partially-depleted (PD) SOI MOSFET on various digital VLSI CMOS circuit families. The parasitic bipolar effect resulting from the floating body is shown to degrade the circuit noise margin and stability in general. In certain dynamic circuits and wide multiplexers, the parasitic bipolar effect is shown to cause logic state error if not properly accounted for.