The growing prevalence and computational demands of Artificial Intelligence (AI) workloads has led to widespread use of hardware accelerators in their execution. Scaling the performance of AI accelerators across generations is pivotal to their success in commercial deployments. The intrinsic error-resilient nature of AI workloads present a unique opportunity for performance/energy improvement through precision scaling. Motivated by the recent algorithmic advances in precision scaling for inference and training, we designed RaPiD1, a 4-core AI accelerator chip supporting a spectrum of precisions, namely, 16 and 8-bit floating-point and 4 and 2-bit fixed-point. The 36mm2RaPiD chip fabricated in 7nm EUV technology delivers a peak 3.5 TFLOPS/W in HFP8 mode and 16.5 TOPS/W in INT4 mode at nominal voltage. Using a performance model calibrated to within 1% of the measurement results, we evaluated DNN inference using 4-bit fixed-point representation for a 4-core 1 RaPiD chip system and DNN training using 8-bit floating point representation for a 768 TFLOPs AI system comprising 4 32-core RaPiD chips. Our results show INT4 inference for batch size of 1 achieves 3 - 13.5 (average 7) TOPS/W and FP8 training for a mini-batch of 512 achieves a sustained 102 - 588 (average 203) TFLOPS across a wide range of applications.

Advances in deep neural networks (DNNs) and the availability of massive real-world data have enabled superhuman levels of accuracy on many AI tasks and ushered the explosive growth of AI workloads across the spectrum of computing devices. However, their superior accuracy comes at a high computational cost, which necessitates approaches beyond traditional computing paradigms to improve their operational efficiency. Leveraging the application-level insight of error resilience, we demonstrate how approximate computing (AxC) can significantly boost the efficiency of AI platforms and play a pivotal role in the broader adoption of AI-based applications and services. To this end, we present RaPiD, a multi-tera operations per second (TOPS) AI hardware accelerator core (fabricated at 14-nm technology) that we built from the ground-up using AxC techniques across the stack including algorithms, architecture, programmability, and hardware. We highlight the workload-guided systematic explorations of AxC techniques for AI, including custom number representations, quantization/pruning methodologies, mixed-precision architecture design, instruction sets, and compiler technologies with quality programmability, employed in the RaPiD accelerator.

The combination of growth in compute capabilities and availability of large datasets has led to a re-birth of deep learning. Deep Neural Networks (DNNs) have become state-of-the-art in a variety of machine learning tasks spanning domains across vision, speech, and machine translation. Deep Learning (DL) achieves high accuracy in these tasks at the expense of 100s of ExaOps of computation; posing significant challenges to efficient large-scale deployment in both resource-constrained environments and data centers.

High frequency microprocessor designs require rigorous design guidelines, design methodology advancements, and novel approaches in circuit design style for processors operating in the high megahertz range. Timing closure becomes the single most important design issue, however other design metrics such as area, power and noise need to be given equal consideration within the design cycle. Custom design techniques were used through out the logic circuits and arrays as well as the overall design planning for the 500 MHz microprocessor cycle time.

This paper describes the CMOS circuit techniques used in the design of the high performance Generation-4 S/390 microprocessor. Successful system operation at frequencies up to 400 MHz was achieved through careful static circuit design and timing optimization, along with the limited use of dynamic circuits for highly critical functions, and several different clocking/latching strategies for cycle time reduction. A variety of innovative full-custom circuit techniques were used in the dataflow design. Timing-driven synthesis of the control logic provided maximum flexibility with minimum turn-around time, while still matching the performance level set by the custom parts of the design. The on-chip LI cache was designed extensively with self-resetting CMOS (SRCMOS) circuitry to provide a 2.0 ns access time and up to 500 MHz operation.

In this paper, we address the off-chip driver delta-I or switching noise problem. We propose a novel approach based on switching codes to reduce this noise. These codes are designed to lower the number of drivers which switch in any given system cycle. A formal theoretic framework for switching codes is developed which provides an upper bound on the number of switching drivers as a function of bit width of the data bus and number of redundant switch bits. Switching codes with error detecting and error correcting capabilities are also presented. Switching codes also reduce the average switching activity and power consumed by the off-chip drivers.