The growing prevalence and computational demands of Artificial Intelligence (AI) workloads has led to widespread use of hardware accelerators in their execution. Scaling the performance of AI accelerators across generations is pivotal to their success in commercial deployments. The intrinsic error-resilient nature of AI workloads present a unique opportunity for performance/energy improvement through precision scaling. Motivated by the recent algorithmic advances in precision scaling for inference and training, we designed RaPiD1, a 4-core AI accelerator chip supporting a spectrum of precisions, namely, 16 and 8-bit floating-point and 4 and 2-bit fixed-point. The 36mm2RaPiD chip fabricated in 7nm EUV technology delivers a peak 3.5 TFLOPS/W in HFP8 mode and 16.5 TOPS/W in INT4 mode at nominal voltage. Using a performance model calibrated to within 1% of the measurement results, we evaluated DNN inference using 4-bit fixed-point representation for a 4-core 1 RaPiD chip system and DNN training using 8-bit floating point representation for a 768 TFLOPs AI system comprising 4 32-core RaPiD chips. Our results show INT4 inference for batch size of 1 achieves 3 - 13.5 (average 7) TOPS/W and FP8 training for a mini-batch of 512 achieves a sustained 102 - 588 (average 203) TFLOPS across a wide range of applications.

Advances in deep neural networks (DNNs) and the availability of massive real-world data have enabled superhuman levels of accuracy on many AI tasks and ushered the explosive growth of AI workloads across the spectrum of computing devices. However, their superior accuracy comes at a high computational cost, which necessitates approaches beyond traditional computing paradigms to improve their operational efficiency. Leveraging the application-level insight of error resilience, we demonstrate how approximate computing (AxC) can significantly boost the efficiency of AI platforms and play a pivotal role in the broader adoption of AI-based applications and services. To this end, we present RaPiD, a multi-tera operations per second (TOPS) AI hardware accelerator core (fabricated at 14-nm technology) that we built from the ground-up using AxC techniques across the stack including algorithms, architecture, programmability, and hardware. We highlight the workload-guided systematic explorations of AxC techniques for AI, including custom number representations, quantization/pruning methodologies, mixed-precision architecture design, instruction sets, and compiler technologies with quality programmability, employed in the RaPiD accelerator.

The combination of growth in compute capabilities and availability of large datasets has led to a re-birth of deep learning. Deep Neural Networks (DNNs) have become state-of-the-art in a variety of machine learning tasks spanning domains across vision, speech, and machine translation. Deep Learning (DL) achieves high accuracy in these tasks at the expense of 100s of ExaOps of computation; posing significant challenges to efficient large-scale deployment in both resource-constrained environments and data centers.

In this paper, we present a transient modeling of electromigration (EM) in TSV and TSV-to-wire interfaces in the power delivery network (PDN) of 3D ICs. In particular, we model atomic depletion and accumulation, effective resistance degradation, and full chip-scale PDN lifetime degradation due to EM. Our major focuses are on: (1) time-dependent multi-physics EM modeling approach to model TSVs and connecting wires under the influence of coupled physical phenomenon including electric field, temperature, and stress; (2) time-dependent EM-aware power integrity analysis methodology, which is integrated with the TSV modeling approach to predict long-term IR-drop degradation in full-chip 3D power delivery networks. Our studies show that voids and hillocks grow at various TSV-to-wire interfaces and degrade the effective resistance of TSVs significantly. In addition, our full-chip PDN lifetime analysis shows significant increase in maximum IR drop during lifetime due to EM effects.

Due to the large geometry of through-silicon-vias (TSVs) and their connections to the power grid, significant current crowding can occur in 3D ICs. Prior works model TSVs and power wire segments as single resistors, which cannot capture the detailed current distribution and may miss trouble spots associated with current crowding. This paper studies DC current crowding and its impact on 3D power integrity. First, we explore the current density distribution within a TSV and its power wire connections. Second, we build and validate effective TSV models for current density distributions. Finally, these models are integrated with global power wires for detailed chip-scale power grid analysis.

Technology scaling has provided the semiconductor industry a recipe to successfully meet the application demands for performance for over three decades. This computational capacity was further fueled by the success of circuit and architecture-level innovation, which provided performance improvement in each processor generation. However, future processor designs face a number of key challenges in sustaining the growth trends. The dawn of the 22nm node, and beyond, marks an era of new trends and challenges; where the cost and complexity associated with each technology node is increasing at much faster rate than the device performance gains. Novel tools and design methodologies are needed to not only compensate for these challenges but also to leverage emerging technologies to achieve the desired performance in future processor architectures. Technology alternatives such as 3D integration have attracted significant interest as an additional way of sustaining the density scaling and performance growth.