The growing prevalence and computational demands of Artificial Intelligence (AI) workloads has led to widespread use of hardware accelerators in their execution. Scaling the performance of AI accelerators across generations is pivotal to their success in commercial deployments. The intrinsic error-resilient nature of AI workloads present a unique opportunity for performance/energy improvement through precision scaling. Motivated by the recent algorithmic advances in precision scaling for inference and training, we designed RaPiD1, a 4-core AI accelerator chip supporting a spectrum of precisions, namely, 16 and 8-bit floating-point and 4 and 2-bit fixed-point. The 36mm2RaPiD chip fabricated in 7nm EUV technology delivers a peak 3.5 TFLOPS/W in HFP8 mode and 16.5 TOPS/W in INT4 mode at nominal voltage. Using a performance model calibrated to within 1% of the measurement results, we evaluated DNN inference using 4-bit fixed-point representation for a 4-core 1 RaPiD chip system and DNN training using 8-bit floating point representation for a 768 TFLOPs AI system comprising 4 32-core RaPiD chips. Our results show INT4 inference for batch size of 1 achieves 3 - 13.5 (average 7) TOPS/W and FP8 training for a mini-batch of 512 achieves a sustained 102 - 588 (average 203) TFLOPS across a wide range of applications.

The design of the modern, enterprise-class IBM z15 branch predictor is described. Implemented as a multilevel look-ahead structure, the branch predictor is capable of predicting branch direction and target addresses, augmented with multiple auxiliary direction, target, and power predictors. Predictions are made asynchronously, and later integrated into the processor pipeline. The design is optimized for the unique workloads executed on these enterprise-class systems, including compute intensive and both large instruction and data footprint workloads. This paper highlights the major operations and functions of the IBM z15 branch predictor, including its pipeline, prediction structures and verification methodology. Explanations as to how the design matured to its current state are also provided.

This paper describes the large capacity hierarchical branch predictor in the 5.5 GHz IBM zEnterprise EC12 microprocessor. Performance analyses in a simulation model and on zEC12 hardware demonstrate the benefit of this hierarchy compared to a smaller one level predictor. Novel structures and algorithms for two level branch prediction are presented. Prediction information about multiple branches is bulk transferred from the second level into the first upon detecting a perceived miss in the first level. The second level does not directly make branch predictions. Access to the second level is limited when it is unlikely to be productive. The second level is systematically searched in an order that is likely to provide hits as early as possible. On the workloads analyzed in the simulation model, measurements show a maximum core performance benefit of 13.8%. On the two workloads analyzed on zEC12 hardware 3.4% and 5.3% system performance improvements are achieved.