Large language models have led to state-of-the-art accuracies across several tasks. However, training these models efficiently is challenging because: a) GPU memory capacity is limited, making it impossible to fit large models on even a multi-GPU server, and b) the number of compute operations required can result in unrealistically long training times. Consequently, new methods of model parallelism such as tensor and pipeline parallelism have been proposed. Unfortunately, naive usage of these methods leads to scaling issues at thousands of GPUs. In this paper, we show how tensor, pipeline, and data parallelism can be composed to scale to thousands of GPUs. We propose a novel interleaved pipelining schedule that can improve throughput by 10+% with memory footprint comparable to existing approaches. Our approach allows us to perform training iterations on a model with 1 trillion parameters at 502 petaFLOP/s on 3072 GPUs (per-GPU throughput of 52% of theoretical peak).

Understanding user experience/satisfaction with mobile systems in order to manage computational resources has become a popular approach in recent years. One of the key challenges in this area is how to gauge user satisfaction. In this paper, we study the impact of CPU configuration on user satisfaction and power consumption with real users. Specifically, we propose a system to save energy by altering active CPU core count and frequency while keeping users satisfied. The system utilizes user-facing metrics such as frame rate and input lag to predict user satisfaction and then configure CPU core count and frequency in real-time to maximize satisfaction while minimizing power consumption. We first study a set of applications in-the-lab and show that we can accurately model satisfaction with the collected user-facing metrics. We then go into-the-wild in order to evaluate the proposed system in real environments. In the wild, we build a user-independent (user-oblivious) and user-dependent (personal) model. Users test the two models and the default scheme for one-week duration, which composes 140 days of worth of data. When compared to default scheme, our results show that, without impacting satisfaction, user-independent and user-dependent models save 12.3% and 11.8% of total system energy on average, respectively.