Processing in-memory (PIM) has emerged as a promising approach to address the increasingly memory bound nature of modern applications like machine learning and genomics. While PIM-enabled memories offer significant performance and energy improvements over host-side execution, integration of such memories into existing systems remains an open challenge. In particular, naively replacing regular memory with a PIM-enabled one in a conventional processor could be detrimental to its performance. PIM applications are optimized to saturate the memory subsystem to maximize speedup. However, since modern processors, including CPUs and GPUs, support multi-tenancy to improve utilization, such saturation can lead to extreme unfairness and denial of service to other applications. In this paper, we characterize the performance of a PIMenabled GPU system when co-executing regular GPU kernels with a PIM kernel. Our characterization shows that PIM kernels can easily overwhelm the interconnect and the memory controller and severely degrade the performance of the non-PIM kernel, hurting system-level fairness and throughput metrics. Based on this characterization, we propose changes to the interconnect that ease the flow of requests from the processor to the memory controller. At the memory controller, we propose a new scheduling policy, called F3FS, that optimizes for fairness and throughput. While F3FS benefits from changes to the interconnect, we show that it performs comparably to existing policies without them. We evaluate and compare the proposed changes to state-of-theart memory controller scheduling policies under both competitive (two kernels from different applications) and collaborative (two kernels from same application) scenarios.

Data movement latency when using on-chip accelerators in emerging heterogeneous architectures is a serious performance bottleneck. While hardware/software mechanisms such as peer-to-peer DMA between producer/consumer accelerators allow bypassing main memory and significantly reduce main memory contention, schedulers in both the hardware and software domains remain oblivious to their presence. Instead, most contemporary schedulers tend to be deadline-driven, with improved utilization and/or throughput serving as secondary or co-primary goals. This lack of focus on data communication will only worsen execution times as accelerator latencies reduce. In this paper, we present RELIEF (RElaxing Least-laxIty to Enable Forwarding), an online least laxity-driven accelerator scheduling policy that relieves memory pressure in accelerator-rich architectures via data movement-aware scheduling. RELIEF leverages laxity (time margin to a deadline) to opportunistically utilize available hardware data forwarding mechanisms while minimizing quality-of-service (QoS) degradation and unfairness. RELIEF achieves up to 50 % more forwards compared to state-of-the-art policies, reducing main memory traffic and energy consumption by up to 32 % and 18 %, respectively. At the same time, RELIEF meets 14% more task deadlines on average and reduces worst-case deadline violation by 14%, highlighting QoS and fairness improvements.

With the advent of edge computing and 5G, multiple mobile applications are being offloaded to cloud servers to meet their computational demands. Computer vision workloads dominate this space. Since the vision workloads are composed of linear algebra kernels, they perform significantly well on SIMT/SIMD architectures such as GPUs. While an application can maximize its performance on a GPU when it is the sole consumer of the GPU's resources, it fails to maintain that performance in a multi-application scenario. The primary cause of this problem is the lack of efficient virtualization techniques for GPUs and contention among the applications for the shared resources. Sadly, most of the prior work in this area is devoted to predicting single application performance. To the best of our knowledge we propose the first machine learning based predictor to predict the performance of an ensemble of applications on a GPU. Our predictor achieves an error of 9% across a suite of representative vision workloads for predicting the execution time. Competing algorithms that primarily work for single application scenarios have significantly inferior prediction accuracy and their error rates are more than 140%.