Graphics Processing Units (GPUs) have become an attractive platform for accelerating challenging applications on a range of platforms, from High Performance Computing (HPC) to full-featured smartphones. They can overcome computational barriers in a wide range of data-parallel kernels. GPUs hide pipeline stalls and memory latency by utilizing efficient thread preemption. But given the demands on the memory hierarchy due to the growth in the number of computing cores on-chip, it has become increasingly difficult to hide all of these stalls. In this article, we propose a novel Hint-Assisted Wavefront Scheduler (HAWS) to bypass long-latency stalls. HAWS starts by enhancing a compiler infrastructure to identify potential opportunities that can bypass memory stalls. HAWS includes a wavefront scheduler that can continue to execute instructions in the shadow of a memory stall, executing instructions speculatively, guided by compiler-generated hints. HAWS increases utilization of GPU resources by aggressively fetching/executing speculatively. Based on our simulation results on the AMD Southern Islands GPU architecture, at an estimated cost of 0.4% total chip area, HAWS can improve application performance by 14.6% on average for memory intensive applications.

Presilicon simulation is one of the key toolsets for computer architects to evaluate and optimize their future designs. As Graphics Processing Units (GPUs) have become the platform of choice in many computing communities due to their impressive processing capabilities, computer architecture researchers need a simulation framework that allows them to quantitatively consider design tradeoffs. In this paper, we present the Multi2Sim Kepler simulator framework, a new detailed GPU microarchitecture performance simulator that supports NVIDIA's Kepler shader assembly (SASS) code execution. The toolset provides a disassembler, a functional simulator and a detailed cycle-based simulator. We provide insight into the architecture of the NVIDIA Kepler GPU, describing the details of the streaming multiprocessor, front-end and instruction pipelines. We compare the performance of this new simulator against an NVIDIA K20X, a high-end Kepler device. We also evaluate the performance of NVIDIA's CUDA benchmark suite on our GPU performance simulator.